Sep 11 17:16:22.668412 ip-10-0-140-51 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Sep 11 17:16:22.668423 ip-10-0-140-51 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Sep 11 17:16:22.668430 ip-10-0-140-51 systemd[1]: kubelet.service: Failed with result 'resources'.
Sep 11 17:16:22.668649 ip-10-0-140-51 systemd[1]: Failed to start Kubernetes Kubelet.
Sep 11 17:16:32.788302 ip-10-0-140-51 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Sep 11 17:16:32.788318 ip-10-0-140-51 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 7dccd2da3e03490bac449e2cadbae22c --
Sep 11 17:18:23.467389 ip-10-0-140-51 systemd[1]: Starting Kubernetes Kubelet...
Sep 11 17:18:23.860750 ip-10-0-140-51 kubenswrapper[2552]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 11 17:18:23.860750 ip-10-0-140-51 kubenswrapper[2552]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 11 17:18:23.860750 ip-10-0-140-51 kubenswrapper[2552]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 11 17:18:23.860750 ip-10-0-140-51 kubenswrapper[2552]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 11 17:18:23.860750 ip-10-0-140-51 kubenswrapper[2552]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 11 17:18:23.862257 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.862164    2552 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 11 17:18:23.865997 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.865977    2552 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 17:18:23.865997 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.865994    2552 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 17:18:23.865997 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.865999    2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 17:18:23.865997 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.866003    2552 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 17:18:23.866261 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.866008    2552 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 17:18:23.866261 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.866014    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 17:18:23.866261 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.866022    2552 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 17:18:23.866261 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.866028    2552 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 17:18:23.866261 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.866032    2552 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 17:18:23.866261 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.866037    2552 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 17:18:23.866261 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.866041    2552 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 17:18:23.866261 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.866046    2552 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 17:18:23.866261 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.866050    2552 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 17:18:23.866261 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.866069    2552 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 17:18:23.866261 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.866075    2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 17:18:23.866261 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.866079    2552 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 17:18:23.866261 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.866083    2552 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 17:18:23.866261 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.866087    2552 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 17:18:23.866261 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.866092    2552 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 17:18:23.866261 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.866096    2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 17:18:23.866261 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.866100    2552 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 17:18:23.866261 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.866104    2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 17:18:23.866261 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.866108    2552 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 17:18:23.867048 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.866112    2552 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 17:18:23.867048 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.866116    2552 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 17:18:23.867048 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.866120    2552 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 17:18:23.867048 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.866125    2552 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 17:18:23.867048 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.866129    2552 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 17:18:23.867048 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.866133    2552 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 17:18:23.867048 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.866137    2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 17:18:23.867048 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.866141    2552 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 17:18:23.867048 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.866147    2552 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 17:18:23.867048 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.866151    2552 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 17:18:23.867048 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.866156    2552 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 17:18:23.867048 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.866160    2552 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 17:18:23.867048 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.866164    2552 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 17:18:23.867048 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.866169    2552 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 17:18:23.867048 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.866174    2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 17:18:23.867048 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.866178    2552 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 17:18:23.867048 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.866182    2552 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 17:18:23.867048 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.866186    2552 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 17:18:23.867048 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.866191    2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 17:18:23.867560 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.866199    2552 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 17:18:23.867560 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.866204    2552 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 17:18:23.867560 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.866209    2552 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 17:18:23.867560 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.866214    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 17:18:23.867560 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.866219    2552 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 17:18:23.867560 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.866223    2552 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 17:18:23.867560 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.866227    2552 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 17:18:23.867560 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.866231    2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 17:18:23.867560 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.866236    2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 17:18:23.867560 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.866240    2552 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 17:18:23.867560 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.866244    2552 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 17:18:23.867560 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.866248    2552 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 17:18:23.867560 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.866253    2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 17:18:23.867560 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.866257    2552 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 17:18:23.867560 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.866262    2552 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 17:18:23.867560 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.866268    2552 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 17:18:23.867560 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.866272    2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 17:18:23.867560 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.866277    2552 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 17:18:23.867560 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.866281    2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 17:18:23.868174 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.866284    2552 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 17:18:23.868174 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.866289    2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 17:18:23.868174 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.866293    2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 17:18:23.868174 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.866298    2552 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 17:18:23.868174 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.866302    2552 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 17:18:23.868174 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.866307    2552 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 17:18:23.868174 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.866311    2552 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 17:18:23.868174 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.866315    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 17:18:23.868174 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.866320    2552 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 17:18:23.868174 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.866325    2552 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 17:18:23.868174 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.866329    2552 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 17:18:23.868174 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.866333    2552 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 17:18:23.868174 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.866338    2552 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 17:18:23.868174 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.866344    2552 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 17:18:23.868174 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.866349    2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 17:18:23.868174 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.866354    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 17:18:23.868174 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.866358    2552 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 17:18:23.868174 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.866363    2552 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 17:18:23.868174 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.866367    2552 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 17:18:23.868976 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.866371    2552 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 17:18:23.868976 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.866375    2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 17:18:23.868976 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.866380    2552 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 17:18:23.868976 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.866384    2552 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 17:18:23.868976 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.866388    2552 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 17:18:23.868976 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.866392    2552 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 17:18:23.868976 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.866396    2552 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 17:18:23.868976 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.866400    2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 17:18:23.868976 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.866404    2552 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 17:18:23.868976 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.866408    2552 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 17:18:23.868976 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.866413    2552 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 17:18:23.868976 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.866417    2552 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 17:18:23.868976 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.866421    2552 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 17:18:23.868976 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.866425    2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 17:18:23.868976 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.866429    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 17:18:23.868976 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.866433    2552 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 17:18:23.868976 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.866437    2552 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 17:18:23.868976 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.866443    2552 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 17:18:23.868976 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.866447    2552 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 17:18:23.868976 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.868966    2552 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 17:18:23.869837 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.868978    2552 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 17:18:23.869837 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.868983    2552 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 17:18:23.869837 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.868988    2552 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 17:18:23.869837 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.868992    2552 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 17:18:23.869837 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.868996    2552 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 17:18:23.869837 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.869000    2552 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 17:18:23.869837 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.869004    2552 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 17:18:23.869837 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.869008    2552 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 17:18:23.869837 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.869012    2552 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 17:18:23.869837 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.869017    2552 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 17:18:23.869837 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.869021    2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 17:18:23.869837 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.869025    2552 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 17:18:23.869837 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.869029    2552 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 17:18:23.869837 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.869032    2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 17:18:23.869837 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.869036    2552 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 17:18:23.869837 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.869040    2552 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 17:18:23.869837 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.869045    2552 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 17:18:23.869837 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.869049    2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 17:18:23.869837 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.869067    2552 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 17:18:23.869837 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.869072    2552 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 17:18:23.870756 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.869077    2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 17:18:23.870756 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.869080    2552 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 17:18:23.870756 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.869084    2552 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 17:18:23.870756 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.869088    2552 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 17:18:23.870756 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.869092    2552 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 17:18:23.870756 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.869096    2552 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 17:18:23.870756 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.869101    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 17:18:23.870756 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.869105    2552 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 17:18:23.870756 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.869109    2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 17:18:23.870756 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.869113    2552 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 17:18:23.870756 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.869119    2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 17:18:23.870756 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.869123    2552 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 17:18:23.870756 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.869127    2552 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 17:18:23.870756 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.869132    2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 17:18:23.870756 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.869137    2552 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 17:18:23.870756 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.869141    2552 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 17:18:23.870756 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.869145    2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 17:18:23.870756 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.869150    2552 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 17:18:23.870756 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.869155    2552 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 17:18:23.870756 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.869159    2552 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 17:18:23.871665 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.869163    2552 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 17:18:23.871665 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.869167    2552 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 17:18:23.871665 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.869171    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 17:18:23.871665 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.869175    2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 17:18:23.871665 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.869180    2552 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 17:18:23.871665 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.869184    2552 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 17:18:23.871665 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.869193    2552 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 17:18:23.871665 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.869199    2552 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 17:18:23.871665 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.869204    2552 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 17:18:23.871665 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.869208    2552 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 17:18:23.871665 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.869213    2552 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 17:18:23.871665 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.869218    2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 17:18:23.871665 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.869222    2552 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 17:18:23.871665 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.869226    2552 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 17:18:23.871665 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.869230    2552 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 17:18:23.871665 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.869234    2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 17:18:23.871665 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.869238    2552 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 17:18:23.871665 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.869242    2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 17:18:23.871665 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.869246    2552 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 17:18:23.872376 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.869250    2552 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 17:18:23.872376 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.869254    2552 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 17:18:23.872376 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.869259    2552 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 17:18:23.872376 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.869263    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 17:18:23.872376 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.869268    2552 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 17:18:23.872376 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.869273    2552 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 17:18:23.872376 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.869279    2552 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 17:18:23.872376 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.869283    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 17:18:23.872376 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.869288    2552 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 17:18:23.872376 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.869292    2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 17:18:23.872376 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.869296    2552 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 17:18:23.872376 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.869300    2552 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 17:18:23.872376 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.869304    2552 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 17:18:23.872376 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.869309    2552 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 17:18:23.872376 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.869313    2552 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 17:18:23.872376 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.869317    2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 17:18:23.872376 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.869321    2552 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 17:18:23.872376 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.869325    2552 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 17:18:23.872376 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.869329    2552 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 17:18:23.872376 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.869333    2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 17:18:23.872963 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.869337    2552 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 17:18:23.872963 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.869342    2552 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 17:18:23.872963 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.869346    2552 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 17:18:23.872963 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.869350    2552 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 17:18:23.872963 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.869355    2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 17:18:23.872963 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.869359    2552 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 17:18:23.872963 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.869363    2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 17:18:23.872963 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.869368    2552 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 17:18:23.872963 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.869372    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 17:18:23.872963 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.869376    2552 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 17:18:23.872963 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.869385    2552 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 17:18:23.872963 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.869392    2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 17:18:23.872963 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.869396    2552 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 17:18:23.872963 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.869401    2552 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 17:18:23.872963 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.869405    2552 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 17:18:23.872963 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.869409    2552 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 17:18:23.872963 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.869414    2552 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 17:18:23.872963 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.869418    2552 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 17:18:23.872963 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.869426    2552 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 17:18:23.873529 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.870028    2552 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 17:18:23.873529 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.870037    2552 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 17:18:23.873529 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.870041    2552 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 17:18:23.873529 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.870046    2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 17:18:23.873529 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.870050    2552 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 17:18:23.873529 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.870074    2552 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 17:18:23.873529 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.870078    2552 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 17:18:23.873529 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.870083    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 17:18:23.873529 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.870087    2552 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 17:18:23.873529 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.870091    2552 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 17:18:23.873529 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.870095    2552 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 17:18:23.873529 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.870099    2552 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 17:18:23.873529 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.870104    2552 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 17:18:23.873529 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.870108    2552 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 17:18:23.873529 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.870113    2552 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 17:18:23.873529 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.870117    2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 17:18:23.873529 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.870121    2552 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 17:18:23.873529 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.870125    2552 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 17:18:23.873529 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.870129    2552 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 17:18:23.873529 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.870133    2552 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 17:18:23.874108 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.870137    2552 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 17:18:23.874108 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.870142    2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 17:18:23.874108 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.870146    2552 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 17:18:23.874108 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.870151    2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 17:18:23.874108 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.870155    2552 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 17:18:23.874108 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.870159    2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 17:18:23.874108 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.870163    2552 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 17:18:23.874108 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.870168    2552 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 17:18:23.874108 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.870172    2552 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 17:18:23.874108 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.870176    2552 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 17:18:23.874108 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.870181    2552 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 17:18:23.874108 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.870186    2552 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 17:18:23.874108 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.870191    2552 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 17:18:23.874108 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.870195    2552 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 17:18:23.874108 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.870199    2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 17:18:23.874108 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.870203    2552 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 17:18:23.874108 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.870207    2552 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 17:18:23.874108 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.870211    2552 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 17:18:23.874108 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.870215    2552 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 17:18:23.874108 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.870220    2552 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 17:18:23.874788 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.870224    2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 17:18:23.874788 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.870228    2552 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 17:18:23.874788 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.870232    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 17:18:23.874788 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.870235    2552 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 17:18:23.874788 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.870240    2552 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 17:18:23.874788 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.870244    2552 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 17:18:23.874788 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.870248    2552 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 17:18:23.874788 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.870252    2552 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 17:18:23.874788 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.870256    2552 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 17:18:23.874788 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.870261    2552 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 17:18:23.874788 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.870267    2552 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 17:18:23.874788 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.870271    2552 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 17:18:23.874788 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.870275    2552 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 17:18:23.874788 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.870279    2552 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 17:18:23.874788 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.870283    2552 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 17:18:23.874788 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.870287    2552 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 17:18:23.874788 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.870291    2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 17:18:23.874788 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.870296    2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 17:18:23.874788 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.870300    2552 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 17:18:23.874788 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.870304    2552 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 17:18:23.875303 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.870311    2552 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 17:18:23.875303 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.870317    2552 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 17:18:23.875303 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.870321    2552 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 17:18:23.875303 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.870329    2552 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 17:18:23.875303 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.870336    2552 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 17:18:23.875303 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.870341    2552 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 17:18:23.875303 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.870346    2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 17:18:23.875303 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.870351    2552 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 17:18:23.875303 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.870355    2552 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 17:18:23.875303 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.870360    2552 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 17:18:23.875303 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.870364    2552 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 17:18:23.875303 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.870369    2552 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 17:18:23.875303 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.870373    2552 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 17:18:23.875303 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.870377    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 17:18:23.875303 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.870383    2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 17:18:23.875303 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.870387    2552 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 17:18:23.875303 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.870392    2552 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 17:18:23.875303 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.870396    2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 17:18:23.875303 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.870400    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 17:18:23.875899 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.870405    2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 17:18:23.875899 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.870409    2552 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 17:18:23.875899 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.870413    2552 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 17:18:23.875899 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.870418    2552 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 17:18:23.875899 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.870422    2552 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 17:18:23.875899 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.870426    2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 17:18:23.875899 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.870430    2552 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 17:18:23.875899 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.870434    2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 17:18:23.875899 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.870439    2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 17:18:23.875899 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.870443    2552 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 17:18:23.875899 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.870447    2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 17:18:23.875899 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.870451    2552 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 17:18:23.875899 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.870455    2552 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 17:18:23.875899 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.870459    2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 17:18:23.875899 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.870463    2552 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 17:18:23.875899 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.870475    2552 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 17:18:23.875899 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.870482    2552 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 17:18:23.875899 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.870487    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 17:18:23.875899 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.870492    2552 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 17:18:23.876420 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.870496    2552 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 17:18:23.876420 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.870604    2552 flags.go:64] FLAG: --address="0.0.0.0"
Sep 11 17:18:23.876420 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.870665    2552 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 11 17:18:23.876420 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.870675    2552 flags.go:64] FLAG: --anonymous-auth="true"
Sep 11 17:18:23.876420 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.870682    2552 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 11 17:18:23.876420 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.870689    2552 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 11 17:18:23.876420 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.870694    2552 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 11 17:18:23.876420 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.870701    2552 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 11 17:18:23.876420 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.870708    2552 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 11 17:18:23.876420 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.870713    2552 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 11 17:18:23.876420 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.870718    2552 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 11 17:18:23.876420 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.870724    2552 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 11 17:18:23.876420 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.870729    2552 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 11 17:18:23.876420 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.870734    2552 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 11 17:18:23.876420 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.870739    2552 flags.go:64] FLAG: --cgroup-root=""
Sep 11 17:18:23.876420 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.870744    2552 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 11 17:18:23.876420 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.870749    2552 flags.go:64] FLAG: --client-ca-file=""
Sep 11 17:18:23.876420 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.870754    2552 flags.go:64] FLAG: --cloud-provider="external"
Sep 11 17:18:23.876420 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.870758    2552 flags.go:64] FLAG: --cluster-dns="[]"
Sep 11 17:18:23.876420 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.870766    2552 flags.go:64] FLAG: --cluster-domain=""
Sep 11 17:18:23.876420 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.870770    2552 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 11 17:18:23.876420 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.870775    2552 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 11 17:18:23.876420 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.870780    2552 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 11 17:18:23.876420 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.870786    2552 flags.go:64] FLAG: --container-log-max-files="5"
Sep 11 17:18:23.877017 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.870793    2552 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 11 17:18:23.877017 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.870798    2552 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 11 17:18:23.877017 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.870803    2552 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 11 17:18:23.877017 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.870809    2552 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 11 17:18:23.877017 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.870814    2552 flags.go:64] FLAG: --contention-profiling="false"
Sep 11 17:18:23.877017 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.870823    2552 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 11 17:18:23.877017 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.870828    2552 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 11 17:18:23.877017 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.870834    2552 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 11 17:18:23.877017 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.870839    2552 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 11 17:18:23.877017 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.870846    2552 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 11 17:18:23.877017 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.870851    2552 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 11 17:18:23.877017 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.870856    2552 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 11 17:18:23.877017 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.870861    2552 flags.go:64] FLAG: --enable-load-reader="false"
Sep 11 17:18:23.877017 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.870865    2552 flags.go:64] FLAG: --enable-server="true"
Sep 11 17:18:23.877017 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.870870    2552 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 11 17:18:23.877017 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.870877    2552 flags.go:64] FLAG: --event-burst="100"
Sep 11 17:18:23.877017 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.870882    2552 flags.go:64] FLAG: --event-qps="50"
Sep 11 17:18:23.877017 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.870887    2552 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 11 17:18:23.877017 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.870892    2552 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 11 17:18:23.877017 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.870896    2552 flags.go:64] FLAG: --eviction-hard=""
Sep 11 17:18:23.877017 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.870903    2552 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 11 17:18:23.877017 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.870907    2552 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 11 17:18:23.877017 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.870912    2552 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 11 17:18:23.877017 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.870917    2552 flags.go:64] FLAG: --eviction-soft=""
Sep 11 17:18:23.877017 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.870922    2552 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 11 17:18:23.877660 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.870927    2552 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 11 17:18:23.877660 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.870932    2552 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 11 17:18:23.877660 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.870937    2552 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 11 17:18:23.877660 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.870942    2552 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 11 17:18:23.877660 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.870947    2552 flags.go:64] FLAG: --fail-swap-on="true"
Sep 11 17:18:23.877660 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.870952    2552 flags.go:64] FLAG: --feature-gates=""
Sep 11 17:18:23.877660 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.870958    2552 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 11 17:18:23.877660 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.870963    2552 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 11 17:18:23.877660 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.870968    2552 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 11 17:18:23.877660 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.870974    2552 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 11 17:18:23.877660 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.870979    2552 flags.go:64] FLAG: --healthz-port="10248"
Sep 11 17:18:23.877660 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.870984    2552 flags.go:64] FLAG: --help="false"
Sep 11 17:18:23.877660 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.870991    2552 flags.go:64] FLAG: --hostname-override="ip-10-0-140-51.ec2.internal"
Sep 11 17:18:23.877660 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.870996    2552 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 11 17:18:23.877660 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.871001    2552 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 11 17:18:23.877660 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.871006    2552 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 11 17:18:23.877660 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.871012    2552 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 11 17:18:23.877660 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.871018    2552 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 11 17:18:23.877660 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.871023    2552 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 11 17:18:23.877660 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.871027    2552 flags.go:64] FLAG: --image-service-endpoint=""
Sep 11 17:18:23.877660 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.871032    2552 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 11 17:18:23.877660 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.871037    2552 flags.go:64] FLAG: --kube-api-burst="100"
Sep 11 17:18:23.877660 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.871042    2552 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 11 17:18:23.878274 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.871047    2552 flags.go:64] FLAG: --kube-api-qps="50"
Sep 11 17:18:23.878274 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.871070    2552 flags.go:64] FLAG: --kube-reserved=""
Sep 11 17:18:23.878274 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.871076    2552 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 11 17:18:23.878274 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.871080    2552 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 11 17:18:23.878274 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.871084    2552 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 11 17:18:23.878274 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.871088    2552 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 11 17:18:23.878274 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.871093    2552 flags.go:64] FLAG: --lock-file=""
Sep 11 17:18:23.878274 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.871097    2552 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 11 17:18:23.878274 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.871103    2552 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 11 17:18:23.878274 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.871108    2552 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 11 17:18:23.878274 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.871117    2552 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 11 17:18:23.878274 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.871122    2552 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 11 17:18:23.878274 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.871126    2552 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 11 17:18:23.878274 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.871131    2552 flags.go:64] FLAG: --logging-format="text"
Sep 11 17:18:23.878274 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.871136    2552 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 11 17:18:23.878274 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.871141    2552 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 11 17:18:23.878274 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.871146    2552 flags.go:64] FLAG: --manifest-url=""
Sep 11 17:18:23.878274 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.871151    2552 flags.go:64] FLAG: --manifest-url-header=""
Sep 11 17:18:23.878274 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.871158    2552 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 11 17:18:23.878274 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.871163    2552 flags.go:64] FLAG: --max-open-files="1000000"
Sep 11 17:18:23.878274 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.871170    2552 flags.go:64] FLAG: --max-pods="110"
Sep 11 17:18:23.878274 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.871177    2552 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 11 17:18:23.878274 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.871183    2552 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 11 17:18:23.878274 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.871188    2552 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 11 17:18:23.878274 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.871193    2552 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 11 17:18:23.878964 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.871206    2552 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 11 17:18:23.878964 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.871211    2552 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 11 17:18:23.878964 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.871216    2552 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 11 17:18:23.878964 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.871227    2552 flags.go:64] FLAG: --node-status-max-images="50"
Sep 11 17:18:23.878964 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.871232    2552 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 11 17:18:23.878964 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.871237    2552 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 11 17:18:23.878964 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.871242    2552 flags.go:64] FLAG: --pod-cidr=""
Sep 11 17:18:23.878964 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.871247    2552 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:eee392dbe3ac2ed4a38bca29ae3bd486f5b7780406c62c600154cd1906871b10"
Sep 11 17:18:23.878964 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.871255    2552 flags.go:64] FLAG: --pod-manifest-path=""
Sep 11 17:18:23.878964 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.871260    2552 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 11 17:18:23.878964 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.871265    2552 flags.go:64] FLAG: --pods-per-core="0"
Sep 11 17:18:23.878964 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.871270    2552 flags.go:64] FLAG: --port="10250"
Sep 11 17:18:23.878964 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.871275    2552 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 11 17:18:23.878964 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.871280    2552 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-08044b7a97d49360e"
Sep 11 17:18:23.878964 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.871285    2552 flags.go:64] FLAG: --qos-reserved=""
Sep 11 17:18:23.878964 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.871290    2552 flags.go:64] FLAG: --read-only-port="10255"
Sep 11 17:18:23.878964 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.871295    2552 flags.go:64] FLAG: --register-node="true"
Sep 11 17:18:23.878964 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.871299    2552 flags.go:64] FLAG: --register-with-taints=""
Sep 11 17:18:23.878964 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.871306    2552 flags.go:64] FLAG: --registry-burst="10"
Sep 11 17:18:23.878964 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.871311    2552 flags.go:64] FLAG: --registry-qps="5"
Sep 11 17:18:23.878964 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.871316    2552 flags.go:64] FLAG: --reserved-cpus=""
Sep 11 17:18:23.878964 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.871320    2552 flags.go:64] FLAG: --reserved-memory=""
Sep 11 17:18:23.878964 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.871327    2552 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 11 17:18:23.878964 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.871331    2552 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 11 17:18:23.878964 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.871337    2552 flags.go:64] FLAG: --rotate-certificates="false"
Sep 11 17:18:23.879666 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.871341    2552 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 11 17:18:23.879666 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.871346    2552 flags.go:64] FLAG: --runonce="false"
Sep 11 17:18:23.879666 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.871351    2552 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 11 17:18:23.879666 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.871357    2552 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 11 17:18:23.879666 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.871362    2552 flags.go:64] FLAG: --seccomp-default="false"
Sep 11 17:18:23.879666 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.871367    2552 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 11 17:18:23.879666 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.871372    2552 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 11 17:18:23.879666 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.871376    2552 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 11 17:18:23.879666 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.871382    2552 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 11 17:18:23.879666 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.871387    2552 flags.go:64] FLAG: --storage-driver-password="root"
Sep 11 17:18:23.879666 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.871392    2552 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 11 17:18:23.879666 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.871397    2552 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 11 17:18:23.879666 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.871402    2552 flags.go:64] FLAG: --storage-driver-user="root"
Sep 11 17:18:23.879666 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.871406    2552 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 11 17:18:23.879666 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.871411    2552 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 11 17:18:23.879666 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.871416    2552 flags.go:64] FLAG: --system-cgroups=""
Sep 11 17:18:23.879666 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.871421    2552 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 11 17:18:23.879666 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.871429    2552 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 11 17:18:23.879666 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.871434    2552 flags.go:64] FLAG: --tls-cert-file=""
Sep 11 17:18:23.879666 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.871439    2552 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 11 17:18:23.879666 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.871446    2552 flags.go:64] FLAG: --tls-min-version=""
Sep 11 17:18:23.879666 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.871451    2552 flags.go:64] FLAG: --tls-private-key-file=""
Sep 11 17:18:23.879666 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.871455    2552 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 11 17:18:23.879666 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.871460    2552 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 11 17:18:23.879666 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.871465    2552 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 11 17:18:23.880363 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.871470    2552 flags.go:64] FLAG: --v="2"
Sep 11 17:18:23.880363 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.871477    2552 flags.go:64] FLAG: --version="false"
Sep 11 17:18:23.880363 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.871483    2552 flags.go:64] FLAG: --vmodule=""
Sep 11 17:18:23.880363 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.871489    2552 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 11 17:18:23.880363 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.871494    2552 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 11 17:18:23.880363 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.871676    2552 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 17:18:23.880363 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.871683    2552 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 17:18:23.880363 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.871689    2552 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 17:18:23.880363 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.871693    2552 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 17:18:23.880363 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.871698    2552 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 17:18:23.880363 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.871706    2552 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 17:18:23.880363 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.871711    2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 17:18:23.880363 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.871715    2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 17:18:23.880363 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.871719    2552 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 17:18:23.880363 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.871724    2552 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 17:18:23.880363 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.871728    2552 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 17:18:23.880363 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.871735    2552 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 17:18:23.880363 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.871741    2552 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 17:18:23.880363 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.871746    2552 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 17:18:23.880363 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.871751    2552 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 17:18:23.880363 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.871756    2552 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 17:18:23.880906 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.871760    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 17:18:23.880906 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.871764    2552 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 17:18:23.880906 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.871768    2552 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 17:18:23.880906 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.871773    2552 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 17:18:23.880906 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.871777    2552 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 17:18:23.880906 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.871781    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 17:18:23.880906 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.871785    2552 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 17:18:23.880906 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.871791    2552 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 17:18:23.880906 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.871797    2552 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 17:18:23.880906 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.871801    2552 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 17:18:23.880906 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.871806    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 17:18:23.880906 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.871810    2552 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 17:18:23.880906 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.871815    2552 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 17:18:23.880906 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.871819    2552 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 17:18:23.880906 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.871823    2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 17:18:23.880906 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.871826    2552 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 17:18:23.880906 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.871831    2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 17:18:23.880906 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.871835    2552 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 17:18:23.880906 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.871840    2552 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 17:18:23.881654 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.871844    2552 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 17:18:23.881654 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.871848    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 17:18:23.881654 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.871856    2552 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 17:18:23.881654 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.871860    2552 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 17:18:23.881654 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.871864    2552 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 17:18:23.881654 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.871869    2552 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 17:18:23.881654 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.871873    2552 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 17:18:23.881654 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.871877    2552 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 17:18:23.881654 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.871881    2552 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 17:18:23.881654 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.871886    2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 17:18:23.881654 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.871890    2552 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 17:18:23.881654 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.871895    2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 17:18:23.881654 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.871899    2552 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 17:18:23.881654 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.871903    2552 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 17:18:23.881654 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.871907    2552 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 17:18:23.881654 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.871911    2552 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 17:18:23.881654 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.871915    2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 17:18:23.881654 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.871919    2552 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 17:18:23.881654 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.871924    2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 17:18:23.881654 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.871928    2552 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 17:18:23.882195 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.871932    2552 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 17:18:23.882195 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.871937    2552 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 17:18:23.882195 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.871941    2552 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 17:18:23.882195 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.871947    2552 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 17:18:23.882195 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.871953    2552 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 17:18:23.882195 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.871957    2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 17:18:23.882195 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.871961    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 17:18:23.882195 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.871965    2552 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 17:18:23.882195 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.871970    2552 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 17:18:23.882195 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.871974    2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 17:18:23.882195 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.871978    2552 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 17:18:23.882195 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.871982    2552 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 17:18:23.882195 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.871986    2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 17:18:23.882195 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.871991    2552 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 17:18:23.882195 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.871999    2552 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 17:18:23.882195 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.872003    2552 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 17:18:23.882195 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.872007    2552 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 17:18:23.882195 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.872011    2552 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 17:18:23.882195 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.872015    2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 17:18:23.882685 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.872020    2552 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 17:18:23.882685 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.872024    2552 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 17:18:23.882685 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.872028    2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 17:18:23.882685 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.872033    2552 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 17:18:23.882685 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.872037    2552 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 17:18:23.882685 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.872041    2552 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 17:18:23.882685 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.872046    2552 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 17:18:23.882685 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.872050    2552 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 17:18:23.882685 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.872072    2552 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 17:18:23.882685 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.872077    2552 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 17:18:23.882685 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.872081    2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 17:18:23.882685 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.872085    2552 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 17:18:23.882685 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.872090    2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 17:18:23.882685 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.872093    2552 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 17:18:23.882685 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.872097    2552 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 17:18:23.882685 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.872102    2552 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 17:18:23.882685 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.872106    2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 17:18:23.882685 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.872110    2552 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 17:18:23.882685 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.872113    2552 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 17:18:23.882685 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.872117    2552 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 17:18:23.883259 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.872121    2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 17:18:23.883259 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.872125    2552 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 17:18:23.883259 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.872129    2552 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 17:18:23.883259 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.872133    2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 17:18:23.883259 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.872138    2552 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 17:18:23.883259 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.872655    2552 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 11 17:18:23.883259 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.881892    2552 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 11 17:18:23.883259 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.881913    2552 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 11 17:18:23.883259 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.883040    2552 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 11 17:18:23.883259 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.883050    2552 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 11 17:18:23.883259 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883129    2552 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 17:18:23.883259 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883135    2552 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 17:18:23.883259 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883138    2552 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 17:18:23.883259 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883142    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 17:18:23.883259 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883145    2552 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 17:18:23.883685 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883148    2552 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 17:18:23.883685 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883151    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 17:18:23.883685 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883154    2552 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 17:18:23.883685 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883157    2552 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 17:18:23.883685 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883159    2552 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 17:18:23.883685 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883162    2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 17:18:23.883685 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883165    2552 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 17:18:23.883685 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883170    2552 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 17:18:23.883685 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883173    2552 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 17:18:23.883685 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883177    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 17:18:23.883685 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883180    2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 17:18:23.883685 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883182    2552 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 17:18:23.883685 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883185    2552 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 17:18:23.883685 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883188    2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 17:18:23.883685 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883191    2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 17:18:23.883685 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883193    2552 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 17:18:23.883685 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883196    2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 17:18:23.883685 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883200    2552 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 17:18:23.883685 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883203    2552 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 17:18:23.884182 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883205    2552 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 17:18:23.884182 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883208    2552 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 17:18:23.884182 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883210    2552 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 17:18:23.884182 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883213    2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 17:18:23.884182 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883216    2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 17:18:23.884182 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883219    2552 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 17:18:23.884182 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883222    2552 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 17:18:23.884182 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883225    2552 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 17:18:23.884182 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883227    2552 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 17:18:23.884182 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883230    2552 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 17:18:23.884182 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883233    2552 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 17:18:23.884182 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883235    2552 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 17:18:23.884182 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883238    2552 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 17:18:23.884182 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883241    2552 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 17:18:23.884182 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883243    2552 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 17:18:23.884182 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883246    2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 17:18:23.884182 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883249    2552 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 17:18:23.884182 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883251    2552 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 17:18:23.884182 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883254    2552 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 17:18:23.884182 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883257    2552 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 17:18:23.884681 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883260    2552 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 17:18:23.884681 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883262    2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 17:18:23.884681 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883266    2552 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 17:18:23.884681 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883269    2552 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 17:18:23.884681 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883272    2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 17:18:23.884681 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883274    2552 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 17:18:23.884681 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883277    2552 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 17:18:23.884681 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883280    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 17:18:23.884681 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883283    2552 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 17:18:23.884681 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883285    2552 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 17:18:23.884681 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883288    2552 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 17:18:23.884681 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883291    2552 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 17:18:23.884681 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883294    2552 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 17:18:23.884681 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883296    2552 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 17:18:23.884681 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883299    2552 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 17:18:23.884681 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883302    2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 17:18:23.884681 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883305    2552 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 17:18:23.884681 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883308    2552 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 17:18:23.884681 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883310    2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 17:18:23.884681 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883314    2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 17:18:23.885197 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883317    2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 17:18:23.885197 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883320    2552 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 17:18:23.885197 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883323    2552 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 17:18:23.885197 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883325    2552 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 17:18:23.885197 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883328    2552 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 17:18:23.885197 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883331    2552 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 17:18:23.885197 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883333    2552 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 17:18:23.885197 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883336    2552 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 17:18:23.885197 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883339    2552 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 17:18:23.885197 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883342    2552 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 17:18:23.885197 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883344    2552 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 17:18:23.885197 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883347    2552 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 17:18:23.885197 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883350    2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 17:18:23.885197 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883352    2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 17:18:23.885197 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883355    2552 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 17:18:23.885197 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883359    2552 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 17:18:23.885197 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883363    2552 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 17:18:23.885197 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883366    2552 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 17:18:23.885197 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883369    2552 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 17:18:23.885197 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883371    2552 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 17:18:23.885694 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883374    2552 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 17:18:23.885694 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883378    2552 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 17:18:23.885694 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883382    2552 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 17:18:23.885694 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883385    2552 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 17:18:23.885694 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883388    2552 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 17:18:23.885694 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883390    2552 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 17:18:23.885694 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883393    2552 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 17:18:23.885694 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883396    2552 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 17:18:23.885694 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883399    2552 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 17:18:23.885694 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883402    2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 17:18:23.885694 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883404    2552 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 17:18:23.885694 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883407    2552 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 17:18:23.885694 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883410    2552 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 17:18:23.885694 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883413    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 17:18:23.885694 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883415    2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 17:18:23.886083 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.883421    2552 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 11 17:18:23.886083 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883556    2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 17:18:23.886083 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883561    2552 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 17:18:23.886083 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883565    2552 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 17:18:23.886083 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883570    2552 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 17:18:23.886083 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883573    2552 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 17:18:23.886083 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883576    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 17:18:23.886083 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883580    2552 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 17:18:23.886083 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883584    2552 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 17:18:23.886083 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883587    2552 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 17:18:23.886083 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883590    2552 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 17:18:23.886083 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883593    2552 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 17:18:23.886083 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883596    2552 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 17:18:23.886083 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883599    2552 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 17:18:23.886437 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883601    2552 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 17:18:23.886437 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883604    2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 17:18:23.886437 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883606    2552 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 17:18:23.886437 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883609    2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 17:18:23.886437 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883611    2552 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 17:18:23.886437 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883614    2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 17:18:23.886437 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883616    2552 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 17:18:23.886437 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883619    2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 17:18:23.886437 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883621    2552 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 17:18:23.886437 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883624    2552 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 17:18:23.886437 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883626    2552 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 17:18:23.886437 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883629    2552 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 17:18:23.886437 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883632    2552 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 17:18:23.886437 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883635    2552 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 17:18:23.886437 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883638    2552 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 17:18:23.886437 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883641    2552 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 17:18:23.886437 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883643    2552 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 17:18:23.886437 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883646    2552 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 17:18:23.886437 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883648    2552 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 17:18:23.886437 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883651    2552 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 17:18:23.886928 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883653    2552 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 17:18:23.886928 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883656    2552 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 17:18:23.886928 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883659    2552 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 17:18:23.886928 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883661    2552 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 17:18:23.886928 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883664    2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 17:18:23.886928 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883666    2552 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 17:18:23.886928 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883669    2552 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 17:18:23.886928 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883671    2552 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 17:18:23.886928 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883674    2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 17:18:23.886928 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883676    2552 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 17:18:23.886928 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883679    2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 17:18:23.886928 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883682    2552 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 17:18:23.886928 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883684    2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 17:18:23.886928 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883687    2552 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 17:18:23.886928 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883690    2552 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 17:18:23.886928 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883693    2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 17:18:23.886928 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883696    2552 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 17:18:23.886928 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883698    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 17:18:23.886928 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883701    2552 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 17:18:23.886928 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883703    2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 17:18:23.887437 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883706    2552 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 17:18:23.887437 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883708    2552 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 17:18:23.887437 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883711    2552 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 17:18:23.887437 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883714    2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 17:18:23.887437 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883717    2552 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 17:18:23.887437 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883719    2552 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 17:18:23.887437 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883722    2552 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 17:18:23.887437 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883725    2552 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 17:18:23.887437 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883727    2552 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 17:18:23.887437 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883730    2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 17:18:23.887437 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883732    2552 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 17:18:23.887437 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883735    2552 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 17:18:23.887437 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883737    2552 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 17:18:23.887437 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883740    2552 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 17:18:23.887437 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883743    2552 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 17:18:23.887437 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883745    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 17:18:23.887437 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883748    2552 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 17:18:23.887437 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883750    2552 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 17:18:23.887437 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883753    2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 17:18:23.887437 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883755    2552 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 17:18:23.887938 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883758    2552 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 17:18:23.887938 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883760    2552 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 17:18:23.887938 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883763    2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 17:18:23.887938 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883765    2552 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 17:18:23.887938 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883768    2552 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 17:18:23.887938 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883771    2552 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 17:18:23.887938 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883774    2552 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 17:18:23.887938 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883777    2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 17:18:23.887938 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883779    2552 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 17:18:23.887938 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883782    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 17:18:23.887938 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883784    2552 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 17:18:23.887938 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883788    2552 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 17:18:23.887938 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883791    2552 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 17:18:23.887938 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883794    2552 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 17:18:23.887938 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883797    2552 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 17:18:23.887938 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883800    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 17:18:23.887938 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883803    2552 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 17:18:23.887938 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883805    2552 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 17:18:23.887938 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883808    2552 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 17:18:23.888430 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883810    2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 17:18:23.888430 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883813    2552 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 17:18:23.888430 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883815    2552 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 17:18:23.888430 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883818    2552 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 17:18:23.888430 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883821    2552 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 17:18:23.888430 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883823    2552 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 17:18:23.888430 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:23.883826    2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 17:18:23.888430 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.883830    2552 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 11 17:18:23.888430 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.883953    2552 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 11 17:18:23.888430 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.886114    2552 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 11 17:18:23.888430 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.887074    2552 server.go:1019] "Starting client certificate rotation"
Sep 11 17:18:23.888430 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.887175    2552 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 11 17:18:23.888430 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.887205    2552 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 11 17:18:23.889491 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.889476    2552 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 11 17:18:23.892533 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.892512    2552 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 11 17:18:23.904758 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.904740    2552 log.go:25] "Validated CRI v1 runtime API"
Sep 11 17:18:23.907769 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.907754    2552 log.go:25] "Validated CRI v1 image API"
Sep 11 17:18:23.908639 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.908624    2552 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 11 17:18:23.912873 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.912855    2552 fs.go:135] Filesystem UUIDs: map[43d7e9eb-3093-456c-a903-36bcfaaa4e9f:/dev/nvme0n1p4 4fd931c7-a0d5-494a-990c-3f9786ffbc89:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2]
Sep 11 17:18:23.912932 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.912872    2552 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 11 17:18:23.918106 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.917980    2552 manager.go:216] Machine: {Timestamp:2026-09-11 17:18:23.917494016 +0000 UTC m=+0.331840401 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3123059 MemoryCapacity:33164480512 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2f1a546439656b4d55e210d845bc8f SystemUUID:ec2f1a54-6439-656b-4d55-e210d845bc8f BootID:7dccd2da-3e03-490b-ac44-9e2cadbae22c Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582242304 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:56:a1:7b:2c:bb Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:56:a1:7b:2c:bb Speed:0 Mtu:9001} {Name:ovs-system MacAddress:e2:8f:f4:91:3a:5b Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164480512 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 11 17:18:23.918106 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.918105    2552 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 11 17:18:23.918253 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.918193    2552 manager.go:232] Version: {KernelVersion:5.14.0-570.140.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260908-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 11 17:18:23.920038 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.920006    2552 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 11 17:18:23.920223 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.920046    2552 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-140-51.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 11 17:18:23.920960 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.920948    2552 topology_manager.go:138] "Creating topology manager with none policy"
Sep 11 17:18:23.921038 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.920965    2552 container_manager_linux.go:309] "Creating device plugin manager"
Sep 11 17:18:23.921038 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.920974    2552 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 11 17:18:23.922884 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.922871    2552 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 11 17:18:23.922931 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.922917    2552 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 11 17:18:23.923702 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.923691    2552 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 11 17:18:23.926565 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.926553    2552 state_mem.go:36] "Initialized new in-memory state store"
Sep 11 17:18:23.926696 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.926687    2552 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 11 17:18:23.926731 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.926718    2552 kubelet.go:486] "Attempting to sync node with API server"
Sep 11 17:18:23.926731 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.926728    2552 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 11 17:18:23.926782 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.926746    2552 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 11 17:18:23.926782 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.926756    2552 kubelet.go:398] "Adding apiserver pod source"
Sep 11 17:18:23.926782 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.926764    2552 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 11 17:18:23.927158 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.927143    2552 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 11 17:18:23.927220 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.927162    2552 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 11 17:18:23.931543 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.931517    2552 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 11 17:18:23.933707 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.933694    2552 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 11 17:18:23.933759 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.933723    2552 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 11 17:18:23.935697 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.935685    2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 11 17:18:23.935755 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.935701    2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 11 17:18:23.935755 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.935708    2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 11 17:18:23.935755 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.935713    2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 11 17:18:23.935755 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.935721    2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 11 17:18:23.935755 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.935727    2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 11 17:18:23.935755 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.935740    2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 11 17:18:23.935755 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.935746    2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 11 17:18:23.935755 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.935753    2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 11 17:18:23.935755 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.935759    2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 11 17:18:23.935987 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.935769    2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 11 17:18:23.935987 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.935774    2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 11 17:18:23.936541 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.936532    2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 11 17:18:23.936541 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.936541    2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 11 17:18:23.939338 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:18:23.939309    2552 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 11 17:18:23.939338 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.939324    2552 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-140-51.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 11 17:18:23.939440 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:18:23.939346    2552 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-140-51.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 11 17:18:23.940351 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.940340    2552 server.go:1268] "Started kubelet"
Sep 11 17:18:23.940524 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.940450    2552 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 11 17:18:23.940583 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.940554    2552 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 11 17:18:23.941104 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.941070    2552 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 11 17:18:23.941397 ip-10-0-140-51 systemd[1]: Started Kubernetes Kubelet.
Sep 11 17:18:23.941614 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.941600    2552 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 11 17:18:23.943370 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.943358    2552 server.go:310] "Adding debug handlers to kubelet server"
Sep 11 17:18:23.945914 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:18:23.945104    2552 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-51.ec2.internal.18d453cfcd480781  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-51.ec2.internal,UID:ip-10-0-140-51.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-140-51.ec2.internal,},FirstTimestamp:2026-09-11 17:18:23.940315009 +0000 UTC m=+0.354661393,LastTimestamp:2026-09-11 17:18:23.940315009 +0000 UTC m=+0.354661393,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-51.ec2.internal,}"
Sep 11 17:18:23.946399 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.946378    2552 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 11 17:18:23.946879 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.946864    2552 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 11 17:18:23.947072 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:18:23.947041    2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-51.ec2.internal\" not found"
Sep 11 17:18:23.947927 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.947911    2552 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 11 17:18:23.947927 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.947914    2552 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 11 17:18:23.948087 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.947948    2552 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 11 17:18:23.948251 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.948126    2552 reconstruct.go:97] "Volume reconstruction finished"
Sep 11 17:18:23.948251 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.948137    2552 reconciler.go:29] "Reconciler: start to sync state"
Sep 11 17:18:23.948449 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:18:23.948336    2552 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 11 17:18:23.948867 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.948840    2552 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 11 17:18:23.948867 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.948857    2552 factory.go:55] Registering systemd factory
Sep 11 17:18:23.948867 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.948866    2552 factory.go:223] Registration of the systemd container factory successfully
Sep 11 17:18:23.949132 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.949114    2552 factory.go:153] Registering CRI-O factory
Sep 11 17:18:23.949132 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.949134    2552 factory.go:223] Registration of the crio container factory successfully
Sep 11 17:18:23.949285 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.949151    2552 factory.go:103] Registering Raw factory
Sep 11 17:18:23.949285 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.949164    2552 manager.go:1196] Started watching for new ooms in manager
Sep 11 17:18:23.949621 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.949604    2552 manager.go:349] Starting recovery of all containers
Sep 11 17:18:23.961290 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:18:23.961262    2552 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-140-51.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 11 17:18:23.961921 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:18:23.961270    2552 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 11 17:18:23.961921 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.961909    2552 manager.go:354] Recovery completed
Sep 11 17:18:23.966518 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.966505    2552 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 17:18:23.968460 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.968444    2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-51.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 17:18:23.968543 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.968476    2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-51.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 17:18:23.968543 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.968487    2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-51.ec2.internal" event="NodeHasSufficientPID"
Sep 11 17:18:23.969217 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.969201    2552 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 11 17:18:23.969217 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.969216    2552 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 11 17:18:23.969311 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.969235    2552 state_mem.go:36] "Initialized new in-memory state store"
Sep 11 17:18:23.969311 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.969246    2552 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 17:18:23.970178 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.970162    2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-51.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 17:18:23.970265 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.970187    2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-51.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 17:18:23.970265 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.970197    2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-51.ec2.internal" event="NodeHasSufficientPID"
Sep 11 17:18:23.970523 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.970511    2552 policy_none.go:49] "None policy: Start"
Sep 11 17:18:23.970523 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.970527    2552 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 11 17:18:23.970523 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.970537    2552 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 11 17:18:23.971622 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.971609    2552 policy_none.go:47] "Start"
Sep 11 17:18:23.980579 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:18:23.980494    2552 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-51.ec2.internal.18d453cfcef57aaa  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-51.ec2.internal,UID:ip-10-0-140-51.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-140-51.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-140-51.ec2.internal,},FirstTimestamp:2026-09-11 17:18:23.968459434 +0000 UTC m=+0.382805820,LastTimestamp:2026-09-11 17:18:23.968459434 +0000 UTC m=+0.382805820,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-51.ec2.internal,}"
Sep 11 17:18:23.985022 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.985003    2552 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-n8x7d"
Sep 11 17:18:23.991889 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:18:23.991821    2552 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-51.ec2.internal.18d453cfcef5d087  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-51.ec2.internal,UID:ip-10-0-140-51.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-140-51.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-140-51.ec2.internal,},FirstTimestamp:2026-09-11 17:18:23.968481415 +0000 UTC m=+0.382827800,LastTimestamp:2026-09-11 17:18:23.968481415 +0000 UTC m=+0.382827800,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-51.ec2.internal,}"
Sep 11 17:18:23.994948 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.994929    2552 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-n8x7d"
Sep 11 17:18:23.996639 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.996625    2552 manager.go:337] "Starting Device Plugin manager"
Sep 11 17:18:23.996714 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:18:23.996667    2552 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 11 17:18:23.996714 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.996681    2552 server.go:87] "Starting device plugin registration server"
Sep 11 17:18:23.997021 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.997006    2552 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 11 17:18:23.997316 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.997024    2552 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 11 17:18:23.997316 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.997200    2552 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 11 17:18:23.997316 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.997271    2552 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 11 17:18:23.997316 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:23.997279    2552 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 11 17:18:23.997842 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:18:23.997821    2552 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 11 17:18:23.997900 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:18:23.997875    2552 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-140-51.ec2.internal\" not found"
Sep 11 17:18:24.047997 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:24.047956    2552 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 11 17:18:24.049237 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:24.049216    2552 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 11 17:18:24.049350 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:24.049266    2552 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 11 17:18:24.049350 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:24.049286    2552 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 11 17:18:24.049438 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:18:24.049378    2552 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 11 17:18:24.053303 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:24.053279    2552 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 17:18:24.097317 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:24.097286    2552 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 17:18:24.098366 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:24.098347    2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-51.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 17:18:24.098498 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:24.098383    2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-51.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 17:18:24.098498 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:24.098398    2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-51.ec2.internal" event="NodeHasSufficientPID"
Sep 11 17:18:24.098498 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:24.098451    2552 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-140-51.ec2.internal"
Sep 11 17:18:24.114029 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:24.113944    2552 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-140-51.ec2.internal"
Sep 11 17:18:24.114029 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:18:24.113971    2552 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-140-51.ec2.internal\": node \"ip-10-0-140-51.ec2.internal\" not found"
Sep 11 17:18:24.150180 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:24.150139    2552 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-51.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-140-51.ec2.internal"]
Sep 11 17:18:24.150314 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:24.150233    2552 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 17:18:24.151233 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:24.151214    2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-51.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 17:18:24.151345 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:24.151249    2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-51.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 17:18:24.151345 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:24.151263    2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-51.ec2.internal" event="NodeHasSufficientPID"
Sep 11 17:18:24.152515 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:24.152499    2552 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 17:18:24.152657 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:24.152643    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-51.ec2.internal"
Sep 11 17:18:24.152707 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:24.152675    2552 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 17:18:24.153271 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:24.153254    2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-51.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 17:18:24.153327 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:24.153283    2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-51.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 17:18:24.153327 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:24.153293    2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-51.ec2.internal" event="NodeHasSufficientPID"
Sep 11 17:18:24.153389 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:24.153254    2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-51.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 17:18:24.153389 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:24.153355    2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-51.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 17:18:24.153389 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:24.153369    2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-51.ec2.internal" event="NodeHasSufficientPID"
Sep 11 17:18:24.154449 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:24.154434    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-51.ec2.internal"
Sep 11 17:18:24.154511 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:24.154457    2552 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 17:18:24.155243 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:24.155224    2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-51.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 17:18:24.155330 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:18:24.155245    2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-51.ec2.internal\" not found"
Sep 11 17:18:24.155330 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:24.155256    2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-51.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 17:18:24.155330 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:24.155271    2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-51.ec2.internal" event="NodeHasSufficientPID"
Sep 11 17:18:24.179735 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:18:24.179713    2552 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-51.ec2.internal\" not found" node="ip-10-0-140-51.ec2.internal"
Sep 11 17:18:24.183981 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:18:24.183961    2552 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-51.ec2.internal\" not found" node="ip-10-0-140-51.ec2.internal"
Sep 11 17:18:24.249145 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:24.249113    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2283d44ad0e3ab74ce157fde8b170250-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-51.ec2.internal\" (UID: \"2283d44ad0e3ab74ce157fde8b170250\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-51.ec2.internal"
Sep 11 17:18:24.249145 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:24.249147    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2283d44ad0e3ab74ce157fde8b170250-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-51.ec2.internal\" (UID: \"2283d44ad0e3ab74ce157fde8b170250\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-51.ec2.internal"
Sep 11 17:18:24.249305 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:24.249164    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/144be191d43f74dcac73bb27936b49ab-config\") pod \"kube-apiserver-proxy-ip-10-0-140-51.ec2.internal\" (UID: \"144be191d43f74dcac73bb27936b49ab\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-51.ec2.internal"
Sep 11 17:18:24.256215 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:18:24.256195    2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-51.ec2.internal\" not found"
Sep 11 17:18:24.349971 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:24.349940    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2283d44ad0e3ab74ce157fde8b170250-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-51.ec2.internal\" (UID: \"2283d44ad0e3ab74ce157fde8b170250\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-51.ec2.internal"
Sep 11 17:18:24.349971 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:24.349957    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2283d44ad0e3ab74ce157fde8b170250-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-51.ec2.internal\" (UID: \"2283d44ad0e3ab74ce157fde8b170250\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-51.ec2.internal"
Sep 11 17:18:24.350171 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:24.349989    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2283d44ad0e3ab74ce157fde8b170250-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-51.ec2.internal\" (UID: \"2283d44ad0e3ab74ce157fde8b170250\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-51.ec2.internal"
Sep 11 17:18:24.350171 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:24.350009    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/144be191d43f74dcac73bb27936b49ab-config\") pod \"kube-apiserver-proxy-ip-10-0-140-51.ec2.internal\" (UID: \"144be191d43f74dcac73bb27936b49ab\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-51.ec2.internal"
Sep 11 17:18:24.350171 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:24.350033    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/144be191d43f74dcac73bb27936b49ab-config\") pod \"kube-apiserver-proxy-ip-10-0-140-51.ec2.internal\" (UID: \"144be191d43f74dcac73bb27936b49ab\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-51.ec2.internal"
Sep 11 17:18:24.350171 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:24.350070    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2283d44ad0e3ab74ce157fde8b170250-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-51.ec2.internal\" (UID: \"2283d44ad0e3ab74ce157fde8b170250\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-51.ec2.internal"
Sep 11 17:18:24.356397 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:18:24.356373    2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-51.ec2.internal\" not found"
Sep 11 17:18:24.456906 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:18:24.456825    2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-51.ec2.internal\" not found"
Sep 11 17:18:24.482023 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:24.481985    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-51.ec2.internal"
Sep 11 17:18:24.486766 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:24.486745    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-51.ec2.internal"
Sep 11 17:18:26.450511 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:18:26.450477    2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-51.ec2.internal\" not found"
Sep 11 17:18:26.551085 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:18:26.551046    2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-51.ec2.internal\" not found"
Sep 11 17:18:26.651613 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:18:26.651534    2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-51.ec2.internal\" not found"
Sep 11 17:18:26.752238 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:18:26.752208    2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-51.ec2.internal\" not found"
Sep 11 17:18:26.773518 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:26.773497    2552 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 17:18:26.779723 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:26.779700    2552 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 11 17:18:26.779825 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:26.779810    2552 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 11 17:18:26.779863 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:26.779849    2552 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 11 17:18:26.852579 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:18:26.852546    2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-51.ec2.internal\" not found"
Sep 11 17:18:26.892612 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:26.892573    2552 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-10 17:13:25 +0000 UTC" deadline="2028-02-22 13:04:50.243921747 +0000 UTC"
Sep 11 17:18:26.892612 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:26.892608    2552 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12691h46m23.351317567s"
Sep 11 17:18:26.953545 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:18:26.953514    2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-51.ec2.internal\" not found"
Sep 11 17:18:26.960684 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:26.960658    2552 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 17:18:26.963433 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:26.963394    2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-51.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 17:18:26.963433 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:26.963437    2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-51.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 17:18:26.963663 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:26.963453    2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-51.ec2.internal" event="NodeHasSufficientPID"
Sep 11 17:18:27.007015 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.006987    2552 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 17:18:27.040878 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.040843    2552 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-51.ec2.internal"
Sep 11 17:18:27.054802 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.054780    2552 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 11 17:18:27.057361 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.057340    2552 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-51.ec2.internal"
Sep 11 17:18:27.062343 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:27.062304    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2283d44ad0e3ab74ce157fde8b170250.slice/crio-6b66574765e6ed5afb385282d08825de67e44c3656b4a9448ebf5a1ba28753b4 WatchSource:0}: Error finding container 6b66574765e6ed5afb385282d08825de67e44c3656b4a9448ebf5a1ba28753b4: Status 404 returned error can't find the container with id 6b66574765e6ed5afb385282d08825de67e44c3656b4a9448ebf5a1ba28753b4
Sep 11 17:18:27.062791 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:27.062773    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod144be191d43f74dcac73bb27936b49ab.slice/crio-973c6c0f8ac2afbb0782f323ca3509a442eb606b28d94496d32d82f2bef11a35 WatchSource:0}: Error finding container 973c6c0f8ac2afbb0782f323ca3509a442eb606b28d94496d32d82f2bef11a35: Status 404 returned error can't find the container with id 973c6c0f8ac2afbb0782f323ca3509a442eb606b28d94496d32d82f2bef11a35
Sep 11 17:18:27.066471 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.066442    2552 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 11 17:18:27.066549 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.066506    2552 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 17:18:27.323399 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.323302    2552 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 17:18:27.820319 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.820232    2552 apiserver.go:52] "Watching apiserver"
Sep 11 17:18:27.825584 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.825559    2552 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 11 17:18:27.825957 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.825935    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-140-51.ec2.internal","openshift-dns/node-resolver-h6lht","openshift-multus/multus-additional-cni-plugins-z559n","openshift-multus/network-metrics-daemon-d5jqp","openshift-network-diagnostics/network-check-target-9gb8k","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-22x74","openshift-cluster-node-tuning-operator/tuned-cph4q","openshift-image-registry/node-ca-gbf8j","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-51.ec2.internal","openshift-multus/multus-q586m","openshift-network-operator/iptables-alerter-fptk9","openshift-ovn-kubernetes/ovnkube-node-6k9tr","kube-system/global-pull-secret-syncer-lx6ht","kube-system/konnectivity-agent-8j6jf"]
Sep 11 17:18:27.828374 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.828353    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-cph4q"
Sep 11 17:18:27.830385 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.830363    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lx6ht"
Sep 11 17:18:27.830534 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:18:27.830433    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lx6ht" podUID="72eae49b-cb97-4819-aaf5-5c6eb23ce9c8"
Sep 11 17:18:27.830690 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.830657    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-kh2s9\""
Sep 11 17:18:27.830778 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.830696    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 11 17:18:27.830933 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.830915    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 11 17:18:27.832565 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.832546    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-z559n"
Sep 11 17:18:27.834715 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.834698    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d5jqp"
Sep 11 17:18:27.834813 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:18:27.834768    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d5jqp" podUID="b2b1c077-b5b8-4bdc-a1b3-6b98a0bcf6a0"
Sep 11 17:18:27.834891 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.834870    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 11 17:18:27.835084 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.834885    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 11 17:18:27.835185 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.834891    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 11 17:18:27.835185 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.834948    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-bp2mg\""
Sep 11 17:18:27.835513 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.835494    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 11 17:18:27.835692 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.835678    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 11 17:18:27.836996 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.836976    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9gb8k"
Sep 11 17:18:27.837081 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:18:27.837047    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9gb8k" podUID="99bdc64a-0517-49d6-acf3-4e9fc5ea47fe"
Sep 11 17:18:27.839168 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.839148    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-22x74"
Sep 11 17:18:27.840506 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.840488    2552 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 11 17:18:27.841387 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.841178    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 11 17:18:27.841387 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.841186    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 11 17:18:27.841387 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.841208    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-jcpk2\""
Sep 11 17:18:27.841387 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.841327    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 11 17:18:27.841633 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.841601    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-8j6jf"
Sep 11 17:18:27.844559 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.844275    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 11 17:18:27.844762 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.844744    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-qtt2x\""
Sep 11 17:18:27.845345 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.845207    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 11 17:18:27.845345 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.845341    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-gbf8j"
Sep 11 17:18:27.847293 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.847274    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 11 17:18:27.848016 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.847726    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 11 17:18:27.848016 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.847994    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-sjwwm\""
Sep 11 17:18:27.848266 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.848249    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-h6lht"
Sep 11 17:18:27.849274 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.849049    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 11 17:18:27.851266 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.851249    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-q586m"
Sep 11 17:18:27.851392 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.851367    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 11 17:18:27.851673 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.851656    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 11 17:18:27.851776 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.851758    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-bjlc9\""
Sep 11 17:18:27.851973 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.851954    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/26e09059-8abc-4e73-bf43-23a51698a0cf-etc-tuned\") pod \"tuned-cph4q\" (UID: \"26e09059-8abc-4e73-bf43-23a51698a0cf\") " pod="openshift-cluster-node-tuning-operator/tuned-cph4q"
Sep 11 17:18:27.852029 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.851986    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/26e09059-8abc-4e73-bf43-23a51698a0cf-tmp\") pod \"tuned-cph4q\" (UID: \"26e09059-8abc-4e73-bf43-23a51698a0cf\") " pod="openshift-cluster-node-tuning-operator/tuned-cph4q"
Sep 11 17:18:27.852029 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.852002    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/26e09059-8abc-4e73-bf43-23a51698a0cf-etc-systemd\") pod \"tuned-cph4q\" (UID: \"26e09059-8abc-4e73-bf43-23a51698a0cf\") " pod="openshift-cluster-node-tuning-operator/tuned-cph4q"
Sep 11 17:18:27.852029 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.852017    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xpv4j\" (UniqueName: \"kubernetes.io/projected/26e09059-8abc-4e73-bf43-23a51698a0cf-kube-api-access-xpv4j\") pod \"tuned-cph4q\" (UID: \"26e09059-8abc-4e73-bf43-23a51698a0cf\") " pod="openshift-cluster-node-tuning-operator/tuned-cph4q"
Sep 11 17:18:27.852310 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.852038    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e6840164-41a5-4423-bdd0-a364186cf962-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-z559n\" (UID: \"e6840164-41a5-4423-bdd0-a364186cf962\") " pod="openshift-multus/multus-additional-cni-plugins-z559n"
Sep 11 17:18:27.852310 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.852063    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e6840164-41a5-4423-bdd0-a364186cf962-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-z559n\" (UID: \"e6840164-41a5-4423-bdd0-a364186cf962\") " pod="openshift-multus/multus-additional-cni-plugins-z559n"
Sep 11 17:18:27.852310 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.852091    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b5a337f0-d706-42c3-b846-1997efc3d693-registration-dir\") pod \"aws-ebs-csi-driver-node-22x74\" (UID: \"b5a337f0-d706-42c3-b846-1997efc3d693\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-22x74"
Sep 11 17:18:27.852310 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.852155    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/26e09059-8abc-4e73-bf43-23a51698a0cf-etc-modprobe-d\") pod \"tuned-cph4q\" (UID: \"26e09059-8abc-4e73-bf43-23a51698a0cf\") " pod="openshift-cluster-node-tuning-operator/tuned-cph4q"
Sep 11 17:18:27.852310 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.852180    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/26e09059-8abc-4e73-bf43-23a51698a0cf-run\") pod \"tuned-cph4q\" (UID: \"26e09059-8abc-4e73-bf43-23a51698a0cf\") " pod="openshift-cluster-node-tuning-operator/tuned-cph4q"
Sep 11 17:18:27.852310 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.852202    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e6840164-41a5-4423-bdd0-a364186cf962-system-cni-dir\") pod \"multus-additional-cni-plugins-z559n\" (UID: \"e6840164-41a5-4423-bdd0-a364186cf962\") " pod="openshift-multus/multus-additional-cni-plugins-z559n"
Sep 11 17:18:27.852310 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.852228    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/26e09059-8abc-4e73-bf43-23a51698a0cf-etc-sysctl-conf\") pod \"tuned-cph4q\" (UID: \"26e09059-8abc-4e73-bf43-23a51698a0cf\") " pod="openshift-cluster-node-tuning-operator/tuned-cph4q"
Sep 11 17:18:27.852310 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.852250    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/26e09059-8abc-4e73-bf43-23a51698a0cf-var-lib-kubelet\") pod \"tuned-cph4q\" (UID: \"26e09059-8abc-4e73-bf43-23a51698a0cf\") " pod="openshift-cluster-node-tuning-operator/tuned-cph4q"
Sep 11 17:18:27.852310 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.852272    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/26e09059-8abc-4e73-bf43-23a51698a0cf-host\") pod \"tuned-cph4q\" (UID: \"26e09059-8abc-4e73-bf43-23a51698a0cf\") " pod="openshift-cluster-node-tuning-operator/tuned-cph4q"
Sep 11 17:18:27.852310 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.852296    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e6840164-41a5-4423-bdd0-a364186cf962-cnibin\") pod \"multus-additional-cni-plugins-z559n\" (UID: \"e6840164-41a5-4423-bdd0-a364186cf962\") " pod="openshift-multus/multus-additional-cni-plugins-z559n"
Sep 11 17:18:27.852795 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.852375    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7jckg\" (UniqueName: \"kubernetes.io/projected/e6840164-41a5-4423-bdd0-a364186cf962-kube-api-access-7jckg\") pod \"multus-additional-cni-plugins-z559n\" (UID: \"e6840164-41a5-4423-bdd0-a364186cf962\") " pod="openshift-multus/multus-additional-cni-plugins-z559n"
Sep 11 17:18:27.852795 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.852426    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b5a337f0-d706-42c3-b846-1997efc3d693-etc-selinux\") pod \"aws-ebs-csi-driver-node-22x74\" (UID: \"b5a337f0-d706-42c3-b846-1997efc3d693\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-22x74"
Sep 11 17:18:27.852795 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.852481    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dx97x\" (UniqueName: \"kubernetes.io/projected/b2b1c077-b5b8-4bdc-a1b3-6b98a0bcf6a0-kube-api-access-dx97x\") pod \"network-metrics-daemon-d5jqp\" (UID: \"b2b1c077-b5b8-4bdc-a1b3-6b98a0bcf6a0\") " pod="openshift-multus/network-metrics-daemon-d5jqp"
Sep 11 17:18:27.852795 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.852517    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-42k58\" (UniqueName: \"kubernetes.io/projected/99bdc64a-0517-49d6-acf3-4e9fc5ea47fe-kube-api-access-42k58\") pod \"network-check-target-9gb8k\" (UID: \"99bdc64a-0517-49d6-acf3-4e9fc5ea47fe\") " pod="openshift-network-diagnostics/network-check-target-9gb8k"
Sep 11 17:18:27.852795 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.852554    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b5a337f0-d706-42c3-b846-1997efc3d693-socket-dir\") pod \"aws-ebs-csi-driver-node-22x74\" (UID: \"b5a337f0-d706-42c3-b846-1997efc3d693\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-22x74"
Sep 11 17:18:27.852795 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.852583    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/26e09059-8abc-4e73-bf43-23a51698a0cf-etc-sysctl-d\") pod \"tuned-cph4q\" (UID: \"26e09059-8abc-4e73-bf43-23a51698a0cf\") " pod="openshift-cluster-node-tuning-operator/tuned-cph4q"
Sep 11 17:18:27.852795 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.852604    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/26e09059-8abc-4e73-bf43-23a51698a0cf-sys\") pod \"tuned-cph4q\" (UID: \"26e09059-8abc-4e73-bf43-23a51698a0cf\") " pod="openshift-cluster-node-tuning-operator/tuned-cph4q"
Sep 11 17:18:27.852795 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.852633    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e6840164-41a5-4423-bdd0-a364186cf962-cni-binary-copy\") pod \"multus-additional-cni-plugins-z559n\" (UID: \"e6840164-41a5-4423-bdd0-a364186cf962\") " pod="openshift-multus/multus-additional-cni-plugins-z559n"
Sep 11 17:18:27.852795 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.852655    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b2b1c077-b5b8-4bdc-a1b3-6b98a0bcf6a0-metrics-certs\") pod \"network-metrics-daemon-d5jqp\" (UID: \"b2b1c077-b5b8-4bdc-a1b3-6b98a0bcf6a0\") " pod="openshift-multus/network-metrics-daemon-d5jqp"
Sep 11 17:18:27.852795 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.852677    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b5a337f0-d706-42c3-b846-1997efc3d693-sys-fs\") pod \"aws-ebs-csi-driver-node-22x74\" (UID: \"b5a337f0-d706-42c3-b846-1997efc3d693\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-22x74"
Sep 11 17:18:27.852795 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.852698    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b5a337f0-d706-42c3-b846-1997efc3d693-kubelet-dir\") pod \"aws-ebs-csi-driver-node-22x74\" (UID: \"b5a337f0-d706-42c3-b846-1997efc3d693\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-22x74"
Sep 11 17:18:27.852795 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.852730    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/5b082ce5-2ea9-46dc-8307-08a722392886-konnectivity-ca\") pod \"konnectivity-agent-8j6jf\" (UID: \"5b082ce5-2ea9-46dc-8307-08a722392886\") " pod="kube-system/konnectivity-agent-8j6jf"
Sep 11 17:18:27.852795 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.852752    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/72eae49b-cb97-4819-aaf5-5c6eb23ce9c8-original-pull-secret\") pod \"global-pull-secret-syncer-lx6ht\" (UID: \"72eae49b-cb97-4819-aaf5-5c6eb23ce9c8\") " pod="kube-system/global-pull-secret-syncer-lx6ht"
Sep 11 17:18:27.852795 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.852773    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/26e09059-8abc-4e73-bf43-23a51698a0cf-etc-sysconfig\") pod \"tuned-cph4q\" (UID: \"26e09059-8abc-4e73-bf43-23a51698a0cf\") " pod="openshift-cluster-node-tuning-operator/tuned-cph4q"
Sep 11 17:18:27.852795 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.852794    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e6840164-41a5-4423-bdd0-a364186cf962-tuning-conf-dir\") pod \"multus-additional-cni-plugins-z559n\" (UID: \"e6840164-41a5-4423-bdd0-a364186cf962\") " pod="openshift-multus/multus-additional-cni-plugins-z559n"
Sep 11 17:18:27.853521 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.852817    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/72eae49b-cb97-4819-aaf5-5c6eb23ce9c8-kubelet-config\") pod \"global-pull-secret-syncer-lx6ht\" (UID: \"72eae49b-cb97-4819-aaf5-5c6eb23ce9c8\") " pod="kube-system/global-pull-secret-syncer-lx6ht"
Sep 11 17:18:27.853521 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.852839    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/26e09059-8abc-4e73-bf43-23a51698a0cf-etc-kubernetes\") pod \"tuned-cph4q\" (UID: \"26e09059-8abc-4e73-bf43-23a51698a0cf\") " pod="openshift-cluster-node-tuning-operator/tuned-cph4q"
Sep 11 17:18:27.853521 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.852893    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lxscx\" (UniqueName: \"kubernetes.io/projected/b5a337f0-d706-42c3-b846-1997efc3d693-kube-api-access-lxscx\") pod \"aws-ebs-csi-driver-node-22x74\" (UID: \"b5a337f0-d706-42c3-b846-1997efc3d693\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-22x74"
Sep 11 17:18:27.853521 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.852928    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/5b082ce5-2ea9-46dc-8307-08a722392886-agent-certs\") pod \"konnectivity-agent-8j6jf\" (UID: \"5b082ce5-2ea9-46dc-8307-08a722392886\") " pod="kube-system/konnectivity-agent-8j6jf"
Sep 11 17:18:27.853521 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.852973    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b5a337f0-d706-42c3-b846-1997efc3d693-device-dir\") pod \"aws-ebs-csi-driver-node-22x74\" (UID: \"b5a337f0-d706-42c3-b846-1997efc3d693\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-22x74"
Sep 11 17:18:27.853521 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.853005    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/72eae49b-cb97-4819-aaf5-5c6eb23ce9c8-dbus\") pod \"global-pull-secret-syncer-lx6ht\" (UID: \"72eae49b-cb97-4819-aaf5-5c6eb23ce9c8\") " pod="kube-system/global-pull-secret-syncer-lx6ht"
Sep 11 17:18:27.853521 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.853030    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/26e09059-8abc-4e73-bf43-23a51698a0cf-lib-modules\") pod \"tuned-cph4q\" (UID: \"26e09059-8abc-4e73-bf43-23a51698a0cf\") " pod="openshift-cluster-node-tuning-operator/tuned-cph4q"
Sep 11 17:18:27.853521 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.853509    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 11 17:18:27.854228 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.854203    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-xm9gh\""
Sep 11 17:18:27.854735 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.854713    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-fptk9"
Sep 11 17:18:27.855184 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.855161    2552 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 11 17:18:27.857001 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.856889    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-km2wv\""
Sep 11 17:18:27.857001 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.856894    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 11 17:18:27.857186 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.857053    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 11 17:18:27.857186 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.857085    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-6k9tr"
Sep 11 17:18:27.857186 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.857169    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 11 17:18:27.861305 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.861079    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 11 17:18:27.861412 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.861329    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 11 17:18:27.861412 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.861359    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 11 17:18:27.861519 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.861505    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-f42nh\""
Sep 11 17:18:27.861600 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.861567    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 11 17:18:27.861600 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.861590    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 11 17:18:27.862610 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.862584    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 11 17:18:27.883251 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.883223    2552 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-qv25v"
Sep 11 17:18:27.892395 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.892364    2552 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-qv25v"
Sep 11 17:18:27.942086 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.942057    2552 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 11 17:18:27.947345 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.947303    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-51.ec2.internal" event={"ID":"144be191d43f74dcac73bb27936b49ab","Type":"ContainerStarted","Data":"973c6c0f8ac2afbb0782f323ca3509a442eb606b28d94496d32d82f2bef11a35"}
Sep 11 17:18:27.948456 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.948432    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-51.ec2.internal" event={"ID":"2283d44ad0e3ab74ce157fde8b170250","Type":"ContainerStarted","Data":"6b66574765e6ed5afb385282d08825de67e44c3656b4a9448ebf5a1ba28753b4"}
Sep 11 17:18:27.953786 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.953760    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b5a337f0-d706-42c3-b846-1997efc3d693-registration-dir\") pod \"aws-ebs-csi-driver-node-22x74\" (UID: \"b5a337f0-d706-42c3-b846-1997efc3d693\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-22x74"
Sep 11 17:18:27.953907 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.953799    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e6840164-41a5-4423-bdd0-a364186cf962-system-cni-dir\") pod \"multus-additional-cni-plugins-z559n\" (UID: \"e6840164-41a5-4423-bdd0-a364186cf962\") " pod="openshift-multus/multus-additional-cni-plugins-z559n"
Sep 11 17:18:27.953907 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.953830    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0620c0a0-8070-4307-86d9-c69a76ea14a9-host-run-ovn-kubernetes\") pod \"ovnkube-node-6k9tr\" (UID: \"0620c0a0-8070-4307-86d9-c69a76ea14a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6k9tr"
Sep 11 17:18:27.953907 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.953858    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/26e09059-8abc-4e73-bf43-23a51698a0cf-etc-sysctl-conf\") pod \"tuned-cph4q\" (UID: \"26e09059-8abc-4e73-bf43-23a51698a0cf\") " pod="openshift-cluster-node-tuning-operator/tuned-cph4q"
Sep 11 17:18:27.953907 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.953880    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/26e09059-8abc-4e73-bf43-23a51698a0cf-var-lib-kubelet\") pod \"tuned-cph4q\" (UID: \"26e09059-8abc-4e73-bf43-23a51698a0cf\") " pod="openshift-cluster-node-tuning-operator/tuned-cph4q"
Sep 11 17:18:27.953907 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.953888    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b5a337f0-d706-42c3-b846-1997efc3d693-registration-dir\") pod \"aws-ebs-csi-driver-node-22x74\" (UID: \"b5a337f0-d706-42c3-b846-1997efc3d693\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-22x74"
Sep 11 17:18:27.954168 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.953912    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/26e09059-8abc-4e73-bf43-23a51698a0cf-host\") pod \"tuned-cph4q\" (UID: \"26e09059-8abc-4e73-bf43-23a51698a0cf\") " pod="openshift-cluster-node-tuning-operator/tuned-cph4q"
Sep 11 17:18:27.954168 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.953926    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e6840164-41a5-4423-bdd0-a364186cf962-system-cni-dir\") pod \"multus-additional-cni-plugins-z559n\" (UID: \"e6840164-41a5-4423-bdd0-a364186cf962\") " pod="openshift-multus/multus-additional-cni-plugins-z559n"
Sep 11 17:18:27.954168 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.953935    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e6840164-41a5-4423-bdd0-a364186cf962-cnibin\") pod \"multus-additional-cni-plugins-z559n\" (UID: \"e6840164-41a5-4423-bdd0-a364186cf962\") " pod="openshift-multus/multus-additional-cni-plugins-z559n"
Sep 11 17:18:27.954168 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.953965    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/41755610-5307-49d4-af8e-a021e026aa13-cnibin\") pod \"multus-q586m\" (UID: \"41755610-5307-49d4-af8e-a021e026aa13\") " pod="openshift-multus/multus-q586m"
Sep 11 17:18:27.954168 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.953977    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/26e09059-8abc-4e73-bf43-23a51698a0cf-var-lib-kubelet\") pod \"tuned-cph4q\" (UID: \"26e09059-8abc-4e73-bf43-23a51698a0cf\") " pod="openshift-cluster-node-tuning-operator/tuned-cph4q"
Sep 11 17:18:27.954168 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.954009    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/41755610-5307-49d4-af8e-a021e026aa13-multus-daemon-config\") pod \"multus-q586m\" (UID: \"41755610-5307-49d4-af8e-a021e026aa13\") " pod="openshift-multus/multus-q586m"
Sep 11 17:18:27.954168 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.954020    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/26e09059-8abc-4e73-bf43-23a51698a0cf-host\") pod \"tuned-cph4q\" (UID: \"26e09059-8abc-4e73-bf43-23a51698a0cf\") " pod="openshift-cluster-node-tuning-operator/tuned-cph4q"
Sep 11 17:18:27.954168 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.954032    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e6840164-41a5-4423-bdd0-a364186cf962-cnibin\") pod \"multus-additional-cni-plugins-z559n\" (UID: \"e6840164-41a5-4423-bdd0-a364186cf962\") " pod="openshift-multus/multus-additional-cni-plugins-z559n"
Sep 11 17:18:27.954168 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.954041    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/26e09059-8abc-4e73-bf43-23a51698a0cf-etc-sysctl-conf\") pod \"tuned-cph4q\" (UID: \"26e09059-8abc-4e73-bf43-23a51698a0cf\") " pod="openshift-cluster-node-tuning-operator/tuned-cph4q"
Sep 11 17:18:27.954168 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.954050    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b5a337f0-d706-42c3-b846-1997efc3d693-etc-selinux\") pod \"aws-ebs-csi-driver-node-22x74\" (UID: \"b5a337f0-d706-42c3-b846-1997efc3d693\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-22x74"
Sep 11 17:18:27.954168 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.954087    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dx97x\" (UniqueName: \"kubernetes.io/projected/b2b1c077-b5b8-4bdc-a1b3-6b98a0bcf6a0-kube-api-access-dx97x\") pod \"network-metrics-daemon-d5jqp\" (UID: \"b2b1c077-b5b8-4bdc-a1b3-6b98a0bcf6a0\") " pod="openshift-multus/network-metrics-daemon-d5jqp"
Sep 11 17:18:27.954168 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.954143    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-42k58\" (UniqueName: \"kubernetes.io/projected/99bdc64a-0517-49d6-acf3-4e9fc5ea47fe-kube-api-access-42k58\") pod \"network-check-target-9gb8k\" (UID: \"99bdc64a-0517-49d6-acf3-4e9fc5ea47fe\") " pod="openshift-network-diagnostics/network-check-target-9gb8k"
Sep 11 17:18:27.954731 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.954188    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b5a337f0-d706-42c3-b846-1997efc3d693-socket-dir\") pod \"aws-ebs-csi-driver-node-22x74\" (UID: \"b5a337f0-d706-42c3-b846-1997efc3d693\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-22x74"
Sep 11 17:18:27.954731 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.954206    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b5a337f0-d706-42c3-b846-1997efc3d693-etc-selinux\") pod \"aws-ebs-csi-driver-node-22x74\" (UID: \"b5a337f0-d706-42c3-b846-1997efc3d693\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-22x74"
Sep 11 17:18:27.954731 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.954220    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0620c0a0-8070-4307-86d9-c69a76ea14a9-host-slash\") pod \"ovnkube-node-6k9tr\" (UID: \"0620c0a0-8070-4307-86d9-c69a76ea14a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6k9tr"
Sep 11 17:18:27.954731 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.954246    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/0620c0a0-8070-4307-86d9-c69a76ea14a9-env-overrides\") pod \"ovnkube-node-6k9tr\" (UID: \"0620c0a0-8070-4307-86d9-c69a76ea14a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6k9tr"
Sep 11 17:18:27.954731 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.954272    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d0ec6f78-bdc6-420a-a6aa-953d61643e27-serviceca\") pod \"node-ca-gbf8j\" (UID: \"d0ec6f78-bdc6-420a-a6aa-953d61643e27\") " pod="openshift-image-registry/node-ca-gbf8j"
Sep 11 17:18:27.954731 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.954301    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/41755610-5307-49d4-af8e-a021e026aa13-host-run-k8s-cni-cncf-io\") pod \"multus-q586m\" (UID: \"41755610-5307-49d4-af8e-a021e026aa13\") " pod="openshift-multus/multus-q586m"
Sep 11 17:18:27.954731 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.954339    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b2b1c077-b5b8-4bdc-a1b3-6b98a0bcf6a0-metrics-certs\") pod \"network-metrics-daemon-d5jqp\" (UID: \"b2b1c077-b5b8-4bdc-a1b3-6b98a0bcf6a0\") " pod="openshift-multus/network-metrics-daemon-d5jqp"
Sep 11 17:18:27.954731 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.954373    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wqxzx\" (UniqueName: \"kubernetes.io/projected/8b4c1f98-046f-47c4-889f-c2bd807dddbe-kube-api-access-wqxzx\") pod \"node-resolver-h6lht\" (UID: \"8b4c1f98-046f-47c4-889f-c2bd807dddbe\") " pod="openshift-dns/node-resolver-h6lht"
Sep 11 17:18:27.954731 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.954341    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b5a337f0-d706-42c3-b846-1997efc3d693-socket-dir\") pod \"aws-ebs-csi-driver-node-22x74\" (UID: \"b5a337f0-d706-42c3-b846-1997efc3d693\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-22x74"
Sep 11 17:18:27.954731 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.954401    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/181608ca-8121-4bf3-8d50-a407708b420a-iptables-alerter-script\") pod \"iptables-alerter-fptk9\" (UID: \"181608ca-8121-4bf3-8d50-a407708b420a\") " pod="openshift-network-operator/iptables-alerter-fptk9"
Sep 11 17:18:27.954731 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.954441    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e6840164-41a5-4423-bdd0-a364186cf962-tuning-conf-dir\") pod \"multus-additional-cni-plugins-z559n\" (UID: \"e6840164-41a5-4423-bdd0-a364186cf962\") " pod="openshift-multus/multus-additional-cni-plugins-z559n"
Sep 11 17:18:27.954731 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:18:27.954445    2552 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 17:18:27.954731 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.954476    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0620c0a0-8070-4307-86d9-c69a76ea14a9-host-cni-netd\") pod \"ovnkube-node-6k9tr\" (UID: \"0620c0a0-8070-4307-86d9-c69a76ea14a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6k9tr"
Sep 11 17:18:27.954731 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:18:27.954526    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b2b1c077-b5b8-4bdc-a1b3-6b98a0bcf6a0-metrics-certs podName:b2b1c077-b5b8-4bdc-a1b3-6b98a0bcf6a0 nodeName:}" failed. No retries permitted until 2026-09-11 17:18:28.454500179 +0000 UTC m=+2.975791688 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b2b1c077-b5b8-4bdc-a1b3-6b98a0bcf6a0-metrics-certs") pod "network-metrics-daemon-d5jqp" (UID: "b2b1c077-b5b8-4bdc-a1b3-6b98a0bcf6a0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 17:18:27.954731 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.954581    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xvcc7\" (UniqueName: \"kubernetes.io/projected/d0ec6f78-bdc6-420a-a6aa-953d61643e27-kube-api-access-xvcc7\") pod \"node-ca-gbf8j\" (UID: \"d0ec6f78-bdc6-420a-a6aa-953d61643e27\") " pod="openshift-image-registry/node-ca-gbf8j"
Sep 11 17:18:27.954731 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.954610    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/41755610-5307-49d4-af8e-a021e026aa13-multus-socket-dir-parent\") pod \"multus-q586m\" (UID: \"41755610-5307-49d4-af8e-a021e026aa13\") " pod="openshift-multus/multus-q586m"
Sep 11 17:18:27.955307 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.954631    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e6840164-41a5-4423-bdd0-a364186cf962-tuning-conf-dir\") pod \"multus-additional-cni-plugins-z559n\" (UID: \"e6840164-41a5-4423-bdd0-a364186cf962\") " pod="openshift-multus/multus-additional-cni-plugins-z559n"
Sep 11 17:18:27.955307 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.954634    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/41755610-5307-49d4-af8e-a021e026aa13-host-run-netns\") pod \"multus-q586m\" (UID: \"41755610-5307-49d4-af8e-a021e026aa13\") " pod="openshift-multus/multus-q586m"
Sep 11 17:18:27.955307 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.954700    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tdhvz\" (UniqueName: \"kubernetes.io/projected/181608ca-8121-4bf3-8d50-a407708b420a-kube-api-access-tdhvz\") pod \"iptables-alerter-fptk9\" (UID: \"181608ca-8121-4bf3-8d50-a407708b420a\") " pod="openshift-network-operator/iptables-alerter-fptk9"
Sep 11 17:18:27.955307 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.954733    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/26e09059-8abc-4e73-bf43-23a51698a0cf-etc-kubernetes\") pod \"tuned-cph4q\" (UID: \"26e09059-8abc-4e73-bf43-23a51698a0cf\") " pod="openshift-cluster-node-tuning-operator/tuned-cph4q"
Sep 11 17:18:27.955307 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.954760    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0620c0a0-8070-4307-86d9-c69a76ea14a9-run-ovn\") pod \"ovnkube-node-6k9tr\" (UID: \"0620c0a0-8070-4307-86d9-c69a76ea14a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6k9tr"
Sep 11 17:18:27.957541 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.954785    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/5b082ce5-2ea9-46dc-8307-08a722392886-agent-certs\") pod \"konnectivity-agent-8j6jf\" (UID: \"5b082ce5-2ea9-46dc-8307-08a722392886\") " pod="kube-system/konnectivity-agent-8j6jf"
Sep 11 17:18:27.957829 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.957796    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/26e09059-8abc-4e73-bf43-23a51698a0cf-lib-modules\") pod \"tuned-cph4q\" (UID: \"26e09059-8abc-4e73-bf43-23a51698a0cf\") " pod="openshift-cluster-node-tuning-operator/tuned-cph4q"
Sep 11 17:18:27.958039 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.958020    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/26e09059-8abc-4e73-bf43-23a51698a0cf-lib-modules\") pod \"tuned-cph4q\" (UID: \"26e09059-8abc-4e73-bf43-23a51698a0cf\") " pod="openshift-cluster-node-tuning-operator/tuned-cph4q"
Sep 11 17:18:27.958247 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.958229    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/26e09059-8abc-4e73-bf43-23a51698a0cf-etc-kubernetes\") pod \"tuned-cph4q\" (UID: \"26e09059-8abc-4e73-bf43-23a51698a0cf\") " pod="openshift-cluster-node-tuning-operator/tuned-cph4q"
Sep 11 17:18:27.958403 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.958384    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/26e09059-8abc-4e73-bf43-23a51698a0cf-tmp\") pod \"tuned-cph4q\" (UID: \"26e09059-8abc-4e73-bf43-23a51698a0cf\") " pod="openshift-cluster-node-tuning-operator/tuned-cph4q"
Sep 11 17:18:27.958489 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.958470    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xpv4j\" (UniqueName: \"kubernetes.io/projected/26e09059-8abc-4e73-bf43-23a51698a0cf-kube-api-access-xpv4j\") pod \"tuned-cph4q\" (UID: \"26e09059-8abc-4e73-bf43-23a51698a0cf\") " pod="openshift-cluster-node-tuning-operator/tuned-cph4q"
Sep 11 17:18:27.958613 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.958588    2552 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 11 17:18:27.959234 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.959074    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e6840164-41a5-4423-bdd0-a364186cf962-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-z559n\" (UID: \"e6840164-41a5-4423-bdd0-a364186cf962\") " pod="openshift-multus/multus-additional-cni-plugins-z559n"
Sep 11 17:18:27.959234 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.959142    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e6840164-41a5-4423-bdd0-a364186cf962-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-z559n\" (UID: \"e6840164-41a5-4423-bdd0-a364186cf962\") " pod="openshift-multus/multus-additional-cni-plugins-z559n"
Sep 11 17:18:27.959234 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.959202    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8b4c1f98-046f-47c4-889f-c2bd807dddbe-tmp-dir\") pod \"node-resolver-h6lht\" (UID: \"8b4c1f98-046f-47c4-889f-c2bd807dddbe\") " pod="openshift-dns/node-resolver-h6lht"
Sep 11 17:18:27.959234 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.959233    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0620c0a0-8070-4307-86d9-c69a76ea14a9-systemd-units\") pod \"ovnkube-node-6k9tr\" (UID: \"0620c0a0-8070-4307-86d9-c69a76ea14a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6k9tr"
Sep 11 17:18:27.959430 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.959262    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/26e09059-8abc-4e73-bf43-23a51698a0cf-etc-systemd\") pod \"tuned-cph4q\" (UID: \"26e09059-8abc-4e73-bf43-23a51698a0cf\") " pod="openshift-cluster-node-tuning-operator/tuned-cph4q"
Sep 11 17:18:27.959430 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.959285    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0620c0a0-8070-4307-86d9-c69a76ea14a9-host-run-netns\") pod \"ovnkube-node-6k9tr\" (UID: \"0620c0a0-8070-4307-86d9-c69a76ea14a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6k9tr"
Sep 11 17:18:27.959430 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.959340    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0620c0a0-8070-4307-86d9-c69a76ea14a9-run-systemd\") pod \"ovnkube-node-6k9tr\" (UID: \"0620c0a0-8070-4307-86d9-c69a76ea14a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6k9tr"
Sep 11 17:18:27.959430 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.959368    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0620c0a0-8070-4307-86d9-c69a76ea14a9-host-cni-bin\") pod \"ovnkube-node-6k9tr\" (UID: \"0620c0a0-8070-4307-86d9-c69a76ea14a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6k9tr"
Sep 11 17:18:27.959430 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.959413    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/0620c0a0-8070-4307-86d9-c69a76ea14a9-ovnkube-script-lib\") pod \"ovnkube-node-6k9tr\" (UID: \"0620c0a0-8070-4307-86d9-c69a76ea14a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6k9tr"
Sep 11 17:18:27.959632 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.959456    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/41755610-5307-49d4-af8e-a021e026aa13-system-cni-dir\") pod \"multus-q586m\" (UID: \"41755610-5307-49d4-af8e-a021e026aa13\") " pod="openshift-multus/multus-q586m"
Sep 11 17:18:27.959632 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.959509    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/41755610-5307-49d4-af8e-a021e026aa13-multus-cni-dir\") pod \"multus-q586m\" (UID: \"41755610-5307-49d4-af8e-a021e026aa13\") " pod="openshift-multus/multus-q586m"
Sep 11 17:18:27.959724 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.959625    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-798k2\" (UniqueName: \"kubernetes.io/projected/41755610-5307-49d4-af8e-a021e026aa13-kube-api-access-798k2\") pod \"multus-q586m\" (UID: \"41755610-5307-49d4-af8e-a021e026aa13\") " pod="openshift-multus/multus-q586m"
Sep 11 17:18:27.959724 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.959676    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/26e09059-8abc-4e73-bf43-23a51698a0cf-etc-modprobe-d\") pod \"tuned-cph4q\" (UID: \"26e09059-8abc-4e73-bf43-23a51698a0cf\") " pod="openshift-cluster-node-tuning-operator/tuned-cph4q"
Sep 11 17:18:27.959813 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.959703    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/26e09059-8abc-4e73-bf43-23a51698a0cf-run\") pod \"tuned-cph4q\" (UID: \"26e09059-8abc-4e73-bf43-23a51698a0cf\") " pod="openshift-cluster-node-tuning-operator/tuned-cph4q"
Sep 11 17:18:27.959813 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.959753    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/8b4c1f98-046f-47c4-889f-c2bd807dddbe-hosts-file\") pod \"node-resolver-h6lht\" (UID: \"8b4c1f98-046f-47c4-889f-c2bd807dddbe\") " pod="openshift-dns/node-resolver-h6lht"
Sep 11 17:18:27.959813 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.959777    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/0620c0a0-8070-4307-86d9-c69a76ea14a9-ovn-node-metrics-cert\") pod \"ovnkube-node-6k9tr\" (UID: \"0620c0a0-8070-4307-86d9-c69a76ea14a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6k9tr"
Sep 11 17:18:27.959937 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.959826    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/41755610-5307-49d4-af8e-a021e026aa13-etc-kubernetes\") pod \"multus-q586m\" (UID: \"41755610-5307-49d4-af8e-a021e026aa13\") " pod="openshift-multus/multus-q586m"
Sep 11 17:18:27.959937 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.959857    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7jckg\" (UniqueName: \"kubernetes.io/projected/e6840164-41a5-4423-bdd0-a364186cf962-kube-api-access-7jckg\") pod \"multus-additional-cni-plugins-z559n\" (UID: \"e6840164-41a5-4423-bdd0-a364186cf962\") " pod="openshift-multus/multus-additional-cni-plugins-z559n"
Sep 11 17:18:27.959937 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.959884    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/41755610-5307-49d4-af8e-a021e026aa13-cni-binary-copy\") pod \"multus-q586m\" (UID: \"41755610-5307-49d4-af8e-a021e026aa13\") " pod="openshift-multus/multus-q586m"
Sep 11 17:18:27.959937 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.959923    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/26e09059-8abc-4e73-bf43-23a51698a0cf-etc-sysctl-d\") pod \"tuned-cph4q\" (UID: \"26e09059-8abc-4e73-bf43-23a51698a0cf\") " pod="openshift-cluster-node-tuning-operator/tuned-cph4q"
Sep 11 17:18:27.960155 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.959945    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/26e09059-8abc-4e73-bf43-23a51698a0cf-sys\") pod \"tuned-cph4q\" (UID: \"26e09059-8abc-4e73-bf43-23a51698a0cf\") " pod="openshift-cluster-node-tuning-operator/tuned-cph4q"
Sep 11 17:18:27.960155 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.959972    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e6840164-41a5-4423-bdd0-a364186cf962-cni-binary-copy\") pod \"multus-additional-cni-plugins-z559n\" (UID: \"e6840164-41a5-4423-bdd0-a364186cf962\") " pod="openshift-multus/multus-additional-cni-plugins-z559n"
Sep 11 17:18:27.960155 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.960000    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b5a337f0-d706-42c3-b846-1997efc3d693-sys-fs\") pod \"aws-ebs-csi-driver-node-22x74\" (UID: \"b5a337f0-d706-42c3-b846-1997efc3d693\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-22x74"
Sep 11 17:18:27.960155 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.960027    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0620c0a0-8070-4307-86d9-c69a76ea14a9-host-kubelet\") pod \"ovnkube-node-6k9tr\" (UID: \"0620c0a0-8070-4307-86d9-c69a76ea14a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6k9tr"
Sep 11 17:18:27.960155 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.960056    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0620c0a0-8070-4307-86d9-c69a76ea14a9-run-openvswitch\") pod \"ovnkube-node-6k9tr\" (UID: \"0620c0a0-8070-4307-86d9-c69a76ea14a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6k9tr"
Sep 11 17:18:27.960155 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.960079    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0620c0a0-8070-4307-86d9-c69a76ea14a9-node-log\") pod \"ovnkube-node-6k9tr\" (UID: \"0620c0a0-8070-4307-86d9-c69a76ea14a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6k9tr"
Sep 11 17:18:27.960421 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.960166    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/41755610-5307-49d4-af8e-a021e026aa13-host-run-multus-certs\") pod \"multus-q586m\" (UID: \"41755610-5307-49d4-af8e-a021e026aa13\") " pod="openshift-multus/multus-q586m"
Sep 11 17:18:27.960421 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.960297    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b5a337f0-d706-42c3-b846-1997efc3d693-kubelet-dir\") pod \"aws-ebs-csi-driver-node-22x74\" (UID: \"b5a337f0-d706-42c3-b846-1997efc3d693\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-22x74"
Sep 11 17:18:27.960509 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.960401    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/5b082ce5-2ea9-46dc-8307-08a722392886-konnectivity-ca\") pod \"konnectivity-agent-8j6jf\" (UID: \"5b082ce5-2ea9-46dc-8307-08a722392886\") " pod="kube-system/konnectivity-agent-8j6jf"
Sep 11 17:18:27.960509 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.960479    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/72eae49b-cb97-4819-aaf5-5c6eb23ce9c8-original-pull-secret\") pod \"global-pull-secret-syncer-lx6ht\" (UID: \"72eae49b-cb97-4819-aaf5-5c6eb23ce9c8\") " pod="kube-system/global-pull-secret-syncer-lx6ht"
Sep 11 17:18:27.960609 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.960557    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/26e09059-8abc-4e73-bf43-23a51698a0cf-etc-sysconfig\") pod \"tuned-cph4q\" (UID: \"26e09059-8abc-4e73-bf43-23a51698a0cf\") " pod="openshift-cluster-node-tuning-operator/tuned-cph4q"
Sep 11 17:18:27.960609 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.960588    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0620c0a0-8070-4307-86d9-c69a76ea14a9-log-socket\") pod \"ovnkube-node-6k9tr\" (UID: \"0620c0a0-8070-4307-86d9-c69a76ea14a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6k9tr"
Sep 11 17:18:27.960713 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.960648    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jf7ct\" (UniqueName: \"kubernetes.io/projected/0620c0a0-8070-4307-86d9-c69a76ea14a9-kube-api-access-jf7ct\") pod \"ovnkube-node-6k9tr\" (UID: \"0620c0a0-8070-4307-86d9-c69a76ea14a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6k9tr"
Sep 11 17:18:27.960713 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.960693    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/72eae49b-cb97-4819-aaf5-5c6eb23ce9c8-kubelet-config\") pod \"global-pull-secret-syncer-lx6ht\" (UID: \"72eae49b-cb97-4819-aaf5-5c6eb23ce9c8\") " pod="kube-system/global-pull-secret-syncer-lx6ht"
Sep 11 17:18:27.960810 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.960728    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0620c0a0-8070-4307-86d9-c69a76ea14a9-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-6k9tr\" (UID: \"0620c0a0-8070-4307-86d9-c69a76ea14a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6k9tr"
Sep 11 17:18:27.960863 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.960814    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/0620c0a0-8070-4307-86d9-c69a76ea14a9-ovnkube-config\") pod \"ovnkube-node-6k9tr\" (UID: \"0620c0a0-8070-4307-86d9-c69a76ea14a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6k9tr"
Sep 11 17:18:27.961034 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.961015    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/26e09059-8abc-4e73-bf43-23a51698a0cf-etc-sysctl-d\") pod \"tuned-cph4q\" (UID: \"26e09059-8abc-4e73-bf43-23a51698a0cf\") " pod="openshift-cluster-node-tuning-operator/tuned-cph4q"
Sep 11 17:18:27.961096 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.961083    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/26e09059-8abc-4e73-bf43-23a51698a0cf-run\") pod \"tuned-cph4q\" (UID: \"26e09059-8abc-4e73-bf43-23a51698a0cf\") " pod="openshift-cluster-node-tuning-operator/tuned-cph4q"
Sep 11 17:18:27.961174 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.961126    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b5a337f0-d706-42c3-b846-1997efc3d693-sys-fs\") pod \"aws-ebs-csi-driver-node-22x74\" (UID: \"b5a337f0-d706-42c3-b846-1997efc3d693\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-22x74"
Sep 11 17:18:27.961227 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.961195    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/41755610-5307-49d4-af8e-a021e026aa13-multus-conf-dir\") pod \"multus-q586m\" (UID: \"41755610-5307-49d4-af8e-a021e026aa13\") " pod="openshift-multus/multus-q586m"
Sep 11 17:18:27.961342 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.961311    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/26e09059-8abc-4e73-bf43-23a51698a0cf-etc-sysconfig\") pod \"tuned-cph4q\" (UID: \"26e09059-8abc-4e73-bf43-23a51698a0cf\") " pod="openshift-cluster-node-tuning-operator/tuned-cph4q"
Sep 11 17:18:27.961342 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.961329    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/26e09059-8abc-4e73-bf43-23a51698a0cf-etc-modprobe-d\") pod \"tuned-cph4q\" (UID: \"26e09059-8abc-4e73-bf43-23a51698a0cf\") " pod="openshift-cluster-node-tuning-operator/tuned-cph4q"
Sep 11 17:18:27.961434 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.961387    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b5a337f0-d706-42c3-b846-1997efc3d693-kubelet-dir\") pod \"aws-ebs-csi-driver-node-22x74\" (UID: \"b5a337f0-d706-42c3-b846-1997efc3d693\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-22x74"
Sep 11 17:18:27.961434 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.961424    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0620c0a0-8070-4307-86d9-c69a76ea14a9-var-lib-openvswitch\") pod \"ovnkube-node-6k9tr\" (UID: \"0620c0a0-8070-4307-86d9-c69a76ea14a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6k9tr"
Sep 11 17:18:27.961528 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.961456    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0620c0a0-8070-4307-86d9-c69a76ea14a9-etc-openvswitch\") pod \"ovnkube-node-6k9tr\" (UID: \"0620c0a0-8070-4307-86d9-c69a76ea14a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6k9tr"
Sep 11 17:18:27.961528 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.961490    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lxscx\" (UniqueName: \"kubernetes.io/projected/b5a337f0-d706-42c3-b846-1997efc3d693-kube-api-access-lxscx\") pod \"aws-ebs-csi-driver-node-22x74\" (UID: \"b5a337f0-d706-42c3-b846-1997efc3d693\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-22x74"
Sep 11 17:18:27.961528 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.961519    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b5a337f0-d706-42c3-b846-1997efc3d693-device-dir\") pod \"aws-ebs-csi-driver-node-22x74\" (UID: \"b5a337f0-d706-42c3-b846-1997efc3d693\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-22x74"
Sep 11 17:18:27.961677 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.961569    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/26e09059-8abc-4e73-bf43-23a51698a0cf-etc-systemd\") pod \"tuned-cph4q\" (UID: \"26e09059-8abc-4e73-bf43-23a51698a0cf\") " pod="openshift-cluster-node-tuning-operator/tuned-cph4q"
Sep 11 17:18:27.961677 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.961596    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/72eae49b-cb97-4819-aaf5-5c6eb23ce9c8-dbus\") pod \"global-pull-secret-syncer-lx6ht\" (UID: \"72eae49b-cb97-4819-aaf5-5c6eb23ce9c8\") " pod="kube-system/global-pull-secret-syncer-lx6ht"
Sep 11 17:18:27.961677 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.961633    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/26e09059-8abc-4e73-bf43-23a51698a0cf-etc-tuned\") pod \"tuned-cph4q\" (UID: \"26e09059-8abc-4e73-bf43-23a51698a0cf\") " pod="openshift-cluster-node-tuning-operator/tuned-cph4q"
Sep 11 17:18:27.961823 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.961702    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d0ec6f78-bdc6-420a-a6aa-953d61643e27-host\") pod \"node-ca-gbf8j\" (UID: \"d0ec6f78-bdc6-420a-a6aa-953d61643e27\") " pod="openshift-image-registry/node-ca-gbf8j"
Sep 11 17:18:27.961823 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.961754    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/41755610-5307-49d4-af8e-a021e026aa13-host-var-lib-cni-bin\") pod \"multus-q586m\" (UID: \"41755610-5307-49d4-af8e-a021e026aa13\") " pod="openshift-multus/multus-q586m"
Sep 11 17:18:27.961823 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.961798    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/41755610-5307-49d4-af8e-a021e026aa13-host-var-lib-cni-multus\") pod \"multus-q586m\" (UID: \"41755610-5307-49d4-af8e-a021e026aa13\") " pod="openshift-multus/multus-q586m"
Sep 11 17:18:27.962015 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.961832    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/41755610-5307-49d4-af8e-a021e026aa13-host-var-lib-kubelet\") pod \"multus-q586m\" (UID: \"41755610-5307-49d4-af8e-a021e026aa13\") " pod="openshift-multus/multus-q586m"
Sep 11 17:18:27.962015 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.961970    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e6840164-41a5-4423-bdd0-a364186cf962-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-z559n\" (UID: \"e6840164-41a5-4423-bdd0-a364186cf962\") " pod="openshift-multus/multus-additional-cni-plugins-z559n"
Sep 11 17:18:27.962015 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.962006    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/41755610-5307-49d4-af8e-a021e026aa13-hostroot\") pod \"multus-q586m\" (UID: \"41755610-5307-49d4-af8e-a021e026aa13\") " pod="openshift-multus/multus-q586m"
Sep 11 17:18:27.962188 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.962028    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/26e09059-8abc-4e73-bf43-23a51698a0cf-sys\") pod \"tuned-cph4q\" (UID: \"26e09059-8abc-4e73-bf43-23a51698a0cf\") " pod="openshift-cluster-node-tuning-operator/tuned-cph4q"
Sep 11 17:18:27.962188 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.962123    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/72eae49b-cb97-4819-aaf5-5c6eb23ce9c8-dbus\") pod \"global-pull-secret-syncer-lx6ht\" (UID: \"72eae49b-cb97-4819-aaf5-5c6eb23ce9c8\") " pod="kube-system/global-pull-secret-syncer-lx6ht"
Sep 11 17:18:27.962188 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.962151    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/181608ca-8121-4bf3-8d50-a407708b420a-host-slash\") pod \"iptables-alerter-fptk9\" (UID: \"181608ca-8121-4bf3-8d50-a407708b420a\") " pod="openshift-network-operator/iptables-alerter-fptk9"
Sep 11 17:18:27.962321 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.962211    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/5b082ce5-2ea9-46dc-8307-08a722392886-konnectivity-ca\") pod \"konnectivity-agent-8j6jf\" (UID: \"5b082ce5-2ea9-46dc-8307-08a722392886\") " pod="kube-system/konnectivity-agent-8j6jf"
Sep 11 17:18:27.962408 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:18:27.962390    2552 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 17:18:27.962846 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:18:27.962825    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/72eae49b-cb97-4819-aaf5-5c6eb23ce9c8-original-pull-secret podName:72eae49b-cb97-4819-aaf5-5c6eb23ce9c8 nodeName:}" failed. No retries permitted until 2026-09-11 17:18:28.462688652 +0000 UTC m=+2.983980169 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/72eae49b-cb97-4819-aaf5-5c6eb23ce9c8-original-pull-secret") pod "global-pull-secret-syncer-lx6ht" (UID: "72eae49b-cb97-4819-aaf5-5c6eb23ce9c8") : object "kube-system"/"original-pull-secret" not registered
Sep 11 17:18:27.962936 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.962879    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b5a337f0-d706-42c3-b846-1997efc3d693-device-dir\") pod \"aws-ebs-csi-driver-node-22x74\" (UID: \"b5a337f0-d706-42c3-b846-1997efc3d693\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-22x74"
Sep 11 17:18:27.962936 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.962906    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e6840164-41a5-4423-bdd0-a364186cf962-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-z559n\" (UID: \"e6840164-41a5-4423-bdd0-a364186cf962\") " pod="openshift-multus/multus-additional-cni-plugins-z559n"
Sep 11 17:18:27.963031 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.962937    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/72eae49b-cb97-4819-aaf5-5c6eb23ce9c8-kubelet-config\") pod \"global-pull-secret-syncer-lx6ht\" (UID: \"72eae49b-cb97-4819-aaf5-5c6eb23ce9c8\") " pod="kube-system/global-pull-secret-syncer-lx6ht"
Sep 11 17:18:27.963031 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.963013    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e6840164-41a5-4423-bdd0-a364186cf962-cni-binary-copy\") pod \"multus-additional-cni-plugins-z559n\" (UID: \"e6840164-41a5-4423-bdd0-a364186cf962\") " pod="openshift-multus/multus-additional-cni-plugins-z559n"
Sep 11 17:18:27.963158 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.963042    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/26e09059-8abc-4e73-bf43-23a51698a0cf-tmp\") pod \"tuned-cph4q\" (UID: \"26e09059-8abc-4e73-bf43-23a51698a0cf\") " pod="openshift-cluster-node-tuning-operator/tuned-cph4q"
Sep 11 17:18:27.963249 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:18:27.963210    2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 17:18:27.963249 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:18:27.963232    2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 17:18:27.963249 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:18:27.963246    2552 projected.go:196] Error preparing data for projected volume kube-api-access-42k58 for pod openshift-network-diagnostics/network-check-target-9gb8k: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 17:18:27.963401 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:18:27.963368    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/99bdc64a-0517-49d6-acf3-4e9fc5ea47fe-kube-api-access-42k58 podName:99bdc64a-0517-49d6-acf3-4e9fc5ea47fe nodeName:}" failed. No retries permitted until 2026-09-11 17:18:28.463336281 +0000 UTC m=+2.984627796 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-42k58" (UniqueName: "kubernetes.io/projected/99bdc64a-0517-49d6-acf3-4e9fc5ea47fe-kube-api-access-42k58") pod "network-check-target-9gb8k" (UID: "99bdc64a-0517-49d6-acf3-4e9fc5ea47fe") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 17:18:27.964712 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.964501    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/5b082ce5-2ea9-46dc-8307-08a722392886-agent-certs\") pod \"konnectivity-agent-8j6jf\" (UID: \"5b082ce5-2ea9-46dc-8307-08a722392886\") " pod="kube-system/konnectivity-agent-8j6jf"
Sep 11 17:18:27.964712 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.964664    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/26e09059-8abc-4e73-bf43-23a51698a0cf-etc-tuned\") pod \"tuned-cph4q\" (UID: \"26e09059-8abc-4e73-bf43-23a51698a0cf\") " pod="openshift-cluster-node-tuning-operator/tuned-cph4q"
Sep 11 17:18:27.965720 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.965698    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dx97x\" (UniqueName: \"kubernetes.io/projected/b2b1c077-b5b8-4bdc-a1b3-6b98a0bcf6a0-kube-api-access-dx97x\") pod \"network-metrics-daemon-d5jqp\" (UID: \"b2b1c077-b5b8-4bdc-a1b3-6b98a0bcf6a0\") " pod="openshift-multus/network-metrics-daemon-d5jqp"
Sep 11 17:18:27.967095 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.967072    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xpv4j\" (UniqueName: \"kubernetes.io/projected/26e09059-8abc-4e73-bf43-23a51698a0cf-kube-api-access-xpv4j\") pod \"tuned-cph4q\" (UID: \"26e09059-8abc-4e73-bf43-23a51698a0cf\") " pod="openshift-cluster-node-tuning-operator/tuned-cph4q"
Sep 11 17:18:27.968633 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.968608    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7jckg\" (UniqueName: \"kubernetes.io/projected/e6840164-41a5-4423-bdd0-a364186cf962-kube-api-access-7jckg\") pod \"multus-additional-cni-plugins-z559n\" (UID: \"e6840164-41a5-4423-bdd0-a364186cf962\") " pod="openshift-multus/multus-additional-cni-plugins-z559n"
Sep 11 17:18:27.972329 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:27.972308    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lxscx\" (UniqueName: \"kubernetes.io/projected/b5a337f0-d706-42c3-b846-1997efc3d693-kube-api-access-lxscx\") pod \"aws-ebs-csi-driver-node-22x74\" (UID: \"b5a337f0-d706-42c3-b846-1997efc3d693\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-22x74"
Sep 11 17:18:28.063070 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:28.063037    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0620c0a0-8070-4307-86d9-c69a76ea14a9-host-cni-netd\") pod \"ovnkube-node-6k9tr\" (UID: \"0620c0a0-8070-4307-86d9-c69a76ea14a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6k9tr"
Sep 11 17:18:28.063070 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:28.063075    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xvcc7\" (UniqueName: \"kubernetes.io/projected/d0ec6f78-bdc6-420a-a6aa-953d61643e27-kube-api-access-xvcc7\") pod \"node-ca-gbf8j\" (UID: \"d0ec6f78-bdc6-420a-a6aa-953d61643e27\") " pod="openshift-image-registry/node-ca-gbf8j"
Sep 11 17:18:28.063340 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:28.063098    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/41755610-5307-49d4-af8e-a021e026aa13-multus-socket-dir-parent\") pod \"multus-q586m\" (UID: \"41755610-5307-49d4-af8e-a021e026aa13\") " pod="openshift-multus/multus-q586m"
Sep 11 17:18:28.063340 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:28.063152    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/41755610-5307-49d4-af8e-a021e026aa13-host-run-netns\") pod \"multus-q586m\" (UID: \"41755610-5307-49d4-af8e-a021e026aa13\") " pod="openshift-multus/multus-q586m"
Sep 11 17:18:28.063340 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:28.063175    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tdhvz\" (UniqueName: \"kubernetes.io/projected/181608ca-8121-4bf3-8d50-a407708b420a-kube-api-access-tdhvz\") pod \"iptables-alerter-fptk9\" (UID: \"181608ca-8121-4bf3-8d50-a407708b420a\") " pod="openshift-network-operator/iptables-alerter-fptk9"
Sep 11 17:18:28.063340 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:28.063174    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0620c0a0-8070-4307-86d9-c69a76ea14a9-host-cni-netd\") pod \"ovnkube-node-6k9tr\" (UID: \"0620c0a0-8070-4307-86d9-c69a76ea14a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6k9tr"
Sep 11 17:18:28.063340 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:28.063197    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0620c0a0-8070-4307-86d9-c69a76ea14a9-run-ovn\") pod \"ovnkube-node-6k9tr\" (UID: \"0620c0a0-8070-4307-86d9-c69a76ea14a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6k9tr"
Sep 11 17:18:28.063340 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:28.063249    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8b4c1f98-046f-47c4-889f-c2bd807dddbe-tmp-dir\") pod \"node-resolver-h6lht\" (UID: \"8b4c1f98-046f-47c4-889f-c2bd807dddbe\") " pod="openshift-dns/node-resolver-h6lht"
Sep 11 17:18:28.063340 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:28.063251    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/41755610-5307-49d4-af8e-a021e026aa13-host-run-netns\") pod \"multus-q586m\" (UID: \"41755610-5307-49d4-af8e-a021e026aa13\") " pod="openshift-multus/multus-q586m"
Sep 11 17:18:28.063340 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:28.063269    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/41755610-5307-49d4-af8e-a021e026aa13-multus-socket-dir-parent\") pod \"multus-q586m\" (UID: \"41755610-5307-49d4-af8e-a021e026aa13\") " pod="openshift-multus/multus-q586m"
Sep 11 17:18:28.063340 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:28.063282    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0620c0a0-8070-4307-86d9-c69a76ea14a9-systemd-units\") pod \"ovnkube-node-6k9tr\" (UID: \"0620c0a0-8070-4307-86d9-c69a76ea14a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6k9tr"
Sep 11 17:18:28.063340 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:28.063331    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0620c0a0-8070-4307-86d9-c69a76ea14a9-systemd-units\") pod \"ovnkube-node-6k9tr\" (UID: \"0620c0a0-8070-4307-86d9-c69a76ea14a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6k9tr"
Sep 11 17:18:28.063807 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:28.063368    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0620c0a0-8070-4307-86d9-c69a76ea14a9-run-ovn\") pod \"ovnkube-node-6k9tr\" (UID: \"0620c0a0-8070-4307-86d9-c69a76ea14a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6k9tr"
Sep 11 17:18:28.063807 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:28.063446    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0620c0a0-8070-4307-86d9-c69a76ea14a9-host-run-netns\") pod \"ovnkube-node-6k9tr\" (UID: \"0620c0a0-8070-4307-86d9-c69a76ea14a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6k9tr"
Sep 11 17:18:28.063807 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:28.063475    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0620c0a0-8070-4307-86d9-c69a76ea14a9-run-systemd\") pod \"ovnkube-node-6k9tr\" (UID: \"0620c0a0-8070-4307-86d9-c69a76ea14a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6k9tr"
Sep 11 17:18:28.063807 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:28.063496    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0620c0a0-8070-4307-86d9-c69a76ea14a9-host-cni-bin\") pod \"ovnkube-node-6k9tr\" (UID: \"0620c0a0-8070-4307-86d9-c69a76ea14a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6k9tr"
Sep 11 17:18:28.063807 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:28.063520    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0620c0a0-8070-4307-86d9-c69a76ea14a9-host-run-netns\") pod \"ovnkube-node-6k9tr\" (UID: \"0620c0a0-8070-4307-86d9-c69a76ea14a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6k9tr"
Sep 11 17:18:28.063807 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:28.063521    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/0620c0a0-8070-4307-86d9-c69a76ea14a9-ovnkube-script-lib\") pod \"ovnkube-node-6k9tr\" (UID: \"0620c0a0-8070-4307-86d9-c69a76ea14a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6k9tr"
Sep 11 17:18:28.063807 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:28.063553    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0620c0a0-8070-4307-86d9-c69a76ea14a9-run-systemd\") pod \"ovnkube-node-6k9tr\" (UID: \"0620c0a0-8070-4307-86d9-c69a76ea14a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6k9tr"
Sep 11 17:18:28.063807 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:28.063568    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/41755610-5307-49d4-af8e-a021e026aa13-system-cni-dir\") pod \"multus-q586m\" (UID: \"41755610-5307-49d4-af8e-a021e026aa13\") " pod="openshift-multus/multus-q586m"
Sep 11 17:18:28.063807 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:28.063584    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8b4c1f98-046f-47c4-889f-c2bd807dddbe-tmp-dir\") pod \"node-resolver-h6lht\" (UID: \"8b4c1f98-046f-47c4-889f-c2bd807dddbe\") " pod="openshift-dns/node-resolver-h6lht"
Sep 11 17:18:28.063807 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:28.063594    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/41755610-5307-49d4-af8e-a021e026aa13-multus-cni-dir\") pod \"multus-q586m\" (UID: \"41755610-5307-49d4-af8e-a021e026aa13\") " pod="openshift-multus/multus-q586m"
Sep 11 17:18:28.063807 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:28.063602    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0620c0a0-8070-4307-86d9-c69a76ea14a9-host-cni-bin\") pod \"ovnkube-node-6k9tr\" (UID: \"0620c0a0-8070-4307-86d9-c69a76ea14a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6k9tr"
Sep 11 17:18:28.063807 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:28.063631    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-798k2\" (UniqueName: \"kubernetes.io/projected/41755610-5307-49d4-af8e-a021e026aa13-kube-api-access-798k2\") pod \"multus-q586m\" (UID: \"41755610-5307-49d4-af8e-a021e026aa13\") " pod="openshift-multus/multus-q586m"
Sep 11 17:18:28.063807 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:28.063662    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/41755610-5307-49d4-af8e-a021e026aa13-multus-cni-dir\") pod \"multus-q586m\" (UID: \"41755610-5307-49d4-af8e-a021e026aa13\") " pod="openshift-multus/multus-q586m"
Sep 11 17:18:28.063807 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:28.063765    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/8b4c1f98-046f-47c4-889f-c2bd807dddbe-hosts-file\") pod \"node-resolver-h6lht\" (UID: \"8b4c1f98-046f-47c4-889f-c2bd807dddbe\") " pod="openshift-dns/node-resolver-h6lht"
Sep 11 17:18:28.063807 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:28.063795    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/41755610-5307-49d4-af8e-a021e026aa13-system-cni-dir\") pod \"multus-q586m\" (UID: \"41755610-5307-49d4-af8e-a021e026aa13\") " pod="openshift-multus/multus-q586m"
Sep 11 17:18:28.063807 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:28.063796    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/0620c0a0-8070-4307-86d9-c69a76ea14a9-ovn-node-metrics-cert\") pod \"ovnkube-node-6k9tr\" (UID: \"0620c0a0-8070-4307-86d9-c69a76ea14a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6k9tr"
Sep 11 17:18:28.064638 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:28.063841    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/41755610-5307-49d4-af8e-a021e026aa13-etc-kubernetes\") pod \"multus-q586m\" (UID: \"41755610-5307-49d4-af8e-a021e026aa13\") " pod="openshift-multus/multus-q586m"
Sep 11 17:18:28.064638 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:28.063866    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/8b4c1f98-046f-47c4-889f-c2bd807dddbe-hosts-file\") pod \"node-resolver-h6lht\" (UID: \"8b4c1f98-046f-47c4-889f-c2bd807dddbe\") " pod="openshift-dns/node-resolver-h6lht"
Sep 11 17:18:28.064638 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:28.063871    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/41755610-5307-49d4-af8e-a021e026aa13-cni-binary-copy\") pod \"multus-q586m\" (UID: \"41755610-5307-49d4-af8e-a021e026aa13\") " pod="openshift-multus/multus-q586m"
Sep 11 17:18:28.064638 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:28.063916    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0620c0a0-8070-4307-86d9-c69a76ea14a9-host-kubelet\") pod \"ovnkube-node-6k9tr\" (UID: \"0620c0a0-8070-4307-86d9-c69a76ea14a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6k9tr"
Sep 11 17:18:28.064638 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:28.063935    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0620c0a0-8070-4307-86d9-c69a76ea14a9-run-openvswitch\") pod \"ovnkube-node-6k9tr\" (UID: \"0620c0a0-8070-4307-86d9-c69a76ea14a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6k9tr"
Sep 11 17:18:28.064638 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:28.063930    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/41755610-5307-49d4-af8e-a021e026aa13-etc-kubernetes\") pod \"multus-q586m\" (UID: \"41755610-5307-49d4-af8e-a021e026aa13\") " pod="openshift-multus/multus-q586m"
Sep 11 17:18:28.064638 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:28.063951    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0620c0a0-8070-4307-86d9-c69a76ea14a9-node-log\") pod \"ovnkube-node-6k9tr\" (UID: \"0620c0a0-8070-4307-86d9-c69a76ea14a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6k9tr"
Sep 11 17:18:28.064638 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:28.063975    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/41755610-5307-49d4-af8e-a021e026aa13-host-run-multus-certs\") pod \"multus-q586m\" (UID: \"41755610-5307-49d4-af8e-a021e026aa13\") " pod="openshift-multus/multus-q586m"
Sep 11 17:18:28.064638 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:28.063993    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0620c0a0-8070-4307-86d9-c69a76ea14a9-host-kubelet\") pod \"ovnkube-node-6k9tr\" (UID: \"0620c0a0-8070-4307-86d9-c69a76ea14a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6k9tr"
Sep 11 17:18:28.064638 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:28.064006    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0620c0a0-8070-4307-86d9-c69a76ea14a9-run-openvswitch\") pod \"ovnkube-node-6k9tr\" (UID: \"0620c0a0-8070-4307-86d9-c69a76ea14a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6k9tr"
Sep 11 17:18:28.064638 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:28.064019    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0620c0a0-8070-4307-86d9-c69a76ea14a9-log-socket\") pod \"ovnkube-node-6k9tr\" (UID: \"0620c0a0-8070-4307-86d9-c69a76ea14a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6k9tr"
Sep 11 17:18:28.064638 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:28.064051    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jf7ct\" (UniqueName: \"kubernetes.io/projected/0620c0a0-8070-4307-86d9-c69a76ea14a9-kube-api-access-jf7ct\") pod \"ovnkube-node-6k9tr\" (UID: \"0620c0a0-8070-4307-86d9-c69a76ea14a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6k9tr"
Sep 11 17:18:28.064638 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:28.064066    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/41755610-5307-49d4-af8e-a021e026aa13-host-run-multus-certs\") pod \"multus-q586m\" (UID: \"41755610-5307-49d4-af8e-a021e026aa13\") " pod="openshift-multus/multus-q586m"
Sep 11 17:18:28.064638 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:28.064104    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0620c0a0-8070-4307-86d9-c69a76ea14a9-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-6k9tr\" (UID: \"0620c0a0-8070-4307-86d9-c69a76ea14a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6k9tr"
Sep 11 17:18:28.064638 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:28.064149    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/0620c0a0-8070-4307-86d9-c69a76ea14a9-ovnkube-config\") pod \"ovnkube-node-6k9tr\" (UID: \"0620c0a0-8070-4307-86d9-c69a76ea14a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6k9tr"
Sep 11 17:18:28.064638 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:28.064123    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0620c0a0-8070-4307-86d9-c69a76ea14a9-log-socket\") pod \"ovnkube-node-6k9tr\" (UID: \"0620c0a0-8070-4307-86d9-c69a76ea14a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6k9tr"
Sep 11 17:18:28.064638 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:28.064172    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/0620c0a0-8070-4307-86d9-c69a76ea14a9-ovnkube-script-lib\") pod \"ovnkube-node-6k9tr\" (UID: \"0620c0a0-8070-4307-86d9-c69a76ea14a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6k9tr"
Sep 11 17:18:28.065450 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:28.064180    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0620c0a0-8070-4307-86d9-c69a76ea14a9-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-6k9tr\" (UID: \"0620c0a0-8070-4307-86d9-c69a76ea14a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6k9tr"
Sep 11 17:18:28.065450 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:28.064176    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/41755610-5307-49d4-af8e-a021e026aa13-multus-conf-dir\") pod \"multus-q586m\" (UID: \"41755610-5307-49d4-af8e-a021e026aa13\") " pod="openshift-multus/multus-q586m"
Sep 11 17:18:28.065450 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:28.064151    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0620c0a0-8070-4307-86d9-c69a76ea14a9-node-log\") pod \"ovnkube-node-6k9tr\" (UID: \"0620c0a0-8070-4307-86d9-c69a76ea14a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6k9tr"
Sep 11 17:18:28.065450 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:28.064232    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0620c0a0-8070-4307-86d9-c69a76ea14a9-var-lib-openvswitch\") pod \"ovnkube-node-6k9tr\" (UID: \"0620c0a0-8070-4307-86d9-c69a76ea14a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6k9tr"
Sep 11 17:18:28.065450 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:28.064244    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/41755610-5307-49d4-af8e-a021e026aa13-multus-conf-dir\") pod \"multus-q586m\" (UID: \"41755610-5307-49d4-af8e-a021e026aa13\") " pod="openshift-multus/multus-q586m"
Sep 11 17:18:28.065450 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:28.064258    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0620c0a0-8070-4307-86d9-c69a76ea14a9-etc-openvswitch\") pod \"ovnkube-node-6k9tr\" (UID: \"0620c0a0-8070-4307-86d9-c69a76ea14a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6k9tr"
Sep 11 17:18:28.065450 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:28.064286    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d0ec6f78-bdc6-420a-a6aa-953d61643e27-host\") pod \"node-ca-gbf8j\" (UID: \"d0ec6f78-bdc6-420a-a6aa-953d61643e27\") " pod="openshift-image-registry/node-ca-gbf8j"
Sep 11 17:18:28.065450 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:28.064311    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/41755610-5307-49d4-af8e-a021e026aa13-host-var-lib-cni-bin\") pod \"multus-q586m\" (UID: \"41755610-5307-49d4-af8e-a021e026aa13\") " pod="openshift-multus/multus-q586m"
Sep 11 17:18:28.065450 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:28.064319    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0620c0a0-8070-4307-86d9-c69a76ea14a9-etc-openvswitch\") pod \"ovnkube-node-6k9tr\" (UID: \"0620c0a0-8070-4307-86d9-c69a76ea14a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6k9tr"
Sep 11 17:18:28.065450 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:28.064286    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0620c0a0-8070-4307-86d9-c69a76ea14a9-var-lib-openvswitch\") pod \"ovnkube-node-6k9tr\" (UID: \"0620c0a0-8070-4307-86d9-c69a76ea14a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6k9tr"
Sep 11 17:18:28.065450 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:28.064335    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/41755610-5307-49d4-af8e-a021e026aa13-host-var-lib-cni-multus\") pod \"multus-q586m\" (UID: \"41755610-5307-49d4-af8e-a021e026aa13\") " pod="openshift-multus/multus-q586m"
Sep 11 17:18:28.065450 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:28.064359    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/41755610-5307-49d4-af8e-a021e026aa13-host-var-lib-kubelet\") pod \"multus-q586m\" (UID: \"41755610-5307-49d4-af8e-a021e026aa13\") " pod="openshift-multus/multus-q586m"
Sep 11 17:18:28.065450 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:28.064366    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d0ec6f78-bdc6-420a-a6aa-953d61643e27-host\") pod \"node-ca-gbf8j\" (UID: \"d0ec6f78-bdc6-420a-a6aa-953d61643e27\") " pod="openshift-image-registry/node-ca-gbf8j"
Sep 11 17:18:28.065450 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:28.064388    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/41755610-5307-49d4-af8e-a021e026aa13-hostroot\") pod \"multus-q586m\" (UID: \"41755610-5307-49d4-af8e-a021e026aa13\") " pod="openshift-multus/multus-q586m"
Sep 11 17:18:28.065450 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:28.064408    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/41755610-5307-49d4-af8e-a021e026aa13-host-var-lib-cni-bin\") pod \"multus-q586m\" (UID: \"41755610-5307-49d4-af8e-a021e026aa13\") " pod="openshift-multus/multus-q586m"
Sep 11 17:18:28.065450 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:28.064413    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/181608ca-8121-4bf3-8d50-a407708b420a-host-slash\") pod \"iptables-alerter-fptk9\" (UID: \"181608ca-8121-4bf3-8d50-a407708b420a\") " pod="openshift-network-operator/iptables-alerter-fptk9"
Sep 11 17:18:28.065450 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:28.064451    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/41755610-5307-49d4-af8e-a021e026aa13-host-var-lib-cni-multus\") pod \"multus-q586m\" (UID: \"41755610-5307-49d4-af8e-a021e026aa13\") " pod="openshift-multus/multus-q586m"
Sep 11 17:18:28.065450 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:28.064453    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0620c0a0-8070-4307-86d9-c69a76ea14a9-host-run-ovn-kubernetes\") pod \"ovnkube-node-6k9tr\" (UID: \"0620c0a0-8070-4307-86d9-c69a76ea14a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6k9tr"
Sep 11 17:18:28.066038 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:28.064457    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/41755610-5307-49d4-af8e-a021e026aa13-host-var-lib-kubelet\") pod \"multus-q586m\" (UID: \"41755610-5307-49d4-af8e-a021e026aa13\") " pod="openshift-multus/multus-q586m"
Sep 11 17:18:28.066038 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:28.064488    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/41755610-5307-49d4-af8e-a021e026aa13-cnibin\") pod \"multus-q586m\" (UID: \"41755610-5307-49d4-af8e-a021e026aa13\") " pod="openshift-multus/multus-q586m"
Sep 11 17:18:28.066038 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:28.064506    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0620c0a0-8070-4307-86d9-c69a76ea14a9-host-run-ovn-kubernetes\") pod \"ovnkube-node-6k9tr\" (UID: \"0620c0a0-8070-4307-86d9-c69a76ea14a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6k9tr"
Sep 11 17:18:28.066038 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:28.064514    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/41755610-5307-49d4-af8e-a021e026aa13-multus-daemon-config\") pod \"multus-q586m\" (UID: \"41755610-5307-49d4-af8e-a021e026aa13\") " pod="openshift-multus/multus-q586m"
Sep 11 17:18:28.066038 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:28.064523    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/41755610-5307-49d4-af8e-a021e026aa13-hostroot\") pod \"multus-q586m\" (UID: \"41755610-5307-49d4-af8e-a021e026aa13\") " pod="openshift-multus/multus-q586m"
Sep 11 17:18:28.066038 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:28.064544    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/41755610-5307-49d4-af8e-a021e026aa13-cni-binary-copy\") pod \"multus-q586m\" (UID: \"41755610-5307-49d4-af8e-a021e026aa13\") " pod="openshift-multus/multus-q586m"
Sep 11 17:18:28.066038 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:28.064548    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/181608ca-8121-4bf3-8d50-a407708b420a-host-slash\") pod \"iptables-alerter-fptk9\" (UID: \"181608ca-8121-4bf3-8d50-a407708b420a\") " pod="openshift-network-operator/iptables-alerter-fptk9"
Sep 11 17:18:28.066038 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:28.064580    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/41755610-5307-49d4-af8e-a021e026aa13-cnibin\") pod \"multus-q586m\" (UID: \"41755610-5307-49d4-af8e-a021e026aa13\") " pod="openshift-multus/multus-q586m"
Sep 11 17:18:28.066038 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:28.064571    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0620c0a0-8070-4307-86d9-c69a76ea14a9-host-slash\") pod \"ovnkube-node-6k9tr\" (UID: \"0620c0a0-8070-4307-86d9-c69a76ea14a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6k9tr"
Sep 11 17:18:28.066038 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:28.064612    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0620c0a0-8070-4307-86d9-c69a76ea14a9-host-slash\") pod \"ovnkube-node-6k9tr\" (UID: \"0620c0a0-8070-4307-86d9-c69a76ea14a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6k9tr"
Sep 11 17:18:28.066038 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:28.064617    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/0620c0a0-8070-4307-86d9-c69a76ea14a9-env-overrides\") pod \"ovnkube-node-6k9tr\" (UID: \"0620c0a0-8070-4307-86d9-c69a76ea14a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6k9tr"
Sep 11 17:18:28.066038 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:28.064649    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d0ec6f78-bdc6-420a-a6aa-953d61643e27-serviceca\") pod \"node-ca-gbf8j\" (UID: \"d0ec6f78-bdc6-420a-a6aa-953d61643e27\") " pod="openshift-image-registry/node-ca-gbf8j"
Sep 11 17:18:28.066038 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:28.064688    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/41755610-5307-49d4-af8e-a021e026aa13-host-run-k8s-cni-cncf-io\") pod \"multus-q586m\" (UID: \"41755610-5307-49d4-af8e-a021e026aa13\") " pod="openshift-multus/multus-q586m"
Sep 11 17:18:28.066038 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:28.064702    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/0620c0a0-8070-4307-86d9-c69a76ea14a9-ovnkube-config\") pod \"ovnkube-node-6k9tr\" (UID: \"0620c0a0-8070-4307-86d9-c69a76ea14a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6k9tr"
Sep 11 17:18:28.066038 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:28.064730    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wqxzx\" (UniqueName: \"kubernetes.io/projected/8b4c1f98-046f-47c4-889f-c2bd807dddbe-kube-api-access-wqxzx\") pod \"node-resolver-h6lht\" (UID: \"8b4c1f98-046f-47c4-889f-c2bd807dddbe\") " pod="openshift-dns/node-resolver-h6lht"
Sep 11 17:18:28.066038 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:28.064758    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/181608ca-8121-4bf3-8d50-a407708b420a-iptables-alerter-script\") pod \"iptables-alerter-fptk9\" (UID: \"181608ca-8121-4bf3-8d50-a407708b420a\") " pod="openshift-network-operator/iptables-alerter-fptk9"
Sep 11 17:18:28.066038 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:28.064968    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/0620c0a0-8070-4307-86d9-c69a76ea14a9-env-overrides\") pod \"ovnkube-node-6k9tr\" (UID: \"0620c0a0-8070-4307-86d9-c69a76ea14a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6k9tr"
Sep 11 17:18:28.066038 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:28.065021    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/41755610-5307-49d4-af8e-a021e026aa13-host-run-k8s-cni-cncf-io\") pod \"multus-q586m\" (UID: \"41755610-5307-49d4-af8e-a021e026aa13\") " pod="openshift-multus/multus-q586m"
Sep 11 17:18:28.066786 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:28.065048    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/41755610-5307-49d4-af8e-a021e026aa13-multus-daemon-config\") pod \"multus-q586m\" (UID: \"41755610-5307-49d4-af8e-a021e026aa13\") " pod="openshift-multus/multus-q586m"
Sep 11 17:18:28.066786 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:28.065145    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d0ec6f78-bdc6-420a-a6aa-953d61643e27-serviceca\") pod \"node-ca-gbf8j\" (UID: \"d0ec6f78-bdc6-420a-a6aa-953d61643e27\") " pod="openshift-image-registry/node-ca-gbf8j"
Sep 11 17:18:28.066786 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:28.065358    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/181608ca-8121-4bf3-8d50-a407708b420a-iptables-alerter-script\") pod \"iptables-alerter-fptk9\" (UID: \"181608ca-8121-4bf3-8d50-a407708b420a\") " pod="openshift-network-operator/iptables-alerter-fptk9"
Sep 11 17:18:28.066895 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:28.066877    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/0620c0a0-8070-4307-86d9-c69a76ea14a9-ovn-node-metrics-cert\") pod \"ovnkube-node-6k9tr\" (UID: \"0620c0a0-8070-4307-86d9-c69a76ea14a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6k9tr"
Sep 11 17:18:28.089752 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:28.089684    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xvcc7\" (UniqueName: \"kubernetes.io/projected/d0ec6f78-bdc6-420a-a6aa-953d61643e27-kube-api-access-xvcc7\") pod \"node-ca-gbf8j\" (UID: \"d0ec6f78-bdc6-420a-a6aa-953d61643e27\") " pod="openshift-image-registry/node-ca-gbf8j"
Sep 11 17:18:28.089877 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:28.089747    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jf7ct\" (UniqueName: \"kubernetes.io/projected/0620c0a0-8070-4307-86d9-c69a76ea14a9-kube-api-access-jf7ct\") pod \"ovnkube-node-6k9tr\" (UID: \"0620c0a0-8070-4307-86d9-c69a76ea14a9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6k9tr"
Sep 11 17:18:28.089877 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:28.089744    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tdhvz\" (UniqueName: \"kubernetes.io/projected/181608ca-8121-4bf3-8d50-a407708b420a-kube-api-access-tdhvz\") pod \"iptables-alerter-fptk9\" (UID: \"181608ca-8121-4bf3-8d50-a407708b420a\") " pod="openshift-network-operator/iptables-alerter-fptk9"
Sep 11 17:18:28.091313 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:28.091291    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-798k2\" (UniqueName: \"kubernetes.io/projected/41755610-5307-49d4-af8e-a021e026aa13-kube-api-access-798k2\") pod \"multus-q586m\" (UID: \"41755610-5307-49d4-af8e-a021e026aa13\") " pod="openshift-multus/multus-q586m"
Sep 11 17:18:28.092082 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:28.092066    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wqxzx\" (UniqueName: \"kubernetes.io/projected/8b4c1f98-046f-47c4-889f-c2bd807dddbe-kube-api-access-wqxzx\") pod \"node-resolver-h6lht\" (UID: \"8b4c1f98-046f-47c4-889f-c2bd807dddbe\") " pod="openshift-dns/node-resolver-h6lht"
Sep 11 17:18:28.139878 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:28.139851    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-cph4q"
Sep 11 17:18:28.147719 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:28.147689    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-z559n"
Sep 11 17:18:28.148072 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:28.148049    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26e09059_8abc_4e73_bf43_23a51698a0cf.slice/crio-cd625f89f705de8d6458143aafeaeef1dad848e67f7c0f8abaaeb49344ee181a WatchSource:0}: Error finding container cd625f89f705de8d6458143aafeaeef1dad848e67f7c0f8abaaeb49344ee181a: Status 404 returned error can't find the container with id cd625f89f705de8d6458143aafeaeef1dad848e67f7c0f8abaaeb49344ee181a
Sep 11 17:18:28.155766 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:28.155742    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6840164_41a5_4423_bdd0_a364186cf962.slice/crio-3214686e3c78389cc2890b79f40a6f0033db6a997a37eaa213c3c9fd518653b0 WatchSource:0}: Error finding container 3214686e3c78389cc2890b79f40a6f0033db6a997a37eaa213c3c9fd518653b0: Status 404 returned error can't find the container with id 3214686e3c78389cc2890b79f40a6f0033db6a997a37eaa213c3c9fd518653b0
Sep 11 17:18:28.162717 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:28.162696    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-22x74"
Sep 11 17:18:28.167581 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:28.167563    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-8j6jf"
Sep 11 17:18:28.170891 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:28.170870    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5a337f0_d706_42c3_b846_1997efc3d693.slice/crio-cc7621b9ee5754b9455328dc4aff43d1423b87af2bbbee65f7eb4252078b8315 WatchSource:0}: Error finding container cc7621b9ee5754b9455328dc4aff43d1423b87af2bbbee65f7eb4252078b8315: Status 404 returned error can't find the container with id cc7621b9ee5754b9455328dc4aff43d1423b87af2bbbee65f7eb4252078b8315
Sep 11 17:18:28.173271 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:28.173255    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-gbf8j"
Sep 11 17:18:28.176004 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:28.175976    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b082ce5_2ea9_46dc_8307_08a722392886.slice/crio-4ca1efe948f56f081f8513ddc2983ac35caf94e0ce7c673d3a7d182951285efb WatchSource:0}: Error finding container 4ca1efe948f56f081f8513ddc2983ac35caf94e0ce7c673d3a7d182951285efb: Status 404 returned error can't find the container with id 4ca1efe948f56f081f8513ddc2983ac35caf94e0ce7c673d3a7d182951285efb
Sep 11 17:18:28.181850 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:28.181829    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-h6lht"
Sep 11 17:18:28.182832 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:28.182811    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0ec6f78_bdc6_420a_a6aa_953d61643e27.slice/crio-d5214f49d9a606d0d76f74fa4dd11d4cd7e75b1dee8f288b10eeabdeb6a6a923 WatchSource:0}: Error finding container d5214f49d9a606d0d76f74fa4dd11d4cd7e75b1dee8f288b10eeabdeb6a6a923: Status 404 returned error can't find the container with id d5214f49d9a606d0d76f74fa4dd11d4cd7e75b1dee8f288b10eeabdeb6a6a923
Sep 11 17:18:28.190429 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:28.190412    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-q586m"
Sep 11 17:18:28.191143 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:28.191101    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b4c1f98_046f_47c4_889f_c2bd807dddbe.slice/crio-d9302cf6d2af1e15df909c0896d133d0f98209a42749a6094e122632a2b689a9 WatchSource:0}: Error finding container d9302cf6d2af1e15df909c0896d133d0f98209a42749a6094e122632a2b689a9: Status 404 returned error can't find the container with id d9302cf6d2af1e15df909c0896d133d0f98209a42749a6094e122632a2b689a9
Sep 11 17:18:28.198234 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:28.198214    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41755610_5307_49d4_af8e_a021e026aa13.slice/crio-b76a60f6c3f68ba01fb86da9d0767355312f7db6d3d59e81989f83ccfc3c1f9b WatchSource:0}: Error finding container b76a60f6c3f68ba01fb86da9d0767355312f7db6d3d59e81989f83ccfc3c1f9b: Status 404 returned error can't find the container with id b76a60f6c3f68ba01fb86da9d0767355312f7db6d3d59e81989f83ccfc3c1f9b
Sep 11 17:18:28.198313 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:28.198271    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-fptk9"
Sep 11 17:18:28.205724 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:28.205702    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-6k9tr"
Sep 11 17:18:28.207961 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:28.207932    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod181608ca_8121_4bf3_8d50_a407708b420a.slice/crio-d6c5065148b60f9d73d248d0472b94e7293fb846db3686f0f19af3db436f2bf6 WatchSource:0}: Error finding container d6c5065148b60f9d73d248d0472b94e7293fb846db3686f0f19af3db436f2bf6: Status 404 returned error can't find the container with id d6c5065148b60f9d73d248d0472b94e7293fb846db3686f0f19af3db436f2bf6
Sep 11 17:18:28.214842 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:28.214808    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0620c0a0_8070_4307_86d9_c69a76ea14a9.slice/crio-8095538755cab36abbbe0c28143a32342870cdec8f1f6f23f0449c1e8079b09f WatchSource:0}: Error finding container 8095538755cab36abbbe0c28143a32342870cdec8f1f6f23f0449c1e8079b09f: Status 404 returned error can't find the container with id 8095538755cab36abbbe0c28143a32342870cdec8f1f6f23f0449c1e8079b09f
Sep 11 17:18:28.268787 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:28.268589    2552 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 17:18:28.467687 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:28.467612    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/72eae49b-cb97-4819-aaf5-5c6eb23ce9c8-original-pull-secret\") pod \"global-pull-secret-syncer-lx6ht\" (UID: \"72eae49b-cb97-4819-aaf5-5c6eb23ce9c8\") " pod="kube-system/global-pull-secret-syncer-lx6ht"
Sep 11 17:18:28.467687 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:28.467668    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-42k58\" (UniqueName: \"kubernetes.io/projected/99bdc64a-0517-49d6-acf3-4e9fc5ea47fe-kube-api-access-42k58\") pod \"network-check-target-9gb8k\" (UID: \"99bdc64a-0517-49d6-acf3-4e9fc5ea47fe\") " pod="openshift-network-diagnostics/network-check-target-9gb8k"
Sep 11 17:18:28.467912 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:28.467699    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b2b1c077-b5b8-4bdc-a1b3-6b98a0bcf6a0-metrics-certs\") pod \"network-metrics-daemon-d5jqp\" (UID: \"b2b1c077-b5b8-4bdc-a1b3-6b98a0bcf6a0\") " pod="openshift-multus/network-metrics-daemon-d5jqp"
Sep 11 17:18:28.467912 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:18:28.467764    2552 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 17:18:28.467912 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:18:28.467822    2552 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 17:18:28.467912 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:18:28.467834    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/72eae49b-cb97-4819-aaf5-5c6eb23ce9c8-original-pull-secret podName:72eae49b-cb97-4819-aaf5-5c6eb23ce9c8 nodeName:}" failed. No retries permitted until 2026-09-11 17:18:29.467818796 +0000 UTC m=+3.989110310 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/72eae49b-cb97-4819-aaf5-5c6eb23ce9c8-original-pull-secret") pod "global-pull-secret-syncer-lx6ht" (UID: "72eae49b-cb97-4819-aaf5-5c6eb23ce9c8") : object "kube-system"/"original-pull-secret" not registered
Sep 11 17:18:28.467912 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:18:28.467853    2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 17:18:28.467912 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:18:28.467872    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b2b1c077-b5b8-4bdc-a1b3-6b98a0bcf6a0-metrics-certs podName:b2b1c077-b5b8-4bdc-a1b3-6b98a0bcf6a0 nodeName:}" failed. No retries permitted until 2026-09-11 17:18:29.467856569 +0000 UTC m=+3.989148085 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b2b1c077-b5b8-4bdc-a1b3-6b98a0bcf6a0-metrics-certs") pod "network-metrics-daemon-d5jqp" (UID: "b2b1c077-b5b8-4bdc-a1b3-6b98a0bcf6a0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 17:18:28.467912 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:18:28.467878    2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 17:18:28.467912 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:18:28.467891    2552 projected.go:196] Error preparing data for projected volume kube-api-access-42k58 for pod openshift-network-diagnostics/network-check-target-9gb8k: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 17:18:28.468261 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:18:28.467927    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/99bdc64a-0517-49d6-acf3-4e9fc5ea47fe-kube-api-access-42k58 podName:99bdc64a-0517-49d6-acf3-4e9fc5ea47fe nodeName:}" failed. No retries permitted until 2026-09-11 17:18:29.467917386 +0000 UTC m=+3.989208885 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-42k58" (UniqueName: "kubernetes.io/projected/99bdc64a-0517-49d6-acf3-4e9fc5ea47fe-kube-api-access-42k58") pod "network-check-target-9gb8k" (UID: "99bdc64a-0517-49d6-acf3-4e9fc5ea47fe") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 17:18:28.814937 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:28.814856    2552 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 17:18:28.896055 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:28.895999    2552 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-10 17:13:27 +0000 UTC" deadline="2028-03-11 20:45:14.030823538 +0000 UTC"
Sep 11 17:18:28.896055 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:28.896050    2552 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13131h26m45.134777467s"
Sep 11 17:18:28.951950 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:28.951910    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-fptk9" event={"ID":"181608ca-8121-4bf3-8d50-a407708b420a","Type":"ContainerStarted","Data":"d6c5065148b60f9d73d248d0472b94e7293fb846db3686f0f19af3db436f2bf6"}
Sep 11 17:18:28.953197 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:28.953166    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-h6lht" event={"ID":"8b4c1f98-046f-47c4-889f-c2bd807dddbe","Type":"ContainerStarted","Data":"d9302cf6d2af1e15df909c0896d133d0f98209a42749a6094e122632a2b689a9"}
Sep 11 17:18:28.954433 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:28.954403    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-8j6jf" event={"ID":"5b082ce5-2ea9-46dc-8307-08a722392886","Type":"ContainerStarted","Data":"4ca1efe948f56f081f8513ddc2983ac35caf94e0ce7c673d3a7d182951285efb"}
Sep 11 17:18:28.955718 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:28.955682    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-22x74" event={"ID":"b5a337f0-d706-42c3-b846-1997efc3d693","Type":"ContainerStarted","Data":"cc7621b9ee5754b9455328dc4aff43d1423b87af2bbbee65f7eb4252078b8315"}
Sep 11 17:18:28.956977 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:28.956947    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-cph4q" event={"ID":"26e09059-8abc-4e73-bf43-23a51698a0cf","Type":"ContainerStarted","Data":"cd625f89f705de8d6458143aafeaeef1dad848e67f7c0f8abaaeb49344ee181a"}
Sep 11 17:18:28.958661 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:28.958635    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6k9tr" event={"ID":"0620c0a0-8070-4307-86d9-c69a76ea14a9","Type":"ContainerStarted","Data":"8095538755cab36abbbe0c28143a32342870cdec8f1f6f23f0449c1e8079b09f"}
Sep 11 17:18:28.960072 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:28.960049    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-q586m" event={"ID":"41755610-5307-49d4-af8e-a021e026aa13","Type":"ContainerStarted","Data":"b76a60f6c3f68ba01fb86da9d0767355312f7db6d3d59e81989f83ccfc3c1f9b"}
Sep 11 17:18:28.961292 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:28.961266    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-gbf8j" event={"ID":"d0ec6f78-bdc6-420a-a6aa-953d61643e27","Type":"ContainerStarted","Data":"d5214f49d9a606d0d76f74fa4dd11d4cd7e75b1dee8f288b10eeabdeb6a6a923"}
Sep 11 17:18:28.962750 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:28.962717    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-z559n" event={"ID":"e6840164-41a5-4423-bdd0-a364186cf962","Type":"ContainerStarted","Data":"3214686e3c78389cc2890b79f40a6f0033db6a997a37eaa213c3c9fd518653b0"}
Sep 11 17:18:29.477478 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:29.477190    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/72eae49b-cb97-4819-aaf5-5c6eb23ce9c8-original-pull-secret\") pod \"global-pull-secret-syncer-lx6ht\" (UID: \"72eae49b-cb97-4819-aaf5-5c6eb23ce9c8\") " pod="kube-system/global-pull-secret-syncer-lx6ht"
Sep 11 17:18:29.477478 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:29.477259    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-42k58\" (UniqueName: \"kubernetes.io/projected/99bdc64a-0517-49d6-acf3-4e9fc5ea47fe-kube-api-access-42k58\") pod \"network-check-target-9gb8k\" (UID: \"99bdc64a-0517-49d6-acf3-4e9fc5ea47fe\") " pod="openshift-network-diagnostics/network-check-target-9gb8k"
Sep 11 17:18:29.477478 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:29.477290    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b2b1c077-b5b8-4bdc-a1b3-6b98a0bcf6a0-metrics-certs\") pod \"network-metrics-daemon-d5jqp\" (UID: \"b2b1c077-b5b8-4bdc-a1b3-6b98a0bcf6a0\") " pod="openshift-multus/network-metrics-daemon-d5jqp"
Sep 11 17:18:29.477478 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:18:29.477352    2552 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 17:18:29.477478 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:18:29.477415    2552 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 17:18:29.477478 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:18:29.477429    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/72eae49b-cb97-4819-aaf5-5c6eb23ce9c8-original-pull-secret podName:72eae49b-cb97-4819-aaf5-5c6eb23ce9c8 nodeName:}" failed. No retries permitted until 2026-09-11 17:18:31.477408873 +0000 UTC m=+5.998700389 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/72eae49b-cb97-4819-aaf5-5c6eb23ce9c8-original-pull-secret") pod "global-pull-secret-syncer-lx6ht" (UID: "72eae49b-cb97-4819-aaf5-5c6eb23ce9c8") : object "kube-system"/"original-pull-secret" not registered
Sep 11 17:18:29.477478 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:18:29.477466    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b2b1c077-b5b8-4bdc-a1b3-6b98a0bcf6a0-metrics-certs podName:b2b1c077-b5b8-4bdc-a1b3-6b98a0bcf6a0 nodeName:}" failed. No retries permitted until 2026-09-11 17:18:31.477450901 +0000 UTC m=+5.998742407 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b2b1c077-b5b8-4bdc-a1b3-6b98a0bcf6a0-metrics-certs") pod "network-metrics-daemon-d5jqp" (UID: "b2b1c077-b5b8-4bdc-a1b3-6b98a0bcf6a0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 17:18:29.477956 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:18:29.477545    2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 17:18:29.477956 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:18:29.477558    2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 17:18:29.477956 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:18:29.477571    2552 projected.go:196] Error preparing data for projected volume kube-api-access-42k58 for pod openshift-network-diagnostics/network-check-target-9gb8k: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 17:18:29.477956 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:18:29.477605    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/99bdc64a-0517-49d6-acf3-4e9fc5ea47fe-kube-api-access-42k58 podName:99bdc64a-0517-49d6-acf3-4e9fc5ea47fe nodeName:}" failed. No retries permitted until 2026-09-11 17:18:31.477593217 +0000 UTC m=+5.998884730 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-42k58" (UniqueName: "kubernetes.io/projected/99bdc64a-0517-49d6-acf3-4e9fc5ea47fe-kube-api-access-42k58") pod "network-check-target-9gb8k" (UID: "99bdc64a-0517-49d6-acf3-4e9fc5ea47fe") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 17:18:29.897068 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:29.896945    2552 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-10 17:13:27 +0000 UTC" deadline="2028-05-14 08:09:28.784744682 +0000 UTC"
Sep 11 17:18:29.897068 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:29.896986    2552 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14654h50m58.887763756s"
Sep 11 17:18:29.943045 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:29.943012    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9gb8k"
Sep 11 17:18:29.943244 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:18:29.943155    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9gb8k" podUID="99bdc64a-0517-49d6-acf3-4e9fc5ea47fe"
Sep 11 17:18:29.943578 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:29.943556    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lx6ht"
Sep 11 17:18:29.943685 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:18:29.943665    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lx6ht" podUID="72eae49b-cb97-4819-aaf5-5c6eb23ce9c8"
Sep 11 17:18:29.943860 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:29.943845    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d5jqp"
Sep 11 17:18:29.943962 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:18:29.943943    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d5jqp" podUID="b2b1c077-b5b8-4bdc-a1b3-6b98a0bcf6a0"
Sep 11 17:18:31.495027 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:31.494597    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/72eae49b-cb97-4819-aaf5-5c6eb23ce9c8-original-pull-secret\") pod \"global-pull-secret-syncer-lx6ht\" (UID: \"72eae49b-cb97-4819-aaf5-5c6eb23ce9c8\") " pod="kube-system/global-pull-secret-syncer-lx6ht"
Sep 11 17:18:31.495027 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:31.494672    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-42k58\" (UniqueName: \"kubernetes.io/projected/99bdc64a-0517-49d6-acf3-4e9fc5ea47fe-kube-api-access-42k58\") pod \"network-check-target-9gb8k\" (UID: \"99bdc64a-0517-49d6-acf3-4e9fc5ea47fe\") " pod="openshift-network-diagnostics/network-check-target-9gb8k"
Sep 11 17:18:31.495027 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:31.494702    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b2b1c077-b5b8-4bdc-a1b3-6b98a0bcf6a0-metrics-certs\") pod \"network-metrics-daemon-d5jqp\" (UID: \"b2b1c077-b5b8-4bdc-a1b3-6b98a0bcf6a0\") " pod="openshift-multus/network-metrics-daemon-d5jqp"
Sep 11 17:18:31.495027 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:18:31.494791    2552 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 17:18:31.495027 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:18:31.494827    2552 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 17:18:31.495027 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:18:31.494849    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/72eae49b-cb97-4819-aaf5-5c6eb23ce9c8-original-pull-secret podName:72eae49b-cb97-4819-aaf5-5c6eb23ce9c8 nodeName:}" failed. No retries permitted until 2026-09-11 17:18:35.494830332 +0000 UTC m=+10.016121839 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/72eae49b-cb97-4819-aaf5-5c6eb23ce9c8-original-pull-secret") pod "global-pull-secret-syncer-lx6ht" (UID: "72eae49b-cb97-4819-aaf5-5c6eb23ce9c8") : object "kube-system"/"original-pull-secret" not registered
Sep 11 17:18:31.495027 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:18:31.494867    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b2b1c077-b5b8-4bdc-a1b3-6b98a0bcf6a0-metrics-certs podName:b2b1c077-b5b8-4bdc-a1b3-6b98a0bcf6a0 nodeName:}" failed. No retries permitted until 2026-09-11 17:18:35.494857656 +0000 UTC m=+10.016149160 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b2b1c077-b5b8-4bdc-a1b3-6b98a0bcf6a0-metrics-certs") pod "network-metrics-daemon-d5jqp" (UID: "b2b1c077-b5b8-4bdc-a1b3-6b98a0bcf6a0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 17:18:31.495027 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:18:31.494943    2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 17:18:31.495027 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:18:31.494957    2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 17:18:31.495027 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:18:31.494968    2552 projected.go:196] Error preparing data for projected volume kube-api-access-42k58 for pod openshift-network-diagnostics/network-check-target-9gb8k: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 17:18:31.495027 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:18:31.495002    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/99bdc64a-0517-49d6-acf3-4e9fc5ea47fe-kube-api-access-42k58 podName:99bdc64a-0517-49d6-acf3-4e9fc5ea47fe nodeName:}" failed. No retries permitted until 2026-09-11 17:18:35.494990921 +0000 UTC m=+10.016282433 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-42k58" (UniqueName: "kubernetes.io/projected/99bdc64a-0517-49d6-acf3-4e9fc5ea47fe-kube-api-access-42k58") pod "network-check-target-9gb8k" (UID: "99bdc64a-0517-49d6-acf3-4e9fc5ea47fe") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 17:18:31.944012 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:31.943242    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lx6ht"
Sep 11 17:18:31.944012 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:18:31.943368    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lx6ht" podUID="72eae49b-cb97-4819-aaf5-5c6eb23ce9c8"
Sep 11 17:18:31.944012 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:31.943792    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d5jqp"
Sep 11 17:18:31.944012 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:18:31.943900    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d5jqp" podUID="b2b1c077-b5b8-4bdc-a1b3-6b98a0bcf6a0"
Sep 11 17:18:31.944706 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:31.944466    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9gb8k"
Sep 11 17:18:31.944706 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:18:31.944563    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9gb8k" podUID="99bdc64a-0517-49d6-acf3-4e9fc5ea47fe"
Sep 11 17:18:33.943101 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:33.943069    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lx6ht"
Sep 11 17:18:33.943101 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:33.943081    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9gb8k"
Sep 11 17:18:33.943542 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:33.943081    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d5jqp"
Sep 11 17:18:33.943542 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:18:33.943200    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lx6ht" podUID="72eae49b-cb97-4819-aaf5-5c6eb23ce9c8"
Sep 11 17:18:33.943542 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:18:33.943310    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d5jqp" podUID="b2b1c077-b5b8-4bdc-a1b3-6b98a0bcf6a0"
Sep 11 17:18:33.943542 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:18:33.943458    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9gb8k" podUID="99bdc64a-0517-49d6-acf3-4e9fc5ea47fe"
Sep 11 17:18:35.526781 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:35.526743    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-42k58\" (UniqueName: \"kubernetes.io/projected/99bdc64a-0517-49d6-acf3-4e9fc5ea47fe-kube-api-access-42k58\") pod \"network-check-target-9gb8k\" (UID: \"99bdc64a-0517-49d6-acf3-4e9fc5ea47fe\") " pod="openshift-network-diagnostics/network-check-target-9gb8k"
Sep 11 17:18:35.527239 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:35.526802    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b2b1c077-b5b8-4bdc-a1b3-6b98a0bcf6a0-metrics-certs\") pod \"network-metrics-daemon-d5jqp\" (UID: \"b2b1c077-b5b8-4bdc-a1b3-6b98a0bcf6a0\") " pod="openshift-multus/network-metrics-daemon-d5jqp"
Sep 11 17:18:35.527239 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:35.526858    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/72eae49b-cb97-4819-aaf5-5c6eb23ce9c8-original-pull-secret\") pod \"global-pull-secret-syncer-lx6ht\" (UID: \"72eae49b-cb97-4819-aaf5-5c6eb23ce9c8\") " pod="kube-system/global-pull-secret-syncer-lx6ht"
Sep 11 17:18:35.527239 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:18:35.526922    2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 17:18:35.527239 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:18:35.526944    2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 17:18:35.527239 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:18:35.526957    2552 projected.go:196] Error preparing data for projected volume kube-api-access-42k58 for pod openshift-network-diagnostics/network-check-target-9gb8k: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 17:18:35.527239 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:18:35.526960    2552 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 17:18:35.527239 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:18:35.527014    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/72eae49b-cb97-4819-aaf5-5c6eb23ce9c8-original-pull-secret podName:72eae49b-cb97-4819-aaf5-5c6eb23ce9c8 nodeName:}" failed. No retries permitted until 2026-09-11 17:18:43.526996823 +0000 UTC m=+18.048288324 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/72eae49b-cb97-4819-aaf5-5c6eb23ce9c8-original-pull-secret") pod "global-pull-secret-syncer-lx6ht" (UID: "72eae49b-cb97-4819-aaf5-5c6eb23ce9c8") : object "kube-system"/"original-pull-secret" not registered
Sep 11 17:18:35.527239 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:18:35.527032    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/99bdc64a-0517-49d6-acf3-4e9fc5ea47fe-kube-api-access-42k58 podName:99bdc64a-0517-49d6-acf3-4e9fc5ea47fe nodeName:}" failed. No retries permitted until 2026-09-11 17:18:43.527021773 +0000 UTC m=+18.048313273 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-42k58" (UniqueName: "kubernetes.io/projected/99bdc64a-0517-49d6-acf3-4e9fc5ea47fe-kube-api-access-42k58") pod "network-check-target-9gb8k" (UID: "99bdc64a-0517-49d6-acf3-4e9fc5ea47fe") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 17:18:35.527239 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:18:35.527046    2552 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 17:18:35.527239 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:18:35.527076    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b2b1c077-b5b8-4bdc-a1b3-6b98a0bcf6a0-metrics-certs podName:b2b1c077-b5b8-4bdc-a1b3-6b98a0bcf6a0 nodeName:}" failed. No retries permitted until 2026-09-11 17:18:43.527065947 +0000 UTC m=+18.048357462 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b2b1c077-b5b8-4bdc-a1b3-6b98a0bcf6a0-metrics-certs") pod "network-metrics-daemon-d5jqp" (UID: "b2b1c077-b5b8-4bdc-a1b3-6b98a0bcf6a0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 17:18:35.944087 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:35.944009    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d5jqp"
Sep 11 17:18:35.944252 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:18:35.944158    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d5jqp" podUID="b2b1c077-b5b8-4bdc-a1b3-6b98a0bcf6a0"
Sep 11 17:18:35.944252 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:35.944228    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9gb8k"
Sep 11 17:18:35.944367 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:18:35.944331    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9gb8k" podUID="99bdc64a-0517-49d6-acf3-4e9fc5ea47fe"
Sep 11 17:18:35.944425 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:35.944370    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lx6ht"
Sep 11 17:18:35.944535 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:18:35.944436    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lx6ht" podUID="72eae49b-cb97-4819-aaf5-5c6eb23ce9c8"
Sep 11 17:18:37.943641 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:37.943605    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d5jqp"
Sep 11 17:18:37.944229 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:37.943660    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lx6ht"
Sep 11 17:18:37.944229 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:37.943605    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9gb8k"
Sep 11 17:18:37.944229 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:18:37.943759    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d5jqp" podUID="b2b1c077-b5b8-4bdc-a1b3-6b98a0bcf6a0"
Sep 11 17:18:37.944229 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:18:37.943813    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9gb8k" podUID="99bdc64a-0517-49d6-acf3-4e9fc5ea47fe"
Sep 11 17:18:37.944229 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:18:37.943882    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lx6ht" podUID="72eae49b-cb97-4819-aaf5-5c6eb23ce9c8"
Sep 11 17:18:39.943601 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:39.943561    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d5jqp"
Sep 11 17:18:39.943601 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:39.943594    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lx6ht"
Sep 11 17:18:39.944145 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:39.943561    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9gb8k"
Sep 11 17:18:39.944145 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:18:39.943718    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d5jqp" podUID="b2b1c077-b5b8-4bdc-a1b3-6b98a0bcf6a0"
Sep 11 17:18:39.944145 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:18:39.943856    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9gb8k" podUID="99bdc64a-0517-49d6-acf3-4e9fc5ea47fe"
Sep 11 17:18:39.944145 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:18:39.943926    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lx6ht" podUID="72eae49b-cb97-4819-aaf5-5c6eb23ce9c8"
Sep 11 17:18:41.943473 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:41.943443    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d5jqp"
Sep 11 17:18:41.943898 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:41.943443    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9gb8k"
Sep 11 17:18:41.943898 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:18:41.943560    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d5jqp" podUID="b2b1c077-b5b8-4bdc-a1b3-6b98a0bcf6a0"
Sep 11 17:18:41.943898 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:41.943440    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lx6ht"
Sep 11 17:18:41.943898 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:18:41.943642    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9gb8k" podUID="99bdc64a-0517-49d6-acf3-4e9fc5ea47fe"
Sep 11 17:18:41.943898 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:18:41.943740    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lx6ht" podUID="72eae49b-cb97-4819-aaf5-5c6eb23ce9c8"
Sep 11 17:18:43.589927 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:43.589887    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-42k58\" (UniqueName: \"kubernetes.io/projected/99bdc64a-0517-49d6-acf3-4e9fc5ea47fe-kube-api-access-42k58\") pod \"network-check-target-9gb8k\" (UID: \"99bdc64a-0517-49d6-acf3-4e9fc5ea47fe\") " pod="openshift-network-diagnostics/network-check-target-9gb8k"
Sep 11 17:18:43.589927 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:43.589935    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b2b1c077-b5b8-4bdc-a1b3-6b98a0bcf6a0-metrics-certs\") pod \"network-metrics-daemon-d5jqp\" (UID: \"b2b1c077-b5b8-4bdc-a1b3-6b98a0bcf6a0\") " pod="openshift-multus/network-metrics-daemon-d5jqp"
Sep 11 17:18:43.590478 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:43.589985    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/72eae49b-cb97-4819-aaf5-5c6eb23ce9c8-original-pull-secret\") pod \"global-pull-secret-syncer-lx6ht\" (UID: \"72eae49b-cb97-4819-aaf5-5c6eb23ce9c8\") " pod="kube-system/global-pull-secret-syncer-lx6ht"
Sep 11 17:18:43.590478 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:18:43.590069    2552 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 17:18:43.590478 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:18:43.590126    2552 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 17:18:43.590478 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:18:43.590072    2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 17:18:43.590478 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:18:43.590203    2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 17:18:43.590478 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:18:43.590214    2552 projected.go:196] Error preparing data for projected volume kube-api-access-42k58 for pod openshift-network-diagnostics/network-check-target-9gb8k: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 17:18:43.590478 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:18:43.590143    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/72eae49b-cb97-4819-aaf5-5c6eb23ce9c8-original-pull-secret podName:72eae49b-cb97-4819-aaf5-5c6eb23ce9c8 nodeName:}" failed. No retries permitted until 2026-09-11 17:18:59.590126072 +0000 UTC m=+34.111417589 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/72eae49b-cb97-4819-aaf5-5c6eb23ce9c8-original-pull-secret") pod "global-pull-secret-syncer-lx6ht" (UID: "72eae49b-cb97-4819-aaf5-5c6eb23ce9c8") : object "kube-system"/"original-pull-secret" not registered
Sep 11 17:18:43.590478 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:18:43.590258    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b2b1c077-b5b8-4bdc-a1b3-6b98a0bcf6a0-metrics-certs podName:b2b1c077-b5b8-4bdc-a1b3-6b98a0bcf6a0 nodeName:}" failed. No retries permitted until 2026-09-11 17:18:59.590243938 +0000 UTC m=+34.111535446 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b2b1c077-b5b8-4bdc-a1b3-6b98a0bcf6a0-metrics-certs") pod "network-metrics-daemon-d5jqp" (UID: "b2b1c077-b5b8-4bdc-a1b3-6b98a0bcf6a0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 17:18:43.590478 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:18:43.590278    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/99bdc64a-0517-49d6-acf3-4e9fc5ea47fe-kube-api-access-42k58 podName:99bdc64a-0517-49d6-acf3-4e9fc5ea47fe nodeName:}" failed. No retries permitted until 2026-09-11 17:18:59.590269846 +0000 UTC m=+34.111561345 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-42k58" (UniqueName: "kubernetes.io/projected/99bdc64a-0517-49d6-acf3-4e9fc5ea47fe-kube-api-access-42k58") pod "network-check-target-9gb8k" (UID: "99bdc64a-0517-49d6-acf3-4e9fc5ea47fe") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 17:18:43.942885 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:43.942793    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9gb8k"
Sep 11 17:18:43.942885 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:43.942831    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lx6ht"
Sep 11 17:18:43.943104 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:43.942794    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d5jqp"
Sep 11 17:18:43.943104 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:18:43.942926    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9gb8k" podUID="99bdc64a-0517-49d6-acf3-4e9fc5ea47fe"
Sep 11 17:18:43.943104 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:18:43.942988    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d5jqp" podUID="b2b1c077-b5b8-4bdc-a1b3-6b98a0bcf6a0"
Sep 11 17:18:43.943104 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:18:43.943067    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lx6ht" podUID="72eae49b-cb97-4819-aaf5-5c6eb23ce9c8"
Sep 11 17:18:45.946471 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:45.944585    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d5jqp"
Sep 11 17:18:45.946471 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:18:45.944749    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d5jqp" podUID="b2b1c077-b5b8-4bdc-a1b3-6b98a0bcf6a0"
Sep 11 17:18:45.946471 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:45.945245    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9gb8k"
Sep 11 17:18:45.946471 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:18:45.945356    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9gb8k" podUID="99bdc64a-0517-49d6-acf3-4e9fc5ea47fe"
Sep 11 17:18:45.946471 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:45.945418    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lx6ht"
Sep 11 17:18:45.946471 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:18:45.945527    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lx6ht" podUID="72eae49b-cb97-4819-aaf5-5c6eb23ce9c8"
Sep 11 17:18:47.006995 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:47.006706    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-gbf8j" event={"ID":"d0ec6f78-bdc6-420a-a6aa-953d61643e27","Type":"ContainerStarted","Data":"f0f4bab957c86b0395297b5b715fbe4ea5d8146f11e17260bdce745c7c6ddf82"}
Sep 11 17:18:47.008824 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:47.008798    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-z559n" event={"ID":"e6840164-41a5-4423-bdd0-a364186cf962","Type":"ContainerStarted","Data":"62f89432e1242628f737aa6f3a650a51a7e0fc1dd0ef9b1e2f408923e4407ead"}
Sep 11 17:18:47.011127 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:47.011077    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-h6lht" event={"ID":"8b4c1f98-046f-47c4-889f-c2bd807dddbe","Type":"ContainerStarted","Data":"72288ef9f185b14bc5640b9321ea6ac006f6a40c0addc832155f71aee8719280"}
Sep 11 17:18:47.012984 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:47.012956    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-8j6jf" event={"ID":"5b082ce5-2ea9-46dc-8307-08a722392886","Type":"ContainerStarted","Data":"0aba5baa5f55d32529dfb7ac617eeef9d41b0e3384b25222e00dc4cc79edf4b1"}
Sep 11 17:18:47.014799 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:47.014777    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-22x74" event={"ID":"b5a337f0-d706-42c3-b846-1997efc3d693","Type":"ContainerStarted","Data":"629026522b6d09d8ba64fdc03ad117302338738aecbc09e4b76108d12ea800f8"}
Sep 11 17:18:47.016347 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:47.016315    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-cph4q" event={"ID":"26e09059-8abc-4e73-bf43-23a51698a0cf","Type":"ContainerStarted","Data":"86024e446fdf2097e52011069433abb873bc6d3b6d31e303b1dbe5e6175bf886"}
Sep 11 17:18:47.017800 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:47.017775    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-51.ec2.internal" event={"ID":"144be191d43f74dcac73bb27936b49ab","Type":"ContainerStarted","Data":"4672ba0ef2ed959949f6e46ddbcdf2b60eb667a0235add359ad497a1ffe41694"}
Sep 11 17:18:47.019580 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:47.019555    2552 generic.go:356] "Generic (PLEG): container finished" podID="2283d44ad0e3ab74ce157fde8b170250" containerID="f3e5b1ccfdfcf4497c218f25ce552278138218d34432a004bdfb1df71ffd5d6a" exitCode=0
Sep 11 17:18:47.019691 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:47.019638    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-51.ec2.internal" event={"ID":"2283d44ad0e3ab74ce157fde8b170250","Type":"ContainerDied","Data":"f3e5b1ccfdfcf4497c218f25ce552278138218d34432a004bdfb1df71ffd5d6a"}
Sep 11 17:18:47.023185 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:47.023167    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6k9tr_0620c0a0-8070-4307-86d9-c69a76ea14a9/ovn-acl-logging/0.log"
Sep 11 17:18:47.023556 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:47.023532    2552 generic.go:356] "Generic (PLEG): container finished" podID="0620c0a0-8070-4307-86d9-c69a76ea14a9" containerID="6d3d35f946cea2fffc3cfea9981cff5eaddd82888cb1b444bac83725e4627019" exitCode=1
Sep 11 17:18:47.023625 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:47.023596    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6k9tr" event={"ID":"0620c0a0-8070-4307-86d9-c69a76ea14a9","Type":"ContainerStarted","Data":"d8ec0e39017c549cc2827506a8a95e08faf7143509376dd0772f1826f3fdb733"}
Sep 11 17:18:47.023691 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:47.023677    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6k9tr" event={"ID":"0620c0a0-8070-4307-86d9-c69a76ea14a9","Type":"ContainerStarted","Data":"7a16cb6590faeaf00545b6edd5debc74710bd9910a9e8cac32dcb743d7b0fd05"}
Sep 11 17:18:47.023738 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:47.023731    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6k9tr" event={"ID":"0620c0a0-8070-4307-86d9-c69a76ea14a9","Type":"ContainerStarted","Data":"23fd24481f292944c1cfaeac86ab20d7096669f47878d75d466659a766624d90"}
Sep 11 17:18:47.023784 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:47.023773    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6k9tr" event={"ID":"0620c0a0-8070-4307-86d9-c69a76ea14a9","Type":"ContainerStarted","Data":"2dda4770cd60e4540c24bd2ac110f054a02db7b9b3fd62c6a54486d726b78fcf"}
Sep 11 17:18:47.023825 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:47.023819    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6k9tr" event={"ID":"0620c0a0-8070-4307-86d9-c69a76ea14a9","Type":"ContainerDied","Data":"6d3d35f946cea2fffc3cfea9981cff5eaddd82888cb1b444bac83725e4627019"}
Sep 11 17:18:47.023876 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:47.023862    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6k9tr" event={"ID":"0620c0a0-8070-4307-86d9-c69a76ea14a9","Type":"ContainerStarted","Data":"c29f73821988eacb60d2c6e9314366d495f4ab69d1932b1a41ad5307afb2a01a"}
Sep 11 17:18:47.025526 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:47.025501    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-q586m" event={"ID":"41755610-5307-49d4-af8e-a021e026aa13","Type":"ContainerStarted","Data":"519a527632164a8f442cafeb471a5cdaf7b872322fe72f127d1dcc47edceca30"}
Sep 11 17:18:47.025850 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:47.025814    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-gbf8j" podStartSLOduration=3.000226348 podStartE2EDuration="21.025801931s" totalImagesPullingTime="18.025575583s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 17:18:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:18:47.024878854 +0000 UTC m=+21.546170375" watchObservedRunningTime="2026-09-11 17:18:47.025801931 +0000 UTC m=+21.547093452"
Sep 11 17:18:47.058390 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:47.058345    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-8j6jf" podStartSLOduration=7.06779741 podStartE2EDuration="21.058331752s" totalImagesPullingTime="13.990534342s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:18:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:18:47.058177866 +0000 UTC m=+21.579469388" watchObservedRunningTime="2026-09-11 17:18:47.058331752 +0000 UTC m=+21.579623272"
Sep 11 17:18:47.083017 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:47.082961    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-h6lht" podStartSLOduration=3.065947134 podStartE2EDuration="21.082944469s" totalImagesPullingTime="18.016997335s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 17:18:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:18:47.082490806 +0000 UTC m=+21.603782320" watchObservedRunningTime="2026-09-11 17:18:47.082944469 +0000 UTC m=+21.604235990"
Sep 11 17:18:47.106548 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:47.106505    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-51.ec2.internal" podStartSLOduration=20.106489679 podStartE2EDuration="20.106489679s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 17:18:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:18:47.10609747 +0000 UTC m=+21.627389003" watchObservedRunningTime="2026-09-11 17:18:47.106489679 +0000 UTC m=+21.627781200"
Sep 11 17:18:47.132650 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:47.132592    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-cph4q" podStartSLOduration=3.069879548 podStartE2EDuration="21.132571971s" totalImagesPullingTime="18.062692423s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 17:18:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:18:47.131751266 +0000 UTC m=+21.653042786" watchObservedRunningTime="2026-09-11 17:18:47.132571971 +0000 UTC m=+21.653863493"
Sep 11 17:18:47.209031 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:47.208977    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-q586m" podStartSLOduration=2.784268504 podStartE2EDuration="21.208957079s" totalImagesPullingTime="18.424688575s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 17:18:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:18:47.208924715 +0000 UTC m=+21.730216236" watchObservedRunningTime="2026-09-11 17:18:47.208957079 +0000 UTC m=+21.730248601"
Sep 11 17:18:47.657959 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:47.657925    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-8j6jf"
Sep 11 17:18:47.943532 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:47.943447    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lx6ht"
Sep 11 17:18:47.943796 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:47.943447    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d5jqp"
Sep 11 17:18:47.943796 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:18:47.943574    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lx6ht" podUID="72eae49b-cb97-4819-aaf5-5c6eb23ce9c8"
Sep 11 17:18:47.943796 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:47.943447    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9gb8k"
Sep 11 17:18:47.943796 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:18:47.943635    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d5jqp" podUID="b2b1c077-b5b8-4bdc-a1b3-6b98a0bcf6a0"
Sep 11 17:18:47.943796 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:18:47.943687    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9gb8k" podUID="99bdc64a-0517-49d6-acf3-4e9fc5ea47fe"
Sep 11 17:18:48.030832 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:48.030794    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-51.ec2.internal" event={"ID":"2283d44ad0e3ab74ce157fde8b170250","Type":"ContainerStarted","Data":"9306cdbbc1eabc23740c6f14c169b57af6427adf4c389e1a3349510c6e8f557a"}
Sep 11 17:18:48.032380 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:48.032357    2552 generic.go:356] "Generic (PLEG): container finished" podID="e6840164-41a5-4423-bdd0-a364186cf962" containerID="62f89432e1242628f737aa6f3a650a51a7e0fc1dd0ef9b1e2f408923e4407ead" exitCode=0
Sep 11 17:18:48.032479 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:48.032425    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-z559n" event={"ID":"e6840164-41a5-4423-bdd0-a364186cf962","Type":"ContainerDied","Data":"62f89432e1242628f737aa6f3a650a51a7e0fc1dd0ef9b1e2f408923e4407ead"}
Sep 11 17:18:48.033941 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:48.033915    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-fptk9" event={"ID":"181608ca-8121-4bf3-8d50-a407708b420a","Type":"ContainerStarted","Data":"8fe480eb4205846ea4989aec451260d888726731782f16102b565311fd0033ac"}
Sep 11 17:18:48.048071 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:48.048031    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-51.ec2.internal" podStartSLOduration=21.04802017 podStartE2EDuration="21.04802017s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 17:18:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:18:48.047806984 +0000 UTC m=+22.569098507" watchObservedRunningTime="2026-09-11 17:18:48.04802017 +0000 UTC m=+22.569311690"
Sep 11 17:18:48.066431 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:48.066331    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-fptk9" podStartSLOduration=4.066001926 podStartE2EDuration="22.066316844s" totalImagesPullingTime="18.000314918s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 17:18:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:18:48.065828271 +0000 UTC m=+22.587119791" watchObservedRunningTime="2026-09-11 17:18:48.066316844 +0000 UTC m=+22.587608364"
Sep 11 17:18:48.615570 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:48.615549    2552 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 11 17:18:48.901576 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:48.901421    2552 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-11T17:18:48.615565963Z","UUID":"db36e02e-1c97-4725-bbbd-df3785dc59f3","Handler":null,"Name":"","Endpoint":""}
Sep 11 17:18:48.903101 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:48.903068    2552 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 11 17:18:48.903233 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:48.903124    2552 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 11 17:18:49.039622 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:49.039554    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-22x74" event={"ID":"b5a337f0-d706-42c3-b846-1997efc3d693","Type":"ContainerStarted","Data":"835dfca1ec73a055d3d8903dd066c66d7e9732920caf8f6bf846761f156d763c"}
Sep 11 17:18:49.943541 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:49.943307    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lx6ht"
Sep 11 17:18:49.943828 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:49.943327    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9gb8k"
Sep 11 17:18:49.943828 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:18:49.943650    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lx6ht" podUID="72eae49b-cb97-4819-aaf5-5c6eb23ce9c8"
Sep 11 17:18:49.943828 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:18:49.943721    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9gb8k" podUID="99bdc64a-0517-49d6-acf3-4e9fc5ea47fe"
Sep 11 17:18:49.943828 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:49.943340    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d5jqp"
Sep 11 17:18:49.944041 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:18:49.943831    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d5jqp" podUID="b2b1c077-b5b8-4bdc-a1b3-6b98a0bcf6a0"
Sep 11 17:18:50.049146 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:50.046911    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6k9tr_0620c0a0-8070-4307-86d9-c69a76ea14a9/ovn-acl-logging/0.log"
Sep 11 17:18:50.049986 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:50.049937    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6k9tr" event={"ID":"0620c0a0-8070-4307-86d9-c69a76ea14a9","Type":"ContainerStarted","Data":"b2e16667ca437e9c97d9cf09cfab89bdf8dcbcd82d00f8dff3867f75237ed65c"}
Sep 11 17:18:51.053517 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:51.053480    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-22x74" event={"ID":"b5a337f0-d706-42c3-b846-1997efc3d693","Type":"ContainerStarted","Data":"f349626d1359ad47e91409e42dd6e4f1a425493bf667fc32a89c078069721c91"}
Sep 11 17:18:51.072068 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:51.072028    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-22x74" podStartSLOduration=3.917117829 podStartE2EDuration="25.072015934s" totalImagesPullingTime="21.154898105s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 17:18:26 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:18:51.071538568 +0000 UTC m=+25.592830089" watchObservedRunningTime="2026-09-11 17:18:51.072015934 +0000 UTC m=+25.593307456"
Sep 11 17:18:51.818711 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:51.818525    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-8j6jf"
Sep 11 17:18:51.819178 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:51.819157    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-8j6jf"
Sep 11 17:18:51.942963 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:51.942931    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lx6ht"
Sep 11 17:18:51.943142 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:51.942931    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d5jqp"
Sep 11 17:18:51.943142 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:18:51.943070    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lx6ht" podUID="72eae49b-cb97-4819-aaf5-5c6eb23ce9c8"
Sep 11 17:18:51.943257 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:51.943142    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9gb8k"
Sep 11 17:18:51.943315 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:18:51.943290    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d5jqp" podUID="b2b1c077-b5b8-4bdc-a1b3-6b98a0bcf6a0"
Sep 11 17:18:51.943447 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:18:51.943420    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9gb8k" podUID="99bdc64a-0517-49d6-acf3-4e9fc5ea47fe"
Sep 11 17:18:52.058911 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:52.058884    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6k9tr_0620c0a0-8070-4307-86d9-c69a76ea14a9/ovn-acl-logging/0.log"
Sep 11 17:18:52.059367 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:52.059287    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6k9tr" event={"ID":"0620c0a0-8070-4307-86d9-c69a76ea14a9","Type":"ContainerStarted","Data":"5b0e6f348835372cbb73824b24ba0182d738e706e84987a8c5d66494e3fc65c3"}
Sep 11 17:18:52.059949 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:52.059926    2552 scope.go:117] "RemoveContainer" containerID="6d3d35f946cea2fffc3cfea9981cff5eaddd82888cb1b444bac83725e4627019"
Sep 11 17:18:52.060268 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:52.060252    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-8j6jf"
Sep 11 17:18:53.066885 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:53.066778    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-lx6ht"]
Sep 11 17:18:53.067597 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:53.066928    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lx6ht"
Sep 11 17:18:53.067597 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:18:53.067027    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lx6ht" podUID="72eae49b-cb97-4819-aaf5-5c6eb23ce9c8"
Sep 11 17:18:53.067689 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:53.067613    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6k9tr_0620c0a0-8070-4307-86d9-c69a76ea14a9/ovn-acl-logging/0.log"
Sep 11 17:18:53.068139 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:53.068091    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6k9tr" event={"ID":"0620c0a0-8070-4307-86d9-c69a76ea14a9","Type":"ContainerStarted","Data":"1d3fe7a34a42d0bbc7142bd7b0c566f024b88059eef0899fd9d376c7d6bf447c"}
Sep 11 17:18:53.068663 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:53.068160    2552 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 11 17:18:53.068663 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:53.068543    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-6k9tr"
Sep 11 17:18:53.068663 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:53.068611    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-6k9tr"
Sep 11 17:18:53.072503 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:53.072474    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-9gb8k"]
Sep 11 17:18:53.072619 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:53.072596    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9gb8k"
Sep 11 17:18:53.074055 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:18:53.072699    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9gb8k" podUID="99bdc64a-0517-49d6-acf3-4e9fc5ea47fe"
Sep 11 17:18:53.074055 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:53.073279    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-d5jqp"]
Sep 11 17:18:53.074055 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:53.073371    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d5jqp"
Sep 11 17:18:53.074055 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:18:53.073462    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d5jqp" podUID="b2b1c077-b5b8-4bdc-a1b3-6b98a0bcf6a0"
Sep 11 17:18:53.086882 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:53.086854    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-6k9tr"
Sep 11 17:18:53.087618 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:53.087596    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-6k9tr"
Sep 11 17:18:53.104495 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:53.104446    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-6k9tr" podStartSLOduration=9.072026956 podStartE2EDuration="27.10442867s" totalImagesPullingTime="18.032401714s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 17:18:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:18:53.103167507 +0000 UTC m=+27.624459029" watchObservedRunningTime="2026-09-11 17:18:53.10442867 +0000 UTC m=+27.625720192"
Sep 11 17:18:54.070981 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:54.070953    2552 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 11 17:18:54.943079 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:54.943040    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lx6ht"
Sep 11 17:18:54.943256 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:54.943087    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d5jqp"
Sep 11 17:18:54.943256 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:54.943161    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9gb8k"
Sep 11 17:18:54.943256 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:18:54.943176    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lx6ht" podUID="72eae49b-cb97-4819-aaf5-5c6eb23ce9c8"
Sep 11 17:18:54.943361 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:18:54.943264    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9gb8k" podUID="99bdc64a-0517-49d6-acf3-4e9fc5ea47fe"
Sep 11 17:18:54.943361 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:18:54.943343    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d5jqp" podUID="b2b1c077-b5b8-4bdc-a1b3-6b98a0bcf6a0"
Sep 11 17:18:55.075007 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:55.074926    2552 generic.go:356] "Generic (PLEG): container finished" podID="e6840164-41a5-4423-bdd0-a364186cf962" containerID="991bdfeecf02d019a5c1a3dcedd0d54f418500a84ee4e22870d98eee9dcc5691" exitCode=0
Sep 11 17:18:55.075581 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:55.075004    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-z559n" event={"ID":"e6840164-41a5-4423-bdd0-a364186cf962","Type":"ContainerDied","Data":"991bdfeecf02d019a5c1a3dcedd0d54f418500a84ee4e22870d98eee9dcc5691"}
Sep 11 17:18:55.075581 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:55.075210    2552 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 11 17:18:56.492149 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:56.491946    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-6k9tr"
Sep 11 17:18:56.492551 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:56.492310    2552 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 11 17:18:56.501743 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:56.501708    2552 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ovn-kubernetes/ovnkube-node-6k9tr" podUID="0620c0a0-8070-4307-86d9-c69a76ea14a9" containerName="ovnkube-controller" probeResult="failure" output=""
Sep 11 17:18:56.509910 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:56.509879    2552 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ovn-kubernetes/ovnkube-node-6k9tr" podUID="0620c0a0-8070-4307-86d9-c69a76ea14a9" containerName="ovnkube-controller" probeResult="failure" output=""
Sep 11 17:18:56.943407 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:56.943377    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9gb8k"
Sep 11 17:18:56.943576 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:18:56.943467    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9gb8k" podUID="99bdc64a-0517-49d6-acf3-4e9fc5ea47fe"
Sep 11 17:18:56.943576 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:56.943537    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lx6ht"
Sep 11 17:18:56.943658 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:18:56.943615    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lx6ht" podUID="72eae49b-cb97-4819-aaf5-5c6eb23ce9c8"
Sep 11 17:18:56.943658 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:56.943650    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d5jqp"
Sep 11 17:18:56.943735 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:18:56.943711    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d5jqp" podUID="b2b1c077-b5b8-4bdc-a1b3-6b98a0bcf6a0"
Sep 11 17:18:57.081329 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:57.081298    2552 generic.go:356] "Generic (PLEG): container finished" podID="e6840164-41a5-4423-bdd0-a364186cf962" containerID="8910cfbd48a9a42aa26e7caf25b36ba07ac3a73b30bb1a0049b428a77a9dd168" exitCode=0
Sep 11 17:18:57.081494 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:57.081347    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-z559n" event={"ID":"e6840164-41a5-4423-bdd0-a364186cf962","Type":"ContainerDied","Data":"8910cfbd48a9a42aa26e7caf25b36ba07ac3a73b30bb1a0049b428a77a9dd168"}
Sep 11 17:18:58.943494 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:58.943457    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9gb8k"
Sep 11 17:18:58.943999 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:58.943514    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d5jqp"
Sep 11 17:18:58.943999 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:58.943474    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lx6ht"
Sep 11 17:18:58.943999 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:18:58.943586    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9gb8k" podUID="99bdc64a-0517-49d6-acf3-4e9fc5ea47fe"
Sep 11 17:18:58.943999 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:18:58.943638    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lx6ht" podUID="72eae49b-cb97-4819-aaf5-5c6eb23ce9c8"
Sep 11 17:18:58.943999 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:18:58.943717    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d5jqp" podUID="b2b1c077-b5b8-4bdc-a1b3-6b98a0bcf6a0"
Sep 11 17:18:59.087672 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.087640    2552 generic.go:356] "Generic (PLEG): container finished" podID="e6840164-41a5-4423-bdd0-a364186cf962" containerID="f6b72e1fff71a2715fa5d31703db6527465d5fe8aea94b0e6a096f3bc81b0936" exitCode=0
Sep 11 17:18:59.087816 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.087687    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-z559n" event={"ID":"e6840164-41a5-4423-bdd0-a364186cf962","Type":"ContainerDied","Data":"f6b72e1fff71a2715fa5d31703db6527465d5fe8aea94b0e6a096f3bc81b0936"}
Sep 11 17:18:59.272621 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.272538    2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-51.ec2.internal" event="NodeReady"
Sep 11 17:18:59.272781 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.272685    2552 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 11 17:18:59.309048 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.309016    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-86bf889f8b-5hnlq"]
Sep 11 17:18:59.310939 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.310924    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-86bf889f8b-5hnlq"
Sep 11 17:18:59.313090 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.313065    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 11 17:18:59.313406 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.313389    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-w4wtw\""
Sep 11 17:18:59.313464 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.313391    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 11 17:18:59.313497 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.313464    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 11 17:18:59.321952 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.321932    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-5sknr"]
Sep 11 17:18:59.324405 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.322560    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 11 17:18:59.324673 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.324401    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5844b856db-88bn9"]
Sep 11 17:18:59.325023 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.324770    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-5sknr"
Sep 11 17:18:59.326463 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.326447    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5844b856db-88bn9"
Sep 11 17:18:59.326694 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.326655    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-6f8f7958c8-2p4xx"]
Sep 11 17:18:59.328277 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.328261    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-6f8f7958c8-2p4xx"
Sep 11 17:18:59.329825 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.329806    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 11 17:18:59.329893 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.329838    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Sep 11 17:18:59.329935 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.329896    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Sep 11 17:18:59.330093 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.330075    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Sep 11 17:18:59.330377 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.330359    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-cdzt2\""
Sep 11 17:18:59.330488 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.330376    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Sep 11 17:18:59.330659 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.330640    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Sep 11 17:18:59.330731 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.330687    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 11 17:18:59.330731 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.330711    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Sep 11 17:18:59.330854 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.330736    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-9m48j\""
Sep 11 17:18:59.331012 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.330997    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Sep 11 17:18:59.333852 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.333815    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-8qq6g\""
Sep 11 17:18:59.333954 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.333849    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 11 17:18:59.333954 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.333820    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 11 17:18:59.333954 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.333927    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Sep 11 17:18:59.334688 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.334669    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-86bf889f8b-5hnlq"]
Sep 11 17:18:59.337724 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.337706    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Sep 11 17:18:59.342316 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.342296    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-5sknr"]
Sep 11 17:18:59.345185 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.344986    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5844b856db-88bn9"]
Sep 11 17:18:59.348163 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.348131    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-6f8f7958c8-2p4xx"]
Sep 11 17:18:59.351160 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.351139    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-6h5ct"]
Sep 11 17:18:59.353174 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.353157    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-6h5ct"
Sep 11 17:18:59.355098 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.355080    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cd954b897-vtd7t"]
Sep 11 17:18:59.356668 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.356650    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-pmvkp"]
Sep 11 17:18:59.356762 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.356750    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-vtd7t"
Sep 11 17:18:59.356887 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.356872    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 11 17:18:59.357647 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.357632    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-bj8s9\""
Sep 11 17:18:59.357647 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.357642    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 11 17:18:59.358759 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.358740    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-5zr8w\""
Sep 11 17:18:59.359410 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.359189    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-q2zxh"]
Sep 11 17:18:59.360212 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.359192    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-pmvkp"
Sep 11 17:18:59.361831 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.361811    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-696884fb9c-hg7p8"]
Sep 11 17:18:59.361987 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.361964    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-q2zxh"
Sep 11 17:18:59.362102 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.362085    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Sep 11 17:18:59.362225 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.362208    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 11 17:18:59.363030 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.363011    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Sep 11 17:18:59.363137 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.363035    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Sep 11 17:18:59.363137 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.363015    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 11 17:18:59.363258 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.363149    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Sep 11 17:18:59.363258 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.363199    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Sep 11 17:18:59.363369 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.363289    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Sep 11 17:18:59.363369 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.363332    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-tp2h8\""
Sep 11 17:18:59.365393 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.365370    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-cf7c776b7-gdzsp"]
Sep 11 17:18:59.367072 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.367054    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-696884fb9c-hg7p8"
Sep 11 17:18:59.367952 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.367868    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Sep 11 17:18:59.367952 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.367894    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-nzjpd"]
Sep 11 17:18:59.367952 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.367927    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Sep 11 17:18:59.368159 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.367958    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-gdzsp"
Sep 11 17:18:59.368159 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.368024    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-xtqnr\""
Sep 11 17:18:59.368159 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.368076    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Sep 11 17:18:59.370327 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.370169    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-5dcbc7df6b-7kvsj"]
Sep 11 17:18:59.370327 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.370187    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Sep 11 17:18:59.370483 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.370379    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Sep 11 17:18:59.370596 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.370574    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Sep 11 17:18:59.370734 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.370715    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-ds666\""
Sep 11 17:18:59.370872 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.370852    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-nzjpd"
Sep 11 17:18:59.370941 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.370870    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Sep 11 17:18:59.371166 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.371146    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-hnlgn\""
Sep 11 17:18:59.371390 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.371365    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Sep 11 17:18:59.371653 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.371636    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Sep 11 17:18:59.371743 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.371643    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Sep 11 17:18:59.371933 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.371917    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Sep 11 17:18:59.372599 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.372578    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-6f8b768455-ltwfd"]
Sep 11 17:18:59.372918 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.372892    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-7kvsj"
Sep 11 17:18:59.374087 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.374064    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 11 17:18:59.375260 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.375148    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 11 17:18:59.375369 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.375272    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-6h5ct"]
Sep 11 17:18:59.375369 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.375363    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f8b768455-ltwfd"
Sep 11 17:18:59.375542 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.375519    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-wkt9z\""
Sep 11 17:18:59.376600 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.376580    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cd954b897-vtd7t"]
Sep 11 17:18:59.377403 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.377384    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Sep 11 17:18:59.378202 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.378183    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 11 17:18:59.380251 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.380232    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Sep 11 17:18:59.381532 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.381514    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Sep 11 17:18:59.383853 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.382784    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-ft2vj\""
Sep 11 17:18:59.383853 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.383186    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 11 17:18:59.383853 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.383321    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-q2zxh"]
Sep 11 17:18:59.383853 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.383773    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 11 17:18:59.384099 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.384018    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-mrznv\""
Sep 11 17:18:59.384531 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.384515    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-696884fb9c-hg7p8"]
Sep 11 17:18:59.385277 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.385242    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-5dcbc7df6b-7kvsj"]
Sep 11 17:18:59.385910 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.385892    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-cf7c776b7-gdzsp"]
Sep 11 17:18:59.386890 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.386858    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f8b768455-ltwfd"]
Sep 11 17:18:59.393218 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.393196    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-pmvkp"]
Sep 11 17:18:59.399458 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.399418    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-nzjpd"]
Sep 11 17:18:59.404529 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.404512    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/65d6125e-1089-4308-bee8-064eefd92dec-installation-pull-secrets\") pod \"image-registry-86bf889f8b-5hnlq\" (UID: \"65d6125e-1089-4308-bee8-064eefd92dec\") " pod="openshift-image-registry/image-registry-86bf889f8b-5hnlq"
Sep 11 17:18:59.404618 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.404536    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7583d484-6dd5-48e3-9485-05e57c62443e-tmp\") pod \"insights-operator-5844b856db-88bn9\" (UID: \"7583d484-6dd5-48e3-9485-05e57c62443e\") " pod="openshift-insights/insights-operator-5844b856db-88bn9"
Sep 11 17:18:59.404618 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.404552    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-brqvg\" (UniqueName: \"kubernetes.io/projected/65d6125e-1089-4308-bee8-064eefd92dec-kube-api-access-brqvg\") pod \"image-registry-86bf889f8b-5hnlq\" (UID: \"65d6125e-1089-4308-bee8-064eefd92dec\") " pod="openshift-image-registry/image-registry-86bf889f8b-5hnlq"
Sep 11 17:18:59.404618 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.404574    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7583d484-6dd5-48e3-9485-05e57c62443e-serving-cert\") pod \"insights-operator-5844b856db-88bn9\" (UID: \"7583d484-6dd5-48e3-9485-05e57c62443e\") " pod="openshift-insights/insights-operator-5844b856db-88bn9"
Sep 11 17:18:59.404618 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.404597    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/2a34a4aa-8cb7-42f3-9f87-da690cfd8031-default-certificate\") pod \"router-default-6f8f7958c8-2p4xx\" (UID: \"2a34a4aa-8cb7-42f3-9f87-da690cfd8031\") " pod="openshift-ingress/router-default-6f8f7958c8-2p4xx"
Sep 11 17:18:59.404742 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.404649    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/65d6125e-1089-4308-bee8-064eefd92dec-registry-tls\") pod \"image-registry-86bf889f8b-5hnlq\" (UID: \"65d6125e-1089-4308-bee8-064eefd92dec\") " pod="openshift-image-registry/image-registry-86bf889f8b-5hnlq"
Sep 11 17:18:59.404742 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.404669    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/65d6125e-1089-4308-bee8-064eefd92dec-registry-certificates\") pod \"image-registry-86bf889f8b-5hnlq\" (UID: \"65d6125e-1089-4308-bee8-064eefd92dec\") " pod="openshift-image-registry/image-registry-86bf889f8b-5hnlq"
Sep 11 17:18:59.404742 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.404697    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/65d6125e-1089-4308-bee8-064eefd92dec-trusted-ca\") pod \"image-registry-86bf889f8b-5hnlq\" (UID: \"65d6125e-1089-4308-bee8-064eefd92dec\") " pod="openshift-image-registry/image-registry-86bf889f8b-5hnlq"
Sep 11 17:18:59.404742 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.404717    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/2a34a4aa-8cb7-42f3-9f87-da690cfd8031-stats-auth\") pod \"router-default-6f8f7958c8-2p4xx\" (UID: \"2a34a4aa-8cb7-42f3-9f87-da690cfd8031\") " pod="openshift-ingress/router-default-6f8f7958c8-2p4xx"
Sep 11 17:18:59.404853 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.404743    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/65d6125e-1089-4308-bee8-064eefd92dec-image-registry-private-configuration\") pod \"image-registry-86bf889f8b-5hnlq\" (UID: \"65d6125e-1089-4308-bee8-064eefd92dec\") " pod="openshift-image-registry/image-registry-86bf889f8b-5hnlq"
Sep 11 17:18:59.404853 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.404761    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2a34a4aa-8cb7-42f3-9f87-da690cfd8031-metrics-certs\") pod \"router-default-6f8f7958c8-2p4xx\" (UID: \"2a34a4aa-8cb7-42f3-9f87-da690cfd8031\") " pod="openshift-ingress/router-default-6f8f7958c8-2p4xx"
Sep 11 17:18:59.404853 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.404784    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7gql9\" (UniqueName: \"kubernetes.io/projected/2a34a4aa-8cb7-42f3-9f87-da690cfd8031-kube-api-access-7gql9\") pod \"router-default-6f8f7958c8-2p4xx\" (UID: \"2a34a4aa-8cb7-42f3-9f87-da690cfd8031\") " pod="openshift-ingress/router-default-6f8f7958c8-2p4xx"
Sep 11 17:18:59.404853 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.404817    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7583d484-6dd5-48e3-9485-05e57c62443e-trusted-ca-bundle\") pod \"insights-operator-5844b856db-88bn9\" (UID: \"7583d484-6dd5-48e3-9485-05e57c62443e\") " pod="openshift-insights/insights-operator-5844b856db-88bn9"
Sep 11 17:18:59.404853 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.404832    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zwn8r\" (UniqueName: \"kubernetes.io/projected/3d5f2687-9edd-426d-bb9d-3c7221a5aeb1-kube-api-access-zwn8r\") pod \"volume-data-source-validator-86695b7cc7-5sknr\" (UID: \"3d5f2687-9edd-426d-bb9d-3c7221a5aeb1\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-5sknr"
Sep 11 17:18:59.404853 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.404848    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/7583d484-6dd5-48e3-9485-05e57c62443e-snapshots\") pod \"insights-operator-5844b856db-88bn9\" (UID: \"7583d484-6dd5-48e3-9485-05e57c62443e\") " pod="openshift-insights/insights-operator-5844b856db-88bn9"
Sep 11 17:18:59.405029 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.404863    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7583d484-6dd5-48e3-9485-05e57c62443e-service-ca-bundle\") pod \"insights-operator-5844b856db-88bn9\" (UID: \"7583d484-6dd5-48e3-9485-05e57c62443e\") " pod="openshift-insights/insights-operator-5844b856db-88bn9"
Sep 11 17:18:59.405029 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.404886    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/65d6125e-1089-4308-bee8-064eefd92dec-bound-sa-token\") pod \"image-registry-86bf889f8b-5hnlq\" (UID: \"65d6125e-1089-4308-bee8-064eefd92dec\") " pod="openshift-image-registry/image-registry-86bf889f8b-5hnlq"
Sep 11 17:18:59.405029 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.404922    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/65d6125e-1089-4308-bee8-064eefd92dec-ca-trust-extracted\") pod \"image-registry-86bf889f8b-5hnlq\" (UID: \"65d6125e-1089-4308-bee8-064eefd92dec\") " pod="openshift-image-registry/image-registry-86bf889f8b-5hnlq"
Sep 11 17:18:59.405029 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.404980    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s6zpm\" (UniqueName: \"kubernetes.io/projected/7583d484-6dd5-48e3-9485-05e57c62443e-kube-api-access-s6zpm\") pod \"insights-operator-5844b856db-88bn9\" (UID: \"7583d484-6dd5-48e3-9485-05e57c62443e\") " pod="openshift-insights/insights-operator-5844b856db-88bn9"
Sep 11 17:18:59.405029 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.404998    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2a34a4aa-8cb7-42f3-9f87-da690cfd8031-service-ca-bundle\") pod \"router-default-6f8f7958c8-2p4xx\" (UID: \"2a34a4aa-8cb7-42f3-9f87-da690cfd8031\") " pod="openshift-ingress/router-default-6f8f7958c8-2p4xx"
Sep 11 17:18:59.505688 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.505649    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/676c8e42-836e-4f84-b5b0-c1a49eb8c5f3-serving-cert\") pod \"kube-storage-version-migrator-operator-6c8f84f897-pmvkp\" (UID: \"676c8e42-836e-4f84-b5b0-c1a49eb8c5f3\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-pmvkp"
Sep 11 17:18:59.505688 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.505688    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9qd6d\" (UniqueName: \"kubernetes.io/projected/676c8e42-836e-4f84-b5b0-c1a49eb8c5f3-kube-api-access-9qd6d\") pod \"kube-storage-version-migrator-operator-6c8f84f897-pmvkp\" (UID: \"676c8e42-836e-4f84-b5b0-c1a49eb8c5f3\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-pmvkp"
Sep 11 17:18:59.505966 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.505719    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/65d6125e-1089-4308-bee8-064eefd92dec-image-registry-private-configuration\") pod \"image-registry-86bf889f8b-5hnlq\" (UID: \"65d6125e-1089-4308-bee8-064eefd92dec\") " pod="openshift-image-registry/image-registry-86bf889f8b-5hnlq"
Sep 11 17:18:59.505966 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.505742    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7gql9\" (UniqueName: \"kubernetes.io/projected/2a34a4aa-8cb7-42f3-9f87-da690cfd8031-kube-api-access-7gql9\") pod \"router-default-6f8f7958c8-2p4xx\" (UID: \"2a34a4aa-8cb7-42f3-9f87-da690cfd8031\") " pod="openshift-ingress/router-default-6f8f7958c8-2p4xx"
Sep 11 17:18:59.505966 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.505765    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/aac7a167-8766-4c22-926e-ef15ceae304d-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-q2zxh\" (UID: \"aac7a167-8766-4c22-926e-ef15ceae304d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-q2zxh"
Sep 11 17:18:59.505966 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.505785    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/eb5f3be1-2e36-40ba-b3a0-3bec413056e8-serving-cert\") pod \"console-operator-696884fb9c-hg7p8\" (UID: \"eb5f3be1-2e36-40ba-b3a0-3bec413056e8\") " pod="openshift-console-operator/console-operator-696884fb9c-hg7p8"
Sep 11 17:18:59.505966 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.505820    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7583d484-6dd5-48e3-9485-05e57c62443e-trusted-ca-bundle\") pod \"insights-operator-5844b856db-88bn9\" (UID: \"7583d484-6dd5-48e3-9485-05e57c62443e\") " pod="openshift-insights/insights-operator-5844b856db-88bn9"
Sep 11 17:18:59.505966 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.505871    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/7583d484-6dd5-48e3-9485-05e57c62443e-snapshots\") pod \"insights-operator-5844b856db-88bn9\" (UID: \"7583d484-6dd5-48e3-9485-05e57c62443e\") " pod="openshift-insights/insights-operator-5844b856db-88bn9"
Sep 11 17:18:59.505966 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.505967    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b7d42fd1-2cb9-429b-a57a-656a5d7ecb91-metrics-tls\") pod \"dns-default-6h5ct\" (UID: \"b7d42fd1-2cb9-429b-a57a-656a5d7ecb91\") " pod="openshift-dns/dns-default-6h5ct"
Sep 11 17:18:59.506336 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.505994    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/24021eec-97e8-4bd2-91b0-c31be7bd9c88-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-vtd7t\" (UID: \"24021eec-97e8-4bd2-91b0-c31be7bd9c88\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-vtd7t"
Sep 11 17:18:59.506336 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.506022    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/65d6125e-1089-4308-bee8-064eefd92dec-bound-sa-token\") pod \"image-registry-86bf889f8b-5hnlq\" (UID: \"65d6125e-1089-4308-bee8-064eefd92dec\") " pod="openshift-image-registry/image-registry-86bf889f8b-5hnlq"
Sep 11 17:18:59.506336 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.506043    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/65d6125e-1089-4308-bee8-064eefd92dec-ca-trust-extracted\") pod \"image-registry-86bf889f8b-5hnlq\" (UID: \"65d6125e-1089-4308-bee8-064eefd92dec\") " pod="openshift-image-registry/image-registry-86bf889f8b-5hnlq"
Sep 11 17:18:59.506336 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.506073    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w7xph\" (UniqueName: \"kubernetes.io/projected/18feb296-ac22-4b0f-9b2a-5d011fd65561-kube-api-access-w7xph\") pod \"network-check-source-6f8b768455-ltwfd\" (UID: \"18feb296-ac22-4b0f-9b2a-5d011fd65561\") " pod="openshift-network-diagnostics/network-check-source-6f8b768455-ltwfd"
Sep 11 17:18:59.506336 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.506101    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s6zpm\" (UniqueName: \"kubernetes.io/projected/7583d484-6dd5-48e3-9485-05e57c62443e-kube-api-access-s6zpm\") pod \"insights-operator-5844b856db-88bn9\" (UID: \"7583d484-6dd5-48e3-9485-05e57c62443e\") " pod="openshift-insights/insights-operator-5844b856db-88bn9"
Sep 11 17:18:59.506336 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.506186    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/65d6125e-1089-4308-bee8-064eefd92dec-installation-pull-secrets\") pod \"image-registry-86bf889f8b-5hnlq\" (UID: \"65d6125e-1089-4308-bee8-064eefd92dec\") " pod="openshift-image-registry/image-registry-86bf889f8b-5hnlq"
Sep 11 17:18:59.506336 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.506244    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/3e8c629b-c41f-4eef-9c3e-7eed516e00a7-nginx-conf\") pod \"networking-console-plugin-5dcbc7df6b-7kvsj\" (UID: \"3e8c629b-c41f-4eef-9c3e-7eed516e00a7\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-7kvsj"
Sep 11 17:18:59.506336 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.506292    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qx5bk\" (UniqueName: \"kubernetes.io/projected/24021eec-97e8-4bd2-91b0-c31be7bd9c88-kube-api-access-qx5bk\") pod \"cluster-monitoring-operator-6cd954b897-vtd7t\" (UID: \"24021eec-97e8-4bd2-91b0-c31be7bd9c88\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-vtd7t"
Sep 11 17:18:59.506336 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.506333    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/2a34a4aa-8cb7-42f3-9f87-da690cfd8031-default-certificate\") pod \"router-default-6f8f7958c8-2p4xx\" (UID: \"2a34a4aa-8cb7-42f3-9f87-da690cfd8031\") " pod="openshift-ingress/router-default-6f8f7958c8-2p4xx"
Sep 11 17:18:59.506738 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.506377    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/676c8e42-836e-4f84-b5b0-c1a49eb8c5f3-config\") pod \"kube-storage-version-migrator-operator-6c8f84f897-pmvkp\" (UID: \"676c8e42-836e-4f84-b5b0-c1a49eb8c5f3\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-pmvkp"
Sep 11 17:18:59.506738 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.506403    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m4nz2\" (UniqueName: \"kubernetes.io/projected/eb5f3be1-2e36-40ba-b3a0-3bec413056e8-kube-api-access-m4nz2\") pod \"console-operator-696884fb9c-hg7p8\" (UID: \"eb5f3be1-2e36-40ba-b3a0-3bec413056e8\") " pod="openshift-console-operator/console-operator-696884fb9c-hg7p8"
Sep 11 17:18:59.506738 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.506433    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/7583d484-6dd5-48e3-9485-05e57c62443e-snapshots\") pod \"insights-operator-5844b856db-88bn9\" (UID: \"7583d484-6dd5-48e3-9485-05e57c62443e\") " pod="openshift-insights/insights-operator-5844b856db-88bn9"
Sep 11 17:18:59.506738 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.506447    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b7d42fd1-2cb9-429b-a57a-656a5d7ecb91-tmp-dir\") pod \"dns-default-6h5ct\" (UID: \"b7d42fd1-2cb9-429b-a57a-656a5d7ecb91\") " pod="openshift-dns/dns-default-6h5ct"
Sep 11 17:18:59.506738 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.506475    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mkcqj\" (UniqueName: \"kubernetes.io/projected/1a431afe-6ad9-4886-b0ac-8d147f276146-kube-api-access-mkcqj\") pod \"service-ca-operator-cf7c776b7-gdzsp\" (UID: \"1a431afe-6ad9-4886-b0ac-8d147f276146\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-gdzsp"
Sep 11 17:18:59.506738 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.506474    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/65d6125e-1089-4308-bee8-064eefd92dec-ca-trust-extracted\") pod \"image-registry-86bf889f8b-5hnlq\" (UID: \"65d6125e-1089-4308-bee8-064eefd92dec\") " pod="openshift-image-registry/image-registry-86bf889f8b-5hnlq"
Sep 11 17:18:59.506738 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.506531    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1a431afe-6ad9-4886-b0ac-8d147f276146-serving-cert\") pod \"service-ca-operator-cf7c776b7-gdzsp\" (UID: \"1a431afe-6ad9-4886-b0ac-8d147f276146\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-gdzsp"
Sep 11 17:18:59.506738 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.506571    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/65d6125e-1089-4308-bee8-064eefd92dec-registry-tls\") pod \"image-registry-86bf889f8b-5hnlq\" (UID: \"65d6125e-1089-4308-bee8-064eefd92dec\") " pod="openshift-image-registry/image-registry-86bf889f8b-5hnlq"
Sep 11 17:18:59.506738 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.506632    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b7d42fd1-2cb9-429b-a57a-656a5d7ecb91-config-volume\") pod \"dns-default-6h5ct\" (UID: \"b7d42fd1-2cb9-429b-a57a-656a5d7ecb91\") " pod="openshift-dns/dns-default-6h5ct"
Sep 11 17:18:59.506738 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:18:59.506662    2552 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 11 17:18:59.506738 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:18:59.506676    2552 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-86bf889f8b-5hnlq: secret "image-registry-tls" not found
Sep 11 17:18:59.506738 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.506659    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nhd45\" (UniqueName: \"kubernetes.io/projected/aac7a167-8766-4c22-926e-ef15ceae304d-kube-api-access-nhd45\") pod \"cluster-samples-operator-86499c89c4-q2zxh\" (UID: \"aac7a167-8766-4c22-926e-ef15ceae304d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-q2zxh"
Sep 11 17:18:59.506738 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:18:59.506731    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/65d6125e-1089-4308-bee8-064eefd92dec-registry-tls podName:65d6125e-1089-4308-bee8-064eefd92dec nodeName:}" failed. No retries permitted until 2026-09-11 17:19:00.006713119 +0000 UTC m=+34.528004633 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/65d6125e-1089-4308-bee8-064eefd92dec-registry-tls") pod "image-registry-86bf889f8b-5hnlq" (UID: "65d6125e-1089-4308-bee8-064eefd92dec") : secret "image-registry-tls" not found
Sep 11 17:18:59.507359 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.506761    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/65d6125e-1089-4308-bee8-064eefd92dec-trusted-ca\") pod \"image-registry-86bf889f8b-5hnlq\" (UID: \"65d6125e-1089-4308-bee8-064eefd92dec\") " pod="openshift-image-registry/image-registry-86bf889f8b-5hnlq"
Sep 11 17:18:59.507359 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.506837    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/eb5f3be1-2e36-40ba-b3a0-3bec413056e8-config\") pod \"console-operator-696884fb9c-hg7p8\" (UID: \"eb5f3be1-2e36-40ba-b3a0-3bec413056e8\") " pod="openshift-console-operator/console-operator-696884fb9c-hg7p8"
Sep 11 17:18:59.507359 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.506862    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/eb5f3be1-2e36-40ba-b3a0-3bec413056e8-trusted-ca\") pod \"console-operator-696884fb9c-hg7p8\" (UID: \"eb5f3be1-2e36-40ba-b3a0-3bec413056e8\") " pod="openshift-console-operator/console-operator-696884fb9c-hg7p8"
Sep 11 17:18:59.507359 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.506890    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v4hp8\" (UniqueName: \"kubernetes.io/projected/b7d42fd1-2cb9-429b-a57a-656a5d7ecb91-kube-api-access-v4hp8\") pod \"dns-default-6h5ct\" (UID: \"b7d42fd1-2cb9-429b-a57a-656a5d7ecb91\") " pod="openshift-dns/dns-default-6h5ct"
Sep 11 17:18:59.507359 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.506897    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7583d484-6dd5-48e3-9485-05e57c62443e-trusted-ca-bundle\") pod \"insights-operator-5844b856db-88bn9\" (UID: \"7583d484-6dd5-48e3-9485-05e57c62443e\") " pod="openshift-insights/insights-operator-5844b856db-88bn9"
Sep 11 17:18:59.507359 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.506920    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zwn8r\" (UniqueName: \"kubernetes.io/projected/3d5f2687-9edd-426d-bb9d-3c7221a5aeb1-kube-api-access-zwn8r\") pod \"volume-data-source-validator-86695b7cc7-5sknr\" (UID: \"3d5f2687-9edd-426d-bb9d-3c7221a5aeb1\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-5sknr"
Sep 11 17:18:59.507359 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.507026    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7583d484-6dd5-48e3-9485-05e57c62443e-service-ca-bundle\") pod \"insights-operator-5844b856db-88bn9\" (UID: \"7583d484-6dd5-48e3-9485-05e57c62443e\") " pod="openshift-insights/insights-operator-5844b856db-88bn9"
Sep 11 17:18:59.507359 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.507068    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3e8c629b-c41f-4eef-9c3e-7eed516e00a7-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-7kvsj\" (UID: \"3e8c629b-c41f-4eef-9c3e-7eed516e00a7\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-7kvsj"
Sep 11 17:18:59.507359 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.507099    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/62af7c53-a9e7-40d3-ac2d-595a8dcfc54f-cert\") pod \"ingress-canary-nzjpd\" (UID: \"62af7c53-a9e7-40d3-ac2d-595a8dcfc54f\") " pod="openshift-ingress-canary/ingress-canary-nzjpd"
Sep 11 17:18:59.507359 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.507145    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2a34a4aa-8cb7-42f3-9f87-da690cfd8031-metrics-certs\") pod \"router-default-6f8f7958c8-2p4xx\" (UID: \"2a34a4aa-8cb7-42f3-9f87-da690cfd8031\") " pod="openshift-ingress/router-default-6f8f7958c8-2p4xx"
Sep 11 17:18:59.507359 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.507189    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2a34a4aa-8cb7-42f3-9f87-da690cfd8031-service-ca-bundle\") pod \"router-default-6f8f7958c8-2p4xx\" (UID: \"2a34a4aa-8cb7-42f3-9f87-da690cfd8031\") " pod="openshift-ingress/router-default-6f8f7958c8-2p4xx"
Sep 11 17:18:59.507359 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:18:59.507232    2552 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 11 17:18:59.507359 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:18:59.507300    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2a34a4aa-8cb7-42f3-9f87-da690cfd8031-metrics-certs podName:2a34a4aa-8cb7-42f3-9f87-da690cfd8031 nodeName:}" failed. No retries permitted until 2026-09-11 17:19:00.007280854 +0000 UTC m=+34.528572370 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2a34a4aa-8cb7-42f3-9f87-da690cfd8031-metrics-certs") pod "router-default-6f8f7958c8-2p4xx" (UID: "2a34a4aa-8cb7-42f3-9f87-da690cfd8031") : secret "router-metrics-certs-default" not found
Sep 11 17:18:59.507359 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.507345    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1a431afe-6ad9-4886-b0ac-8d147f276146-config\") pod \"service-ca-operator-cf7c776b7-gdzsp\" (UID: \"1a431afe-6ad9-4886-b0ac-8d147f276146\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-gdzsp"
Sep 11 17:18:59.508022 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.507385    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/24021eec-97e8-4bd2-91b0-c31be7bd9c88-telemetry-config\") pod \"cluster-monitoring-operator-6cd954b897-vtd7t\" (UID: \"24021eec-97e8-4bd2-91b0-c31be7bd9c88\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-vtd7t"
Sep 11 17:18:59.508022 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.507418    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7583d484-6dd5-48e3-9485-05e57c62443e-tmp\") pod \"insights-operator-5844b856db-88bn9\" (UID: \"7583d484-6dd5-48e3-9485-05e57c62443e\") " pod="openshift-insights/insights-operator-5844b856db-88bn9"
Sep 11 17:18:59.508022 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.507444    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-brqvg\" (UniqueName: \"kubernetes.io/projected/65d6125e-1089-4308-bee8-064eefd92dec-kube-api-access-brqvg\") pod \"image-registry-86bf889f8b-5hnlq\" (UID: \"65d6125e-1089-4308-bee8-064eefd92dec\") " pod="openshift-image-registry/image-registry-86bf889f8b-5hnlq"
Sep 11 17:18:59.508022 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.507471    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fh25t\" (UniqueName: \"kubernetes.io/projected/62af7c53-a9e7-40d3-ac2d-595a8dcfc54f-kube-api-access-fh25t\") pod \"ingress-canary-nzjpd\" (UID: \"62af7c53-a9e7-40d3-ac2d-595a8dcfc54f\") " pod="openshift-ingress-canary/ingress-canary-nzjpd"
Sep 11 17:18:59.508022 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.507500    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7583d484-6dd5-48e3-9485-05e57c62443e-serving-cert\") pod \"insights-operator-5844b856db-88bn9\" (UID: \"7583d484-6dd5-48e3-9485-05e57c62443e\") " pod="openshift-insights/insights-operator-5844b856db-88bn9"
Sep 11 17:18:59.508022 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.507524    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/2a34a4aa-8cb7-42f3-9f87-da690cfd8031-stats-auth\") pod \"router-default-6f8f7958c8-2p4xx\" (UID: \"2a34a4aa-8cb7-42f3-9f87-da690cfd8031\") " pod="openshift-ingress/router-default-6f8f7958c8-2p4xx"
Sep 11 17:18:59.508022 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.507561    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/65d6125e-1089-4308-bee8-064eefd92dec-registry-certificates\") pod \"image-registry-86bf889f8b-5hnlq\" (UID: \"65d6125e-1089-4308-bee8-064eefd92dec\") " pod="openshift-image-registry/image-registry-86bf889f8b-5hnlq"
Sep 11 17:18:59.508022 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.507602    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/65d6125e-1089-4308-bee8-064eefd92dec-trusted-ca\") pod \"image-registry-86bf889f8b-5hnlq\" (UID: \"65d6125e-1089-4308-bee8-064eefd92dec\") " pod="openshift-image-registry/image-registry-86bf889f8b-5hnlq"
Sep 11 17:18:59.508022 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:18:59.507710    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2a34a4aa-8cb7-42f3-9f87-da690cfd8031-service-ca-bundle podName:2a34a4aa-8cb7-42f3-9f87-da690cfd8031 nodeName:}" failed. No retries permitted until 2026-09-11 17:19:00.007683855 +0000 UTC m=+34.528975374 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/2a34a4aa-8cb7-42f3-9f87-da690cfd8031-service-ca-bundle") pod "router-default-6f8f7958c8-2p4xx" (UID: "2a34a4aa-8cb7-42f3-9f87-da690cfd8031") : configmap references non-existent config key: service-ca.crt
Sep 11 17:18:59.508022 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.507763    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7583d484-6dd5-48e3-9485-05e57c62443e-service-ca-bundle\") pod \"insights-operator-5844b856db-88bn9\" (UID: \"7583d484-6dd5-48e3-9485-05e57c62443e\") " pod="openshift-insights/insights-operator-5844b856db-88bn9"
Sep 11 17:18:59.508022 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.507878    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7583d484-6dd5-48e3-9485-05e57c62443e-tmp\") pod \"insights-operator-5844b856db-88bn9\" (UID: \"7583d484-6dd5-48e3-9485-05e57c62443e\") " pod="openshift-insights/insights-operator-5844b856db-88bn9"
Sep 11 17:18:59.508470 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.508151    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/65d6125e-1089-4308-bee8-064eefd92dec-registry-certificates\") pod \"image-registry-86bf889f8b-5hnlq\" (UID: \"65d6125e-1089-4308-bee8-064eefd92dec\") " pod="openshift-image-registry/image-registry-86bf889f8b-5hnlq"
Sep 11 17:18:59.510543 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.510401    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7583d484-6dd5-48e3-9485-05e57c62443e-serving-cert\") pod \"insights-operator-5844b856db-88bn9\" (UID: \"7583d484-6dd5-48e3-9485-05e57c62443e\") " pod="openshift-insights/insights-operator-5844b856db-88bn9"
Sep 11 17:18:59.510543 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.510503    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/65d6125e-1089-4308-bee8-064eefd92dec-image-registry-private-configuration\") pod \"image-registry-86bf889f8b-5hnlq\" (UID: \"65d6125e-1089-4308-bee8-064eefd92dec\") " pod="openshift-image-registry/image-registry-86bf889f8b-5hnlq"
Sep 11 17:18:59.510543 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.510518    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/2a34a4aa-8cb7-42f3-9f87-da690cfd8031-default-certificate\") pod \"router-default-6f8f7958c8-2p4xx\" (UID: \"2a34a4aa-8cb7-42f3-9f87-da690cfd8031\") " pod="openshift-ingress/router-default-6f8f7958c8-2p4xx"
Sep 11 17:18:59.510727 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.510617    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/65d6125e-1089-4308-bee8-064eefd92dec-installation-pull-secrets\") pod \"image-registry-86bf889f8b-5hnlq\" (UID: \"65d6125e-1089-4308-bee8-064eefd92dec\") " pod="openshift-image-registry/image-registry-86bf889f8b-5hnlq"
Sep 11 17:18:59.511083 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.511063    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/2a34a4aa-8cb7-42f3-9f87-da690cfd8031-stats-auth\") pod \"router-default-6f8f7958c8-2p4xx\" (UID: \"2a34a4aa-8cb7-42f3-9f87-da690cfd8031\") " pod="openshift-ingress/router-default-6f8f7958c8-2p4xx"
Sep 11 17:18:59.518977 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.518955    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s6zpm\" (UniqueName: \"kubernetes.io/projected/7583d484-6dd5-48e3-9485-05e57c62443e-kube-api-access-s6zpm\") pod \"insights-operator-5844b856db-88bn9\" (UID: \"7583d484-6dd5-48e3-9485-05e57c62443e\") " pod="openshift-insights/insights-operator-5844b856db-88bn9"
Sep 11 17:18:59.519066 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.518998    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zwn8r\" (UniqueName: \"kubernetes.io/projected/3d5f2687-9edd-426d-bb9d-3c7221a5aeb1-kube-api-access-zwn8r\") pod \"volume-data-source-validator-86695b7cc7-5sknr\" (UID: \"3d5f2687-9edd-426d-bb9d-3c7221a5aeb1\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-5sknr"
Sep 11 17:18:59.519066 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.519046    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7gql9\" (UniqueName: \"kubernetes.io/projected/2a34a4aa-8cb7-42f3-9f87-da690cfd8031-kube-api-access-7gql9\") pod \"router-default-6f8f7958c8-2p4xx\" (UID: \"2a34a4aa-8cb7-42f3-9f87-da690cfd8031\") " pod="openshift-ingress/router-default-6f8f7958c8-2p4xx"
Sep 11 17:18:59.519186 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.519140    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-brqvg\" (UniqueName: \"kubernetes.io/projected/65d6125e-1089-4308-bee8-064eefd92dec-kube-api-access-brqvg\") pod \"image-registry-86bf889f8b-5hnlq\" (UID: \"65d6125e-1089-4308-bee8-064eefd92dec\") " pod="openshift-image-registry/image-registry-86bf889f8b-5hnlq"
Sep 11 17:18:59.519186 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.519171    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/65d6125e-1089-4308-bee8-064eefd92dec-bound-sa-token\") pod \"image-registry-86bf889f8b-5hnlq\" (UID: \"65d6125e-1089-4308-bee8-064eefd92dec\") " pod="openshift-image-registry/image-registry-86bf889f8b-5hnlq"
Sep 11 17:18:59.537272 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.537202    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:18:59.608243 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.608202    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1a431afe-6ad9-4886-b0ac-8d147f276146-config\") pod \"service-ca-operator-cf7c776b7-gdzsp\" (UID: \"1a431afe-6ad9-4886-b0ac-8d147f276146\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-gdzsp"
Sep 11 17:18:59.608243 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.608245    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b2b1c077-b5b8-4bdc-a1b3-6b98a0bcf6a0-metrics-certs\") pod \"network-metrics-daemon-d5jqp\" (UID: \"b2b1c077-b5b8-4bdc-a1b3-6b98a0bcf6a0\") " pod="openshift-multus/network-metrics-daemon-d5jqp"
Sep 11 17:18:59.608515 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.608262    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/24021eec-97e8-4bd2-91b0-c31be7bd9c88-telemetry-config\") pod \"cluster-monitoring-operator-6cd954b897-vtd7t\" (UID: \"24021eec-97e8-4bd2-91b0-c31be7bd9c88\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-vtd7t"
Sep 11 17:18:59.608515 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.608284    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fh25t\" (UniqueName: \"kubernetes.io/projected/62af7c53-a9e7-40d3-ac2d-595a8dcfc54f-kube-api-access-fh25t\") pod \"ingress-canary-nzjpd\" (UID: \"62af7c53-a9e7-40d3-ac2d-595a8dcfc54f\") " pod="openshift-ingress-canary/ingress-canary-nzjpd"
Sep 11 17:18:59.608515 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.608315    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/676c8e42-836e-4f84-b5b0-c1a49eb8c5f3-serving-cert\") pod \"kube-storage-version-migrator-operator-6c8f84f897-pmvkp\" (UID: \"676c8e42-836e-4f84-b5b0-c1a49eb8c5f3\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-pmvkp"
Sep 11 17:18:59.608515 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.608339    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9qd6d\" (UniqueName: \"kubernetes.io/projected/676c8e42-836e-4f84-b5b0-c1a49eb8c5f3-kube-api-access-9qd6d\") pod \"kube-storage-version-migrator-operator-6c8f84f897-pmvkp\" (UID: \"676c8e42-836e-4f84-b5b0-c1a49eb8c5f3\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-pmvkp"
Sep 11 17:18:59.608515 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:18:59.608364    2552 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 17:18:59.608515 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:18:59.608431    2552 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 11 17:18:59.608515 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:18:59.608439    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b2b1c077-b5b8-4bdc-a1b3-6b98a0bcf6a0-metrics-certs podName:b2b1c077-b5b8-4bdc-a1b3-6b98a0bcf6a0 nodeName:}" failed. No retries permitted until 2026-09-11 17:19:31.608420094 +0000 UTC m=+66.129711609 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b2b1c077-b5b8-4bdc-a1b3-6b98a0bcf6a0-metrics-certs") pod "network-metrics-daemon-d5jqp" (UID: "b2b1c077-b5b8-4bdc-a1b3-6b98a0bcf6a0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 17:18:59.608515 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:18:59.608475    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aac7a167-8766-4c22-926e-ef15ceae304d-samples-operator-tls podName:aac7a167-8766-4c22-926e-ef15ceae304d nodeName:}" failed. No retries permitted until 2026-09-11 17:19:00.108461274 +0000 UTC m=+34.629752775 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/aac7a167-8766-4c22-926e-ef15ceae304d-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-q2zxh" (UID: "aac7a167-8766-4c22-926e-ef15ceae304d") : secret "samples-operator-tls" not found
Sep 11 17:18:59.608515 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.608373    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/aac7a167-8766-4c22-926e-ef15ceae304d-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-q2zxh\" (UID: \"aac7a167-8766-4c22-926e-ef15ceae304d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-q2zxh"
Sep 11 17:18:59.608515 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.608504    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/eb5f3be1-2e36-40ba-b3a0-3bec413056e8-serving-cert\") pod \"console-operator-696884fb9c-hg7p8\" (UID: \"eb5f3be1-2e36-40ba-b3a0-3bec413056e8\") " pod="openshift-console-operator/console-operator-696884fb9c-hg7p8"
Sep 11 17:18:59.609001 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.608530    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/72eae49b-cb97-4819-aaf5-5c6eb23ce9c8-original-pull-secret\") pod \"global-pull-secret-syncer-lx6ht\" (UID: \"72eae49b-cb97-4819-aaf5-5c6eb23ce9c8\") " pod="kube-system/global-pull-secret-syncer-lx6ht"
Sep 11 17:18:59.609001 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.608555    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b7d42fd1-2cb9-429b-a57a-656a5d7ecb91-metrics-tls\") pod \"dns-default-6h5ct\" (UID: \"b7d42fd1-2cb9-429b-a57a-656a5d7ecb91\") " pod="openshift-dns/dns-default-6h5ct"
Sep 11 17:18:59.609001 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.608581    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/24021eec-97e8-4bd2-91b0-c31be7bd9c88-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-vtd7t\" (UID: \"24021eec-97e8-4bd2-91b0-c31be7bd9c88\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-vtd7t"
Sep 11 17:18:59.609001 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.608613    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w7xph\" (UniqueName: \"kubernetes.io/projected/18feb296-ac22-4b0f-9b2a-5d011fd65561-kube-api-access-w7xph\") pod \"network-check-source-6f8b768455-ltwfd\" (UID: \"18feb296-ac22-4b0f-9b2a-5d011fd65561\") " pod="openshift-network-diagnostics/network-check-source-6f8b768455-ltwfd"
Sep 11 17:18:59.609001 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.608643    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-42k58\" (UniqueName: \"kubernetes.io/projected/99bdc64a-0517-49d6-acf3-4e9fc5ea47fe-kube-api-access-42k58\") pod \"network-check-target-9gb8k\" (UID: \"99bdc64a-0517-49d6-acf3-4e9fc5ea47fe\") " pod="openshift-network-diagnostics/network-check-target-9gb8k"
Sep 11 17:18:59.609001 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.608677    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/3e8c629b-c41f-4eef-9c3e-7eed516e00a7-nginx-conf\") pod \"networking-console-plugin-5dcbc7df6b-7kvsj\" (UID: \"3e8c629b-c41f-4eef-9c3e-7eed516e00a7\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-7kvsj"
Sep 11 17:18:59.609001 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:18:59.608692    2552 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 17:18:59.609001 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.608701    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qx5bk\" (UniqueName: \"kubernetes.io/projected/24021eec-97e8-4bd2-91b0-c31be7bd9c88-kube-api-access-qx5bk\") pod \"cluster-monitoring-operator-6cd954b897-vtd7t\" (UID: \"24021eec-97e8-4bd2-91b0-c31be7bd9c88\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-vtd7t"
Sep 11 17:18:59.609001 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:18:59.608735    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b7d42fd1-2cb9-429b-a57a-656a5d7ecb91-metrics-tls podName:b7d42fd1-2cb9-429b-a57a-656a5d7ecb91 nodeName:}" failed. No retries permitted until 2026-09-11 17:19:00.108721193 +0000 UTC m=+34.630012692 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b7d42fd1-2cb9-429b-a57a-656a5d7ecb91-metrics-tls") pod "dns-default-6h5ct" (UID: "b7d42fd1-2cb9-429b-a57a-656a5d7ecb91") : secret "dns-default-metrics-tls" not found
Sep 11 17:18:59.609001 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.608756    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/676c8e42-836e-4f84-b5b0-c1a49eb8c5f3-config\") pod \"kube-storage-version-migrator-operator-6c8f84f897-pmvkp\" (UID: \"676c8e42-836e-4f84-b5b0-c1a49eb8c5f3\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-pmvkp"
Sep 11 17:18:59.609001 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:18:59.608764    2552 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 17:18:59.609001 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.608789    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m4nz2\" (UniqueName: \"kubernetes.io/projected/eb5f3be1-2e36-40ba-b3a0-3bec413056e8-kube-api-access-m4nz2\") pod \"console-operator-696884fb9c-hg7p8\" (UID: \"eb5f3be1-2e36-40ba-b3a0-3bec413056e8\") " pod="openshift-console-operator/console-operator-696884fb9c-hg7p8"
Sep 11 17:18:59.609001 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:18:59.608816    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/72eae49b-cb97-4819-aaf5-5c6eb23ce9c8-original-pull-secret podName:72eae49b-cb97-4819-aaf5-5c6eb23ce9c8 nodeName:}" failed. No retries permitted until 2026-09-11 17:19:31.608803663 +0000 UTC m=+66.130095176 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/72eae49b-cb97-4819-aaf5-5c6eb23ce9c8-original-pull-secret") pod "global-pull-secret-syncer-lx6ht" (UID: "72eae49b-cb97-4819-aaf5-5c6eb23ce9c8") : object "kube-system"/"original-pull-secret" not registered
Sep 11 17:18:59.609001 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.608845    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b7d42fd1-2cb9-429b-a57a-656a5d7ecb91-tmp-dir\") pod \"dns-default-6h5ct\" (UID: \"b7d42fd1-2cb9-429b-a57a-656a5d7ecb91\") " pod="openshift-dns/dns-default-6h5ct"
Sep 11 17:18:59.609001 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.608871    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mkcqj\" (UniqueName: \"kubernetes.io/projected/1a431afe-6ad9-4886-b0ac-8d147f276146-kube-api-access-mkcqj\") pod \"service-ca-operator-cf7c776b7-gdzsp\" (UID: \"1a431afe-6ad9-4886-b0ac-8d147f276146\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-gdzsp"
Sep 11 17:18:59.609001 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.608903    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1a431afe-6ad9-4886-b0ac-8d147f276146-serving-cert\") pod \"service-ca-operator-cf7c776b7-gdzsp\" (UID: \"1a431afe-6ad9-4886-b0ac-8d147f276146\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-gdzsp"
Sep 11 17:18:59.609793 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.608944    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b7d42fd1-2cb9-429b-a57a-656a5d7ecb91-config-volume\") pod \"dns-default-6h5ct\" (UID: \"b7d42fd1-2cb9-429b-a57a-656a5d7ecb91\") " pod="openshift-dns/dns-default-6h5ct"
Sep 11 17:18:59.609793 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.608970    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nhd45\" (UniqueName: \"kubernetes.io/projected/aac7a167-8766-4c22-926e-ef15ceae304d-kube-api-access-nhd45\") pod \"cluster-samples-operator-86499c89c4-q2zxh\" (UID: \"aac7a167-8766-4c22-926e-ef15ceae304d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-q2zxh"
Sep 11 17:18:59.609793 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.609000    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/eb5f3be1-2e36-40ba-b3a0-3bec413056e8-config\") pod \"console-operator-696884fb9c-hg7p8\" (UID: \"eb5f3be1-2e36-40ba-b3a0-3bec413056e8\") " pod="openshift-console-operator/console-operator-696884fb9c-hg7p8"
Sep 11 17:18:59.609793 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.609023    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/eb5f3be1-2e36-40ba-b3a0-3bec413056e8-trusted-ca\") pod \"console-operator-696884fb9c-hg7p8\" (UID: \"eb5f3be1-2e36-40ba-b3a0-3bec413056e8\") " pod="openshift-console-operator/console-operator-696884fb9c-hg7p8"
Sep 11 17:18:59.609793 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:18:59.609040    2552 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 11 17:18:59.609793 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.609049    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v4hp8\" (UniqueName: \"kubernetes.io/projected/b7d42fd1-2cb9-429b-a57a-656a5d7ecb91-kube-api-access-v4hp8\") pod \"dns-default-6h5ct\" (UID: \"b7d42fd1-2cb9-429b-a57a-656a5d7ecb91\") " pod="openshift-dns/dns-default-6h5ct"
Sep 11 17:18:59.609793 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:18:59.609086    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24021eec-97e8-4bd2-91b0-c31be7bd9c88-cluster-monitoring-operator-tls podName:24021eec-97e8-4bd2-91b0-c31be7bd9c88 nodeName:}" failed. No retries permitted until 2026-09-11 17:19:00.109071858 +0000 UTC m=+34.630363378 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/24021eec-97e8-4bd2-91b0-c31be7bd9c88-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-vtd7t" (UID: "24021eec-97e8-4bd2-91b0-c31be7bd9c88") : secret "cluster-monitoring-operator-tls" not found
Sep 11 17:18:59.609793 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.609160    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3e8c629b-c41f-4eef-9c3e-7eed516e00a7-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-7kvsj\" (UID: \"3e8c629b-c41f-4eef-9c3e-7eed516e00a7\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-7kvsj"
Sep 11 17:18:59.609793 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.609191    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/62af7c53-a9e7-40d3-ac2d-595a8dcfc54f-cert\") pod \"ingress-canary-nzjpd\" (UID: \"62af7c53-a9e7-40d3-ac2d-595a8dcfc54f\") " pod="openshift-ingress-canary/ingress-canary-nzjpd"
Sep 11 17:18:59.609793 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.608967    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1a431afe-6ad9-4886-b0ac-8d147f276146-config\") pod \"service-ca-operator-cf7c776b7-gdzsp\" (UID: \"1a431afe-6ad9-4886-b0ac-8d147f276146\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-gdzsp"
Sep 11 17:18:59.609793 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.609539    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b7d42fd1-2cb9-429b-a57a-656a5d7ecb91-tmp-dir\") pod \"dns-default-6h5ct\" (UID: \"b7d42fd1-2cb9-429b-a57a-656a5d7ecb91\") " pod="openshift-dns/dns-default-6h5ct"
Sep 11 17:18:59.609793 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.609587    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/676c8e42-836e-4f84-b5b0-c1a49eb8c5f3-config\") pod \"kube-storage-version-migrator-operator-6c8f84f897-pmvkp\" (UID: \"676c8e42-836e-4f84-b5b0-c1a49eb8c5f3\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-pmvkp"
Sep 11 17:18:59.609793 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.609081    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/24021eec-97e8-4bd2-91b0-c31be7bd9c88-telemetry-config\") pod \"cluster-monitoring-operator-6cd954b897-vtd7t\" (UID: \"24021eec-97e8-4bd2-91b0-c31be7bd9c88\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-vtd7t"
Sep 11 17:18:59.610441 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.609815    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/3e8c629b-c41f-4eef-9c3e-7eed516e00a7-nginx-conf\") pod \"networking-console-plugin-5dcbc7df6b-7kvsj\" (UID: \"3e8c629b-c41f-4eef-9c3e-7eed516e00a7\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-7kvsj"
Sep 11 17:18:59.610441 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.610020    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/eb5f3be1-2e36-40ba-b3a0-3bec413056e8-config\") pod \"console-operator-696884fb9c-hg7p8\" (UID: \"eb5f3be1-2e36-40ba-b3a0-3bec413056e8\") " pod="openshift-console-operator/console-operator-696884fb9c-hg7p8"
Sep 11 17:18:59.610441 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:18:59.610127    2552 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 11 17:18:59.610441 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:18:59.610169    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3e8c629b-c41f-4eef-9c3e-7eed516e00a7-networking-console-plugin-cert podName:3e8c629b-c41f-4eef-9c3e-7eed516e00a7 nodeName:}" failed. No retries permitted until 2026-09-11 17:19:00.110155762 +0000 UTC m=+34.631447262 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/3e8c629b-c41f-4eef-9c3e-7eed516e00a7-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-7kvsj" (UID: "3e8c629b-c41f-4eef-9c3e-7eed516e00a7") : secret "networking-console-plugin-cert" not found
Sep 11 17:18:59.610441 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.610363    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b7d42fd1-2cb9-429b-a57a-656a5d7ecb91-config-volume\") pod \"dns-default-6h5ct\" (UID: \"b7d42fd1-2cb9-429b-a57a-656a5d7ecb91\") " pod="openshift-dns/dns-default-6h5ct"
Sep 11 17:18:59.610705 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.610596    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/eb5f3be1-2e36-40ba-b3a0-3bec413056e8-trusted-ca\") pod \"console-operator-696884fb9c-hg7p8\" (UID: \"eb5f3be1-2e36-40ba-b3a0-3bec413056e8\") " pod="openshift-console-operator/console-operator-696884fb9c-hg7p8"
Sep 11 17:18:59.611447 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.611421    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/676c8e42-836e-4f84-b5b0-c1a49eb8c5f3-serving-cert\") pod \"kube-storage-version-migrator-operator-6c8f84f897-pmvkp\" (UID: \"676c8e42-836e-4f84-b5b0-c1a49eb8c5f3\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-pmvkp"
Sep 11 17:18:59.611863 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.611845    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-42k58\" (UniqueName: \"kubernetes.io/projected/99bdc64a-0517-49d6-acf3-4e9fc5ea47fe-kube-api-access-42k58\") pod \"network-check-target-9gb8k\" (UID: \"99bdc64a-0517-49d6-acf3-4e9fc5ea47fe\") " pod="openshift-network-diagnostics/network-check-target-9gb8k"
Sep 11 17:18:59.612017 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.612001    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/62af7c53-a9e7-40d3-ac2d-595a8dcfc54f-cert\") pod \"ingress-canary-nzjpd\" (UID: \"62af7c53-a9e7-40d3-ac2d-595a8dcfc54f\") " pod="openshift-ingress-canary/ingress-canary-nzjpd"
Sep 11 17:18:59.612070 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.612023    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1a431afe-6ad9-4886-b0ac-8d147f276146-serving-cert\") pod \"service-ca-operator-cf7c776b7-gdzsp\" (UID: \"1a431afe-6ad9-4886-b0ac-8d147f276146\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-gdzsp"
Sep 11 17:18:59.612356 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.612341    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/eb5f3be1-2e36-40ba-b3a0-3bec413056e8-serving-cert\") pod \"console-operator-696884fb9c-hg7p8\" (UID: \"eb5f3be1-2e36-40ba-b3a0-3bec413056e8\") " pod="openshift-console-operator/console-operator-696884fb9c-hg7p8"
Sep 11 17:18:59.625129 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.625095    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m4nz2\" (UniqueName: \"kubernetes.io/projected/eb5f3be1-2e36-40ba-b3a0-3bec413056e8-kube-api-access-m4nz2\") pod \"console-operator-696884fb9c-hg7p8\" (UID: \"eb5f3be1-2e36-40ba-b3a0-3bec413056e8\") " pod="openshift-console-operator/console-operator-696884fb9c-hg7p8"
Sep 11 17:18:59.625251 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.625177    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v4hp8\" (UniqueName: \"kubernetes.io/projected/b7d42fd1-2cb9-429b-a57a-656a5d7ecb91-kube-api-access-v4hp8\") pod \"dns-default-6h5ct\" (UID: \"b7d42fd1-2cb9-429b-a57a-656a5d7ecb91\") " pod="openshift-dns/dns-default-6h5ct"
Sep 11 17:18:59.625772 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.625740    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mkcqj\" (UniqueName: \"kubernetes.io/projected/1a431afe-6ad9-4886-b0ac-8d147f276146-kube-api-access-mkcqj\") pod \"service-ca-operator-cf7c776b7-gdzsp\" (UID: \"1a431afe-6ad9-4886-b0ac-8d147f276146\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-gdzsp"
Sep 11 17:18:59.625856 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.625802    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9qd6d\" (UniqueName: \"kubernetes.io/projected/676c8e42-836e-4f84-b5b0-c1a49eb8c5f3-kube-api-access-9qd6d\") pod \"kube-storage-version-migrator-operator-6c8f84f897-pmvkp\" (UID: \"676c8e42-836e-4f84-b5b0-c1a49eb8c5f3\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-pmvkp"
Sep 11 17:18:59.625965 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.625949    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w7xph\" (UniqueName: \"kubernetes.io/projected/18feb296-ac22-4b0f-9b2a-5d011fd65561-kube-api-access-w7xph\") pod \"network-check-source-6f8b768455-ltwfd\" (UID: \"18feb296-ac22-4b0f-9b2a-5d011fd65561\") " pod="openshift-network-diagnostics/network-check-source-6f8b768455-ltwfd"
Sep 11 17:18:59.626168 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.626146    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fh25t\" (UniqueName: \"kubernetes.io/projected/62af7c53-a9e7-40d3-ac2d-595a8dcfc54f-kube-api-access-fh25t\") pod \"ingress-canary-nzjpd\" (UID: \"62af7c53-a9e7-40d3-ac2d-595a8dcfc54f\") " pod="openshift-ingress-canary/ingress-canary-nzjpd"
Sep 11 17:18:59.626234 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.626218    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qx5bk\" (UniqueName: \"kubernetes.io/projected/24021eec-97e8-4bd2-91b0-c31be7bd9c88-kube-api-access-qx5bk\") pod \"cluster-monitoring-operator-6cd954b897-vtd7t\" (UID: \"24021eec-97e8-4bd2-91b0-c31be7bd9c88\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-vtd7t"
Sep 11 17:18:59.627030 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.627014    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nhd45\" (UniqueName: \"kubernetes.io/projected/aac7a167-8766-4c22-926e-ef15ceae304d-kube-api-access-nhd45\") pod \"cluster-samples-operator-86499c89c4-q2zxh\" (UID: \"aac7a167-8766-4c22-926e-ef15ceae304d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-q2zxh"
Sep 11 17:18:59.634662 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.634645    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-5sknr"
Sep 11 17:18:59.640954 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.640925    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:18:59.641080 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.641064    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5844b856db-88bn9"
Sep 11 17:18:59.684761 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.684714    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-pmvkp"
Sep 11 17:18:59.697946 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.697237    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-696884fb9c-hg7p8"
Sep 11 17:18:59.704702 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.704487    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-gdzsp"
Sep 11 17:18:59.710829 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.710133    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-nzjpd"
Sep 11 17:18:59.720583 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.720553    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f8b768455-ltwfd"
Sep 11 17:18:59.924478 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.924448    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-5sknr"]
Sep 11 17:18:59.926994 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.926968    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5844b856db-88bn9"]
Sep 11 17:18:59.928205 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:59.928173    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d5f2687_9edd_426d_bb9d_3c7221a5aeb1.slice/crio-6cda3e06d23e5cf814bf3dd2f1b82e69dff43d558767d7fcdfc2eb781e82da3e WatchSource:0}: Error finding container 6cda3e06d23e5cf814bf3dd2f1b82e69dff43d558767d7fcdfc2eb781e82da3e: Status 404 returned error can't find the container with id 6cda3e06d23e5cf814bf3dd2f1b82e69dff43d558767d7fcdfc2eb781e82da3e
Sep 11 17:18:59.929635 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:59.929612    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7583d484_6dd5_48e3_9485_05e57c62443e.slice/crio-01ce9c326fa63e98b5ec247633de008d242288d928aac4fd97cf20ed17342006 WatchSource:0}: Error finding container 01ce9c326fa63e98b5ec247633de008d242288d928aac4fd97cf20ed17342006: Status 404 returned error can't find the container with id 01ce9c326fa63e98b5ec247633de008d242288d928aac4fd97cf20ed17342006
Sep 11 17:18:59.978327 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:18:59.978291    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-pmvkp"]
Sep 11 17:18:59.981593 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:18:59.981567    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod676c8e42_836e_4f84_b5b0_c1a49eb8c5f3.slice/crio-0912614449ee6eed1a128fe3630d3ec7da6e0c09e693d0a5757412f37f306bae WatchSource:0}: Error finding container 0912614449ee6eed1a128fe3630d3ec7da6e0c09e693d0a5757412f37f306bae: Status 404 returned error can't find the container with id 0912614449ee6eed1a128fe3630d3ec7da6e0c09e693d0a5757412f37f306bae
Sep 11 17:19:00.009337 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:00.009239    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-nzjpd"]
Sep 11 17:19:00.012502 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:00.012479    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-cf7c776b7-gdzsp"]
Sep 11 17:19:00.013866 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:00.013477    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/65d6125e-1089-4308-bee8-064eefd92dec-registry-tls\") pod \"image-registry-86bf889f8b-5hnlq\" (UID: \"65d6125e-1089-4308-bee8-064eefd92dec\") " pod="openshift-image-registry/image-registry-86bf889f8b-5hnlq"
Sep 11 17:19:00.015046 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:00.014175    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2a34a4aa-8cb7-42f3-9f87-da690cfd8031-metrics-certs\") pod \"router-default-6f8f7958c8-2p4xx\" (UID: \"2a34a4aa-8cb7-42f3-9f87-da690cfd8031\") " pod="openshift-ingress/router-default-6f8f7958c8-2p4xx"
Sep 11 17:19:00.015046 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:00.014232    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2a34a4aa-8cb7-42f3-9f87-da690cfd8031-service-ca-bundle\") pod \"router-default-6f8f7958c8-2p4xx\" (UID: \"2a34a4aa-8cb7-42f3-9f87-da690cfd8031\") " pod="openshift-ingress/router-default-6f8f7958c8-2p4xx"
Sep 11 17:19:00.015046 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:19:00.014449    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2a34a4aa-8cb7-42f3-9f87-da690cfd8031-service-ca-bundle podName:2a34a4aa-8cb7-42f3-9f87-da690cfd8031 nodeName:}" failed. No retries permitted until 2026-09-11 17:19:01.014423378 +0000 UTC m=+35.535714899 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/2a34a4aa-8cb7-42f3-9f87-da690cfd8031-service-ca-bundle") pod "router-default-6f8f7958c8-2p4xx" (UID: "2a34a4aa-8cb7-42f3-9f87-da690cfd8031") : configmap references non-existent config key: service-ca.crt
Sep 11 17:19:00.015046 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:19:00.014513    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62af7c53_a9e7_40d3_ac2d_595a8dcfc54f.slice/crio-87c91f2d75d519d4b371551a2d319de18d816b1c98c126aa27f13c29bb33adc8 WatchSource:0}: Error finding container 87c91f2d75d519d4b371551a2d319de18d816b1c98c126aa27f13c29bb33adc8: Status 404 returned error can't find the container with id 87c91f2d75d519d4b371551a2d319de18d816b1c98c126aa27f13c29bb33adc8
Sep 11 17:19:00.015046 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:19:00.014541    2552 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 11 17:19:00.015046 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:19:00.014605    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2a34a4aa-8cb7-42f3-9f87-da690cfd8031-metrics-certs podName:2a34a4aa-8cb7-42f3-9f87-da690cfd8031 nodeName:}" failed. No retries permitted until 2026-09-11 17:19:01.014587258 +0000 UTC m=+35.535878787 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2a34a4aa-8cb7-42f3-9f87-da690cfd8031-metrics-certs") pod "router-default-6f8f7958c8-2p4xx" (UID: "2a34a4aa-8cb7-42f3-9f87-da690cfd8031") : secret "router-metrics-certs-default" not found
Sep 11 17:19:00.015046 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:19:00.015043    2552 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 11 17:19:00.015405 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:19:00.015064    2552 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-86bf889f8b-5hnlq: secret "image-registry-tls" not found
Sep 11 17:19:00.015405 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:19:00.015379    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/65d6125e-1089-4308-bee8-064eefd92dec-registry-tls podName:65d6125e-1089-4308-bee8-064eefd92dec nodeName:}" failed. No retries permitted until 2026-09-11 17:19:01.015362709 +0000 UTC m=+35.536654208 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/65d6125e-1089-4308-bee8-064eefd92dec-registry-tls") pod "image-registry-86bf889f8b-5hnlq" (UID: "65d6125e-1089-4308-bee8-064eefd92dec") : secret "image-registry-tls" not found
Sep 11 17:19:00.015504 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:00.015462    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-696884fb9c-hg7p8"]
Sep 11 17:19:00.032491 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:00.032466    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f8b768455-ltwfd"]
Sep 11 17:19:00.035671 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:19:00.035644    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18feb296_ac22_4b0f_9b2a_5d011fd65561.slice/crio-efb65730cb13f0d4d7a68f30004b6eeace17f9867ac84a383fa62a7395d5487e WatchSource:0}: Error finding container efb65730cb13f0d4d7a68f30004b6eeace17f9867ac84a383fa62a7395d5487e: Status 404 returned error can't find the container with id efb65730cb13f0d4d7a68f30004b6eeace17f9867ac84a383fa62a7395d5487e
Sep 11 17:19:00.091630 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:00.091594    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-88bn9" event={"ID":"7583d484-6dd5-48e3-9485-05e57c62443e","Type":"ContainerStarted","Data":"01ce9c326fa63e98b5ec247633de008d242288d928aac4fd97cf20ed17342006"}
Sep 11 17:19:00.092689 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:00.092658    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-pmvkp" event={"ID":"676c8e42-836e-4f84-b5b0-c1a49eb8c5f3","Type":"ContainerStarted","Data":"0912614449ee6eed1a128fe3630d3ec7da6e0c09e693d0a5757412f37f306bae"}
Sep 11 17:19:00.093780 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:00.093759    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-5sknr" event={"ID":"3d5f2687-9edd-426d-bb9d-3c7221a5aeb1","Type":"ContainerStarted","Data":"6cda3e06d23e5cf814bf3dd2f1b82e69dff43d558767d7fcdfc2eb781e82da3e"}
Sep 11 17:19:00.094906 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:00.094884    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-nzjpd" event={"ID":"62af7c53-a9e7-40d3-ac2d-595a8dcfc54f","Type":"ContainerStarted","Data":"87c91f2d75d519d4b371551a2d319de18d816b1c98c126aa27f13c29bb33adc8"}
Sep 11 17:19:00.096011 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:00.095993    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-gdzsp" event={"ID":"1a431afe-6ad9-4886-b0ac-8d147f276146","Type":"ContainerStarted","Data":"89375c90d841c87772f317620e5232a68062c1ef5b511e84224217bc761d9a81"}
Sep 11 17:19:00.097047 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:00.097030    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f8b768455-ltwfd" event={"ID":"18feb296-ac22-4b0f-9b2a-5d011fd65561","Type":"ContainerStarted","Data":"efb65730cb13f0d4d7a68f30004b6eeace17f9867ac84a383fa62a7395d5487e"}
Sep 11 17:19:00.098124 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:00.098097    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-hg7p8" event={"ID":"eb5f3be1-2e36-40ba-b3a0-3bec413056e8","Type":"ContainerStarted","Data":"d799cb441d09029408005dae81e058c1163bb32431992f6edb03473e5dd98479"}
Sep 11 17:19:00.115612 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:00.115582    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/aac7a167-8766-4c22-926e-ef15ceae304d-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-q2zxh\" (UID: \"aac7a167-8766-4c22-926e-ef15ceae304d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-q2zxh"
Sep 11 17:19:00.115753 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:00.115627    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b7d42fd1-2cb9-429b-a57a-656a5d7ecb91-metrics-tls\") pod \"dns-default-6h5ct\" (UID: \"b7d42fd1-2cb9-429b-a57a-656a5d7ecb91\") " pod="openshift-dns/dns-default-6h5ct"
Sep 11 17:19:00.115753 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:00.115651    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/24021eec-97e8-4bd2-91b0-c31be7bd9c88-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-vtd7t\" (UID: \"24021eec-97e8-4bd2-91b0-c31be7bd9c88\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-vtd7t"
Sep 11 17:19:00.115753 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:00.115743    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3e8c629b-c41f-4eef-9c3e-7eed516e00a7-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-7kvsj\" (UID: \"3e8c629b-c41f-4eef-9c3e-7eed516e00a7\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-7kvsj"
Sep 11 17:19:00.116035 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:19:00.115772    2552 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 11 17:19:00.116035 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:19:00.115827    2552 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 11 17:19:00.116035 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:19:00.115856    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aac7a167-8766-4c22-926e-ef15ceae304d-samples-operator-tls podName:aac7a167-8766-4c22-926e-ef15ceae304d nodeName:}" failed. No retries permitted until 2026-09-11 17:19:01.115836227 +0000 UTC m=+35.637127759 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/aac7a167-8766-4c22-926e-ef15ceae304d-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-q2zxh" (UID: "aac7a167-8766-4c22-926e-ef15ceae304d") : secret "samples-operator-tls" not found
Sep 11 17:19:00.116035 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:19:00.115829    2552 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 11 17:19:00.116035 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:19:00.115894    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3e8c629b-c41f-4eef-9c3e-7eed516e00a7-networking-console-plugin-cert podName:3e8c629b-c41f-4eef-9c3e-7eed516e00a7 nodeName:}" failed. No retries permitted until 2026-09-11 17:19:01.115878477 +0000 UTC m=+35.637169976 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/3e8c629b-c41f-4eef-9c3e-7eed516e00a7-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-7kvsj" (UID: "3e8c629b-c41f-4eef-9c3e-7eed516e00a7") : secret "networking-console-plugin-cert" not found
Sep 11 17:19:00.116035 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:19:00.115776    2552 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 17:19:00.116035 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:19:00.115963    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b7d42fd1-2cb9-429b-a57a-656a5d7ecb91-metrics-tls podName:b7d42fd1-2cb9-429b-a57a-656a5d7ecb91 nodeName:}" failed. No retries permitted until 2026-09-11 17:19:01.115945417 +0000 UTC m=+35.637236919 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b7d42fd1-2cb9-429b-a57a-656a5d7ecb91-metrics-tls") pod "dns-default-6h5ct" (UID: "b7d42fd1-2cb9-429b-a57a-656a5d7ecb91") : secret "dns-default-metrics-tls" not found
Sep 11 17:19:00.116035 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:19:00.115981    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24021eec-97e8-4bd2-91b0-c31be7bd9c88-cluster-monitoring-operator-tls podName:24021eec-97e8-4bd2-91b0-c31be7bd9c88 nodeName:}" failed. No retries permitted until 2026-09-11 17:19:01.115971736 +0000 UTC m=+35.637263239 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/24021eec-97e8-4bd2-91b0-c31be7bd9c88-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-vtd7t" (UID: "24021eec-97e8-4bd2-91b0-c31be7bd9c88") : secret "cluster-monitoring-operator-tls" not found
Sep 11 17:19:00.756223 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:00.756185    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:19:00.858342 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:00.858249    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:19:00.942786 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:00.942751    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d5jqp"
Sep 11 17:19:00.944525 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:00.943269    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9gb8k"
Sep 11 17:19:00.944525 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:00.943683    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lx6ht"
Sep 11 17:19:00.946206 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:00.945925    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-vthbj\""
Sep 11 17:19:00.949795 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:00.948803    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 11 17:19:00.949795 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:00.949051    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 11 17:19:00.949795 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:00.949280    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-gdccs\""
Sep 11 17:19:00.964938 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:00.964698    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:19:00.974171 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:00.973840    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9gb8k"
Sep 11 17:19:01.026156 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:01.025731    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2a34a4aa-8cb7-42f3-9f87-da690cfd8031-metrics-certs\") pod \"router-default-6f8f7958c8-2p4xx\" (UID: \"2a34a4aa-8cb7-42f3-9f87-da690cfd8031\") " pod="openshift-ingress/router-default-6f8f7958c8-2p4xx"
Sep 11 17:19:01.026156 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:01.025781    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2a34a4aa-8cb7-42f3-9f87-da690cfd8031-service-ca-bundle\") pod \"router-default-6f8f7958c8-2p4xx\" (UID: \"2a34a4aa-8cb7-42f3-9f87-da690cfd8031\") " pod="openshift-ingress/router-default-6f8f7958c8-2p4xx"
Sep 11 17:19:01.026156 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:19:01.025891    2552 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 11 17:19:01.026156 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:01.025940    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/65d6125e-1089-4308-bee8-064eefd92dec-registry-tls\") pod \"image-registry-86bf889f8b-5hnlq\" (UID: \"65d6125e-1089-4308-bee8-064eefd92dec\") " pod="openshift-image-registry/image-registry-86bf889f8b-5hnlq"
Sep 11 17:19:01.026156 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:19:01.025958    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2a34a4aa-8cb7-42f3-9f87-da690cfd8031-service-ca-bundle podName:2a34a4aa-8cb7-42f3-9f87-da690cfd8031 nodeName:}" failed. No retries permitted until 2026-09-11 17:19:03.025934564 +0000 UTC m=+37.547226082 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/2a34a4aa-8cb7-42f3-9f87-da690cfd8031-service-ca-bundle") pod "router-default-6f8f7958c8-2p4xx" (UID: "2a34a4aa-8cb7-42f3-9f87-da690cfd8031") : configmap references non-existent config key: service-ca.crt
Sep 11 17:19:01.026156 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:19:01.025991    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2a34a4aa-8cb7-42f3-9f87-da690cfd8031-metrics-certs podName:2a34a4aa-8cb7-42f3-9f87-da690cfd8031 nodeName:}" failed. No retries permitted until 2026-09-11 17:19:03.025980782 +0000 UTC m=+37.547272288 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2a34a4aa-8cb7-42f3-9f87-da690cfd8031-metrics-certs") pod "router-default-6f8f7958c8-2p4xx" (UID: "2a34a4aa-8cb7-42f3-9f87-da690cfd8031") : secret "router-metrics-certs-default" not found
Sep 11 17:19:01.026156 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:19:01.026006    2552 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 11 17:19:01.026156 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:19:01.026018    2552 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-86bf889f8b-5hnlq: secret "image-registry-tls" not found
Sep 11 17:19:01.026156 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:19:01.026058    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/65d6125e-1089-4308-bee8-064eefd92dec-registry-tls podName:65d6125e-1089-4308-bee8-064eefd92dec nodeName:}" failed. No retries permitted until 2026-09-11 17:19:03.026043059 +0000 UTC m=+37.547334573 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/65d6125e-1089-4308-bee8-064eefd92dec-registry-tls") pod "image-registry-86bf889f8b-5hnlq" (UID: "65d6125e-1089-4308-bee8-064eefd92dec") : secret "image-registry-tls" not found
Sep 11 17:19:01.129701 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:01.128198    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3e8c629b-c41f-4eef-9c3e-7eed516e00a7-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-7kvsj\" (UID: \"3e8c629b-c41f-4eef-9c3e-7eed516e00a7\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-7kvsj"
Sep 11 17:19:01.129701 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:01.128346    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/aac7a167-8766-4c22-926e-ef15ceae304d-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-q2zxh\" (UID: \"aac7a167-8766-4c22-926e-ef15ceae304d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-q2zxh"
Sep 11 17:19:01.129701 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:01.128391    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b7d42fd1-2cb9-429b-a57a-656a5d7ecb91-metrics-tls\") pod \"dns-default-6h5ct\" (UID: \"b7d42fd1-2cb9-429b-a57a-656a5d7ecb91\") " pod="openshift-dns/dns-default-6h5ct"
Sep 11 17:19:01.129701 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:01.128422    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/24021eec-97e8-4bd2-91b0-c31be7bd9c88-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-vtd7t\" (UID: \"24021eec-97e8-4bd2-91b0-c31be7bd9c88\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-vtd7t"
Sep 11 17:19:01.129701 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:19:01.128555    2552 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 11 17:19:01.129701 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:19:01.128620    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24021eec-97e8-4bd2-91b0-c31be7bd9c88-cluster-monitoring-operator-tls podName:24021eec-97e8-4bd2-91b0-c31be7bd9c88 nodeName:}" failed. No retries permitted until 2026-09-11 17:19:03.128600709 +0000 UTC m=+37.649892215 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/24021eec-97e8-4bd2-91b0-c31be7bd9c88-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-vtd7t" (UID: "24021eec-97e8-4bd2-91b0-c31be7bd9c88") : secret "cluster-monitoring-operator-tls" not found
Sep 11 17:19:01.129701 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:19:01.129311    2552 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 11 17:19:01.129701 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:19:01.129368    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3e8c629b-c41f-4eef-9c3e-7eed516e00a7-networking-console-plugin-cert podName:3e8c629b-c41f-4eef-9c3e-7eed516e00a7 nodeName:}" failed. No retries permitted until 2026-09-11 17:19:03.129352109 +0000 UTC m=+37.650643610 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/3e8c629b-c41f-4eef-9c3e-7eed516e00a7-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-7kvsj" (UID: "3e8c629b-c41f-4eef-9c3e-7eed516e00a7") : secret "networking-console-plugin-cert" not found
Sep 11 17:19:01.129701 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:19:01.129457    2552 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 17:19:01.129701 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:19:01.129514    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b7d42fd1-2cb9-429b-a57a-656a5d7ecb91-metrics-tls podName:b7d42fd1-2cb9-429b-a57a-656a5d7ecb91 nodeName:}" failed. No retries permitted until 2026-09-11 17:19:03.12949486 +0000 UTC m=+37.650786362 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b7d42fd1-2cb9-429b-a57a-656a5d7ecb91-metrics-tls") pod "dns-default-6h5ct" (UID: "b7d42fd1-2cb9-429b-a57a-656a5d7ecb91") : secret "dns-default-metrics-tls" not found
Sep 11 17:19:01.129701 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:19:01.129577    2552 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 11 17:19:01.129701 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:19:01.129614    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aac7a167-8766-4c22-926e-ef15ceae304d-samples-operator-tls podName:aac7a167-8766-4c22-926e-ef15ceae304d nodeName:}" failed. No retries permitted until 2026-09-11 17:19:03.129602429 +0000 UTC m=+37.650893933 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/aac7a167-8766-4c22-926e-ef15ceae304d-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-q2zxh" (UID: "aac7a167-8766-4c22-926e-ef15ceae304d") : secret "samples-operator-tls" not found
Sep 11 17:19:01.318102 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:01.317982    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-9gb8k"]
Sep 11 17:19:01.325165 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:19:01.325127    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99bdc64a_0517_49d6_acf3_4e9fc5ea47fe.slice/crio-d5e8fe20472d47ecab68ef436d202ef01fd873a4b17e372bc4580ed7b302a6e2 WatchSource:0}: Error finding container d5e8fe20472d47ecab68ef436d202ef01fd873a4b17e372bc4580ed7b302a6e2: Status 404 returned error can't find the container with id d5e8fe20472d47ecab68ef436d202ef01fd873a4b17e372bc4580ed7b302a6e2
Sep 11 17:19:01.440069 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:01.439994    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:19:01.543352 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:01.543308    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:19:01.643642 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:01.643576    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:19:02.110648 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:02.110568    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-9gb8k" event={"ID":"99bdc64a-0517-49d6-acf3-4e9fc5ea47fe","Type":"ContainerStarted","Data":"d5e8fe20472d47ecab68ef436d202ef01fd873a4b17e372bc4580ed7b302a6e2"}
Sep 11 17:19:02.504451 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:02.504220    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:19:02.604056 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:02.603837    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:19:03.052106 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:03.051771    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/65d6125e-1089-4308-bee8-064eefd92dec-registry-tls\") pod \"image-registry-86bf889f8b-5hnlq\" (UID: \"65d6125e-1089-4308-bee8-064eefd92dec\") " pod="openshift-image-registry/image-registry-86bf889f8b-5hnlq"
Sep 11 17:19:03.052106 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:03.052064    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2a34a4aa-8cb7-42f3-9f87-da690cfd8031-metrics-certs\") pod \"router-default-6f8f7958c8-2p4xx\" (UID: \"2a34a4aa-8cb7-42f3-9f87-da690cfd8031\") " pod="openshift-ingress/router-default-6f8f7958c8-2p4xx"
Sep 11 17:19:03.052106 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:19:03.052079    2552 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 11 17:19:03.052106 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:19:03.052101    2552 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-86bf889f8b-5hnlq: secret "image-registry-tls" not found
Sep 11 17:19:03.052571 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:19:03.052241    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/65d6125e-1089-4308-bee8-064eefd92dec-registry-tls podName:65d6125e-1089-4308-bee8-064eefd92dec nodeName:}" failed. No retries permitted until 2026-09-11 17:19:07.052162391 +0000 UTC m=+41.573453917 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/65d6125e-1089-4308-bee8-064eefd92dec-registry-tls") pod "image-registry-86bf889f8b-5hnlq" (UID: "65d6125e-1089-4308-bee8-064eefd92dec") : secret "image-registry-tls" not found
Sep 11 17:19:03.052571 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:19:03.052258    2552 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 11 17:19:03.052571 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:03.052097    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2a34a4aa-8cb7-42f3-9f87-da690cfd8031-service-ca-bundle\") pod \"router-default-6f8f7958c8-2p4xx\" (UID: \"2a34a4aa-8cb7-42f3-9f87-da690cfd8031\") " pod="openshift-ingress/router-default-6f8f7958c8-2p4xx"
Sep 11 17:19:03.052571 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:19:03.052265    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2a34a4aa-8cb7-42f3-9f87-da690cfd8031-service-ca-bundle podName:2a34a4aa-8cb7-42f3-9f87-da690cfd8031 nodeName:}" failed. No retries permitted until 2026-09-11 17:19:07.052255121 +0000 UTC m=+41.573546624 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/2a34a4aa-8cb7-42f3-9f87-da690cfd8031-service-ca-bundle") pod "router-default-6f8f7958c8-2p4xx" (UID: "2a34a4aa-8cb7-42f3-9f87-da690cfd8031") : configmap references non-existent config key: service-ca.crt
Sep 11 17:19:03.052571 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:19:03.052315    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2a34a4aa-8cb7-42f3-9f87-da690cfd8031-metrics-certs podName:2a34a4aa-8cb7-42f3-9f87-da690cfd8031 nodeName:}" failed. No retries permitted until 2026-09-11 17:19:07.052299724 +0000 UTC m=+41.573591228 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2a34a4aa-8cb7-42f3-9f87-da690cfd8031-metrics-certs") pod "router-default-6f8f7958c8-2p4xx" (UID: "2a34a4aa-8cb7-42f3-9f87-da690cfd8031") : secret "router-metrics-certs-default" not found
Sep 11 17:19:03.120035 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:03.119263    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-5sknr" event={"ID":"3d5f2687-9edd-426d-bb9d-3c7221a5aeb1","Type":"ContainerStarted","Data":"7a22e4f491cbc3ff9e394be0dcb471b59f4d9e4438aca9fcb78212b22a25da34"}
Sep 11 17:19:03.136899 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:03.136829    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-5sknr" podStartSLOduration=32.384846268 podStartE2EDuration="35.136807166s" totalImagesPullingTime="2.751960898s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:18:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:19:03.134995714 +0000 UTC m=+37.656287236" watchObservedRunningTime="2026-09-11 17:19:03.136807166 +0000 UTC m=+37.658098686"
Sep 11 17:19:03.153748 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:03.153707    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3e8c629b-c41f-4eef-9c3e-7eed516e00a7-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-7kvsj\" (UID: \"3e8c629b-c41f-4eef-9c3e-7eed516e00a7\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-7kvsj"
Sep 11 17:19:03.153936 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:03.153824    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/aac7a167-8766-4c22-926e-ef15ceae304d-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-q2zxh\" (UID: \"aac7a167-8766-4c22-926e-ef15ceae304d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-q2zxh"
Sep 11 17:19:03.153936 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:03.153869    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b7d42fd1-2cb9-429b-a57a-656a5d7ecb91-metrics-tls\") pod \"dns-default-6h5ct\" (UID: \"b7d42fd1-2cb9-429b-a57a-656a5d7ecb91\") " pod="openshift-dns/dns-default-6h5ct"
Sep 11 17:19:03.153936 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:03.153903    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/24021eec-97e8-4bd2-91b0-c31be7bd9c88-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-vtd7t\" (UID: \"24021eec-97e8-4bd2-91b0-c31be7bd9c88\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-vtd7t"
Sep 11 17:19:03.154101 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:19:03.154052    2552 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 11 17:19:03.154172 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:19:03.154131    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24021eec-97e8-4bd2-91b0-c31be7bd9c88-cluster-monitoring-operator-tls podName:24021eec-97e8-4bd2-91b0-c31be7bd9c88 nodeName:}" failed. No retries permitted until 2026-09-11 17:19:07.154098178 +0000 UTC m=+41.675389701 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/24021eec-97e8-4bd2-91b0-c31be7bd9c88-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-vtd7t" (UID: "24021eec-97e8-4bd2-91b0-c31be7bd9c88") : secret "cluster-monitoring-operator-tls" not found
Sep 11 17:19:03.154900 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:19:03.154539    2552 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 11 17:19:03.154900 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:19:03.154596    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3e8c629b-c41f-4eef-9c3e-7eed516e00a7-networking-console-plugin-cert podName:3e8c629b-c41f-4eef-9c3e-7eed516e00a7 nodeName:}" failed. No retries permitted until 2026-09-11 17:19:07.154580936 +0000 UTC m=+41.675872439 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/3e8c629b-c41f-4eef-9c3e-7eed516e00a7-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-7kvsj" (UID: "3e8c629b-c41f-4eef-9c3e-7eed516e00a7") : secret "networking-console-plugin-cert" not found
Sep 11 17:19:03.154900 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:19:03.154661    2552 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 11 17:19:03.154900 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:19:03.154698    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aac7a167-8766-4c22-926e-ef15ceae304d-samples-operator-tls podName:aac7a167-8766-4c22-926e-ef15ceae304d nodeName:}" failed. No retries permitted until 2026-09-11 17:19:07.154686846 +0000 UTC m=+41.675978361 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/aac7a167-8766-4c22-926e-ef15ceae304d-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-q2zxh" (UID: "aac7a167-8766-4c22-926e-ef15ceae304d") : secret "samples-operator-tls" not found
Sep 11 17:19:03.154900 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:19:03.154750    2552 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 17:19:03.154900 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:19:03.154781    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b7d42fd1-2cb9-429b-a57a-656a5d7ecb91-metrics-tls podName:b7d42fd1-2cb9-429b-a57a-656a5d7ecb91 nodeName:}" failed. No retries permitted until 2026-09-11 17:19:07.154772077 +0000 UTC m=+41.676063580 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b7d42fd1-2cb9-429b-a57a-656a5d7ecb91-metrics-tls") pod "dns-default-6h5ct" (UID: "b7d42fd1-2cb9-429b-a57a-656a5d7ecb91") : secret "dns-default-metrics-tls" not found
Sep 11 17:19:03.241041 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:03.240983    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:19:03.341228 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:03.341142    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:19:04.787832 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:04.787787    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:19:05.030168 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:05.030129    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:19:05.310648 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:05.310611    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:19:05.412585 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:05.412553    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:19:07.090794 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:07.090755    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/65d6125e-1089-4308-bee8-064eefd92dec-registry-tls\") pod \"image-registry-86bf889f8b-5hnlq\" (UID: \"65d6125e-1089-4308-bee8-064eefd92dec\") " pod="openshift-image-registry/image-registry-86bf889f8b-5hnlq"
Sep 11 17:19:07.091221 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:07.090811    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2a34a4aa-8cb7-42f3-9f87-da690cfd8031-metrics-certs\") pod \"router-default-6f8f7958c8-2p4xx\" (UID: \"2a34a4aa-8cb7-42f3-9f87-da690cfd8031\") " pod="openshift-ingress/router-default-6f8f7958c8-2p4xx"
Sep 11 17:19:07.091221 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:07.090834    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2a34a4aa-8cb7-42f3-9f87-da690cfd8031-service-ca-bundle\") pod \"router-default-6f8f7958c8-2p4xx\" (UID: \"2a34a4aa-8cb7-42f3-9f87-da690cfd8031\") " pod="openshift-ingress/router-default-6f8f7958c8-2p4xx"
Sep 11 17:19:07.091221 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:19:07.090927    2552 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 11 17:19:07.091221 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:19:07.090953    2552 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-86bf889f8b-5hnlq: secret "image-registry-tls" not found
Sep 11 17:19:07.091221 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:19:07.090955    2552 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 11 17:19:07.091221 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:19:07.090992    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2a34a4aa-8cb7-42f3-9f87-da690cfd8031-service-ca-bundle podName:2a34a4aa-8cb7-42f3-9f87-da690cfd8031 nodeName:}" failed. No retries permitted until 2026-09-11 17:19:15.090973958 +0000 UTC m=+49.612265457 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/2a34a4aa-8cb7-42f3-9f87-da690cfd8031-service-ca-bundle") pod "router-default-6f8f7958c8-2p4xx" (UID: "2a34a4aa-8cb7-42f3-9f87-da690cfd8031") : configmap references non-existent config key: service-ca.crt
Sep 11 17:19:07.091221 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:19:07.091012    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2a34a4aa-8cb7-42f3-9f87-da690cfd8031-metrics-certs podName:2a34a4aa-8cb7-42f3-9f87-da690cfd8031 nodeName:}" failed. No retries permitted until 2026-09-11 17:19:15.091004428 +0000 UTC m=+49.612295927 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2a34a4aa-8cb7-42f3-9f87-da690cfd8031-metrics-certs") pod "router-default-6f8f7958c8-2p4xx" (UID: "2a34a4aa-8cb7-42f3-9f87-da690cfd8031") : secret "router-metrics-certs-default" not found
Sep 11 17:19:07.091221 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:19:07.091021    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/65d6125e-1089-4308-bee8-064eefd92dec-registry-tls podName:65d6125e-1089-4308-bee8-064eefd92dec nodeName:}" failed. No retries permitted until 2026-09-11 17:19:15.091015986 +0000 UTC m=+49.612307485 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/65d6125e-1089-4308-bee8-064eefd92dec-registry-tls") pod "image-registry-86bf889f8b-5hnlq" (UID: "65d6125e-1089-4308-bee8-064eefd92dec") : secret "image-registry-tls" not found
Sep 11 17:19:07.191741 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:07.191537    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3e8c629b-c41f-4eef-9c3e-7eed516e00a7-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-7kvsj\" (UID: \"3e8c629b-c41f-4eef-9c3e-7eed516e00a7\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-7kvsj"
Sep 11 17:19:07.191741 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:07.191651    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/aac7a167-8766-4c22-926e-ef15ceae304d-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-q2zxh\" (UID: \"aac7a167-8766-4c22-926e-ef15ceae304d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-q2zxh"
Sep 11 17:19:07.191741 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:07.191699    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b7d42fd1-2cb9-429b-a57a-656a5d7ecb91-metrics-tls\") pod \"dns-default-6h5ct\" (UID: \"b7d42fd1-2cb9-429b-a57a-656a5d7ecb91\") " pod="openshift-dns/dns-default-6h5ct"
Sep 11 17:19:07.191741 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:07.191728    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/24021eec-97e8-4bd2-91b0-c31be7bd9c88-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-vtd7t\" (UID: \"24021eec-97e8-4bd2-91b0-c31be7bd9c88\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-vtd7t"
Sep 11 17:19:07.192010 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:19:07.191851    2552 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 11 17:19:07.192010 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:19:07.191886    2552 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 11 17:19:07.192010 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:19:07.191908    2552 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 17:19:07.192010 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:19:07.191927    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aac7a167-8766-4c22-926e-ef15ceae304d-samples-operator-tls podName:aac7a167-8766-4c22-926e-ef15ceae304d nodeName:}" failed. No retries permitted until 2026-09-11 17:19:15.191906251 +0000 UTC m=+49.713197769 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/aac7a167-8766-4c22-926e-ef15ceae304d-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-q2zxh" (UID: "aac7a167-8766-4c22-926e-ef15ceae304d") : secret "samples-operator-tls" not found
Sep 11 17:19:07.192010 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:19:07.191851    2552 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 11 17:19:07.192010 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:19:07.191945    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b7d42fd1-2cb9-429b-a57a-656a5d7ecb91-metrics-tls podName:b7d42fd1-2cb9-429b-a57a-656a5d7ecb91 nodeName:}" failed. No retries permitted until 2026-09-11 17:19:15.191934746 +0000 UTC m=+49.713226251 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b7d42fd1-2cb9-429b-a57a-656a5d7ecb91-metrics-tls") pod "dns-default-6h5ct" (UID: "b7d42fd1-2cb9-429b-a57a-656a5d7ecb91") : secret "dns-default-metrics-tls" not found
Sep 11 17:19:07.192010 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:19:07.191961    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24021eec-97e8-4bd2-91b0-c31be7bd9c88-cluster-monitoring-operator-tls podName:24021eec-97e8-4bd2-91b0-c31be7bd9c88 nodeName:}" failed. No retries permitted until 2026-09-11 17:19:15.191952615 +0000 UTC m=+49.713244114 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/24021eec-97e8-4bd2-91b0-c31be7bd9c88-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-vtd7t" (UID: "24021eec-97e8-4bd2-91b0-c31be7bd9c88") : secret "cluster-monitoring-operator-tls" not found
Sep 11 17:19:07.192010 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:19:07.191976    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3e8c629b-c41f-4eef-9c3e-7eed516e00a7-networking-console-plugin-cert podName:3e8c629b-c41f-4eef-9c3e-7eed516e00a7 nodeName:}" failed. No retries permitted until 2026-09-11 17:19:15.191967593 +0000 UTC m=+49.713259093 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/3e8c629b-c41f-4eef-9c3e-7eed516e00a7-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-7kvsj" (UID: "3e8c629b-c41f-4eef-9c3e-7eed516e00a7") : secret "networking-console-plugin-cert" not found
Sep 11 17:19:07.286320 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:07.286281    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:19:07.366250 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:07.366159    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:19:11.145100 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:11.145066    2552 generic.go:356] "Generic (PLEG): container finished" podID="e6840164-41a5-4423-bdd0-a364186cf962" containerID="c62540708c38e8f2feca4f4b1fb58d43b42d10bb27d16fb6b55fe36e18949d92" exitCode=0
Sep 11 17:19:11.145604 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:11.145152    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-z559n" event={"ID":"e6840164-41a5-4423-bdd0-a364186cf962","Type":"ContainerDied","Data":"c62540708c38e8f2feca4f4b1fb58d43b42d10bb27d16fb6b55fe36e18949d92"}
Sep 11 17:19:11.147564 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:11.147505    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-nzjpd" event={"ID":"62af7c53-a9e7-40d3-ac2d-595a8dcfc54f","Type":"ContainerStarted","Data":"74931c50e1148749a1025f500736d2d200ec1a160457cff07e10e524f9c90205"}
Sep 11 17:19:11.149143 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:11.149084    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-gdzsp" event={"ID":"1a431afe-6ad9-4886-b0ac-8d147f276146","Type":"ContainerStarted","Data":"e40838db22480c574a5f8b4163a9157178a18c6f3e5e1a4adcfeaf9a7ead88e6"}
Sep 11 17:19:11.151164 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:11.151140    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f8b768455-ltwfd" event={"ID":"18feb296-ac22-4b0f-9b2a-5d011fd65561","Type":"ContainerStarted","Data":"0e40cd548147dc30628890429e23d96ebef4f24f275d5c5c7b0d2720576f5690"}
Sep 11 17:19:11.152914 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:11.152898    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-hg7p8_eb5f3be1-2e36-40ba-b3a0-3bec413056e8/console-operator/0.log"
Sep 11 17:19:11.153009 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:11.152931    2552 generic.go:356] "Generic (PLEG): container finished" podID="eb5f3be1-2e36-40ba-b3a0-3bec413056e8" containerID="2454b7abd2adc071639147c0ebe91736b55d0d9cea8c1160ddf3c2bd02240096" exitCode=255
Sep 11 17:19:11.153009 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:11.153000    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-hg7p8" event={"ID":"eb5f3be1-2e36-40ba-b3a0-3bec413056e8","Type":"ContainerDied","Data":"2454b7abd2adc071639147c0ebe91736b55d0d9cea8c1160ddf3c2bd02240096"}
Sep 11 17:19:11.153180 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:11.153164    2552 scope.go:117] "RemoveContainer" containerID="2454b7abd2adc071639147c0ebe91736b55d0d9cea8c1160ddf3c2bd02240096"
Sep 11 17:19:11.155905 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:11.155881    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-88bn9" event={"ID":"7583d484-6dd5-48e3-9485-05e57c62443e","Type":"ContainerStarted","Data":"e418bdaad64f64da336af4eb4c265596d5bb9125b82775b2913b7408fe3d34d0"}
Sep 11 17:19:11.157518 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:11.157497    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-pmvkp" event={"ID":"676c8e42-836e-4f84-b5b0-c1a49eb8c5f3","Type":"ContainerStarted","Data":"2144e3de6c1bf9fc393ea498ca1bfbc74ac20c543e7842955f8b137fed1ea3b3"}
Sep 11 17:19:11.159158 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:11.159136    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-9gb8k" event={"ID":"99bdc64a-0517-49d6-acf3-4e9fc5ea47fe","Type":"ContainerStarted","Data":"13c2cb75562f7f1c7ca682ccfbaacf7ca7fc85b8a8ec574bfb9d808bde1c795a"}
Sep 11 17:19:11.173712 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:11.173657    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-9gb8k"
Sep 11 17:19:11.183283 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:11.183237    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-gdzsp" podStartSLOduration=32.804696344999996 podStartE2EDuration="43.18321771s" totalImagesPullingTime="10.378521365s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:18:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:19:11.182985749 +0000 UTC m=+45.704277270" watchObservedRunningTime="2026-09-11 17:19:11.18321771 +0000 UTC m=+45.704509231"
Sep 11 17:19:11.201758 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:11.201711    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-nzjpd" podStartSLOduration=1.823965506 podStartE2EDuration="12.201696826s" totalImagesPullingTime="10.37773132s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:18:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:19:11.201629657 +0000 UTC m=+45.722921177" watchObservedRunningTime="2026-09-11 17:19:11.201696826 +0000 UTC m=+45.722988343"
Sep 11 17:19:11.259338 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:11.259285    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-6f8b768455-ltwfd" podStartSLOduration=32.898679621 podStartE2EDuration="43.259260877s" totalImagesPullingTime="10.360581256s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:18:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:19:11.23689667 +0000 UTC m=+45.758188191" watchObservedRunningTime="2026-09-11 17:19:11.259260877 +0000 UTC m=+45.780552399"
Sep 11 17:19:11.277769 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:11.277523    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-9gb8k" podStartSLOduration=36.202738855 podStartE2EDuration="45.277501442s" totalImagesPullingTime="9.074762587s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:18:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:19:11.27698788 +0000 UTC m=+45.798279402" watchObservedRunningTime="2026-09-11 17:19:11.277501442 +0000 UTC m=+45.798792966"
Sep 11 17:19:11.300032 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:11.299956    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-pmvkp" podStartSLOduration=32.917209622 podStartE2EDuration="43.299934289s" totalImagesPullingTime="10.382724667s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:18:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:19:11.299739086 +0000 UTC m=+45.821030608" watchObservedRunningTime="2026-09-11 17:19:11.299934289 +0000 UTC m=+45.821225802"
Sep 11 17:19:11.351827 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:11.351586    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5844b856db-88bn9" podStartSLOduration=32.88551673 podStartE2EDuration="43.351563952s" totalImagesPullingTime="10.466047222s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:18:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:19:11.349253124 +0000 UTC m=+45.870544644" watchObservedRunningTime="2026-09-11 17:19:11.351563952 +0000 UTC m=+45.872855474"
Sep 11 17:19:12.043756 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:12.043719    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-99f6z"]
Sep 11 17:19:12.048147 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:12.048100    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:19:12.063481 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:12.063452    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-99f6z"
Sep 11 17:19:12.064423 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:12.064397    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-99f6z"]
Sep 11 17:19:12.068432 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:12.068372    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 11 17:19:12.068767 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:12.068577    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-p99b8\""
Sep 11 17:19:12.068767 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:12.068634    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 11 17:19:12.137667 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:12.137629    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/3f805648-71b2-435c-9942-1701f83fe7a3-crio-socket\") pod \"insights-runtime-extractor-99f6z\" (UID: \"3f805648-71b2-435c-9942-1701f83fe7a3\") " pod="openshift-insights/insights-runtime-extractor-99f6z"
Sep 11 17:19:12.137843 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:12.137669    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/3f805648-71b2-435c-9942-1701f83fe7a3-data-volume\") pod \"insights-runtime-extractor-99f6z\" (UID: \"3f805648-71b2-435c-9942-1701f83fe7a3\") " pod="openshift-insights/insights-runtime-extractor-99f6z"
Sep 11 17:19:12.137843 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:12.137807    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3f805648-71b2-435c-9942-1701f83fe7a3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-99f6z\" (UID: \"3f805648-71b2-435c-9942-1701f83fe7a3\") " pod="openshift-insights/insights-runtime-extractor-99f6z"
Sep 11 17:19:12.137946 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:12.137854    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3f805648-71b2-435c-9942-1701f83fe7a3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-99f6z\" (UID: \"3f805648-71b2-435c-9942-1701f83fe7a3\") " pod="openshift-insights/insights-runtime-extractor-99f6z"
Sep 11 17:19:12.137946 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:12.137933    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-whvkh\" (UniqueName: \"kubernetes.io/projected/3f805648-71b2-435c-9942-1701f83fe7a3-kube-api-access-whvkh\") pod \"insights-runtime-extractor-99f6z\" (UID: \"3f805648-71b2-435c-9942-1701f83fe7a3\") " pod="openshift-insights/insights-runtime-extractor-99f6z"
Sep 11 17:19:12.165018 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:12.164991    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-hg7p8_eb5f3be1-2e36-40ba-b3a0-3bec413056e8/console-operator/1.log"
Sep 11 17:19:12.165481 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:12.165467    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-hg7p8_eb5f3be1-2e36-40ba-b3a0-3bec413056e8/console-operator/0.log"
Sep 11 17:19:12.165538 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:12.165502    2552 generic.go:356] "Generic (PLEG): container finished" podID="eb5f3be1-2e36-40ba-b3a0-3bec413056e8" containerID="71d5dc7d53a1930226de81bcec2343f11d64510aed37865e8b006149fd5704f4" exitCode=255
Sep 11 17:19:12.165620 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:12.165592    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-hg7p8" event={"ID":"eb5f3be1-2e36-40ba-b3a0-3bec413056e8","Type":"ContainerDied","Data":"71d5dc7d53a1930226de81bcec2343f11d64510aed37865e8b006149fd5704f4"}
Sep 11 17:19:12.165675 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:12.165661    2552 scope.go:117] "RemoveContainer" containerID="2454b7abd2adc071639147c0ebe91736b55d0d9cea8c1160ddf3c2bd02240096"
Sep 11 17:19:12.165966 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:12.165947    2552 scope.go:117] "RemoveContainer" containerID="71d5dc7d53a1930226de81bcec2343f11d64510aed37865e8b006149fd5704f4"
Sep 11 17:19:12.166339 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:19:12.166312    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-696884fb9c-hg7p8_openshift-console-operator(eb5f3be1-2e36-40ba-b3a0-3bec413056e8)\"" pod="openshift-console-operator/console-operator-696884fb9c-hg7p8" podUID="eb5f3be1-2e36-40ba-b3a0-3bec413056e8"
Sep 11 17:19:12.168162 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:12.168134    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:19:12.169756 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:12.169733    2552 generic.go:356] "Generic (PLEG): container finished" podID="e6840164-41a5-4423-bdd0-a364186cf962" containerID="9bf16285f8ee6aeb359e3303af54ffa21ca52df23f88c4457e672a3863c5b33f" exitCode=0
Sep 11 17:19:12.169861 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:12.169831    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-z559n" event={"ID":"e6840164-41a5-4423-bdd0-a364186cf962","Type":"ContainerDied","Data":"9bf16285f8ee6aeb359e3303af54ffa21ca52df23f88c4457e672a3863c5b33f"}
Sep 11 17:19:12.239287 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:12.239248    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/3f805648-71b2-435c-9942-1701f83fe7a3-crio-socket\") pod \"insights-runtime-extractor-99f6z\" (UID: \"3f805648-71b2-435c-9942-1701f83fe7a3\") " pod="openshift-insights/insights-runtime-extractor-99f6z"
Sep 11 17:19:12.239287 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:12.239293    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/3f805648-71b2-435c-9942-1701f83fe7a3-data-volume\") pod \"insights-runtime-extractor-99f6z\" (UID: \"3f805648-71b2-435c-9942-1701f83fe7a3\") " pod="openshift-insights/insights-runtime-extractor-99f6z"
Sep 11 17:19:12.239525 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:12.239510    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/3f805648-71b2-435c-9942-1701f83fe7a3-crio-socket\") pod \"insights-runtime-extractor-99f6z\" (UID: \"3f805648-71b2-435c-9942-1701f83fe7a3\") " pod="openshift-insights/insights-runtime-extractor-99f6z"
Sep 11 17:19:12.239655 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:12.239631    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/3f805648-71b2-435c-9942-1701f83fe7a3-data-volume\") pod \"insights-runtime-extractor-99f6z\" (UID: \"3f805648-71b2-435c-9942-1701f83fe7a3\") " pod="openshift-insights/insights-runtime-extractor-99f6z"
Sep 11 17:19:12.240184 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:12.240157    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3f805648-71b2-435c-9942-1701f83fe7a3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-99f6z\" (UID: \"3f805648-71b2-435c-9942-1701f83fe7a3\") " pod="openshift-insights/insights-runtime-extractor-99f6z"
Sep 11 17:19:12.240298 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:12.240222    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3f805648-71b2-435c-9942-1701f83fe7a3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-99f6z\" (UID: \"3f805648-71b2-435c-9942-1701f83fe7a3\") " pod="openshift-insights/insights-runtime-extractor-99f6z"
Sep 11 17:19:12.241527 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:12.240777    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-whvkh\" (UniqueName: \"kubernetes.io/projected/3f805648-71b2-435c-9942-1701f83fe7a3-kube-api-access-whvkh\") pod \"insights-runtime-extractor-99f6z\" (UID: \"3f805648-71b2-435c-9942-1701f83fe7a3\") " pod="openshift-insights/insights-runtime-extractor-99f6z"
Sep 11 17:19:12.241527 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:19:12.240837    2552 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 17:19:12.241527 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:19:12.240903    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f805648-71b2-435c-9942-1701f83fe7a3-insights-runtime-extractor-tls podName:3f805648-71b2-435c-9942-1701f83fe7a3 nodeName:}" failed. No retries permitted until 2026-09-11 17:19:12.740875538 +0000 UTC m=+47.262167054 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3f805648-71b2-435c-9942-1701f83fe7a3-insights-runtime-extractor-tls") pod "insights-runtime-extractor-99f6z" (UID: "3f805648-71b2-435c-9942-1701f83fe7a3") : secret "insights-runtime-extractor-tls" not found
Sep 11 17:19:12.241527 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:19:12.241311    2552 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 11 17:19:12.241527 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:19:12.241374    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3f805648-71b2-435c-9942-1701f83fe7a3-kube-rbac-proxy-cm podName:3f805648-71b2-435c-9942-1701f83fe7a3 nodeName:}" failed. No retries permitted until 2026-09-11 17:19:12.741356279 +0000 UTC m=+47.262647792 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/3f805648-71b2-435c-9942-1701f83fe7a3-kube-rbac-proxy-cm") pod "insights-runtime-extractor-99f6z" (UID: "3f805648-71b2-435c-9942-1701f83fe7a3") : configmap "kube-rbac-proxy" not found
Sep 11 17:19:12.249953 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:12.249931    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-whvkh\" (UniqueName: \"kubernetes.io/projected/3f805648-71b2-435c-9942-1701f83fe7a3-kube-api-access-whvkh\") pod \"insights-runtime-extractor-99f6z\" (UID: \"3f805648-71b2-435c-9942-1701f83fe7a3\") " pod="openshift-insights/insights-runtime-extractor-99f6z"
Sep 11 17:19:12.266337 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:12.266307    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:19:12.388826 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:12.388777    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:19:12.405935 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:12.405904    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-8b4d697c5-f5bc2"]
Sep 11 17:19:12.425430 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:12.425402    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-f5bc2"
Sep 11 17:19:12.426501 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:12.426476    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-8b4d697c5-f5bc2"]
Sep 11 17:19:12.427886 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:12.427867    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-d4dhf\""
Sep 11 17:19:12.427996 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:12.427914    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Sep 11 17:19:12.428240 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:12.428221    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Sep 11 17:19:12.442965 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:12.442941    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5cxkm\" (UniqueName: \"kubernetes.io/projected/c41df48e-7ab3-41d3-a549-ddd96b123d99-kube-api-access-5cxkm\") pod \"migrator-8b4d697c5-f5bc2\" (UID: \"c41df48e-7ab3-41d3-a549-ddd96b123d99\") " pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-f5bc2"
Sep 11 17:19:12.544318 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:12.544277    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5cxkm\" (UniqueName: \"kubernetes.io/projected/c41df48e-7ab3-41d3-a549-ddd96b123d99-kube-api-access-5cxkm\") pod \"migrator-8b4d697c5-f5bc2\" (UID: \"c41df48e-7ab3-41d3-a549-ddd96b123d99\") " pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-f5bc2"
Sep 11 17:19:12.555826 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:12.555747    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5cxkm\" (UniqueName: \"kubernetes.io/projected/c41df48e-7ab3-41d3-a549-ddd96b123d99-kube-api-access-5cxkm\") pod \"migrator-8b4d697c5-f5bc2\" (UID: \"c41df48e-7ab3-41d3-a549-ddd96b123d99\") " pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-f5bc2"
Sep 11 17:19:12.736595 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:12.736557    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-f5bc2"
Sep 11 17:19:12.746381 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:12.746353    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3f805648-71b2-435c-9942-1701f83fe7a3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-99f6z\" (UID: \"3f805648-71b2-435c-9942-1701f83fe7a3\") " pod="openshift-insights/insights-runtime-extractor-99f6z"
Sep 11 17:19:12.746460 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:12.746394    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3f805648-71b2-435c-9942-1701f83fe7a3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-99f6z\" (UID: \"3f805648-71b2-435c-9942-1701f83fe7a3\") " pod="openshift-insights/insights-runtime-extractor-99f6z"
Sep 11 17:19:12.746505 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:19:12.746483    2552 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 11 17:19:12.746540 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:19:12.746508    2552 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 17:19:12.746572 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:19:12.746560    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3f805648-71b2-435c-9942-1701f83fe7a3-kube-rbac-proxy-cm podName:3f805648-71b2-435c-9942-1701f83fe7a3 nodeName:}" failed. No retries permitted until 2026-09-11 17:19:13.746544575 +0000 UTC m=+48.267836093 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/3f805648-71b2-435c-9942-1701f83fe7a3-kube-rbac-proxy-cm") pod "insights-runtime-extractor-99f6z" (UID: "3f805648-71b2-435c-9942-1701f83fe7a3") : configmap "kube-rbac-proxy" not found
Sep 11 17:19:12.746618 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:19:12.746585    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f805648-71b2-435c-9942-1701f83fe7a3-insights-runtime-extractor-tls podName:3f805648-71b2-435c-9942-1701f83fe7a3 nodeName:}" failed. No retries permitted until 2026-09-11 17:19:13.746570087 +0000 UTC m=+48.267861585 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3f805648-71b2-435c-9942-1701f83fe7a3-insights-runtime-extractor-tls") pod "insights-runtime-extractor-99f6z" (UID: "3f805648-71b2-435c-9942-1701f83fe7a3") : secret "insights-runtime-extractor-tls" not found
Sep 11 17:19:12.958134 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:12.958087    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:19:13.050567 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:13.050545    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-8b4d697c5-f5bc2"]
Sep 11 17:19:13.052797 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:19:13.052769    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc41df48e_7ab3_41d3_a549_ddd96b123d99.slice/crio-bcb248971fc26e381e10d9db7ab05407b4b22d19582b4f20d4edc3567569db88 WatchSource:0}: Error finding container bcb248971fc26e381e10d9db7ab05407b4b22d19582b4f20d4edc3567569db88: Status 404 returned error can't find the container with id bcb248971fc26e381e10d9db7ab05407b4b22d19582b4f20d4edc3567569db88
Sep 11 17:19:13.100346 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:13.100316    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:19:13.177679 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:13.177640    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-z559n" event={"ID":"e6840164-41a5-4423-bdd0-a364186cf962","Type":"ContainerStarted","Data":"614f60f3ff175732f755f7c0e0abd4eae3647781a2738d9ee8fea3598fc656ee"}
Sep 11 17:19:13.178832 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:13.178806    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-f5bc2" event={"ID":"c41df48e-7ab3-41d3-a549-ddd96b123d99","Type":"ContainerStarted","Data":"bcb248971fc26e381e10d9db7ab05407b4b22d19582b4f20d4edc3567569db88"}
Sep 11 17:19:13.180301 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:13.180282    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-hg7p8_eb5f3be1-2e36-40ba-b3a0-3bec413056e8/console-operator/1.log"
Sep 11 17:19:13.180564 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:13.180551    2552 scope.go:117] "RemoveContainer" containerID="71d5dc7d53a1930226de81bcec2343f11d64510aed37865e8b006149fd5704f4"
Sep 11 17:19:13.180771 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:19:13.180754    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-696884fb9c-hg7p8_openshift-console-operator(eb5f3be1-2e36-40ba-b3a0-3bec413056e8)\"" pod="openshift-console-operator/console-operator-696884fb9c-hg7p8" podUID="eb5f3be1-2e36-40ba-b3a0-3bec413056e8"
Sep 11 17:19:13.226165 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:13.226063    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-z559n" podStartSLOduration=13.365762873 podStartE2EDuration="47.226045642s" totalImagesPullingTime="33.860282769s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 17:18:26 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:19:13.21993491 +0000 UTC m=+47.741226431" watchObservedRunningTime="2026-09-11 17:19:13.226045642 +0000 UTC m=+47.747337204"
Sep 11 17:19:13.271195 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:13.271169    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-h6lht_8b4c1f98-046f-47c4-889f-c2bd807dddbe/dns-node-resolver/0.log"
Sep 11 17:19:13.432079 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:13.432042    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:19:13.575289 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:13.575202    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:19:13.689700 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:13.689664    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:19:13.756815 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:13.756783    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3f805648-71b2-435c-9942-1701f83fe7a3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-99f6z\" (UID: \"3f805648-71b2-435c-9942-1701f83fe7a3\") " pod="openshift-insights/insights-runtime-extractor-99f6z"
Sep 11 17:19:13.756815 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:13.756828    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3f805648-71b2-435c-9942-1701f83fe7a3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-99f6z\" (UID: \"3f805648-71b2-435c-9942-1701f83fe7a3\") " pod="openshift-insights/insights-runtime-extractor-99f6z"
Sep 11 17:19:13.757066 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:19:13.756911    2552 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 11 17:19:13.757066 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:19:13.756968    2552 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 17:19:13.757066 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:19:13.756978    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3f805648-71b2-435c-9942-1701f83fe7a3-kube-rbac-proxy-cm podName:3f805648-71b2-435c-9942-1701f83fe7a3 nodeName:}" failed. No retries permitted until 2026-09-11 17:19:15.756962739 +0000 UTC m=+50.278254239 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/3f805648-71b2-435c-9942-1701f83fe7a3-kube-rbac-proxy-cm") pod "insights-runtime-extractor-99f6z" (UID: "3f805648-71b2-435c-9942-1701f83fe7a3") : configmap "kube-rbac-proxy" not found
Sep 11 17:19:13.757066 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:19:13.757019    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f805648-71b2-435c-9942-1701f83fe7a3-insights-runtime-extractor-tls podName:3f805648-71b2-435c-9942-1701f83fe7a3 nodeName:}" failed. No retries permitted until 2026-09-11 17:19:15.757003129 +0000 UTC m=+50.278294631 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3f805648-71b2-435c-9942-1701f83fe7a3-insights-runtime-extractor-tls") pod "insights-runtime-extractor-99f6z" (UID: "3f805648-71b2-435c-9942-1701f83fe7a3") : secret "insights-runtime-extractor-tls" not found
Sep 11 17:19:13.785323 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:13.785286    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:19:14.268298 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:14.268266    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-gbf8j_d0ec6f78-bdc6-420a-a6aa-953d61643e27/node-ca/0.log"
Sep 11 17:19:14.687529 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:14.687496    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:19:14.805001 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:14.804918    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:19:14.872312 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:14.872279    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-nzjpd_62af7c53-a9e7-40d3-ac2d-595a8dcfc54f/serve-healthcheck-canary/0.log"
Sep 11 17:19:15.168154 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:15.168098    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/65d6125e-1089-4308-bee8-064eefd92dec-registry-tls\") pod \"image-registry-86bf889f8b-5hnlq\" (UID: \"65d6125e-1089-4308-bee8-064eefd92dec\") " pod="openshift-image-registry/image-registry-86bf889f8b-5hnlq"
Sep 11 17:19:15.168353 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:15.168202    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2a34a4aa-8cb7-42f3-9f87-da690cfd8031-metrics-certs\") pod \"router-default-6f8f7958c8-2p4xx\" (UID: \"2a34a4aa-8cb7-42f3-9f87-da690cfd8031\") " pod="openshift-ingress/router-default-6f8f7958c8-2p4xx"
Sep 11 17:19:15.168353 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:15.168237    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2a34a4aa-8cb7-42f3-9f87-da690cfd8031-service-ca-bundle\") pod \"router-default-6f8f7958c8-2p4xx\" (UID: \"2a34a4aa-8cb7-42f3-9f87-da690cfd8031\") " pod="openshift-ingress/router-default-6f8f7958c8-2p4xx"
Sep 11 17:19:15.168353 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:19:15.168267    2552 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 11 17:19:15.168353 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:19:15.168288    2552 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-86bf889f8b-5hnlq: secret "image-registry-tls" not found
Sep 11 17:19:15.168353 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:19:15.168350    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/65d6125e-1089-4308-bee8-064eefd92dec-registry-tls podName:65d6125e-1089-4308-bee8-064eefd92dec nodeName:}" failed. No retries permitted until 2026-09-11 17:19:31.168331315 +0000 UTC m=+65.689622829 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/65d6125e-1089-4308-bee8-064eefd92dec-registry-tls") pod "image-registry-86bf889f8b-5hnlq" (UID: "65d6125e-1089-4308-bee8-064eefd92dec") : secret "image-registry-tls" not found
Sep 11 17:19:15.168570 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:19:15.168362    2552 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 11 17:19:15.168570 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:19:15.168414    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2a34a4aa-8cb7-42f3-9f87-da690cfd8031-metrics-certs podName:2a34a4aa-8cb7-42f3-9f87-da690cfd8031 nodeName:}" failed. No retries permitted until 2026-09-11 17:19:31.16840183 +0000 UTC m=+65.689693329 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2a34a4aa-8cb7-42f3-9f87-da690cfd8031-metrics-certs") pod "router-default-6f8f7958c8-2p4xx" (UID: "2a34a4aa-8cb7-42f3-9f87-da690cfd8031") : secret "router-metrics-certs-default" not found
Sep 11 17:19:15.168570 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:19:15.168438    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2a34a4aa-8cb7-42f3-9f87-da690cfd8031-service-ca-bundle podName:2a34a4aa-8cb7-42f3-9f87-da690cfd8031 nodeName:}" failed. No retries permitted until 2026-09-11 17:19:31.168429352 +0000 UTC m=+65.689720850 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/2a34a4aa-8cb7-42f3-9f87-da690cfd8031-service-ca-bundle") pod "router-default-6f8f7958c8-2p4xx" (UID: "2a34a4aa-8cb7-42f3-9f87-da690cfd8031") : configmap references non-existent config key: service-ca.crt
Sep 11 17:19:15.269608 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:15.269570    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/aac7a167-8766-4c22-926e-ef15ceae304d-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-q2zxh\" (UID: \"aac7a167-8766-4c22-926e-ef15ceae304d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-q2zxh"
Sep 11 17:19:15.270030 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:15.269623    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b7d42fd1-2cb9-429b-a57a-656a5d7ecb91-metrics-tls\") pod \"dns-default-6h5ct\" (UID: \"b7d42fd1-2cb9-429b-a57a-656a5d7ecb91\") " pod="openshift-dns/dns-default-6h5ct"
Sep 11 17:19:15.270030 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:15.269644    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/24021eec-97e8-4bd2-91b0-c31be7bd9c88-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-vtd7t\" (UID: \"24021eec-97e8-4bd2-91b0-c31be7bd9c88\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-vtd7t"
Sep 11 17:19:15.270030 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:19:15.269743    2552 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 11 17:19:15.270030 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:19:15.269760    2552 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 11 17:19:15.270030 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:19:15.269769    2552 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 17:19:15.270030 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:15.269807    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3e8c629b-c41f-4eef-9c3e-7eed516e00a7-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-7kvsj\" (UID: \"3e8c629b-c41f-4eef-9c3e-7eed516e00a7\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-7kvsj"
Sep 11 17:19:15.270030 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:19:15.269815    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24021eec-97e8-4bd2-91b0-c31be7bd9c88-cluster-monitoring-operator-tls podName:24021eec-97e8-4bd2-91b0-c31be7bd9c88 nodeName:}" failed. No retries permitted until 2026-09-11 17:19:31.26980127 +0000 UTC m=+65.791092770 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/24021eec-97e8-4bd2-91b0-c31be7bd9c88-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-vtd7t" (UID: "24021eec-97e8-4bd2-91b0-c31be7bd9c88") : secret "cluster-monitoring-operator-tls" not found
Sep 11 17:19:15.270030 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:19:15.269849    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b7d42fd1-2cb9-429b-a57a-656a5d7ecb91-metrics-tls podName:b7d42fd1-2cb9-429b-a57a-656a5d7ecb91 nodeName:}" failed. No retries permitted until 2026-09-11 17:19:31.269832739 +0000 UTC m=+65.791124245 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b7d42fd1-2cb9-429b-a57a-656a5d7ecb91-metrics-tls") pod "dns-default-6h5ct" (UID: "b7d42fd1-2cb9-429b-a57a-656a5d7ecb91") : secret "dns-default-metrics-tls" not found
Sep 11 17:19:15.270030 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:19:15.269863    2552 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 11 17:19:15.270030 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:19:15.269866    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aac7a167-8766-4c22-926e-ef15ceae304d-samples-operator-tls podName:aac7a167-8766-4c22-926e-ef15ceae304d nodeName:}" failed. No retries permitted until 2026-09-11 17:19:31.269859447 +0000 UTC m=+65.791150945 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/aac7a167-8766-4c22-926e-ef15ceae304d-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-q2zxh" (UID: "aac7a167-8766-4c22-926e-ef15ceae304d") : secret "samples-operator-tls" not found
Sep 11 17:19:15.270030 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:19:15.269888    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3e8c629b-c41f-4eef-9c3e-7eed516e00a7-networking-console-plugin-cert podName:3e8c629b-c41f-4eef-9c3e-7eed516e00a7 nodeName:}" failed. No retries permitted until 2026-09-11 17:19:31.269881098 +0000 UTC m=+65.791172597 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/3e8c629b-c41f-4eef-9c3e-7eed516e00a7-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-7kvsj" (UID: "3e8c629b-c41f-4eef-9c3e-7eed516e00a7") : secret "networking-console-plugin-cert" not found
Sep 11 17:19:15.773930 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:15.773892    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3f805648-71b2-435c-9942-1701f83fe7a3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-99f6z\" (UID: \"3f805648-71b2-435c-9942-1701f83fe7a3\") " pod="openshift-insights/insights-runtime-extractor-99f6z"
Sep 11 17:19:15.774097 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:15.773937    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3f805648-71b2-435c-9942-1701f83fe7a3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-99f6z\" (UID: \"3f805648-71b2-435c-9942-1701f83fe7a3\") " pod="openshift-insights/insights-runtime-extractor-99f6z"
Sep 11 17:19:15.774182 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:19:15.774126    2552 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 17:19:15.774221 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:19:15.774210    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f805648-71b2-435c-9942-1701f83fe7a3-insights-runtime-extractor-tls podName:3f805648-71b2-435c-9942-1701f83fe7a3 nodeName:}" failed. No retries permitted until 2026-09-11 17:19:19.77418776 +0000 UTC m=+54.295479278 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3f805648-71b2-435c-9942-1701f83fe7a3-insights-runtime-extractor-tls") pod "insights-runtime-extractor-99f6z" (UID: "3f805648-71b2-435c-9942-1701f83fe7a3") : secret "insights-runtime-extractor-tls" not found
Sep 11 17:19:15.774609 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:15.774589    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3f805648-71b2-435c-9942-1701f83fe7a3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-99f6z\" (UID: \"3f805648-71b2-435c-9942-1701f83fe7a3\") " pod="openshift-insights/insights-runtime-extractor-99f6z"
Sep 11 17:19:15.858182 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:15.858157    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:19:16.194760 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:16.194722    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-f5bc2" event={"ID":"c41df48e-7ab3-41d3-a549-ddd96b123d99","Type":"ContainerStarted","Data":"c9c0844296fdc1955c27ca178f0f060e747509e4836cf788876e9fe603affc14"}
Sep 11 17:19:16.195023 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:16.194774    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-f5bc2" event={"ID":"c41df48e-7ab3-41d3-a549-ddd96b123d99","Type":"ContainerStarted","Data":"6b6948038010667bcf96b1526884b1e68c903671ebca9f8040ed820a8d8b3200"}
Sep 11 17:19:16.220967 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:16.220917    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-f5bc2" podStartSLOduration=1.587219246 podStartE2EDuration="4.220900969s" totalImagesPullingTime="2.633681723s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:19:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:19:16.21969257 +0000 UTC m=+50.740984091" watchObservedRunningTime="2026-09-11 17:19:16.220900969 +0000 UTC m=+50.742192633"
Sep 11 17:19:16.616762 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:16.616733    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:19:18.583435 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:18.583406    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:19:18.680559 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:18.680524    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:19:19.697818 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:19.697771    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-696884fb9c-hg7p8"
Sep 11 17:19:19.698244 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:19.697875    2552 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-696884fb9c-hg7p8"
Sep 11 17:19:19.698294 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:19.698243    2552 scope.go:117] "RemoveContainer" containerID="71d5dc7d53a1930226de81bcec2343f11d64510aed37865e8b006149fd5704f4"
Sep 11 17:19:19.698405 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:19:19.698386    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-696884fb9c-hg7p8_openshift-console-operator(eb5f3be1-2e36-40ba-b3a0-3bec413056e8)\"" pod="openshift-console-operator/console-operator-696884fb9c-hg7p8" podUID="eb5f3be1-2e36-40ba-b3a0-3bec413056e8"
Sep 11 17:19:19.810260 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:19.810222    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3f805648-71b2-435c-9942-1701f83fe7a3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-99f6z\" (UID: \"3f805648-71b2-435c-9942-1701f83fe7a3\") " pod="openshift-insights/insights-runtime-extractor-99f6z"
Sep 11 17:19:19.810399 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:19:19.810321    2552 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 17:19:19.810399 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:19:19.810376    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f805648-71b2-435c-9942-1701f83fe7a3-insights-runtime-extractor-tls podName:3f805648-71b2-435c-9942-1701f83fe7a3 nodeName:}" failed. No retries permitted until 2026-09-11 17:19:27.81036245 +0000 UTC m=+62.331653954 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3f805648-71b2-435c-9942-1701f83fe7a3-insights-runtime-extractor-tls") pod "insights-runtime-extractor-99f6z" (UID: "3f805648-71b2-435c-9942-1701f83fe7a3") : secret "insights-runtime-extractor-tls" not found
Sep 11 17:19:21.021561 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:21.021522    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:19:21.882803 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:21.882768    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:19:22.130336 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:22.130298    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:19:22.240669 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:22.240576    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:19:22.596471 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:22.596437    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:19:22.903501 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:22.903413    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:19:23.023169 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:23.023138    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:19:23.531229 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:23.531198    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:19:23.752772 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:23.752728    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:19:24.187102 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:24.187071    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:19:24.559736 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:24.559648    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:19:24.930424 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:24.930390    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:19:25.390357 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:25.390329    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:19:25.540479 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:25.540446    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:19:25.671256 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:25.671172    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:19:25.768056 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:25.768025    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:19:26.141834 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:26.141800    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:19:26.511646 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:26.511622    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-6k9tr"
Sep 11 17:19:26.601725 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:26.601691    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:19:26.803433 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:26.803349    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:19:26.895942 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:26.895916    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:19:27.147067 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:27.147032    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:19:27.880078 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:27.880036    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3f805648-71b2-435c-9942-1701f83fe7a3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-99f6z\" (UID: \"3f805648-71b2-435c-9942-1701f83fe7a3\") " pod="openshift-insights/insights-runtime-extractor-99f6z"
Sep 11 17:19:27.882290 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:27.882273    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3f805648-71b2-435c-9942-1701f83fe7a3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-99f6z\" (UID: \"3f805648-71b2-435c-9942-1701f83fe7a3\") " pod="openshift-insights/insights-runtime-extractor-99f6z"
Sep 11 17:19:27.961480 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:27.961444    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:19:27.976848 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:27.976823    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-p99b8\""
Sep 11 17:19:27.984800 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:27.984778    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-99f6z"
Sep 11 17:19:28.150592 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:28.150497    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:19:28.246530 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:28.246498    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:19:28.272848 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:28.272818    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-99f6z"]
Sep 11 17:19:28.275924 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:19:28.275899    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f805648_71b2_435c_9942_1701f83fe7a3.slice/crio-9d03c3093c1cf2a564151a5c5d8994b16f98e710cba0174f900f9116d0381b08 WatchSource:0}: Error finding container 9d03c3093c1cf2a564151a5c5d8994b16f98e710cba0174f900f9116d0381b08: Status 404 returned error can't find the container with id 9d03c3093c1cf2a564151a5c5d8994b16f98e710cba0174f900f9116d0381b08
Sep 11 17:19:28.406348 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:28.406267    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:19:28.579438 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:28.579405    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:19:28.769513 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:28.769429    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:19:28.940542 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:28.940502    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:19:29.236643 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:29.236611    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-99f6z" event={"ID":"3f805648-71b2-435c-9942-1701f83fe7a3","Type":"ContainerStarted","Data":"e5ae6826edf104682df035c87a112353cada16511a5c6e0cc2aa92bb8bd25750"}
Sep 11 17:19:29.236804 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:29.236664    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-99f6z" event={"ID":"3f805648-71b2-435c-9942-1701f83fe7a3","Type":"ContainerStarted","Data":"9d03c3093c1cf2a564151a5c5d8994b16f98e710cba0174f900f9116d0381b08"}
Sep 11 17:19:29.735888 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:29.735849    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:19:30.242782 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:30.242748    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-99f6z" event={"ID":"3f805648-71b2-435c-9942-1701f83fe7a3","Type":"ContainerStarted","Data":"bbebe7056d2b5015cf40a331aec9f2c294ac8ceb5c704bee824ee07747e774bd"}
Sep 11 17:19:30.329860 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:30.329825    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:19:30.526281 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:30.526196    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:19:30.730325 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:30.730281    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:19:30.869107 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:30.869033    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:19:30.975756 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:30.975721    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:19:31.211971 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:31.211888    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/65d6125e-1089-4308-bee8-064eefd92dec-registry-tls\") pod \"image-registry-86bf889f8b-5hnlq\" (UID: \"65d6125e-1089-4308-bee8-064eefd92dec\") " pod="openshift-image-registry/image-registry-86bf889f8b-5hnlq"
Sep 11 17:19:31.211971 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:31.211947    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2a34a4aa-8cb7-42f3-9f87-da690cfd8031-metrics-certs\") pod \"router-default-6f8f7958c8-2p4xx\" (UID: \"2a34a4aa-8cb7-42f3-9f87-da690cfd8031\") " pod="openshift-ingress/router-default-6f8f7958c8-2p4xx"
Sep 11 17:19:31.212231 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:31.212058    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2a34a4aa-8cb7-42f3-9f87-da690cfd8031-service-ca-bundle\") pod \"router-default-6f8f7958c8-2p4xx\" (UID: \"2a34a4aa-8cb7-42f3-9f87-da690cfd8031\") " pod="openshift-ingress/router-default-6f8f7958c8-2p4xx"
Sep 11 17:19:31.212678 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:31.212656    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2a34a4aa-8cb7-42f3-9f87-da690cfd8031-service-ca-bundle\") pod \"router-default-6f8f7958c8-2p4xx\" (UID: \"2a34a4aa-8cb7-42f3-9f87-da690cfd8031\") " pod="openshift-ingress/router-default-6f8f7958c8-2p4xx"
Sep 11 17:19:31.214383 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:31.214356    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/65d6125e-1089-4308-bee8-064eefd92dec-registry-tls\") pod \"image-registry-86bf889f8b-5hnlq\" (UID: \"65d6125e-1089-4308-bee8-064eefd92dec\") " pod="openshift-image-registry/image-registry-86bf889f8b-5hnlq"
Sep 11 17:19:31.214383 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:31.214377    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2a34a4aa-8cb7-42f3-9f87-da690cfd8031-metrics-certs\") pod \"router-default-6f8f7958c8-2p4xx\" (UID: \"2a34a4aa-8cb7-42f3-9f87-da690cfd8031\") " pod="openshift-ingress/router-default-6f8f7958c8-2p4xx"
Sep 11 17:19:31.313120 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:31.313086    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3e8c629b-c41f-4eef-9c3e-7eed516e00a7-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-7kvsj\" (UID: \"3e8c629b-c41f-4eef-9c3e-7eed516e00a7\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-7kvsj"
Sep 11 17:19:31.313504 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:31.313196    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/aac7a167-8766-4c22-926e-ef15ceae304d-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-q2zxh\" (UID: \"aac7a167-8766-4c22-926e-ef15ceae304d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-q2zxh"
Sep 11 17:19:31.313504 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:31.313224    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b7d42fd1-2cb9-429b-a57a-656a5d7ecb91-metrics-tls\") pod \"dns-default-6h5ct\" (UID: \"b7d42fd1-2cb9-429b-a57a-656a5d7ecb91\") " pod="openshift-dns/dns-default-6h5ct"
Sep 11 17:19:31.313504 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:31.313242    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/24021eec-97e8-4bd2-91b0-c31be7bd9c88-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-vtd7t\" (UID: \"24021eec-97e8-4bd2-91b0-c31be7bd9c88\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-vtd7t"
Sep 11 17:19:31.315634 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:31.315606    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/aac7a167-8766-4c22-926e-ef15ceae304d-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-q2zxh\" (UID: \"aac7a167-8766-4c22-926e-ef15ceae304d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-q2zxh"
Sep 11 17:19:31.315732 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:31.315606    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3e8c629b-c41f-4eef-9c3e-7eed516e00a7-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-7kvsj\" (UID: \"3e8c629b-c41f-4eef-9c3e-7eed516e00a7\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-7kvsj"
Sep 11 17:19:31.315732 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:31.315670    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/24021eec-97e8-4bd2-91b0-c31be7bd9c88-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-vtd7t\" (UID: \"24021eec-97e8-4bd2-91b0-c31be7bd9c88\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-vtd7t"
Sep 11 17:19:31.315732 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:31.315726    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b7d42fd1-2cb9-429b-a57a-656a5d7ecb91-metrics-tls\") pod \"dns-default-6h5ct\" (UID: \"b7d42fd1-2cb9-429b-a57a-656a5d7ecb91\") " pod="openshift-dns/dns-default-6h5ct"
Sep 11 17:19:31.422387 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:31.422358    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-w4wtw\""
Sep 11 17:19:31.430175 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:31.430156    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-86bf889f8b-5hnlq"
Sep 11 17:19:31.452129 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:31.452093    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-8qq6g\""
Sep 11 17:19:31.459296 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:31.459278    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-6f8f7958c8-2p4xx"
Sep 11 17:19:31.471144 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:31.471102    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-bj8s9\""
Sep 11 17:19:31.477788 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:31.477639    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-5zr8w\""
Sep 11 17:19:31.479346 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:31.479206    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-6h5ct"
Sep 11 17:19:31.485933 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:31.485803    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-vtd7t"
Sep 11 17:19:31.493603 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:31.493391    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-xtqnr\""
Sep 11 17:19:31.502285 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:31.502233    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-q2zxh"
Sep 11 17:19:31.518124 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:31.517785    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-ft2vj\""
Sep 11 17:19:31.526923 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:31.526537    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-7kvsj"
Sep 11 17:19:31.616643 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:31.616611    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b2b1c077-b5b8-4bdc-a1b3-6b98a0bcf6a0-metrics-certs\") pod \"network-metrics-daemon-d5jqp\" (UID: \"b2b1c077-b5b8-4bdc-a1b3-6b98a0bcf6a0\") " pod="openshift-multus/network-metrics-daemon-d5jqp"
Sep 11 17:19:31.616750 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:31.616659    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/72eae49b-cb97-4819-aaf5-5c6eb23ce9c8-original-pull-secret\") pod \"global-pull-secret-syncer-lx6ht\" (UID: \"72eae49b-cb97-4819-aaf5-5c6eb23ce9c8\") " pod="kube-system/global-pull-secret-syncer-lx6ht"
Sep 11 17:19:31.619265 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:31.619069    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 11 17:19:31.619265 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:31.619091    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 11 17:19:31.628935 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:31.628914    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b2b1c077-b5b8-4bdc-a1b3-6b98a0bcf6a0-metrics-certs\") pod \"network-metrics-daemon-d5jqp\" (UID: \"b2b1c077-b5b8-4bdc-a1b3-6b98a0bcf6a0\") " pod="openshift-multus/network-metrics-daemon-d5jqp"
Sep 11 17:19:31.629161 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:31.629141    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/72eae49b-cb97-4819-aaf5-5c6eb23ce9c8-original-pull-secret\") pod \"global-pull-secret-syncer-lx6ht\" (UID: \"72eae49b-cb97-4819-aaf5-5c6eb23ce9c8\") " pod="kube-system/global-pull-secret-syncer-lx6ht"
Sep 11 17:19:31.702516 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:31.702484    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-86bf889f8b-5hnlq"]
Sep 11 17:19:31.706028 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:19:31.706000    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65d6125e_1089_4308_bee8_064eefd92dec.slice/crio-dad05c01f82ca8bdd39853766e70907d13181ce0d098ccb06f3353e98e9a3f03 WatchSource:0}: Error finding container dad05c01f82ca8bdd39853766e70907d13181ce0d098ccb06f3353e98e9a3f03: Status 404 returned error can't find the container with id dad05c01f82ca8bdd39853766e70907d13181ce0d098ccb06f3353e98e9a3f03
Sep 11 17:19:31.732827 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:31.732795    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:19:31.745371 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:31.745344    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-6f8f7958c8-2p4xx"]
Sep 11 17:19:31.752038 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:19:31.752014    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a34a4aa_8cb7_42f3_9f87_da690cfd8031.slice/crio-82546e2c46e471fd39f2ab8d50f8d1ba05e7b5e930c9e64f89de96be5206ce44 WatchSource:0}: Error finding container 82546e2c46e471fd39f2ab8d50f8d1ba05e7b5e930c9e64f89de96be5206ce44: Status 404 returned error can't find the container with id 82546e2c46e471fd39f2ab8d50f8d1ba05e7b5e930c9e64f89de96be5206ce44
Sep 11 17:19:31.779034 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:31.779006    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-6h5ct"]
Sep 11 17:19:31.780453 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:31.780396    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cd954b897-vtd7t"]
Sep 11 17:19:31.782630 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:19:31.782602    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7d42fd1_2cb9_429b_a57a_656a5d7ecb91.slice/crio-94bcf81f29a7e9efa4a355a265879292d5b90f0b914c401192d3cec7046043dd WatchSource:0}: Error finding container 94bcf81f29a7e9efa4a355a265879292d5b90f0b914c401192d3cec7046043dd: Status 404 returned error can't find the container with id 94bcf81f29a7e9efa4a355a265879292d5b90f0b914c401192d3cec7046043dd
Sep 11 17:19:31.783476 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:19:31.783447    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24021eec_97e8_4bd2_91b0_c31be7bd9c88.slice/crio-a9e63b36b7123cd248970197a8c7be9d4abc491fc7a2a7d97995b5499751a6ef WatchSource:0}: Error finding container a9e63b36b7123cd248970197a8c7be9d4abc491fc7a2a7d97995b5499751a6ef: Status 404 returned error can't find the container with id a9e63b36b7123cd248970197a8c7be9d4abc491fc7a2a7d97995b5499751a6ef
Sep 11 17:19:31.810562 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:31.810531    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-q2zxh"]
Sep 11 17:19:31.831105 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:31.831075    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-5dcbc7df6b-7kvsj"]
Sep 11 17:19:31.835225 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:19:31.835187    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e8c629b_c41f_4eef_9c3e_7eed516e00a7.slice/crio-83951f27f146e77d788c87f565ef5aa0c9467cc970a903d2dab83aa0401fb849 WatchSource:0}: Error finding container 83951f27f146e77d788c87f565ef5aa0c9467cc970a903d2dab83aa0401fb849: Status 404 returned error can't find the container with id 83951f27f146e77d788c87f565ef5aa0c9467cc970a903d2dab83aa0401fb849
Sep 11 17:19:31.869425 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:31.869401    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-gdccs\""
Sep 11 17:19:31.878334 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:31.878312    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d5jqp"
Sep 11 17:19:31.880311 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:31.880292    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lx6ht"
Sep 11 17:19:32.141124 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:32.141079    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:19:32.154856 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:32.154824    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-d5jqp"]
Sep 11 17:19:32.158714 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:19:32.158682    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2b1c077_b5b8_4bdc_a1b3_6b98a0bcf6a0.slice/crio-5f550797ce1ae2a8fc1b9f136d4b1f0a364d3e3b9ce0c38173fe42d1dddccb8e WatchSource:0}: Error finding container 5f550797ce1ae2a8fc1b9f136d4b1f0a364d3e3b9ce0c38173fe42d1dddccb8e: Status 404 returned error can't find the container with id 5f550797ce1ae2a8fc1b9f136d4b1f0a364d3e3b9ce0c38173fe42d1dddccb8e
Sep 11 17:19:32.159295 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:32.159212    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-lx6ht"]
Sep 11 17:19:32.166011 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:19:32.165988    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72eae49b_cb97_4819_aaf5_5c6eb23ce9c8.slice/crio-726856e9a72ca5fa6d90c9b1048420010a57b2eb021203ae183972d727ce201b WatchSource:0}: Error finding container 726856e9a72ca5fa6d90c9b1048420010a57b2eb021203ae183972d727ce201b: Status 404 returned error can't find the container with id 726856e9a72ca5fa6d90c9b1048420010a57b2eb021203ae183972d727ce201b
Sep 11 17:19:32.251686 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:32.251620    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-d5jqp" event={"ID":"b2b1c077-b5b8-4bdc-a1b3-6b98a0bcf6a0","Type":"ContainerStarted","Data":"5f550797ce1ae2a8fc1b9f136d4b1f0a364d3e3b9ce0c38173fe42d1dddccb8e"}
Sep 11 17:19:32.253005 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:32.252974    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-7kvsj" event={"ID":"3e8c629b-c41f-4eef-9c3e-7eed516e00a7","Type":"ContainerStarted","Data":"83951f27f146e77d788c87f565ef5aa0c9467cc970a903d2dab83aa0401fb849"}
Sep 11 17:19:32.254357 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:32.254331    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-6h5ct" event={"ID":"b7d42fd1-2cb9-429b-a57a-656a5d7ecb91","Type":"ContainerStarted","Data":"94bcf81f29a7e9efa4a355a265879292d5b90f0b914c401192d3cec7046043dd"}
Sep 11 17:19:32.256896 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:32.256871    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-99f6z" event={"ID":"3f805648-71b2-435c-9942-1701f83fe7a3","Type":"ContainerStarted","Data":"ab6d9ecdebf9577bc8b19f024b1e5eb8fd1b17777ddc5bfba8c005a1952758c2"}
Sep 11 17:19:32.258327 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:32.258302    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-vtd7t" event={"ID":"24021eec-97e8-4bd2-91b0-c31be7bd9c88","Type":"ContainerStarted","Data":"a9e63b36b7123cd248970197a8c7be9d4abc491fc7a2a7d97995b5499751a6ef"}
Sep 11 17:19:32.259658 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:32.259629    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-lx6ht" event={"ID":"72eae49b-cb97-4819-aaf5-5c6eb23ce9c8","Type":"ContainerStarted","Data":"726856e9a72ca5fa6d90c9b1048420010a57b2eb021203ae183972d727ce201b"}
Sep 11 17:19:32.260964 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:32.260935    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-q2zxh" event={"ID":"aac7a167-8766-4c22-926e-ef15ceae304d","Type":"ContainerStarted","Data":"4fc7c9f4ad7f4b8be72cd404e9b51b9d64000a693e200623baed6f07f95504e6"}
Sep 11 17:19:32.262784 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:32.262719    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-86bf889f8b-5hnlq" event={"ID":"65d6125e-1089-4308-bee8-064eefd92dec","Type":"ContainerStarted","Data":"edc435e2ba3d0bd8ece5b118caac330c6e6bc35851691af5fd0aabf36bd0be74"}
Sep 11 17:19:32.262784 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:32.262754    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-86bf889f8b-5hnlq" event={"ID":"65d6125e-1089-4308-bee8-064eefd92dec","Type":"ContainerStarted","Data":"dad05c01f82ca8bdd39853766e70907d13181ce0d098ccb06f3353e98e9a3f03"}
Sep 11 17:19:32.262936 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:32.262832    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-86bf889f8b-5hnlq"
Sep 11 17:19:32.264465 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:32.264445    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-6f8f7958c8-2p4xx" event={"ID":"2a34a4aa-8cb7-42f3-9f87-da690cfd8031","Type":"ContainerStarted","Data":"969545da5bb7c1b62851e98d8f0d671a29683464b1ce42acb63394576417987f"}
Sep 11 17:19:32.264555 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:32.264479    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-6f8f7958c8-2p4xx" event={"ID":"2a34a4aa-8cb7-42f3-9f87-da690cfd8031","Type":"ContainerStarted","Data":"82546e2c46e471fd39f2ab8d50f8d1ba05e7b5e930c9e64f89de96be5206ce44"}
Sep 11 17:19:32.285335 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:32.285276    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-99f6z" podStartSLOduration=17.44470193 podStartE2EDuration="20.285222219s" totalImagesPullingTime="2.840520289s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 17:19:12 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:19:32.284434256 +0000 UTC m=+66.805725778" watchObservedRunningTime="2026-09-11 17:19:32.285222219 +0000 UTC m=+66.806513740"
Sep 11 17:19:32.307765 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:32.307708    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-86bf889f8b-5hnlq" podStartSLOduration=66.307690147 podStartE2EDuration="1m6.307690147s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:18:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:19:32.306332866 +0000 UTC m=+66.827624388" watchObservedRunningTime="2026-09-11 17:19:32.307690147 +0000 UTC m=+66.828981667"
Sep 11 17:19:32.330892 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:32.330842    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-6f8f7958c8-2p4xx" podStartSLOduration=64.330826466 podStartE2EDuration="1m4.330826466s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:18:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:19:32.330689782 +0000 UTC m=+66.851981305" watchObservedRunningTime="2026-09-11 17:19:32.330826466 +0000 UTC m=+66.852117988"
Sep 11 17:19:32.338776 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:32.338742    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:19:32.460583 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:32.460496    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-6f8f7958c8-2p4xx"
Sep 11 17:19:32.463458 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:32.463428    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-6f8f7958c8-2p4xx"
Sep 11 17:19:32.529134 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:32.529019    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:19:32.944333 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:32.943929    2552 scope.go:117] "RemoveContainer" containerID="71d5dc7d53a1930226de81bcec2343f11d64510aed37865e8b006149fd5704f4"
Sep 11 17:19:33.271893 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:33.271833    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-6f8f7958c8-2p4xx"
Sep 11 17:19:33.273598 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:33.273576    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-6f8f7958c8-2p4xx"
Sep 11 17:19:35.850937 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:35.850896    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:19:35.951825 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:35.951781    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:19:36.055584 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:36.055545    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:19:36.199444 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:36.199353    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:19:36.282328 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:36.282277    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:19:36.285565 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:36.285532    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-7kvsj" event={"ID":"3e8c629b-c41f-4eef-9c3e-7eed516e00a7","Type":"ContainerStarted","Data":"114e14209f352559d020fe4aea7e03a1781b6470da6f4b15d54be2b256b0da2e"}
Sep 11 17:19:36.287339 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:36.287314    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-6h5ct" event={"ID":"b7d42fd1-2cb9-429b-a57a-656a5d7ecb91","Type":"ContainerStarted","Data":"8c14a36ac233c33ad1af83710671c0cdb471bd1f75197629836a5dcfdf9ff8a4"}
Sep 11 17:19:36.289504 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:36.289481    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-hg7p8_eb5f3be1-2e36-40ba-b3a0-3bec413056e8/console-operator/1.log"
Sep 11 17:19:36.289606 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:36.289567    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-hg7p8" event={"ID":"eb5f3be1-2e36-40ba-b3a0-3bec413056e8","Type":"ContainerStarted","Data":"7f1a0749561df7e622e754f5c17ac6aaf27d2be1fca868ff211f88128776bd88"}
Sep 11 17:19:36.290084 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:36.290056    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-696884fb9c-hg7p8"
Sep 11 17:19:36.292054 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:36.292031    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-vtd7t" event={"ID":"24021eec-97e8-4bd2-91b0-c31be7bd9c88","Type":"ContainerStarted","Data":"49a608ba398b376e91c8c0cc4f76d1f33fbcdc4343c6f68dd699deadc8445f73"}
Sep 11 17:19:36.294204 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:36.294179    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-q2zxh" event={"ID":"aac7a167-8766-4c22-926e-ef15ceae304d","Type":"ContainerStarted","Data":"0e4cabc43d6c498997699a23c1a84227cbb940f3c434838febf00f67320ec45a"}
Sep 11 17:19:36.296355 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:36.296336    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-696884fb9c-hg7p8"
Sep 11 17:19:36.306800 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:36.306750    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-7kvsj" podStartSLOduration=39.779621464 podStartE2EDuration="43.306732484s" totalImagesPullingTime="3.52711102s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:18:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:19:36.305781531 +0000 UTC m=+70.827073063" watchObservedRunningTime="2026-09-11 17:19:36.306732484 +0000 UTC m=+70.828024006"
Sep 11 17:19:36.323946 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:36.323692    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-696884fb9c-hg7p8" podStartSLOduration=57.945721717 podStartE2EDuration="1m8.323675977s" totalImagesPullingTime="10.37795426s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:18:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:19:36.322935122 +0000 UTC m=+70.844226642" watchObservedRunningTime="2026-09-11 17:19:36.323675977 +0000 UTC m=+70.844967494"
Sep 11 17:19:36.338533 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:36.338270    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-vtd7t" podStartSLOduration=64.759891644 podStartE2EDuration="1m8.338254632s" totalImagesPullingTime="3.578362988s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:18:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:19:36.337889483 +0000 UTC m=+70.859181017" watchObservedRunningTime="2026-09-11 17:19:36.338254632 +0000 UTC m=+70.859546153"
Sep 11 17:19:36.367602 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:36.367555    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:19:36.538848 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:36.538706    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:19:36.758660 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:36.758620    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:19:36.833979 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:36.833892    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:19:37.318336 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:37.318308    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-76f99584f4-s92rh"]
Sep 11 17:19:37.330605 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:37.330473    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-76f99584f4-s92rh"
Sep 11 17:19:37.334309 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:37.334281    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:19:37.335882 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:37.335861    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-lqncw\""
Sep 11 17:19:37.335967 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:37.335925    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 11 17:19:37.336061 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:37.335977    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 11 17:19:37.351978 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:37.351940    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-76f99584f4-s92rh"]
Sep 11 17:19:37.392795 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:37.392770    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-7bhdv"]
Sep 11 17:19:37.418177 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:37.417600    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-7bhdv"]
Sep 11 17:19:37.418177 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:37.417727    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-7bhdv"
Sep 11 17:19:37.422591 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:37.422433    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Sep 11 17:19:37.422902 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:37.422881    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-v2gwr\""
Sep 11 17:19:37.468958 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:37.468890    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c2kxk\" (UniqueName: \"kubernetes.io/projected/4e82ca32-df13-4755-9e88-08a339c69abe-kube-api-access-c2kxk\") pod \"downloads-76f99584f4-s92rh\" (UID: \"4e82ca32-df13-4755-9e88-08a339c69abe\") " pod="openshift-console/downloads-76f99584f4-s92rh"
Sep 11 17:19:37.569558 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:37.569527    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c2kxk\" (UniqueName: \"kubernetes.io/projected/4e82ca32-df13-4755-9e88-08a339c69abe-kube-api-access-c2kxk\") pod \"downloads-76f99584f4-s92rh\" (UID: \"4e82ca32-df13-4755-9e88-08a339c69abe\") " pod="openshift-console/downloads-76f99584f4-s92rh"
Sep 11 17:19:37.569725 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:37.569568    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/3277d6b0-b7ff-4851-917c-a46c45c3a25d-tls-certificates\") pod \"prometheus-operator-admission-webhook-5864cd99d7-7bhdv\" (UID: \"3277d6b0-b7ff-4851-917c-a46c45c3a25d\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-7bhdv"
Sep 11 17:19:37.588495 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:37.588468    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c2kxk\" (UniqueName: \"kubernetes.io/projected/4e82ca32-df13-4755-9e88-08a339c69abe-kube-api-access-c2kxk\") pod \"downloads-76f99584f4-s92rh\" (UID: \"4e82ca32-df13-4755-9e88-08a339c69abe\") " pod="openshift-console/downloads-76f99584f4-s92rh"
Sep 11 17:19:37.641061 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:37.641017    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-76f99584f4-s92rh"
Sep 11 17:19:37.670996 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:37.670918    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/3277d6b0-b7ff-4851-917c-a46c45c3a25d-tls-certificates\") pod \"prometheus-operator-admission-webhook-5864cd99d7-7bhdv\" (UID: \"3277d6b0-b7ff-4851-917c-a46c45c3a25d\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-7bhdv"
Sep 11 17:19:37.673404 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:37.673378    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/3277d6b0-b7ff-4851-917c-a46c45c3a25d-tls-certificates\") pod \"prometheus-operator-admission-webhook-5864cd99d7-7bhdv\" (UID: \"3277d6b0-b7ff-4851-917c-a46c45c3a25d\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-7bhdv"
Sep 11 17:19:37.766589 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:37.766422    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-7bhdv"
Sep 11 17:19:38.032504 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:38.032397    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-76f99584f4-s92rh"]
Sep 11 17:19:38.037336 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:19:38.037306    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e82ca32_df13_4755_9e88_08a339c69abe.slice/crio-fe60282532dc8eacff98d832fce1667dfd4240b5dec1cd70ca04ebead6eef241 WatchSource:0}: Error finding container fe60282532dc8eacff98d832fce1667dfd4240b5dec1cd70ca04ebead6eef241: Status 404 returned error can't find the container with id fe60282532dc8eacff98d832fce1667dfd4240b5dec1cd70ca04ebead6eef241
Sep 11 17:19:38.102725 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:38.102696    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-7bhdv"]
Sep 11 17:19:38.106157 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:19:38.106132    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3277d6b0_b7ff_4851_917c_a46c45c3a25d.slice/crio-b65fa340f0fbbc99248deeefde10d7bf140edde7f584b6142e1aa394f53c9b12 WatchSource:0}: Error finding container b65fa340f0fbbc99248deeefde10d7bf140edde7f584b6142e1aa394f53c9b12: Status 404 returned error can't find the container with id b65fa340f0fbbc99248deeefde10d7bf140edde7f584b6142e1aa394f53c9b12
Sep 11 17:19:38.304246 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:38.304153    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-q2zxh" event={"ID":"aac7a167-8766-4c22-926e-ef15ceae304d","Type":"ContainerStarted","Data":"2a38cf9f06fff433755c33f3bc6a43261b0663691f98a6fe8a1e3f4ba6359ba9"}
Sep 11 17:19:38.305534 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:38.305509    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-7bhdv" event={"ID":"3277d6b0-b7ff-4851-917c-a46c45c3a25d","Type":"ContainerStarted","Data":"b65fa340f0fbbc99248deeefde10d7bf140edde7f584b6142e1aa394f53c9b12"}
Sep 11 17:19:38.306551 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:38.306530    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-76f99584f4-s92rh" event={"ID":"4e82ca32-df13-4755-9e88-08a339c69abe","Type":"ContainerStarted","Data":"fe60282532dc8eacff98d832fce1667dfd4240b5dec1cd70ca04ebead6eef241"}
Sep 11 17:19:38.308242 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:38.308223    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-d5jqp" event={"ID":"b2b1c077-b5b8-4bdc-a1b3-6b98a0bcf6a0","Type":"ContainerStarted","Data":"e3aa763a4ec9f92754defa7238eb45ee65454b9d5ecff92beef574116600127b"}
Sep 11 17:19:38.308340 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:38.308257    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-d5jqp" event={"ID":"b2b1c077-b5b8-4bdc-a1b3-6b98a0bcf6a0","Type":"ContainerStarted","Data":"b8d4c5df1479b9d66cf3ec3d367df1005ca4db8613db64d85393c1a5a0877c5c"}
Sep 11 17:19:38.309930 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:38.309907    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-6h5ct" event={"ID":"b7d42fd1-2cb9-429b-a57a-656a5d7ecb91","Type":"ContainerStarted","Data":"889be4cb97c177520ec60689b16d27a7c0c87afa251bfa642d2483cad2d7a2e1"}
Sep 11 17:19:38.310020 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:38.309956    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-6h5ct"
Sep 11 17:19:38.311293 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:38.311264    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-lx6ht" event={"ID":"72eae49b-cb97-4819-aaf5-5c6eb23ce9c8","Type":"ContainerStarted","Data":"8a21525f45afd48a4e02129cafbc31e47e66c7cae7dd6f317ddb83a1f86e7702"}
Sep 11 17:19:38.318347 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:38.318328    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:19:38.347697 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:38.347636    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-q2zxh" podStartSLOduration=66.842221457 podStartE2EDuration="1m10.347618459s" totalImagesPullingTime="3.505397002s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:18:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:19:38.347535585 +0000 UTC m=+72.868827107" watchObservedRunningTime="2026-09-11 17:19:38.347618459 +0000 UTC m=+72.868909990"
Sep 11 17:19:38.393349 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:38.393292    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-d5jqp" podStartSLOduration=67.225382026 podStartE2EDuration="1m12.393273676s" totalImagesPullingTime="5.16789165s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:18:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:19:38.392329469 +0000 UTC m=+72.913620989" watchObservedRunningTime="2026-09-11 17:19:38.393273676 +0000 UTC m=+72.914565197"
Sep 11 17:19:38.429069 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:38.429011    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-lx6ht" podStartSLOduration=66.876922583 podStartE2EDuration="1m12.428995624s" totalImagesPullingTime="5.552073041s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 17:18:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:19:38.427343499 +0000 UTC m=+72.948635021" watchObservedRunningTime="2026-09-11 17:19:38.428995624 +0000 UTC m=+72.950287172"
Sep 11 17:19:38.465668 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:38.465619    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-6h5ct" podStartSLOduration=35.885865822 podStartE2EDuration="39.465600013s" totalImagesPullingTime="3.579734191s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:18:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:19:38.464169326 +0000 UTC m=+72.985460846" watchObservedRunningTime="2026-09-11 17:19:38.465600013 +0000 UTC m=+72.986891533"
Sep 11 17:19:38.483999 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:38.483959    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:19:38.603322 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:38.603289    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:19:38.758314 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:38.758257    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:19:39.017778 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:39.017698    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:19:39.318349 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:39.318256    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:19:39.477449 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:39.477418    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:19:39.719232 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:39.719152    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:19:39.825719 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:39.825642    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:19:40.145081 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:40.145040    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:19:40.242777 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:40.242742    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:19:40.323902 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:40.323865    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-7bhdv" event={"ID":"3277d6b0-b7ff-4851-917c-a46c45c3a25d","Type":"ContainerStarted","Data":"917555b0013d5ba59cac5c72568af1e46c22a6aa0aa72ac50f4e9558711092c8"}
Sep 11 17:19:40.324159 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:40.324139    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-7bhdv"
Sep 11 17:19:40.325390 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:40.325360    2552 patch_prober.go:28] interesting pod/prometheus-operator-admission-webhook-5864cd99d7-7bhdv container/prometheus-operator-admission-webhook namespace/openshift-monitoring: Readiness probe status=failure output="Get \"https://10.132.0.22:8443/healthz\": dial tcp 10.132.0.22:8443: connect: connection refused" start-of-body=
Sep 11 17:19:40.325493 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:40.325414    2552 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-7bhdv" podUID="3277d6b0-b7ff-4851-917c-a46c45c3a25d" containerName="prometheus-operator-admission-webhook" probeResult="failure" output="Get \"https://10.132.0.22:8443/healthz\": dial tcp 10.132.0.22:8443: connect: connection refused"
Sep 11 17:19:40.357539 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:40.357478    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-7bhdv" podStartSLOduration=1.27087029 podStartE2EDuration="3.357457127s" totalImagesPullingTime="2.086586837s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:19:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:19:40.357030408 +0000 UTC m=+74.878321931" watchObservedRunningTime="2026-09-11 17:19:40.357457127 +0000 UTC m=+74.878748647"
Sep 11 17:19:41.332086 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:41.332056    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-7bhdv"
Sep 11 17:19:42.039964 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:42.039924    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:19:42.122247 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:42.122211    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:19:42.497455 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:42.497379    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:19:43.183578 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:43.183548    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-9gb8k"
Sep 11 17:19:44.320363 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:44.320323    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:19:44.388225 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:44.388184    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:19:45.318262 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:45.318231    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-6h5ct"
Sep 11 17:19:45.932534 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:45.932489    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:19:46.037593 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:46.037555    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:19:46.842024 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:46.841986    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:19:48.574158 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:48.574015    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:19:48.659777 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:48.659739    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:19:48.757571 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:48.757537    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:19:48.843611 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:48.843578    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:19:49.254641 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:49.254544    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:19:49.477708 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:49.477649    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:19:49.595846 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:49.595796    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:19:49.663379 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:49.663342    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-78gb7"]
Sep 11 17:19:49.665869 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:49.665842    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-78gb7"
Sep 11 17:19:49.670868 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:49.670843    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 11 17:19:49.671019 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:49.670844    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 11 17:19:49.671208 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:49.671188    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 11 17:19:49.671290 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:49.671196    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-rqg8q\""
Sep 11 17:19:49.671646 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:49.671630    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 11 17:19:49.673557 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:49.673531    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/781705c1-5c64-4a55-b898-1943a2c11a1a-node-exporter-wtmp\") pod \"node-exporter-78gb7\" (UID: \"781705c1-5c64-4a55-b898-1943a2c11a1a\") " pod="openshift-monitoring/node-exporter-78gb7"
Sep 11 17:19:49.673659 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:49.673629    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/781705c1-5c64-4a55-b898-1943a2c11a1a-node-exporter-accelerators-collector-config\") pod \"node-exporter-78gb7\" (UID: \"781705c1-5c64-4a55-b898-1943a2c11a1a\") " pod="openshift-monitoring/node-exporter-78gb7"
Sep 11 17:19:49.673734 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:49.673666    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/781705c1-5c64-4a55-b898-1943a2c11a1a-node-exporter-textfile\") pod \"node-exporter-78gb7\" (UID: \"781705c1-5c64-4a55-b898-1943a2c11a1a\") " pod="openshift-monitoring/node-exporter-78gb7"
Sep 11 17:19:49.673734 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:49.673702    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/781705c1-5c64-4a55-b898-1943a2c11a1a-root\") pod \"node-exporter-78gb7\" (UID: \"781705c1-5c64-4a55-b898-1943a2c11a1a\") " pod="openshift-monitoring/node-exporter-78gb7"
Sep 11 17:19:49.673847 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:49.673740    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/781705c1-5c64-4a55-b898-1943a2c11a1a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-78gb7\" (UID: \"781705c1-5c64-4a55-b898-1943a2c11a1a\") " pod="openshift-monitoring/node-exporter-78gb7"
Sep 11 17:19:49.673847 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:49.673788    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/781705c1-5c64-4a55-b898-1943a2c11a1a-sys\") pod \"node-exporter-78gb7\" (UID: \"781705c1-5c64-4a55-b898-1943a2c11a1a\") " pod="openshift-monitoring/node-exporter-78gb7"
Sep 11 17:19:49.673847 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:49.673817    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/781705c1-5c64-4a55-b898-1943a2c11a1a-metrics-client-ca\") pod \"node-exporter-78gb7\" (UID: \"781705c1-5c64-4a55-b898-1943a2c11a1a\") " pod="openshift-monitoring/node-exporter-78gb7"
Sep 11 17:19:49.674293 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:49.673848    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/781705c1-5c64-4a55-b898-1943a2c11a1a-node-exporter-tls\") pod \"node-exporter-78gb7\" (UID: \"781705c1-5c64-4a55-b898-1943a2c11a1a\") " pod="openshift-monitoring/node-exporter-78gb7"
Sep 11 17:19:49.674293 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:49.673896    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d8z7n\" (UniqueName: \"kubernetes.io/projected/781705c1-5c64-4a55-b898-1943a2c11a1a-kube-api-access-d8z7n\") pod \"node-exporter-78gb7\" (UID: \"781705c1-5c64-4a55-b898-1943a2c11a1a\") " pod="openshift-monitoring/node-exporter-78gb7"
Sep 11 17:19:49.702850 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:49.702804    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:19:49.774947 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:49.774910    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/781705c1-5c64-4a55-b898-1943a2c11a1a-sys\") pod \"node-exporter-78gb7\" (UID: \"781705c1-5c64-4a55-b898-1943a2c11a1a\") " pod="openshift-monitoring/node-exporter-78gb7"
Sep 11 17:19:49.774947 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:49.774949    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/781705c1-5c64-4a55-b898-1943a2c11a1a-metrics-client-ca\") pod \"node-exporter-78gb7\" (UID: \"781705c1-5c64-4a55-b898-1943a2c11a1a\") " pod="openshift-monitoring/node-exporter-78gb7"
Sep 11 17:19:49.775206 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:49.774981    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/781705c1-5c64-4a55-b898-1943a2c11a1a-node-exporter-tls\") pod \"node-exporter-78gb7\" (UID: \"781705c1-5c64-4a55-b898-1943a2c11a1a\") " pod="openshift-monitoring/node-exporter-78gb7"
Sep 11 17:19:49.775206 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:49.775013    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d8z7n\" (UniqueName: \"kubernetes.io/projected/781705c1-5c64-4a55-b898-1943a2c11a1a-kube-api-access-d8z7n\") pod \"node-exporter-78gb7\" (UID: \"781705c1-5c64-4a55-b898-1943a2c11a1a\") " pod="openshift-monitoring/node-exporter-78gb7"
Sep 11 17:19:49.775206 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:49.775032    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/781705c1-5c64-4a55-b898-1943a2c11a1a-sys\") pod \"node-exporter-78gb7\" (UID: \"781705c1-5c64-4a55-b898-1943a2c11a1a\") " pod="openshift-monitoring/node-exporter-78gb7"
Sep 11 17:19:49.775206 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:19:49.775132    2552 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Sep 11 17:19:49.775206 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:49.775166    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/781705c1-5c64-4a55-b898-1943a2c11a1a-node-exporter-wtmp\") pod \"node-exporter-78gb7\" (UID: \"781705c1-5c64-4a55-b898-1943a2c11a1a\") " pod="openshift-monitoring/node-exporter-78gb7"
Sep 11 17:19:49.775206 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:19:49.775201    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/781705c1-5c64-4a55-b898-1943a2c11a1a-node-exporter-tls podName:781705c1-5c64-4a55-b898-1943a2c11a1a nodeName:}" failed. No retries permitted until 2026-09-11 17:19:50.275182311 +0000 UTC m=+84.796473810 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/781705c1-5c64-4a55-b898-1943a2c11a1a-node-exporter-tls") pod "node-exporter-78gb7" (UID: "781705c1-5c64-4a55-b898-1943a2c11a1a") : secret "node-exporter-tls" not found
Sep 11 17:19:49.775487 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:49.775278    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/781705c1-5c64-4a55-b898-1943a2c11a1a-node-exporter-accelerators-collector-config\") pod \"node-exporter-78gb7\" (UID: \"781705c1-5c64-4a55-b898-1943a2c11a1a\") " pod="openshift-monitoring/node-exporter-78gb7"
Sep 11 17:19:49.775487 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:49.775308    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/781705c1-5c64-4a55-b898-1943a2c11a1a-node-exporter-textfile\") pod \"node-exporter-78gb7\" (UID: \"781705c1-5c64-4a55-b898-1943a2c11a1a\") " pod="openshift-monitoring/node-exporter-78gb7"
Sep 11 17:19:49.775487 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:49.775315    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/781705c1-5c64-4a55-b898-1943a2c11a1a-node-exporter-wtmp\") pod \"node-exporter-78gb7\" (UID: \"781705c1-5c64-4a55-b898-1943a2c11a1a\") " pod="openshift-monitoring/node-exporter-78gb7"
Sep 11 17:19:49.775487 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:49.775402    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/781705c1-5c64-4a55-b898-1943a2c11a1a-root\") pod \"node-exporter-78gb7\" (UID: \"781705c1-5c64-4a55-b898-1943a2c11a1a\") " pod="openshift-monitoring/node-exporter-78gb7"
Sep 11 17:19:49.775487 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:49.775462    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/781705c1-5c64-4a55-b898-1943a2c11a1a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-78gb7\" (UID: \"781705c1-5c64-4a55-b898-1943a2c11a1a\") " pod="openshift-monitoring/node-exporter-78gb7"
Sep 11 17:19:49.775742 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:49.775496    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/781705c1-5c64-4a55-b898-1943a2c11a1a-root\") pod \"node-exporter-78gb7\" (UID: \"781705c1-5c64-4a55-b898-1943a2c11a1a\") " pod="openshift-monitoring/node-exporter-78gb7"
Sep 11 17:19:49.775742 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:49.775641    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/781705c1-5c64-4a55-b898-1943a2c11a1a-node-exporter-textfile\") pod \"node-exporter-78gb7\" (UID: \"781705c1-5c64-4a55-b898-1943a2c11a1a\") " pod="openshift-monitoring/node-exporter-78gb7"
Sep 11 17:19:49.775742 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:49.775659    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/781705c1-5c64-4a55-b898-1943a2c11a1a-metrics-client-ca\") pod \"node-exporter-78gb7\" (UID: \"781705c1-5c64-4a55-b898-1943a2c11a1a\") " pod="openshift-monitoring/node-exporter-78gb7"
Sep 11 17:19:49.775909 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:49.775870    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/781705c1-5c64-4a55-b898-1943a2c11a1a-node-exporter-accelerators-collector-config\") pod \"node-exporter-78gb7\" (UID: \"781705c1-5c64-4a55-b898-1943a2c11a1a\") " pod="openshift-monitoring/node-exporter-78gb7"
Sep 11 17:19:49.778013 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:49.777989    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/781705c1-5c64-4a55-b898-1943a2c11a1a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-78gb7\" (UID: \"781705c1-5c64-4a55-b898-1943a2c11a1a\") " pod="openshift-monitoring/node-exporter-78gb7"
Sep 11 17:19:49.795052 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:49.794893    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d8z7n\" (UniqueName: \"kubernetes.io/projected/781705c1-5c64-4a55-b898-1943a2c11a1a-kube-api-access-d8z7n\") pod \"node-exporter-78gb7\" (UID: \"781705c1-5c64-4a55-b898-1943a2c11a1a\") " pod="openshift-monitoring/node-exporter-78gb7"
Sep 11 17:19:49.929977 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:49.929879    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:19:50.249197 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:50.249096    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:19:50.280291 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:50.280246    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/781705c1-5c64-4a55-b898-1943a2c11a1a-node-exporter-tls\") pod \"node-exporter-78gb7\" (UID: \"781705c1-5c64-4a55-b898-1943a2c11a1a\") " pod="openshift-monitoring/node-exporter-78gb7"
Sep 11 17:19:50.282911 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:50.282887    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/781705c1-5c64-4a55-b898-1943a2c11a1a-node-exporter-tls\") pod \"node-exporter-78gb7\" (UID: \"781705c1-5c64-4a55-b898-1943a2c11a1a\") " pod="openshift-monitoring/node-exporter-78gb7"
Sep 11 17:19:50.402384 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:50.402348    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:19:50.527373 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:50.527278    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:19:50.577784 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:50.577755    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-78gb7"
Sep 11 17:19:50.646247 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:50.646212    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:19:50.881409 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:50.881296    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:19:51.038707 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:51.038670    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:19:51.648613 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:51.648567    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:19:51.767596 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:51.767564    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:19:52.126672 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:52.126626    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:19:52.234922 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:52.234889    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:19:52.820012 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:52.819973    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:19:53.276918 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:53.276889    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-86bf889f8b-5hnlq"
Sep 11 17:19:53.391317 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:53.391274    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:19:53.545722 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:53.545624    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:19:53.652648 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:53.652613    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:19:54.269682 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:54.269638    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:19:54.619367 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:54.619320    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:19:54.824562 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:54.824527    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:19:55.869709 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:19:55.869677    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod781705c1_5c64_4a55_b898_1943a2c11a1a.slice/crio-bb320bb918e33cccb5d3a722bfd185bc32b8cd8a6ae60ecd23b771a61622727c WatchSource:0}: Error finding container bb320bb918e33cccb5d3a722bfd185bc32b8cd8a6ae60ecd23b771a61622727c: Status 404 returned error can't find the container with id bb320bb918e33cccb5d3a722bfd185bc32b8cd8a6ae60ecd23b771a61622727c
Sep 11 17:19:56.027343 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:56.027315    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:19:56.241982 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:56.241888    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:19:56.395408 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:56.395354    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-76f99584f4-s92rh" event={"ID":"4e82ca32-df13-4755-9e88-08a339c69abe","Type":"ContainerStarted","Data":"bd7a66393953988a05958ff7b6b0f5da0441b2122ca4cc34ff16a9326ab85479"}
Sep 11 17:19:56.395608 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:56.395528    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-76f99584f4-s92rh"
Sep 11 17:19:56.396937 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:56.396908    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-78gb7" event={"ID":"781705c1-5c64-4a55-b898-1943a2c11a1a","Type":"ContainerStarted","Data":"bb320bb918e33cccb5d3a722bfd185bc32b8cd8a6ae60ecd23b771a61622727c"}
Sep 11 17:19:56.408163 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:56.408131    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-76f99584f4-s92rh"
Sep 11 17:19:56.416014 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:56.415962    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-76f99584f4-s92rh" podStartSLOduration=1.4826183849999999 podStartE2EDuration="19.415943006s" totalImagesPullingTime="17.933324621s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:19:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:19:56.415007047 +0000 UTC m=+90.936298569" watchObservedRunningTime="2026-09-11 17:19:56.415943006 +0000 UTC m=+90.937234528"
Sep 11 17:19:57.403206 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:57.403170    2552 generic.go:356] "Generic (PLEG): container finished" podID="781705c1-5c64-4a55-b898-1943a2c11a1a" containerID="af77e96fec424bc79ab8aaa185b25dbc5b6d98c7a9d89c6c617ab08f0ca38b90" exitCode=0
Sep 11 17:19:57.403602 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:57.403238    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-78gb7" event={"ID":"781705c1-5c64-4a55-b898-1943a2c11a1a","Type":"ContainerDied","Data":"af77e96fec424bc79ab8aaa185b25dbc5b6d98c7a9d89c6c617ab08f0ca38b90"}
Sep 11 17:19:57.813987 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:57.813897    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:19:57.920668 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:57.920631    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:19:58.411616 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:58.411574    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-78gb7" event={"ID":"781705c1-5c64-4a55-b898-1943a2c11a1a","Type":"ContainerStarted","Data":"0284d2ebacf66f2b855c889af668f209b8457685018368999431573cbf827587"}
Sep 11 17:19:58.412180 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:58.411645    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-78gb7" event={"ID":"781705c1-5c64-4a55-b898-1943a2c11a1a","Type":"ContainerStarted","Data":"1a64dafcd345d98ccbc6732c29ee475a878364f4388f537b92c6019afe7ea379"}
Sep 11 17:19:58.437888 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:58.437824    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-78gb7" podStartSLOduration=8.016995151 podStartE2EDuration="9.437802467s" totalImagesPullingTime="1.34094171s" totalInitContainerRuntime="79.865606ms" isStatefulPod=true podCreationTimestamp="2026-09-11 17:19:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:19:58.437023032 +0000 UTC m=+92.958314553" watchObservedRunningTime="2026-09-11 17:19:58.437802467 +0000 UTC m=+92.959093989"
Sep 11 17:19:58.917726 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:58.917683    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:19:58.991072 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:58.991028    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:19:59.443466 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:59.443433    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-86bf889f8b-5hnlq"]
Sep 11 17:19:59.980589 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:19:59.980552    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:20:00.084400 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:20:00.084360    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:20:00.684590 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:20:00.684551    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-75c57466fd-6k4ls"]
Sep 11 17:20:00.712352 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:20:00.712319    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-75c57466fd-6k4ls"]
Sep 11 17:20:00.712532 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:20:00.712478    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-75c57466fd-6k4ls"
Sep 11 17:20:00.721877 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:20:00.721842    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 11 17:20:00.722032 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:20:00.721842    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 11 17:20:00.722522 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:20:00.722501    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 11 17:20:00.722635 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:20:00.722502    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 11 17:20:00.722890 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:20:00.722866    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-tk8qn\""
Sep 11 17:20:00.723028 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:20:00.722994    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 11 17:20:00.726999 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:20:00.726983    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 11 17:20:00.772479 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:20:00.772443    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c7bzl\" (UniqueName: \"kubernetes.io/projected/c770a0fe-5881-40e3-b0c9-4e35427cf1fb-kube-api-access-c7bzl\") pod \"console-75c57466fd-6k4ls\" (UID: \"c770a0fe-5881-40e3-b0c9-4e35427cf1fb\") " pod="openshift-console/console-75c57466fd-6k4ls"
Sep 11 17:20:00.772651 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:20:00.772548    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c770a0fe-5881-40e3-b0c9-4e35427cf1fb-console-config\") pod \"console-75c57466fd-6k4ls\" (UID: \"c770a0fe-5881-40e3-b0c9-4e35427cf1fb\") " pod="openshift-console/console-75c57466fd-6k4ls"
Sep 11 17:20:00.772651 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:20:00.772588    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c770a0fe-5881-40e3-b0c9-4e35427cf1fb-trusted-ca-bundle\") pod \"console-75c57466fd-6k4ls\" (UID: \"c770a0fe-5881-40e3-b0c9-4e35427cf1fb\") " pod="openshift-console/console-75c57466fd-6k4ls"
Sep 11 17:20:00.772651 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:20:00.772627    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c770a0fe-5881-40e3-b0c9-4e35427cf1fb-oauth-serving-cert\") pod \"console-75c57466fd-6k4ls\" (UID: \"c770a0fe-5881-40e3-b0c9-4e35427cf1fb\") " pod="openshift-console/console-75c57466fd-6k4ls"
Sep 11 17:20:00.772816 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:20:00.772750    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c770a0fe-5881-40e3-b0c9-4e35427cf1fb-service-ca\") pod \"console-75c57466fd-6k4ls\" (UID: \"c770a0fe-5881-40e3-b0c9-4e35427cf1fb\") " pod="openshift-console/console-75c57466fd-6k4ls"
Sep 11 17:20:00.772816 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:20:00.772781    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c770a0fe-5881-40e3-b0c9-4e35427cf1fb-console-serving-cert\") pod \"console-75c57466fd-6k4ls\" (UID: \"c770a0fe-5881-40e3-b0c9-4e35427cf1fb\") " pod="openshift-console/console-75c57466fd-6k4ls"
Sep 11 17:20:00.772896 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:20:00.772844    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c770a0fe-5881-40e3-b0c9-4e35427cf1fb-console-oauth-config\") pod \"console-75c57466fd-6k4ls\" (UID: \"c770a0fe-5881-40e3-b0c9-4e35427cf1fb\") " pod="openshift-console/console-75c57466fd-6k4ls"
Sep 11 17:20:00.874146 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:20:00.874087    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c770a0fe-5881-40e3-b0c9-4e35427cf1fb-console-serving-cert\") pod \"console-75c57466fd-6k4ls\" (UID: \"c770a0fe-5881-40e3-b0c9-4e35427cf1fb\") " pod="openshift-console/console-75c57466fd-6k4ls"
Sep 11 17:20:00.874308 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:20:00.874155    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c770a0fe-5881-40e3-b0c9-4e35427cf1fb-console-oauth-config\") pod \"console-75c57466fd-6k4ls\" (UID: \"c770a0fe-5881-40e3-b0c9-4e35427cf1fb\") " pod="openshift-console/console-75c57466fd-6k4ls"
Sep 11 17:20:00.874652 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:20:00.874612    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c7bzl\" (UniqueName: \"kubernetes.io/projected/c770a0fe-5881-40e3-b0c9-4e35427cf1fb-kube-api-access-c7bzl\") pod \"console-75c57466fd-6k4ls\" (UID: \"c770a0fe-5881-40e3-b0c9-4e35427cf1fb\") " pod="openshift-console/console-75c57466fd-6k4ls"
Sep 11 17:20:00.874763 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:20:00.874709    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c770a0fe-5881-40e3-b0c9-4e35427cf1fb-console-config\") pod \"console-75c57466fd-6k4ls\" (UID: \"c770a0fe-5881-40e3-b0c9-4e35427cf1fb\") " pod="openshift-console/console-75c57466fd-6k4ls"
Sep 11 17:20:00.874823 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:20:00.874785    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c770a0fe-5881-40e3-b0c9-4e35427cf1fb-trusted-ca-bundle\") pod \"console-75c57466fd-6k4ls\" (UID: \"c770a0fe-5881-40e3-b0c9-4e35427cf1fb\") " pod="openshift-console/console-75c57466fd-6k4ls"
Sep 11 17:20:00.874879 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:20:00.874832    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c770a0fe-5881-40e3-b0c9-4e35427cf1fb-oauth-serving-cert\") pod \"console-75c57466fd-6k4ls\" (UID: \"c770a0fe-5881-40e3-b0c9-4e35427cf1fb\") " pod="openshift-console/console-75c57466fd-6k4ls"
Sep 11 17:20:00.875941 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:20:00.875031    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c770a0fe-5881-40e3-b0c9-4e35427cf1fb-service-ca\") pod \"console-75c57466fd-6k4ls\" (UID: \"c770a0fe-5881-40e3-b0c9-4e35427cf1fb\") " pod="openshift-console/console-75c57466fd-6k4ls"
Sep 11 17:20:00.875941 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:20:00.875914    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c770a0fe-5881-40e3-b0c9-4e35427cf1fb-oauth-serving-cert\") pod \"console-75c57466fd-6k4ls\" (UID: \"c770a0fe-5881-40e3-b0c9-4e35427cf1fb\") " pod="openshift-console/console-75c57466fd-6k4ls"
Sep 11 17:20:00.876095 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:20:00.875914    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c770a0fe-5881-40e3-b0c9-4e35427cf1fb-service-ca\") pod \"console-75c57466fd-6k4ls\" (UID: \"c770a0fe-5881-40e3-b0c9-4e35427cf1fb\") " pod="openshift-console/console-75c57466fd-6k4ls"
Sep 11 17:20:00.878033 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:20:00.876617    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c770a0fe-5881-40e3-b0c9-4e35427cf1fb-console-config\") pod \"console-75c57466fd-6k4ls\" (UID: \"c770a0fe-5881-40e3-b0c9-4e35427cf1fb\") " pod="openshift-console/console-75c57466fd-6k4ls"
Sep 11 17:20:00.878033 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:20:00.877756    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c770a0fe-5881-40e3-b0c9-4e35427cf1fb-console-serving-cert\") pod \"console-75c57466fd-6k4ls\" (UID: \"c770a0fe-5881-40e3-b0c9-4e35427cf1fb\") " pod="openshift-console/console-75c57466fd-6k4ls"
Sep 11 17:20:00.878033 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:20:00.877880    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c770a0fe-5881-40e3-b0c9-4e35427cf1fb-console-oauth-config\") pod \"console-75c57466fd-6k4ls\" (UID: \"c770a0fe-5881-40e3-b0c9-4e35427cf1fb\") " pod="openshift-console/console-75c57466fd-6k4ls"
Sep 11 17:20:00.882835 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:20:00.882760    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c770a0fe-5881-40e3-b0c9-4e35427cf1fb-trusted-ca-bundle\") pod \"console-75c57466fd-6k4ls\" (UID: \"c770a0fe-5881-40e3-b0c9-4e35427cf1fb\") " pod="openshift-console/console-75c57466fd-6k4ls"
Sep 11 17:20:00.897218 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:20:00.897191    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c7bzl\" (UniqueName: \"kubernetes.io/projected/c770a0fe-5881-40e3-b0c9-4e35427cf1fb-kube-api-access-c7bzl\") pod \"console-75c57466fd-6k4ls\" (UID: \"c770a0fe-5881-40e3-b0c9-4e35427cf1fb\") " pod="openshift-console/console-75c57466fd-6k4ls"
Sep 11 17:20:00.991930 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:20:00.991848    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:20:01.024348 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:20:01.024314    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-75c57466fd-6k4ls"
Sep 11 17:20:01.350044 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:20:01.349970    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-75c57466fd-6k4ls"]
Sep 11 17:20:01.353362 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:20:01.353319    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc770a0fe_5881_40e3_b0c9_4e35427cf1fb.slice/crio-1a446d4ef074555d82eaa1cd75ace5a901c44e9fda509f902ae1e102ff4c338f WatchSource:0}: Error finding container 1a446d4ef074555d82eaa1cd75ace5a901c44e9fda509f902ae1e102ff4c338f: Status 404 returned error can't find the container with id 1a446d4ef074555d82eaa1cd75ace5a901c44e9fda509f902ae1e102ff4c338f
Sep 11 17:20:01.431744 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:20:01.431702    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-75c57466fd-6k4ls" event={"ID":"c770a0fe-5881-40e3-b0c9-4e35427cf1fb","Type":"ContainerStarted","Data":"1a446d4ef074555d82eaa1cd75ace5a901c44e9fda509f902ae1e102ff4c338f"}
Sep 11 17:20:01.803730 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:20:01.803650    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:20:03.352141 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:20:03.352047    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:20:03.530432 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:20:03.530365    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:20:05.452984 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:20:05.452943    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-75c57466fd-6k4ls" event={"ID":"c770a0fe-5881-40e3-b0c9-4e35427cf1fb","Type":"ContainerStarted","Data":"d173542c4901005b9033f0296dc0f700110a1ab1ba44996803a3b866e5ddf02d"}
Sep 11 17:20:08.122393 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:20:08.122356    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:20:08.227706 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:20:08.227669    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:20:09.617600 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:20:09.617566    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:20:09.729532 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:20:09.729488    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:20:11.024533 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:20:11.024489    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-75c57466fd-6k4ls"
Sep 11 17:20:11.025137 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:20:11.024598    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-75c57466fd-6k4ls"
Sep 11 17:20:11.029469 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:20:11.029448    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-75c57466fd-6k4ls"
Sep 11 17:20:11.049960 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:20:11.049911    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-75c57466fd-6k4ls" podStartSLOduration=7.155404003 podStartE2EDuration="11.049894284s" totalImagesPullingTime="3.894490281s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:20:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:20:05.48681188 +0000 UTC m=+100.008103402" watchObservedRunningTime="2026-09-11 17:20:11.049894284 +0000 UTC m=+105.571185861"
Sep 11 17:20:11.479462 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:20:11.479430    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-75c57466fd-6k4ls"
Sep 11 17:20:11.701276 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:20:11.701238    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:20:11.801693 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:20:11.801595    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:20:15.903697 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:20:15.903659    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:20:16.011418 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:20:16.011385    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:20:16.498103 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:20:16.498073    2552 generic.go:356] "Generic (PLEG): container finished" podID="676c8e42-836e-4f84-b5b0-c1a49eb8c5f3" containerID="2144e3de6c1bf9fc393ea498ca1bfbc74ac20c543e7842955f8b137fed1ea3b3" exitCode=0
Sep 11 17:20:16.498290 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:20:16.498156    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-pmvkp" event={"ID":"676c8e42-836e-4f84-b5b0-c1a49eb8c5f3","Type":"ContainerDied","Data":"2144e3de6c1bf9fc393ea498ca1bfbc74ac20c543e7842955f8b137fed1ea3b3"}
Sep 11 17:20:16.498520 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:20:16.498505    2552 scope.go:117] "RemoveContainer" containerID="2144e3de6c1bf9fc393ea498ca1bfbc74ac20c543e7842955f8b137fed1ea3b3"
Sep 11 17:20:17.504896 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:20:17.504859    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-pmvkp" event={"ID":"676c8e42-836e-4f84-b5b0-c1a49eb8c5f3","Type":"ContainerStarted","Data":"baa07e89e30554b5a9b5214e0e17e4e638795298e3a229376c689070d98cd6ac"}
Sep 11 17:20:21.989370 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:20:21.989332    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:20:22.524590 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:20:22.524557    2552 generic.go:356] "Generic (PLEG): container finished" podID="1a431afe-6ad9-4886-b0ac-8d147f276146" containerID="e40838db22480c574a5f8b4163a9157178a18c6f3e5e1a4adcfeaf9a7ead88e6" exitCode=0
Sep 11 17:20:22.524759 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:20:22.524627    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-gdzsp" event={"ID":"1a431afe-6ad9-4886-b0ac-8d147f276146","Type":"ContainerDied","Data":"e40838db22480c574a5f8b4163a9157178a18c6f3e5e1a4adcfeaf9a7ead88e6"}
Sep 11 17:20:22.524950 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:20:22.524938    2552 scope.go:117] "RemoveContainer" containerID="e40838db22480c574a5f8b4163a9157178a18c6f3e5e1a4adcfeaf9a7ead88e6"
Sep 11 17:20:23.530396 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:20:23.530354    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-gdzsp" event={"ID":"1a431afe-6ad9-4886-b0ac-8d147f276146","Type":"ContainerStarted","Data":"7266f5ff8c84b572a30ddc32bba95a4955b932915a17dcc45f113f2a7dcc520a"}
Sep 11 17:20:24.466647 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:20:24.466583    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-86bf889f8b-5hnlq" podUID="65d6125e-1089-4308-bee8-064eefd92dec" containerName="registry" containerID="cri-o://edc435e2ba3d0bd8ece5b118caac330c6e6bc35851691af5fd0aabf36bd0be74" gracePeriod=30
Sep 11 17:20:24.703628 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:20:24.703605    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-86bf889f8b-5hnlq"
Sep 11 17:20:24.889635 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:20:24.889606    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/65d6125e-1089-4308-bee8-064eefd92dec-trusted-ca\") pod \"65d6125e-1089-4308-bee8-064eefd92dec\" (UID: \"65d6125e-1089-4308-bee8-064eefd92dec\") "
Sep 11 17:20:24.889822 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:20:24.889652    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/65d6125e-1089-4308-bee8-064eefd92dec-image-registry-private-configuration\") pod \"65d6125e-1089-4308-bee8-064eefd92dec\" (UID: \"65d6125e-1089-4308-bee8-064eefd92dec\") "
Sep 11 17:20:24.889822 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:20:24.889673    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/65d6125e-1089-4308-bee8-064eefd92dec-installation-pull-secrets\") pod \"65d6125e-1089-4308-bee8-064eefd92dec\" (UID: \"65d6125e-1089-4308-bee8-064eefd92dec\") "
Sep 11 17:20:24.889822 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:20:24.889690    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/65d6125e-1089-4308-bee8-064eefd92dec-registry-tls\") pod \"65d6125e-1089-4308-bee8-064eefd92dec\" (UID: \"65d6125e-1089-4308-bee8-064eefd92dec\") "
Sep 11 17:20:24.889822 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:20:24.889725    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/65d6125e-1089-4308-bee8-064eefd92dec-registry-certificates\") pod \"65d6125e-1089-4308-bee8-064eefd92dec\" (UID: \"65d6125e-1089-4308-bee8-064eefd92dec\") "
Sep 11 17:20:24.889822 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:20:24.889759    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-brqvg\" (UniqueName: \"kubernetes.io/projected/65d6125e-1089-4308-bee8-064eefd92dec-kube-api-access-brqvg\") pod \"65d6125e-1089-4308-bee8-064eefd92dec\" (UID: \"65d6125e-1089-4308-bee8-064eefd92dec\") "
Sep 11 17:20:24.889822 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:20:24.889781    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/65d6125e-1089-4308-bee8-064eefd92dec-bound-sa-token\") pod \"65d6125e-1089-4308-bee8-064eefd92dec\" (UID: \"65d6125e-1089-4308-bee8-064eefd92dec\") "
Sep 11 17:20:24.890153 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:20:24.889869    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/65d6125e-1089-4308-bee8-064eefd92dec-ca-trust-extracted\") pod \"65d6125e-1089-4308-bee8-064eefd92dec\" (UID: \"65d6125e-1089-4308-bee8-064eefd92dec\") "
Sep 11 17:20:24.890236 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:20:24.890205    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/65d6125e-1089-4308-bee8-064eefd92dec-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "65d6125e-1089-4308-bee8-064eefd92dec" (UID: "65d6125e-1089-4308-bee8-064eefd92dec"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 17:20:24.890302 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:20:24.890260    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/65d6125e-1089-4308-bee8-064eefd92dec-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "65d6125e-1089-4308-bee8-064eefd92dec" (UID: "65d6125e-1089-4308-bee8-064eefd92dec"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 17:20:24.890519 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:20:24.890499    2552 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/65d6125e-1089-4308-bee8-064eefd92dec-trusted-ca\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:20:24.890602 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:20:24.890527    2552 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/65d6125e-1089-4308-bee8-064eefd92dec-registry-certificates\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:20:24.892423 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:20:24.892390    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/65d6125e-1089-4308-bee8-064eefd92dec-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "65d6125e-1089-4308-bee8-064eefd92dec" (UID: "65d6125e-1089-4308-bee8-064eefd92dec"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:20:24.892600 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:20:24.892575    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/65d6125e-1089-4308-bee8-064eefd92dec-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "65d6125e-1089-4308-bee8-064eefd92dec" (UID: "65d6125e-1089-4308-bee8-064eefd92dec"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 17:20:24.892672 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:20:24.892641    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/65d6125e-1089-4308-bee8-064eefd92dec-kube-api-access-brqvg" (OuterVolumeSpecName: "kube-api-access-brqvg") pod "65d6125e-1089-4308-bee8-064eefd92dec" (UID: "65d6125e-1089-4308-bee8-064eefd92dec"). InnerVolumeSpecName "kube-api-access-brqvg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:20:24.892725 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:20:24.892685    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/65d6125e-1089-4308-bee8-064eefd92dec-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "65d6125e-1089-4308-bee8-064eefd92dec" (UID: "65d6125e-1089-4308-bee8-064eefd92dec"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 17:20:24.892725 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:20:24.892719    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/65d6125e-1089-4308-bee8-064eefd92dec-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "65d6125e-1089-4308-bee8-064eefd92dec" (UID: "65d6125e-1089-4308-bee8-064eefd92dec"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:20:24.897954 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:20:24.897929    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/65d6125e-1089-4308-bee8-064eefd92dec-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "65d6125e-1089-4308-bee8-064eefd92dec" (UID: "65d6125e-1089-4308-bee8-064eefd92dec"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:20:24.991821 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:20:24.991784    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-brqvg\" (UniqueName: \"kubernetes.io/projected/65d6125e-1089-4308-bee8-064eefd92dec-kube-api-access-brqvg\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:20:24.991821 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:20:24.991817    2552 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/65d6125e-1089-4308-bee8-064eefd92dec-bound-sa-token\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:20:24.991821 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:20:24.991828    2552 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/65d6125e-1089-4308-bee8-064eefd92dec-ca-trust-extracted\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:20:24.992057 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:20:24.991839    2552 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/65d6125e-1089-4308-bee8-064eefd92dec-image-registry-private-configuration\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:20:24.992057 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:20:24.991849    2552 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/65d6125e-1089-4308-bee8-064eefd92dec-installation-pull-secrets\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:20:24.992057 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:20:24.991864    2552 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/65d6125e-1089-4308-bee8-064eefd92dec-registry-tls\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:20:25.539343 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:20:25.539312    2552 generic.go:356] "Generic (PLEG): container finished" podID="65d6125e-1089-4308-bee8-064eefd92dec" containerID="edc435e2ba3d0bd8ece5b118caac330c6e6bc35851691af5fd0aabf36bd0be74" exitCode=0
Sep 11 17:20:25.539517 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:20:25.539354    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-86bf889f8b-5hnlq"
Sep 11 17:20:25.539517 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:20:25.539367    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-86bf889f8b-5hnlq" event={"ID":"65d6125e-1089-4308-bee8-064eefd92dec","Type":"ContainerDied","Data":"edc435e2ba3d0bd8ece5b118caac330c6e6bc35851691af5fd0aabf36bd0be74"}
Sep 11 17:20:25.539517 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:20:25.539408    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-86bf889f8b-5hnlq" event={"ID":"65d6125e-1089-4308-bee8-064eefd92dec","Type":"ContainerDied","Data":"dad05c01f82ca8bdd39853766e70907d13181ce0d098ccb06f3353e98e9a3f03"}
Sep 11 17:20:25.539517 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:20:25.539428    2552 scope.go:117] "RemoveContainer" containerID="edc435e2ba3d0bd8ece5b118caac330c6e6bc35851691af5fd0aabf36bd0be74"
Sep 11 17:20:25.550733 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:20:25.550710    2552 scope.go:117] "RemoveContainer" containerID="edc435e2ba3d0bd8ece5b118caac330c6e6bc35851691af5fd0aabf36bd0be74"
Sep 11 17:20:25.551071 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:20:25.551049    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"edc435e2ba3d0bd8ece5b118caac330c6e6bc35851691af5fd0aabf36bd0be74\": container with ID starting with edc435e2ba3d0bd8ece5b118caac330c6e6bc35851691af5fd0aabf36bd0be74 not found: ID does not exist" containerID="edc435e2ba3d0bd8ece5b118caac330c6e6bc35851691af5fd0aabf36bd0be74"
Sep 11 17:20:25.551168 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:20:25.551079    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"edc435e2ba3d0bd8ece5b118caac330c6e6bc35851691af5fd0aabf36bd0be74"} err="failed to get container status \"edc435e2ba3d0bd8ece5b118caac330c6e6bc35851691af5fd0aabf36bd0be74\": rpc error: code = NotFound desc = could not find container \"edc435e2ba3d0bd8ece5b118caac330c6e6bc35851691af5fd0aabf36bd0be74\": container with ID starting with edc435e2ba3d0bd8ece5b118caac330c6e6bc35851691af5fd0aabf36bd0be74 not found: ID does not exist"
Sep 11 17:20:25.563694 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:20:25.563669    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-86bf889f8b-5hnlq"]
Sep 11 17:20:25.568346 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:20:25.568321    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-86bf889f8b-5hnlq"]
Sep 11 17:20:25.948402 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:20:25.948374    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="65d6125e-1089-4308-bee8-064eefd92dec" path="/var/lib/kubelet/pods/65d6125e-1089-4308-bee8-064eefd92dec/volumes"
Sep 11 17:20:28.781651 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:20:28.781594    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:20:28.885541 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:20:28.885498    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:20:30.972238 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:20:30.972198    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:20:31.154267 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:20:31.154230    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:20:36.581431 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:20:36.581399    2552 generic.go:356] "Generic (PLEG): container finished" podID="7583d484-6dd5-48e3-9485-05e57c62443e" containerID="e418bdaad64f64da336af4eb4c265596d5bb9125b82775b2913b7408fe3d34d0" exitCode=0
Sep 11 17:20:36.581843 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:20:36.581477    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-88bn9" event={"ID":"7583d484-6dd5-48e3-9485-05e57c62443e","Type":"ContainerDied","Data":"e418bdaad64f64da336af4eb4c265596d5bb9125b82775b2913b7408fe3d34d0"}
Sep 11 17:20:36.581843 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:20:36.581835    2552 scope.go:117] "RemoveContainer" containerID="e418bdaad64f64da336af4eb4c265596d5bb9125b82775b2913b7408fe3d34d0"
Sep 11 17:20:37.587141 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:20:37.587091    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-88bn9" event={"ID":"7583d484-6dd5-48e3-9485-05e57c62443e","Type":"ContainerStarted","Data":"834cdbb5e916dddee350742bfd831fa5ebd1568fbdd1a44074119cf1af793e63"}
Sep 11 17:20:40.061337 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:20:40.061300    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:20:40.177397 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:20:40.177365    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:20:40.286313 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:20:40.286280    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:20:42.020340 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:20:42.020302    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:21:05.965338 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:21:05.965296    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:21:06.040479 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:21:06.040444    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:21:12.967906 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:21:12.967868    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:21:18.028494 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:21:18.028459    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:21:18.128962 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:21:18.128928    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:21:30.877441 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:21:30.877405    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:21:30.975293 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:21:30.975253    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:21:32.638369 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:21:32.638332    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:21:32.709861 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:21:32.709826    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:21:36.234528 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:21:36.234494    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-55dd4dd97b-pczxw"]
Sep 11 17:21:36.234973 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:21:36.234931    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="65d6125e-1089-4308-bee8-064eefd92dec" containerName="registry"
Sep 11 17:21:36.234973 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:21:36.234950    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="65d6125e-1089-4308-bee8-064eefd92dec" containerName="registry"
Sep 11 17:21:36.235060 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:21:36.235035    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="65d6125e-1089-4308-bee8-064eefd92dec" containerName="registry"
Sep 11 17:21:36.237369 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:21:36.237351    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-55dd4dd97b-pczxw"
Sep 11 17:21:36.247868 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:21:36.247843    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-55dd4dd97b-pczxw"]
Sep 11 17:21:36.326089 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:21:36.326037    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:21:36.355210 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:21:36.355175    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8737406a-9e72-4560-8331-f791e2d118ca-oauth-serving-cert\") pod \"console-55dd4dd97b-pczxw\" (UID: \"8737406a-9e72-4560-8331-f791e2d118ca\") " pod="openshift-console/console-55dd4dd97b-pczxw"
Sep 11 17:21:36.355381 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:21:36.355224    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8737406a-9e72-4560-8331-f791e2d118ca-console-serving-cert\") pod \"console-55dd4dd97b-pczxw\" (UID: \"8737406a-9e72-4560-8331-f791e2d118ca\") " pod="openshift-console/console-55dd4dd97b-pczxw"
Sep 11 17:21:36.355381 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:21:36.355250    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8737406a-9e72-4560-8331-f791e2d118ca-console-config\") pod \"console-55dd4dd97b-pczxw\" (UID: \"8737406a-9e72-4560-8331-f791e2d118ca\") " pod="openshift-console/console-55dd4dd97b-pczxw"
Sep 11 17:21:36.355381 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:21:36.355288    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8737406a-9e72-4560-8331-f791e2d118ca-service-ca\") pod \"console-55dd4dd97b-pczxw\" (UID: \"8737406a-9e72-4560-8331-f791e2d118ca\") " pod="openshift-console/console-55dd4dd97b-pczxw"
Sep 11 17:21:36.355381 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:21:36.355367    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xckwg\" (UniqueName: \"kubernetes.io/projected/8737406a-9e72-4560-8331-f791e2d118ca-kube-api-access-xckwg\") pod \"console-55dd4dd97b-pczxw\" (UID: \"8737406a-9e72-4560-8331-f791e2d118ca\") " pod="openshift-console/console-55dd4dd97b-pczxw"
Sep 11 17:21:36.355559 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:21:36.355428    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8737406a-9e72-4560-8331-f791e2d118ca-trusted-ca-bundle\") pod \"console-55dd4dd97b-pczxw\" (UID: \"8737406a-9e72-4560-8331-f791e2d118ca\") " pod="openshift-console/console-55dd4dd97b-pczxw"
Sep 11 17:21:36.355559 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:21:36.355495    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8737406a-9e72-4560-8331-f791e2d118ca-console-oauth-config\") pod \"console-55dd4dd97b-pczxw\" (UID: \"8737406a-9e72-4560-8331-f791e2d118ca\") " pod="openshift-console/console-55dd4dd97b-pczxw"
Sep 11 17:21:36.456498 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:21:36.456461    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xckwg\" (UniqueName: \"kubernetes.io/projected/8737406a-9e72-4560-8331-f791e2d118ca-kube-api-access-xckwg\") pod \"console-55dd4dd97b-pczxw\" (UID: \"8737406a-9e72-4560-8331-f791e2d118ca\") " pod="openshift-console/console-55dd4dd97b-pczxw"
Sep 11 17:21:36.456498 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:21:36.456510    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8737406a-9e72-4560-8331-f791e2d118ca-trusted-ca-bundle\") pod \"console-55dd4dd97b-pczxw\" (UID: \"8737406a-9e72-4560-8331-f791e2d118ca\") " pod="openshift-console/console-55dd4dd97b-pczxw"
Sep 11 17:21:36.456781 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:21:36.456529    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8737406a-9e72-4560-8331-f791e2d118ca-console-oauth-config\") pod \"console-55dd4dd97b-pczxw\" (UID: \"8737406a-9e72-4560-8331-f791e2d118ca\") " pod="openshift-console/console-55dd4dd97b-pczxw"
Sep 11 17:21:36.456781 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:21:36.456657    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8737406a-9e72-4560-8331-f791e2d118ca-oauth-serving-cert\") pod \"console-55dd4dd97b-pczxw\" (UID: \"8737406a-9e72-4560-8331-f791e2d118ca\") " pod="openshift-console/console-55dd4dd97b-pczxw"
Sep 11 17:21:36.456781 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:21:36.456723    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8737406a-9e72-4560-8331-f791e2d118ca-console-serving-cert\") pod \"console-55dd4dd97b-pczxw\" (UID: \"8737406a-9e72-4560-8331-f791e2d118ca\") " pod="openshift-console/console-55dd4dd97b-pczxw"
Sep 11 17:21:36.456781 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:21:36.456748    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8737406a-9e72-4560-8331-f791e2d118ca-console-config\") pod \"console-55dd4dd97b-pczxw\" (UID: \"8737406a-9e72-4560-8331-f791e2d118ca\") " pod="openshift-console/console-55dd4dd97b-pczxw"
Sep 11 17:21:36.456781 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:21:36.456782    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8737406a-9e72-4560-8331-f791e2d118ca-service-ca\") pod \"console-55dd4dd97b-pczxw\" (UID: \"8737406a-9e72-4560-8331-f791e2d118ca\") " pod="openshift-console/console-55dd4dd97b-pczxw"
Sep 11 17:21:36.457428 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:21:36.457401    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8737406a-9e72-4560-8331-f791e2d118ca-oauth-serving-cert\") pod \"console-55dd4dd97b-pczxw\" (UID: \"8737406a-9e72-4560-8331-f791e2d118ca\") " pod="openshift-console/console-55dd4dd97b-pczxw"
Sep 11 17:21:36.457540 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:21:36.457451    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8737406a-9e72-4560-8331-f791e2d118ca-console-config\") pod \"console-55dd4dd97b-pczxw\" (UID: \"8737406a-9e72-4560-8331-f791e2d118ca\") " pod="openshift-console/console-55dd4dd97b-pczxw"
Sep 11 17:21:36.457588 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:21:36.457570    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8737406a-9e72-4560-8331-f791e2d118ca-trusted-ca-bundle\") pod \"console-55dd4dd97b-pczxw\" (UID: \"8737406a-9e72-4560-8331-f791e2d118ca\") " pod="openshift-console/console-55dd4dd97b-pczxw"
Sep 11 17:21:36.457626 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:21:36.457586    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8737406a-9e72-4560-8331-f791e2d118ca-service-ca\") pod \"console-55dd4dd97b-pczxw\" (UID: \"8737406a-9e72-4560-8331-f791e2d118ca\") " pod="openshift-console/console-55dd4dd97b-pczxw"
Sep 11 17:21:36.459100 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:21:36.459075    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8737406a-9e72-4560-8331-f791e2d118ca-console-oauth-config\") pod \"console-55dd4dd97b-pczxw\" (UID: \"8737406a-9e72-4560-8331-f791e2d118ca\") " pod="openshift-console/console-55dd4dd97b-pczxw"
Sep 11 17:21:36.459252 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:21:36.459234    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8737406a-9e72-4560-8331-f791e2d118ca-console-serving-cert\") pod \"console-55dd4dd97b-pczxw\" (UID: \"8737406a-9e72-4560-8331-f791e2d118ca\") " pod="openshift-console/console-55dd4dd97b-pczxw"
Sep 11 17:21:36.465069 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:21:36.465047    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xckwg\" (UniqueName: \"kubernetes.io/projected/8737406a-9e72-4560-8331-f791e2d118ca-kube-api-access-xckwg\") pod \"console-55dd4dd97b-pczxw\" (UID: \"8737406a-9e72-4560-8331-f791e2d118ca\") " pod="openshift-console/console-55dd4dd97b-pczxw"
Sep 11 17:21:36.548667 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:21:36.548576    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-55dd4dd97b-pczxw"
Sep 11 17:21:36.822440 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:21:36.822407    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-55dd4dd97b-pczxw"]
Sep 11 17:21:36.825195 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:21:36.825168    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8737406a_9e72_4560_8331_f791e2d118ca.slice/crio-2254b5398a354b4ab1685920cf6ed8f7e11147c6bc8c0f4dd54941e9c95cf0f9 WatchSource:0}: Error finding container 2254b5398a354b4ab1685920cf6ed8f7e11147c6bc8c0f4dd54941e9c95cf0f9: Status 404 returned error can't find the container with id 2254b5398a354b4ab1685920cf6ed8f7e11147c6bc8c0f4dd54941e9c95cf0f9
Sep 11 17:21:37.812187 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:21:37.812148    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-55dd4dd97b-pczxw" event={"ID":"8737406a-9e72-4560-8331-f791e2d118ca","Type":"ContainerStarted","Data":"e2ba79364c83b5234026e718ef2b0fd49f5ff0c3e07ae0d36d93f4d2651082fe"}
Sep 11 17:21:37.812567 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:21:37.812201    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-55dd4dd97b-pczxw" event={"ID":"8737406a-9e72-4560-8331-f791e2d118ca","Type":"ContainerStarted","Data":"2254b5398a354b4ab1685920cf6ed8f7e11147c6bc8c0f4dd54941e9c95cf0f9"}
Sep 11 17:21:37.832422 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:21:37.832365    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-55dd4dd97b-pczxw" podStartSLOduration=1.832348219 podStartE2EDuration="1.832348219s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:21:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:21:37.831039472 +0000 UTC m=+192.352330993" watchObservedRunningTime="2026-09-11 17:21:37.832348219 +0000 UTC m=+192.353639740"
Sep 11 17:21:46.549678 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:21:46.549645    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-55dd4dd97b-pczxw"
Sep 11 17:21:46.550202 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:21:46.549728    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-55dd4dd97b-pczxw"
Sep 11 17:21:46.555268 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:21:46.555240    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-55dd4dd97b-pczxw"
Sep 11 17:21:46.847523 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:21:46.847492    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-55dd4dd97b-pczxw"
Sep 11 17:21:46.965805 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:21:46.965767    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-75c57466fd-6k4ls"]
Sep 11 17:21:47.135288 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:21:47.135206    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:21:47.232206 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:21:47.232173    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:22:00.349873 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:00.349833    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:22:11.985362 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:11.985301    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-75c57466fd-6k4ls" podUID="c770a0fe-5881-40e3-b0c9-4e35427cf1fb" containerName="console" containerID="cri-o://d173542c4901005b9033f0296dc0f700110a1ab1ba44996803a3b866e5ddf02d" gracePeriod=15
Sep 11 17:22:12.224917 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:12.224889    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-75c57466fd-6k4ls"
Sep 11 17:22:12.325741 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:12.325654    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c7bzl\" (UniqueName: \"kubernetes.io/projected/c770a0fe-5881-40e3-b0c9-4e35427cf1fb-kube-api-access-c7bzl\") pod \"c770a0fe-5881-40e3-b0c9-4e35427cf1fb\" (UID: \"c770a0fe-5881-40e3-b0c9-4e35427cf1fb\") "
Sep 11 17:22:12.325741 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:12.325700    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c770a0fe-5881-40e3-b0c9-4e35427cf1fb-console-config\") pod \"c770a0fe-5881-40e3-b0c9-4e35427cf1fb\" (UID: \"c770a0fe-5881-40e3-b0c9-4e35427cf1fb\") "
Sep 11 17:22:12.325954 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:12.325744    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c770a0fe-5881-40e3-b0c9-4e35427cf1fb-console-serving-cert\") pod \"c770a0fe-5881-40e3-b0c9-4e35427cf1fb\" (UID: \"c770a0fe-5881-40e3-b0c9-4e35427cf1fb\") "
Sep 11 17:22:12.325954 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:12.325805    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c770a0fe-5881-40e3-b0c9-4e35427cf1fb-console-oauth-config\") pod \"c770a0fe-5881-40e3-b0c9-4e35427cf1fb\" (UID: \"c770a0fe-5881-40e3-b0c9-4e35427cf1fb\") "
Sep 11 17:22:12.325954 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:12.325860    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c770a0fe-5881-40e3-b0c9-4e35427cf1fb-trusted-ca-bundle\") pod \"c770a0fe-5881-40e3-b0c9-4e35427cf1fb\" (UID: \"c770a0fe-5881-40e3-b0c9-4e35427cf1fb\") "
Sep 11 17:22:12.325954 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:12.325913    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c770a0fe-5881-40e3-b0c9-4e35427cf1fb-oauth-serving-cert\") pod \"c770a0fe-5881-40e3-b0c9-4e35427cf1fb\" (UID: \"c770a0fe-5881-40e3-b0c9-4e35427cf1fb\") "
Sep 11 17:22:12.325954 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:12.325935    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c770a0fe-5881-40e3-b0c9-4e35427cf1fb-service-ca\") pod \"c770a0fe-5881-40e3-b0c9-4e35427cf1fb\" (UID: \"c770a0fe-5881-40e3-b0c9-4e35427cf1fb\") "
Sep 11 17:22:12.326291 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:12.326270    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c770a0fe-5881-40e3-b0c9-4e35427cf1fb-console-config" (OuterVolumeSpecName: "console-config") pod "c770a0fe-5881-40e3-b0c9-4e35427cf1fb" (UID: "c770a0fe-5881-40e3-b0c9-4e35427cf1fb"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 17:22:12.326400 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:12.326368    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c770a0fe-5881-40e3-b0c9-4e35427cf1fb-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "c770a0fe-5881-40e3-b0c9-4e35427cf1fb" (UID: "c770a0fe-5881-40e3-b0c9-4e35427cf1fb"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 17:22:12.326460 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:12.326407    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c770a0fe-5881-40e3-b0c9-4e35427cf1fb-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "c770a0fe-5881-40e3-b0c9-4e35427cf1fb" (UID: "c770a0fe-5881-40e3-b0c9-4e35427cf1fb"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 17:22:12.326512 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:12.326472    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c770a0fe-5881-40e3-b0c9-4e35427cf1fb-service-ca" (OuterVolumeSpecName: "service-ca") pod "c770a0fe-5881-40e3-b0c9-4e35427cf1fb" (UID: "c770a0fe-5881-40e3-b0c9-4e35427cf1fb"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 17:22:12.328227 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:12.328194    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c770a0fe-5881-40e3-b0c9-4e35427cf1fb-kube-api-access-c7bzl" (OuterVolumeSpecName: "kube-api-access-c7bzl") pod "c770a0fe-5881-40e3-b0c9-4e35427cf1fb" (UID: "c770a0fe-5881-40e3-b0c9-4e35427cf1fb"). InnerVolumeSpecName "kube-api-access-c7bzl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:22:12.328227 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:12.328194    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c770a0fe-5881-40e3-b0c9-4e35427cf1fb-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "c770a0fe-5881-40e3-b0c9-4e35427cf1fb" (UID: "c770a0fe-5881-40e3-b0c9-4e35427cf1fb"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 17:22:12.328405 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:12.328241    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c770a0fe-5881-40e3-b0c9-4e35427cf1fb-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "c770a0fe-5881-40e3-b0c9-4e35427cf1fb" (UID: "c770a0fe-5881-40e3-b0c9-4e35427cf1fb"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 17:22:12.426774 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:12.426734    2552 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c770a0fe-5881-40e3-b0c9-4e35427cf1fb-console-config\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:22:12.426774 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:12.426766    2552 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c770a0fe-5881-40e3-b0c9-4e35427cf1fb-console-serving-cert\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:22:12.426774 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:12.426776    2552 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c770a0fe-5881-40e3-b0c9-4e35427cf1fb-console-oauth-config\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:22:12.426774 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:12.426784    2552 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c770a0fe-5881-40e3-b0c9-4e35427cf1fb-trusted-ca-bundle\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:22:12.427036 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:12.426794    2552 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c770a0fe-5881-40e3-b0c9-4e35427cf1fb-oauth-serving-cert\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:22:12.427036 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:12.426802    2552 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c770a0fe-5881-40e3-b0c9-4e35427cf1fb-service-ca\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:22:12.427036 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:12.426810    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c7bzl\" (UniqueName: \"kubernetes.io/projected/c770a0fe-5881-40e3-b0c9-4e35427cf1fb-kube-api-access-c7bzl\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:22:12.940711 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:12.940672    2552 generic.go:356] "Generic (PLEG): container finished" podID="c770a0fe-5881-40e3-b0c9-4e35427cf1fb" containerID="d173542c4901005b9033f0296dc0f700110a1ab1ba44996803a3b866e5ddf02d" exitCode=0
Sep 11 17:22:12.940882 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:12.940720    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-75c57466fd-6k4ls"
Sep 11 17:22:12.940882 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:12.940760    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-75c57466fd-6k4ls" event={"ID":"c770a0fe-5881-40e3-b0c9-4e35427cf1fb","Type":"ContainerDied","Data":"d173542c4901005b9033f0296dc0f700110a1ab1ba44996803a3b866e5ddf02d"}
Sep 11 17:22:12.940882 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:12.940817    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-75c57466fd-6k4ls" event={"ID":"c770a0fe-5881-40e3-b0c9-4e35427cf1fb","Type":"ContainerDied","Data":"1a446d4ef074555d82eaa1cd75ace5a901c44e9fda509f902ae1e102ff4c338f"}
Sep 11 17:22:12.940882 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:12.940840    2552 scope.go:117] "RemoveContainer" containerID="d173542c4901005b9033f0296dc0f700110a1ab1ba44996803a3b866e5ddf02d"
Sep 11 17:22:12.949919 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:12.949901    2552 scope.go:117] "RemoveContainer" containerID="d173542c4901005b9033f0296dc0f700110a1ab1ba44996803a3b866e5ddf02d"
Sep 11 17:22:12.950329 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:22:12.950311    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d173542c4901005b9033f0296dc0f700110a1ab1ba44996803a3b866e5ddf02d\": container with ID starting with d173542c4901005b9033f0296dc0f700110a1ab1ba44996803a3b866e5ddf02d not found: ID does not exist" containerID="d173542c4901005b9033f0296dc0f700110a1ab1ba44996803a3b866e5ddf02d"
Sep 11 17:22:12.950377 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:12.950338    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d173542c4901005b9033f0296dc0f700110a1ab1ba44996803a3b866e5ddf02d"} err="failed to get container status \"d173542c4901005b9033f0296dc0f700110a1ab1ba44996803a3b866e5ddf02d\": rpc error: code = NotFound desc = could not find container \"d173542c4901005b9033f0296dc0f700110a1ab1ba44996803a3b866e5ddf02d\": container with ID starting with d173542c4901005b9033f0296dc0f700110a1ab1ba44996803a3b866e5ddf02d not found: ID does not exist"
Sep 11 17:22:12.961901 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:12.961872    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-75c57466fd-6k4ls"]
Sep 11 17:22:12.970206 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:12.970183    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-75c57466fd-6k4ls"]
Sep 11 17:22:13.947052 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:13.947016    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c770a0fe-5881-40e3-b0c9-4e35427cf1fb" path="/var/lib/kubelet/pods/c770a0fe-5881-40e3-b0c9-4e35427cf1fb/volumes"
Sep 11 17:22:15.303378 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:15.303347    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:22:15.398421 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:15.398387    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:22:16.681062 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:16.681022    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mfjbw"]
Sep 11 17:22:16.681563 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:16.681493    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c770a0fe-5881-40e3-b0c9-4e35427cf1fb" containerName="console"
Sep 11 17:22:16.681563 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:16.681511    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="c770a0fe-5881-40e3-b0c9-4e35427cf1fb" containerName="console"
Sep 11 17:22:16.681676 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:16.681597    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="c770a0fe-5881-40e3-b0c9-4e35427cf1fb" containerName="console"
Sep 11 17:22:16.684543 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:16.684524    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mfjbw"
Sep 11 17:22:16.686784 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:16.686755    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-cjjsx\""
Sep 11 17:22:16.686938 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:16.686915    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 11 17:22:16.687353 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:16.687334    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 11 17:22:16.716990 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:16.716954    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mfjbw"]
Sep 11 17:22:16.761536 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:16.761498    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6076e64f-cf7b-437d-8878-4285fdd99c71-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mfjbw\" (UID: \"6076e64f-cf7b-437d-8878-4285fdd99c71\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mfjbw"
Sep 11 17:22:16.761708 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:16.761537    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g7pf8\" (UniqueName: \"kubernetes.io/projected/6076e64f-cf7b-437d-8878-4285fdd99c71-kube-api-access-g7pf8\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mfjbw\" (UID: \"6076e64f-cf7b-437d-8878-4285fdd99c71\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mfjbw"
Sep 11 17:22:16.761708 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:16.761691    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6076e64f-cf7b-437d-8878-4285fdd99c71-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mfjbw\" (UID: \"6076e64f-cf7b-437d-8878-4285fdd99c71\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mfjbw"
Sep 11 17:22:16.862257 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:16.862213    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6076e64f-cf7b-437d-8878-4285fdd99c71-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mfjbw\" (UID: \"6076e64f-cf7b-437d-8878-4285fdd99c71\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mfjbw"
Sep 11 17:22:16.862257 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:16.862259    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g7pf8\" (UniqueName: \"kubernetes.io/projected/6076e64f-cf7b-437d-8878-4285fdd99c71-kube-api-access-g7pf8\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mfjbw\" (UID: \"6076e64f-cf7b-437d-8878-4285fdd99c71\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mfjbw"
Sep 11 17:22:16.862481 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:16.862326    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6076e64f-cf7b-437d-8878-4285fdd99c71-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mfjbw\" (UID: \"6076e64f-cf7b-437d-8878-4285fdd99c71\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mfjbw"
Sep 11 17:22:16.862640 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:16.862619    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6076e64f-cf7b-437d-8878-4285fdd99c71-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mfjbw\" (UID: \"6076e64f-cf7b-437d-8878-4285fdd99c71\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mfjbw"
Sep 11 17:22:16.862692 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:16.862673    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6076e64f-cf7b-437d-8878-4285fdd99c71-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mfjbw\" (UID: \"6076e64f-cf7b-437d-8878-4285fdd99c71\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mfjbw"
Sep 11 17:22:16.870857 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:16.870824    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g7pf8\" (UniqueName: \"kubernetes.io/projected/6076e64f-cf7b-437d-8878-4285fdd99c71-kube-api-access-g7pf8\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mfjbw\" (UID: \"6076e64f-cf7b-437d-8878-4285fdd99c71\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mfjbw"
Sep 11 17:22:16.993720 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:16.993625    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mfjbw"
Sep 11 17:22:17.274960 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:17.274902    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mfjbw"]
Sep 11 17:22:17.277334 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:22:17.277294    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6076e64f_cf7b_437d_8878_4285fdd99c71.slice/crio-8ab8522ae80c5695aff68bdad9bb1cfb6587fefcc7c417a4bccdeb4f36d0d123 WatchSource:0}: Error finding container 8ab8522ae80c5695aff68bdad9bb1cfb6587fefcc7c417a4bccdeb4f36d0d123: Status 404 returned error can't find the container with id 8ab8522ae80c5695aff68bdad9bb1cfb6587fefcc7c417a4bccdeb4f36d0d123
Sep 11 17:22:17.961659 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:17.961615    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mfjbw" event={"ID":"6076e64f-cf7b-437d-8878-4285fdd99c71","Type":"ContainerStarted","Data":"8ab8522ae80c5695aff68bdad9bb1cfb6587fefcc7c417a4bccdeb4f36d0d123"}
Sep 11 17:22:24.994395 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:24.994362    2552 generic.go:356] "Generic (PLEG): container finished" podID="6076e64f-cf7b-437d-8878-4285fdd99c71" containerID="54d244c7f3e0c285af2aa4dce39f5495b5d4572de062aee083e0d38bd1f8facd" exitCode=0
Sep 11 17:22:24.994749 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:24.994449    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mfjbw" event={"ID":"6076e64f-cf7b-437d-8878-4285fdd99c71","Type":"ContainerDied","Data":"54d244c7f3e0c285af2aa4dce39f5495b5d4572de062aee083e0d38bd1f8facd"}
Sep 11 17:22:28.009405 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:28.009350    2552 generic.go:356] "Generic (PLEG): container finished" podID="6076e64f-cf7b-437d-8878-4285fdd99c71" containerID="cbe05d6ed41e3cbd567004b99677f918849784f75b9a7a06585dfbcfe50f6079" exitCode=0
Sep 11 17:22:28.009819 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:28.009410    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mfjbw" event={"ID":"6076e64f-cf7b-437d-8878-4285fdd99c71","Type":"ContainerDied","Data":"cbe05d6ed41e3cbd567004b99677f918849784f75b9a7a06585dfbcfe50f6079"}
Sep 11 17:22:40.063851 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:40.063800    2552 generic.go:356] "Generic (PLEG): container finished" podID="6076e64f-cf7b-437d-8878-4285fdd99c71" containerID="66c0ae99003f93c518b8788be0ee36efaa7617eb653ece12cc1566df995d80ad" exitCode=0
Sep 11 17:22:40.064352 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:40.063891    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mfjbw" event={"ID":"6076e64f-cf7b-437d-8878-4285fdd99c71","Type":"ContainerDied","Data":"66c0ae99003f93c518b8788be0ee36efaa7617eb653ece12cc1566df995d80ad"}
Sep 11 17:22:41.206551 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:41.206517    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mfjbw"
Sep 11 17:22:41.376362 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:41.376320    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g7pf8\" (UniqueName: \"kubernetes.io/projected/6076e64f-cf7b-437d-8878-4285fdd99c71-kube-api-access-g7pf8\") pod \"6076e64f-cf7b-437d-8878-4285fdd99c71\" (UID: \"6076e64f-cf7b-437d-8878-4285fdd99c71\") "
Sep 11 17:22:41.376362 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:41.376371    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6076e64f-cf7b-437d-8878-4285fdd99c71-util\") pod \"6076e64f-cf7b-437d-8878-4285fdd99c71\" (UID: \"6076e64f-cf7b-437d-8878-4285fdd99c71\") "
Sep 11 17:22:41.376583 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:41.376420    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6076e64f-cf7b-437d-8878-4285fdd99c71-bundle\") pod \"6076e64f-cf7b-437d-8878-4285fdd99c71\" (UID: \"6076e64f-cf7b-437d-8878-4285fdd99c71\") "
Sep 11 17:22:41.377164 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:41.377139    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6076e64f-cf7b-437d-8878-4285fdd99c71-bundle" (OuterVolumeSpecName: "bundle") pod "6076e64f-cf7b-437d-8878-4285fdd99c71" (UID: "6076e64f-cf7b-437d-8878-4285fdd99c71"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:22:41.378687 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:41.378651    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6076e64f-cf7b-437d-8878-4285fdd99c71-kube-api-access-g7pf8" (OuterVolumeSpecName: "kube-api-access-g7pf8") pod "6076e64f-cf7b-437d-8878-4285fdd99c71" (UID: "6076e64f-cf7b-437d-8878-4285fdd99c71"). InnerVolumeSpecName "kube-api-access-g7pf8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:22:41.380700 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:41.380678    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6076e64f-cf7b-437d-8878-4285fdd99c71-util" (OuterVolumeSpecName: "util") pod "6076e64f-cf7b-437d-8878-4285fdd99c71" (UID: "6076e64f-cf7b-437d-8878-4285fdd99c71"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:22:41.477742 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:41.477705    2552 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6076e64f-cf7b-437d-8878-4285fdd99c71-bundle\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:22:41.477742 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:41.477737    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g7pf8\" (UniqueName: \"kubernetes.io/projected/6076e64f-cf7b-437d-8878-4285fdd99c71-kube-api-access-g7pf8\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:22:41.477742 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:41.477749    2552 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6076e64f-cf7b-437d-8878-4285fdd99c71-util\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:22:42.073403 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:42.073365    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mfjbw" event={"ID":"6076e64f-cf7b-437d-8878-4285fdd99c71","Type":"ContainerDied","Data":"8ab8522ae80c5695aff68bdad9bb1cfb6587fefcc7c417a4bccdeb4f36d0d123"}
Sep 11 17:22:42.073588 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:42.073446    2552 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8ab8522ae80c5695aff68bdad9bb1cfb6587fefcc7c417a4bccdeb4f36d0d123"
Sep 11 17:22:42.073588 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:42.073378    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mfjbw"
Sep 11 17:22:42.348826 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:42.348798    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:22:42.420391 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:42.420348    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:22:47.928440 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:47.928390    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:22:48.734340 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:48.734306    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:22:49.821896 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:49.821858    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-bl82f"]
Sep 11 17:22:49.822321 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:49.822178    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6076e64f-cf7b-437d-8878-4285fdd99c71" containerName="util"
Sep 11 17:22:49.822321 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:49.822189    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="6076e64f-cf7b-437d-8878-4285fdd99c71" containerName="util"
Sep 11 17:22:49.822321 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:49.822199    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6076e64f-cf7b-437d-8878-4285fdd99c71" containerName="extract"
Sep 11 17:22:49.822321 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:49.822204    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="6076e64f-cf7b-437d-8878-4285fdd99c71" containerName="extract"
Sep 11 17:22:49.822321 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:49.822214    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6076e64f-cf7b-437d-8878-4285fdd99c71" containerName="pull"
Sep 11 17:22:49.822321 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:49.822220    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="6076e64f-cf7b-437d-8878-4285fdd99c71" containerName="pull"
Sep 11 17:22:49.822321 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:49.822273    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="6076e64f-cf7b-437d-8878-4285fdd99c71" containerName="extract"
Sep 11 17:22:49.824120 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:49.824094    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-bl82f"
Sep 11 17:22:49.827906 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:49.827882    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Sep 11 17:22:49.827906 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:49.827889    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Sep 11 17:22:49.828142 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:49.827967    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-txz96\""
Sep 11 17:22:49.840372 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:49.840348    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/88956bbe-b27d-42a3-98fa-3eb491b2c650-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-bl82f\" (UID: \"88956bbe-b27d-42a3-98fa-3eb491b2c650\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-bl82f"
Sep 11 17:22:49.840494 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:49.840419    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7jrs7\" (UniqueName: \"kubernetes.io/projected/88956bbe-b27d-42a3-98fa-3eb491b2c650-kube-api-access-7jrs7\") pod \"cert-manager-operator-controller-manager-55c444bf86-bl82f\" (UID: \"88956bbe-b27d-42a3-98fa-3eb491b2c650\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-bl82f"
Sep 11 17:22:49.845341 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:49.845318    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-bl82f"]
Sep 11 17:22:49.941443 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:49.941401    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/88956bbe-b27d-42a3-98fa-3eb491b2c650-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-bl82f\" (UID: \"88956bbe-b27d-42a3-98fa-3eb491b2c650\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-bl82f"
Sep 11 17:22:49.941650 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:49.941471    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7jrs7\" (UniqueName: \"kubernetes.io/projected/88956bbe-b27d-42a3-98fa-3eb491b2c650-kube-api-access-7jrs7\") pod \"cert-manager-operator-controller-manager-55c444bf86-bl82f\" (UID: \"88956bbe-b27d-42a3-98fa-3eb491b2c650\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-bl82f"
Sep 11 17:22:49.941851 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:49.941828    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/88956bbe-b27d-42a3-98fa-3eb491b2c650-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-bl82f\" (UID: \"88956bbe-b27d-42a3-98fa-3eb491b2c650\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-bl82f"
Sep 11 17:22:49.953010 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:49.952978    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7jrs7\" (UniqueName: \"kubernetes.io/projected/88956bbe-b27d-42a3-98fa-3eb491b2c650-kube-api-access-7jrs7\") pod \"cert-manager-operator-controller-manager-55c444bf86-bl82f\" (UID: \"88956bbe-b27d-42a3-98fa-3eb491b2c650\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-bl82f"
Sep 11 17:22:50.132508 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:50.132464    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-bl82f"
Sep 11 17:22:50.425340 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:50.425246    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-bl82f"]
Sep 11 17:22:50.430064 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:22:50.430037    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88956bbe_b27d_42a3_98fa_3eb491b2c650.slice/crio-2416ab7cbe60d17b82114a34b01ffd4f773db9411f77eee1da14813cd5c9a8af WatchSource:0}: Error finding container 2416ab7cbe60d17b82114a34b01ffd4f773db9411f77eee1da14813cd5c9a8af: Status 404 returned error can't find the container with id 2416ab7cbe60d17b82114a34b01ffd4f773db9411f77eee1da14813cd5c9a8af
Sep 11 17:22:51.109974 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:51.109935    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-bl82f" event={"ID":"88956bbe-b27d-42a3-98fa-3eb491b2c650","Type":"ContainerStarted","Data":"2416ab7cbe60d17b82114a34b01ffd4f773db9411f77eee1da14813cd5c9a8af"}
Sep 11 17:22:51.495095 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:51.494991    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:22:51.598960 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:51.598920    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:22:53.060662 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:53.060623    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:22:53.126969 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:53.126932    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-bl82f" event={"ID":"88956bbe-b27d-42a3-98fa-3eb491b2c650","Type":"ContainerStarted","Data":"b1dce5680289bcde981aec353a92867ab2a4d999366d8ee50d38698b50b0f699"}
Sep 11 17:22:53.157426 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:53.157355    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-bl82f" podStartSLOduration=2.176538944 podStartE2EDuration="4.157338114s" totalImagesPullingTime="1.98079917s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:22:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:22:53.156287428 +0000 UTC m=+267.677578961" watchObservedRunningTime="2026-09-11 17:22:53.157338114 +0000 UTC m=+267.678629633"
Sep 11 17:22:53.166503 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:53.166468    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:22:53.661058 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:53.661018    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:22:53.781934 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:53.781901    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:22:54.247684 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:54.247645    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpzsr8"]
Sep 11 17:22:54.250333 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:54.250311    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpzsr8"
Sep 11 17:22:54.252474 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:54.252449    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 11 17:22:54.252594 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:54.252545    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-cjjsx\""
Sep 11 17:22:54.255437 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:54.255416    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 11 17:22:54.260094 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:54.260068    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpzsr8"]
Sep 11 17:22:54.272545 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:54.272518    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2272e005-61d4-4436-a24a-0c4b32f81992-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpzsr8\" (UID: \"2272e005-61d4-4436-a24a-0c4b32f81992\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpzsr8"
Sep 11 17:22:54.272680 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:54.272551    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rjnrh\" (UniqueName: \"kubernetes.io/projected/2272e005-61d4-4436-a24a-0c4b32f81992-kube-api-access-rjnrh\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpzsr8\" (UID: \"2272e005-61d4-4436-a24a-0c4b32f81992\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpzsr8"
Sep 11 17:22:54.272680 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:54.272580    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2272e005-61d4-4436-a24a-0c4b32f81992-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpzsr8\" (UID: \"2272e005-61d4-4436-a24a-0c4b32f81992\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpzsr8"
Sep 11 17:22:54.372961 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:54.372919    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2272e005-61d4-4436-a24a-0c4b32f81992-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpzsr8\" (UID: \"2272e005-61d4-4436-a24a-0c4b32f81992\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpzsr8"
Sep 11 17:22:54.372961 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:54.372960    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rjnrh\" (UniqueName: \"kubernetes.io/projected/2272e005-61d4-4436-a24a-0c4b32f81992-kube-api-access-rjnrh\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpzsr8\" (UID: \"2272e005-61d4-4436-a24a-0c4b32f81992\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpzsr8"
Sep 11 17:22:54.373210 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:54.372997    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2272e005-61d4-4436-a24a-0c4b32f81992-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpzsr8\" (UID: \"2272e005-61d4-4436-a24a-0c4b32f81992\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpzsr8"
Sep 11 17:22:54.373429 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:54.373409    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2272e005-61d4-4436-a24a-0c4b32f81992-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpzsr8\" (UID: \"2272e005-61d4-4436-a24a-0c4b32f81992\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpzsr8"
Sep 11 17:22:54.373429 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:54.373419    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2272e005-61d4-4436-a24a-0c4b32f81992-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpzsr8\" (UID: \"2272e005-61d4-4436-a24a-0c4b32f81992\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpzsr8"
Sep 11 17:22:54.381344 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:54.381321    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rjnrh\" (UniqueName: \"kubernetes.io/projected/2272e005-61d4-4436-a24a-0c4b32f81992-kube-api-access-rjnrh\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpzsr8\" (UID: \"2272e005-61d4-4436-a24a-0c4b32f81992\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpzsr8"
Sep 11 17:22:54.559592 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:54.559509    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpzsr8"
Sep 11 17:22:54.829512 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:54.829486    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpzsr8"]
Sep 11 17:22:54.831888 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:22:54.831857    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2272e005_61d4_4436_a24a_0c4b32f81992.slice/crio-2c3466094801aa70589f5252b463093c5477eb7b90611c4738359d0eaaf3bf81 WatchSource:0}: Error finding container 2c3466094801aa70589f5252b463093c5477eb7b90611c4738359d0eaaf3bf81: Status 404 returned error can't find the container with id 2c3466094801aa70589f5252b463093c5477eb7b90611c4738359d0eaaf3bf81
Sep 11 17:22:55.137331 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:55.137298    2552 generic.go:356] "Generic (PLEG): container finished" podID="2272e005-61d4-4436-a24a-0c4b32f81992" containerID="994958aa398329ee16749ab9967d463c24b9b38fe89e4de2c0ea6edae3a07038" exitCode=0
Sep 11 17:22:55.137498 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:55.137388    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpzsr8" event={"ID":"2272e005-61d4-4436-a24a-0c4b32f81992","Type":"ContainerDied","Data":"994958aa398329ee16749ab9967d463c24b9b38fe89e4de2c0ea6edae3a07038"}
Sep 11 17:22:55.137498 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:55.137439    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpzsr8" event={"ID":"2272e005-61d4-4436-a24a-0c4b32f81992","Type":"ContainerStarted","Data":"2c3466094801aa70589f5252b463093c5477eb7b90611c4738359d0eaaf3bf81"}
Sep 11 17:22:55.319441 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:55.319394    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:22:55.473186 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:55.473081    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:22:58.074828 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:58.074794    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:22:58.135939 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:58.135907    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:22:58.153961 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:58.153929    2552 generic.go:356] "Generic (PLEG): container finished" podID="2272e005-61d4-4436-a24a-0c4b32f81992" containerID="babe875efb205138df732612bd2a7810fd3f1eae50a9ea38cc3c9e26ae122e54" exitCode=0
Sep 11 17:22:58.154175 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:58.153983    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpzsr8" event={"ID":"2272e005-61d4-4436-a24a-0c4b32f81992","Type":"ContainerDied","Data":"babe875efb205138df732612bd2a7810fd3f1eae50a9ea38cc3c9e26ae122e54"}
Sep 11 17:22:59.161101 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:59.161063    2552 generic.go:356] "Generic (PLEG): container finished" podID="2272e005-61d4-4436-a24a-0c4b32f81992" containerID="744cb1e98f21e2a680f4a67c800c90a03f574088f0fc49690705c4089211c5b1" exitCode=0
Sep 11 17:22:59.161541 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:59.161207    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpzsr8" event={"ID":"2272e005-61d4-4436-a24a-0c4b32f81992","Type":"ContainerDied","Data":"744cb1e98f21e2a680f4a67c800c90a03f574088f0fc49690705c4089211c5b1"}
Sep 11 17:22:59.210205 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:59.210171    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-9msfk"]
Sep 11 17:22:59.212491 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:59.212468    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-9msfk"
Sep 11 17:22:59.214769 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:59.214750    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-gfzbb\""
Sep 11 17:22:59.215054 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:59.215036    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Sep 11 17:22:59.215137 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:59.215038    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Sep 11 17:22:59.227978 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:59.227945    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-9msfk"]
Sep 11 17:22:59.311193 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:59.311163    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nsxwx\" (UniqueName: \"kubernetes.io/projected/00a327f0-2d62-4ff5-b34e-add55c0924e1-kube-api-access-nsxwx\") pod \"cert-manager-webhook-6486ffd997-9msfk\" (UID: \"00a327f0-2d62-4ff5-b34e-add55c0924e1\") " pod="cert-manager/cert-manager-webhook-6486ffd997-9msfk"
Sep 11 17:22:59.311355 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:59.311212    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/00a327f0-2d62-4ff5-b34e-add55c0924e1-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-9msfk\" (UID: \"00a327f0-2d62-4ff5-b34e-add55c0924e1\") " pod="cert-manager/cert-manager-webhook-6486ffd997-9msfk"
Sep 11 17:22:59.412486 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:59.412392    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/00a327f0-2d62-4ff5-b34e-add55c0924e1-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-9msfk\" (UID: \"00a327f0-2d62-4ff5-b34e-add55c0924e1\") " pod="cert-manager/cert-manager-webhook-6486ffd997-9msfk"
Sep 11 17:22:59.412486 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:59.412471    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nsxwx\" (UniqueName: \"kubernetes.io/projected/00a327f0-2d62-4ff5-b34e-add55c0924e1-kube-api-access-nsxwx\") pod \"cert-manager-webhook-6486ffd997-9msfk\" (UID: \"00a327f0-2d62-4ff5-b34e-add55c0924e1\") " pod="cert-manager/cert-manager-webhook-6486ffd997-9msfk"
Sep 11 17:22:59.422151 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:59.422095    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/00a327f0-2d62-4ff5-b34e-add55c0924e1-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-9msfk\" (UID: \"00a327f0-2d62-4ff5-b34e-add55c0924e1\") " pod="cert-manager/cert-manager-webhook-6486ffd997-9msfk"
Sep 11 17:22:59.422556 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:59.422535    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nsxwx\" (UniqueName: \"kubernetes.io/projected/00a327f0-2d62-4ff5-b34e-add55c0924e1-kube-api-access-nsxwx\") pod \"cert-manager-webhook-6486ffd997-9msfk\" (UID: \"00a327f0-2d62-4ff5-b34e-add55c0924e1\") " pod="cert-manager/cert-manager-webhook-6486ffd997-9msfk"
Sep 11 17:22:59.527609 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:59.527570    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-9msfk"
Sep 11 17:22:59.818950 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:22:59.818867    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-9msfk"]
Sep 11 17:22:59.822442 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:22:59.822412    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod00a327f0_2d62_4ff5_b34e_add55c0924e1.slice/crio-304f7f7dbab470efcb595c4965c3c35394f80f86db4a3c2c7896a0c32bdf32f8 WatchSource:0}: Error finding container 304f7f7dbab470efcb595c4965c3c35394f80f86db4a3c2c7896a0c32bdf32f8: Status 404 returned error can't find the container with id 304f7f7dbab470efcb595c4965c3c35394f80f86db4a3c2c7896a0c32bdf32f8
Sep 11 17:23:00.166612 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:00.166578    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-9msfk" event={"ID":"00a327f0-2d62-4ff5-b34e-add55c0924e1","Type":"ContainerStarted","Data":"304f7f7dbab470efcb595c4965c3c35394f80f86db4a3c2c7896a0c32bdf32f8"}
Sep 11 17:23:00.286125 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:00.286085    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpzsr8"
Sep 11 17:23:00.318987 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:00.318953    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2272e005-61d4-4436-a24a-0c4b32f81992-bundle\") pod \"2272e005-61d4-4436-a24a-0c4b32f81992\" (UID: \"2272e005-61d4-4436-a24a-0c4b32f81992\") "
Sep 11 17:23:00.319181 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:00.319000    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rjnrh\" (UniqueName: \"kubernetes.io/projected/2272e005-61d4-4436-a24a-0c4b32f81992-kube-api-access-rjnrh\") pod \"2272e005-61d4-4436-a24a-0c4b32f81992\" (UID: \"2272e005-61d4-4436-a24a-0c4b32f81992\") "
Sep 11 17:23:00.319181 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:00.319054    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2272e005-61d4-4436-a24a-0c4b32f81992-util\") pod \"2272e005-61d4-4436-a24a-0c4b32f81992\" (UID: \"2272e005-61d4-4436-a24a-0c4b32f81992\") "
Sep 11 17:23:00.319494 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:00.319461    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2272e005-61d4-4436-a24a-0c4b32f81992-bundle" (OuterVolumeSpecName: "bundle") pod "2272e005-61d4-4436-a24a-0c4b32f81992" (UID: "2272e005-61d4-4436-a24a-0c4b32f81992"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:23:00.321366 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:00.321336    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2272e005-61d4-4436-a24a-0c4b32f81992-kube-api-access-rjnrh" (OuterVolumeSpecName: "kube-api-access-rjnrh") pod "2272e005-61d4-4436-a24a-0c4b32f81992" (UID: "2272e005-61d4-4436-a24a-0c4b32f81992"). InnerVolumeSpecName "kube-api-access-rjnrh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:23:00.323802 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:00.323756    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2272e005-61d4-4436-a24a-0c4b32f81992-util" (OuterVolumeSpecName: "util") pod "2272e005-61d4-4436-a24a-0c4b32f81992" (UID: "2272e005-61d4-4436-a24a-0c4b32f81992"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:23:00.420477 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:00.420376    2552 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2272e005-61d4-4436-a24a-0c4b32f81992-util\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:23:00.420477 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:00.420419    2552 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2272e005-61d4-4436-a24a-0c4b32f81992-bundle\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:23:00.420477 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:00.420428    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rjnrh\" (UniqueName: \"kubernetes.io/projected/2272e005-61d4-4436-a24a-0c4b32f81992-kube-api-access-rjnrh\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:23:01.173051 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:01.173025    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpzsr8"
Sep 11 17:23:01.173468 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:01.173023    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpzsr8" event={"ID":"2272e005-61d4-4436-a24a-0c4b32f81992","Type":"ContainerDied","Data":"2c3466094801aa70589f5252b463093c5477eb7b90611c4738359d0eaaf3bf81"}
Sep 11 17:23:01.173468 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:01.173197    2552 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2c3466094801aa70589f5252b463093c5477eb7b90611c4738359d0eaaf3bf81"
Sep 11 17:23:03.037735 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:03.037637    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:23:09.091960 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:09.091923    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:23:09.189581 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:09.189545    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:23:10.216684 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:10.216639    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-9msfk" event={"ID":"00a327f0-2d62-4ff5-b34e-add55c0924e1","Type":"ContainerStarted","Data":"08c36132c8074c922bda461bce157d9643b4bb1e44ee8b1e66ce2ddb27652cf9"}
Sep 11 17:23:10.216684 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:10.216691    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-9msfk"
Sep 11 17:23:10.237427 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:10.237373    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-9msfk" podStartSLOduration=1.693442315 podStartE2EDuration="11.237354621s" totalImagesPullingTime="9.543912306s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:22:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:23:10.23616925 +0000 UTC m=+284.757460773" watchObservedRunningTime="2026-09-11 17:23:10.237354621 +0000 UTC m=+284.758646141"
Sep 11 17:23:11.103723 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:11.103690    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-mfsgs"]
Sep 11 17:23:11.104011 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:11.103998    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2272e005-61d4-4436-a24a-0c4b32f81992" containerName="extract"
Sep 11 17:23:11.104056 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:11.104012    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="2272e005-61d4-4436-a24a-0c4b32f81992" containerName="extract"
Sep 11 17:23:11.104056 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:11.104022    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2272e005-61d4-4436-a24a-0c4b32f81992" containerName="util"
Sep 11 17:23:11.104056 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:11.104027    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="2272e005-61d4-4436-a24a-0c4b32f81992" containerName="util"
Sep 11 17:23:11.104056 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:11.104038    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2272e005-61d4-4436-a24a-0c4b32f81992" containerName="pull"
Sep 11 17:23:11.104056 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:11.104044    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="2272e005-61d4-4436-a24a-0c4b32f81992" containerName="pull"
Sep 11 17:23:11.104244 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:11.104093    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="2272e005-61d4-4436-a24a-0c4b32f81992" containerName="extract"
Sep 11 17:23:11.122671 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:11.122641    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-mfsgs"]
Sep 11 17:23:11.122823 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:11.122736    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-mfsgs"
Sep 11 17:23:11.125157 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:11.125125    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 11 17:23:11.125308 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:11.125173    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-29kqg\""
Sep 11 17:23:11.125308 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:11.125179    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 11 17:23:11.209095 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:11.209054    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/da644d9f-2b46-44dc-8757-60e68db81d23-tmp\") pod \"openshift-lws-operator-fd8ccff4c-mfsgs\" (UID: \"da644d9f-2b46-44dc-8757-60e68db81d23\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-mfsgs"
Sep 11 17:23:11.209300 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:11.209136    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-drjkc\" (UniqueName: \"kubernetes.io/projected/da644d9f-2b46-44dc-8757-60e68db81d23-kube-api-access-drjkc\") pod \"openshift-lws-operator-fd8ccff4c-mfsgs\" (UID: \"da644d9f-2b46-44dc-8757-60e68db81d23\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-mfsgs"
Sep 11 17:23:11.310342 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:11.310302    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-drjkc\" (UniqueName: \"kubernetes.io/projected/da644d9f-2b46-44dc-8757-60e68db81d23-kube-api-access-drjkc\") pod \"openshift-lws-operator-fd8ccff4c-mfsgs\" (UID: \"da644d9f-2b46-44dc-8757-60e68db81d23\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-mfsgs"
Sep 11 17:23:11.310797 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:11.310408    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/da644d9f-2b46-44dc-8757-60e68db81d23-tmp\") pod \"openshift-lws-operator-fd8ccff4c-mfsgs\" (UID: \"da644d9f-2b46-44dc-8757-60e68db81d23\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-mfsgs"
Sep 11 17:23:11.310890 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:11.310867    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/da644d9f-2b46-44dc-8757-60e68db81d23-tmp\") pod \"openshift-lws-operator-fd8ccff4c-mfsgs\" (UID: \"da644d9f-2b46-44dc-8757-60e68db81d23\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-mfsgs"
Sep 11 17:23:11.319044 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:11.319012    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-drjkc\" (UniqueName: \"kubernetes.io/projected/da644d9f-2b46-44dc-8757-60e68db81d23-kube-api-access-drjkc\") pod \"openshift-lws-operator-fd8ccff4c-mfsgs\" (UID: \"da644d9f-2b46-44dc-8757-60e68db81d23\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-mfsgs"
Sep 11 17:23:11.433987 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:11.433887    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-mfsgs"
Sep 11 17:23:11.719037 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:11.719008    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-mfsgs"]
Sep 11 17:23:11.721460 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:23:11.721425    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podda644d9f_2b46_44dc_8757_60e68db81d23.slice/crio-cadefb1f4e527b752687244f1f01b0c5faca5a1ad00777c7d5aab36ab6d27c27 WatchSource:0}: Error finding container cadefb1f4e527b752687244f1f01b0c5faca5a1ad00777c7d5aab36ab6d27c27: Status 404 returned error can't find the container with id cadefb1f4e527b752687244f1f01b0c5faca5a1ad00777c7d5aab36ab6d27c27
Sep 11 17:23:12.226966 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:12.226928    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-mfsgs" event={"ID":"da644d9f-2b46-44dc-8757-60e68db81d23","Type":"ContainerStarted","Data":"cadefb1f4e527b752687244f1f01b0c5faca5a1ad00777c7d5aab36ab6d27c27"}
Sep 11 17:23:14.239523 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:14.239477    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-mfsgs" event={"ID":"da644d9f-2b46-44dc-8757-60e68db81d23","Type":"ContainerStarted","Data":"0ff0653034608704b6470cb654092a2c279ae41ac8b65ba5fad889bfafe676e1"}
Sep 11 17:23:14.268287 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:14.268223    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-mfsgs" podStartSLOduration=1.673515166 podStartE2EDuration="3.268205566s" totalImagesPullingTime="1.5946904s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:23:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:23:14.267742046 +0000 UTC m=+288.789033568" watchObservedRunningTime="2026-09-11 17:23:14.268205566 +0000 UTC m=+288.789497086"
Sep 11 17:23:14.980059 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:14.980022    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:23:15.134186 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:15.134149    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:23:16.223719 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:16.223689    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-9msfk"
Sep 11 17:23:17.220779 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:17.220738    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:23:18.085827 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:18.085796    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnhsmt"]
Sep 11 17:23:18.088082 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:18.088064    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnhsmt"
Sep 11 17:23:18.090186 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:18.090165    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 11 17:23:18.090308 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:18.090222    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 11 17:23:18.090308 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:18.090286    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-cjjsx\""
Sep 11 17:23:18.098713 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:18.098687    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnhsmt"]
Sep 11 17:23:18.167140 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:18.167091    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qw2fx\" (UniqueName: \"kubernetes.io/projected/0904f7f8-6e45-4796-a906-3920ac0d7932-kube-api-access-qw2fx\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnhsmt\" (UID: \"0904f7f8-6e45-4796-a906-3920ac0d7932\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnhsmt"
Sep 11 17:23:18.167321 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:18.167199    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0904f7f8-6e45-4796-a906-3920ac0d7932-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnhsmt\" (UID: \"0904f7f8-6e45-4796-a906-3920ac0d7932\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnhsmt"
Sep 11 17:23:18.167321 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:18.167249    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0904f7f8-6e45-4796-a906-3920ac0d7932-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnhsmt\" (UID: \"0904f7f8-6e45-4796-a906-3920ac0d7932\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnhsmt"
Sep 11 17:23:18.267724 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:18.267685    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qw2fx\" (UniqueName: \"kubernetes.io/projected/0904f7f8-6e45-4796-a906-3920ac0d7932-kube-api-access-qw2fx\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnhsmt\" (UID: \"0904f7f8-6e45-4796-a906-3920ac0d7932\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnhsmt"
Sep 11 17:23:18.267724 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:18.267743    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0904f7f8-6e45-4796-a906-3920ac0d7932-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnhsmt\" (UID: \"0904f7f8-6e45-4796-a906-3920ac0d7932\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnhsmt"
Sep 11 17:23:18.267950 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:18.267784    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0904f7f8-6e45-4796-a906-3920ac0d7932-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnhsmt\" (UID: \"0904f7f8-6e45-4796-a906-3920ac0d7932\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnhsmt"
Sep 11 17:23:18.268176 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:18.268160    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0904f7f8-6e45-4796-a906-3920ac0d7932-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnhsmt\" (UID: \"0904f7f8-6e45-4796-a906-3920ac0d7932\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnhsmt"
Sep 11 17:23:18.268224 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:18.268195    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0904f7f8-6e45-4796-a906-3920ac0d7932-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnhsmt\" (UID: \"0904f7f8-6e45-4796-a906-3920ac0d7932\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnhsmt"
Sep 11 17:23:18.276465 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:18.276437    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qw2fx\" (UniqueName: \"kubernetes.io/projected/0904f7f8-6e45-4796-a906-3920ac0d7932-kube-api-access-qw2fx\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnhsmt\" (UID: \"0904f7f8-6e45-4796-a906-3920ac0d7932\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnhsmt"
Sep 11 17:23:18.397711 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:18.397679    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnhsmt"
Sep 11 17:23:18.672423 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:18.672342    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnhsmt"]
Sep 11 17:23:18.675330 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:23:18.675305    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0904f7f8_6e45_4796_a906_3920ac0d7932.slice/crio-6b296128811d4a4154827fad0d9ec1399aae5d39a039fc7abd1870d2fa2f2d0b WatchSource:0}: Error finding container 6b296128811d4a4154827fad0d9ec1399aae5d39a039fc7abd1870d2fa2f2d0b: Status 404 returned error can't find the container with id 6b296128811d4a4154827fad0d9ec1399aae5d39a039fc7abd1870d2fa2f2d0b
Sep 11 17:23:19.262267 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:19.262227    2552 generic.go:356] "Generic (PLEG): container finished" podID="0904f7f8-6e45-4796-a906-3920ac0d7932" containerID="d0cff19f888f75b2743a462756cc00363cf965d6bfedf23335f9d4996973de91" exitCode=0
Sep 11 17:23:19.262659 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:19.262320    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnhsmt" event={"ID":"0904f7f8-6e45-4796-a906-3920ac0d7932","Type":"ContainerDied","Data":"d0cff19f888f75b2743a462756cc00363cf965d6bfedf23335f9d4996973de91"}
Sep 11 17:23:19.262659 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:19.262378    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnhsmt" event={"ID":"0904f7f8-6e45-4796-a906-3920ac0d7932","Type":"ContainerStarted","Data":"6b296128811d4a4154827fad0d9ec1399aae5d39a039fc7abd1870d2fa2f2d0b"}
Sep 11 17:23:20.269228 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:20.269129    2552 generic.go:356] "Generic (PLEG): container finished" podID="0904f7f8-6e45-4796-a906-3920ac0d7932" containerID="7265a5e3855688c8b82c1668e7403e422d9fc8ed71a8c23609bb5f9e2aa51323" exitCode=0
Sep 11 17:23:20.269228 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:20.269209    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnhsmt" event={"ID":"0904f7f8-6e45-4796-a906-3920ac0d7932","Type":"ContainerDied","Data":"7265a5e3855688c8b82c1668e7403e422d9fc8ed71a8c23609bb5f9e2aa51323"}
Sep 11 17:23:21.276761 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:21.276729    2552 generic.go:356] "Generic (PLEG): container finished" podID="0904f7f8-6e45-4796-a906-3920ac0d7932" containerID="5dec9dfcd1f798da42ff0b2de33a3776c9404dc1bec1b4045829c86c0a09d5e0" exitCode=0
Sep 11 17:23:21.277171 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:21.276808    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnhsmt" event={"ID":"0904f7f8-6e45-4796-a906-3920ac0d7932","Type":"ContainerDied","Data":"5dec9dfcd1f798da42ff0b2de33a3776c9404dc1bec1b4045829c86c0a09d5e0"}
Sep 11 17:23:22.402784 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:22.402757    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnhsmt"
Sep 11 17:23:22.501157 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:22.501092    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0904f7f8-6e45-4796-a906-3920ac0d7932-bundle\") pod \"0904f7f8-6e45-4796-a906-3920ac0d7932\" (UID: \"0904f7f8-6e45-4796-a906-3920ac0d7932\") "
Sep 11 17:23:22.501359 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:22.501182    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0904f7f8-6e45-4796-a906-3920ac0d7932-util\") pod \"0904f7f8-6e45-4796-a906-3920ac0d7932\" (UID: \"0904f7f8-6e45-4796-a906-3920ac0d7932\") "
Sep 11 17:23:22.501359 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:22.501222    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qw2fx\" (UniqueName: \"kubernetes.io/projected/0904f7f8-6e45-4796-a906-3920ac0d7932-kube-api-access-qw2fx\") pod \"0904f7f8-6e45-4796-a906-3920ac0d7932\" (UID: \"0904f7f8-6e45-4796-a906-3920ac0d7932\") "
Sep 11 17:23:22.501889 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:22.501865    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0904f7f8-6e45-4796-a906-3920ac0d7932-bundle" (OuterVolumeSpecName: "bundle") pod "0904f7f8-6e45-4796-a906-3920ac0d7932" (UID: "0904f7f8-6e45-4796-a906-3920ac0d7932"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:23:22.503263 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:22.503232    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0904f7f8-6e45-4796-a906-3920ac0d7932-kube-api-access-qw2fx" (OuterVolumeSpecName: "kube-api-access-qw2fx") pod "0904f7f8-6e45-4796-a906-3920ac0d7932" (UID: "0904f7f8-6e45-4796-a906-3920ac0d7932"). InnerVolumeSpecName "kube-api-access-qw2fx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:23:22.506719 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:22.506684    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0904f7f8-6e45-4796-a906-3920ac0d7932-util" (OuterVolumeSpecName: "util") pod "0904f7f8-6e45-4796-a906-3920ac0d7932" (UID: "0904f7f8-6e45-4796-a906-3920ac0d7932"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:23:22.602435 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:22.602401    2552 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0904f7f8-6e45-4796-a906-3920ac0d7932-bundle\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:23:22.602435 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:22.602433    2552 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0904f7f8-6e45-4796-a906-3920ac0d7932-util\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:23:22.602619 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:22.602442    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qw2fx\" (UniqueName: \"kubernetes.io/projected/0904f7f8-6e45-4796-a906-3920ac0d7932-kube-api-access-qw2fx\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:23:23.289579 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:23.289538    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnhsmt" event={"ID":"0904f7f8-6e45-4796-a906-3920ac0d7932","Type":"ContainerDied","Data":"6b296128811d4a4154827fad0d9ec1399aae5d39a039fc7abd1870d2fa2f2d0b"}
Sep 11 17:23:23.289579 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:23.289574    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnhsmt"
Sep 11 17:23:23.289808 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:23.289597    2552 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6b296128811d4a4154827fad0d9ec1399aae5d39a039fc7abd1870d2fa2f2d0b"
Sep 11 17:23:24.797039 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:24.796985    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:23:24.897523 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:24.897492    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:23:25.862712 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:25.862676    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-hg7p8_eb5f3be1-2e36-40ba-b3a0-3bec413056e8/console-operator/1.log"
Sep 11 17:23:25.863210 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:25.862868    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-hg7p8_eb5f3be1-2e36-40ba-b3a0-3bec413056e8/console-operator/1.log"
Sep 11 17:23:25.866614 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:25.866590    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6k9tr_0620c0a0-8070-4307-86d9-c69a76ea14a9/ovn-acl-logging/0.log"
Sep 11 17:23:25.866920 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:25.866905    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6k9tr_0620c0a0-8070-4307-86d9-c69a76ea14a9/ovn-acl-logging/0.log"
Sep 11 17:23:25.875778 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:25.875752    2552 kubelet.go:1648] "Image garbage collection succeeded"
Sep 11 17:23:27.948289 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:27.948261    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mlpc8"]
Sep 11 17:23:27.949713 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:27.948527    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0904f7f8-6e45-4796-a906-3920ac0d7932" containerName="pull"
Sep 11 17:23:27.949713 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:27.948537    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="0904f7f8-6e45-4796-a906-3920ac0d7932" containerName="pull"
Sep 11 17:23:27.949713 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:27.948548    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0904f7f8-6e45-4796-a906-3920ac0d7932" containerName="util"
Sep 11 17:23:27.949713 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:27.948554    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="0904f7f8-6e45-4796-a906-3920ac0d7932" containerName="util"
Sep 11 17:23:27.949713 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:27.948568    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0904f7f8-6e45-4796-a906-3920ac0d7932" containerName="extract"
Sep 11 17:23:27.949713 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:27.948573    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="0904f7f8-6e45-4796-a906-3920ac0d7932" containerName="extract"
Sep 11 17:23:27.949713 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:27.948630    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="0904f7f8-6e45-4796-a906-3920ac0d7932" containerName="extract"
Sep 11 17:23:27.950798 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:27.950781    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mlpc8"
Sep 11 17:23:27.953273 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:27.953249    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 11 17:23:27.953414 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:27.953310    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 11 17:23:27.953521 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:27.953507    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-cjjsx\""
Sep 11 17:23:27.966126 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:27.966084    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mlpc8"]
Sep 11 17:23:28.048766 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:28.048724    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1a34bedc-d296-4116-8cfd-b192c94c8dbd-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mlpc8\" (UID: \"1a34bedc-d296-4116-8cfd-b192c94c8dbd\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mlpc8"
Sep 11 17:23:28.048953 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:28.048803    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1a34bedc-d296-4116-8cfd-b192c94c8dbd-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mlpc8\" (UID: \"1a34bedc-d296-4116-8cfd-b192c94c8dbd\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mlpc8"
Sep 11 17:23:28.048953 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:28.048824    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9btlv\" (UniqueName: \"kubernetes.io/projected/1a34bedc-d296-4116-8cfd-b192c94c8dbd-kube-api-access-9btlv\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mlpc8\" (UID: \"1a34bedc-d296-4116-8cfd-b192c94c8dbd\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mlpc8"
Sep 11 17:23:28.149374 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:28.149334    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1a34bedc-d296-4116-8cfd-b192c94c8dbd-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mlpc8\" (UID: \"1a34bedc-d296-4116-8cfd-b192c94c8dbd\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mlpc8"
Sep 11 17:23:28.149374 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:28.149377    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9btlv\" (UniqueName: \"kubernetes.io/projected/1a34bedc-d296-4116-8cfd-b192c94c8dbd-kube-api-access-9btlv\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mlpc8\" (UID: \"1a34bedc-d296-4116-8cfd-b192c94c8dbd\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mlpc8"
Sep 11 17:23:28.149626 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:28.149422    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1a34bedc-d296-4116-8cfd-b192c94c8dbd-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mlpc8\" (UID: \"1a34bedc-d296-4116-8cfd-b192c94c8dbd\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mlpc8"
Sep 11 17:23:28.149761 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:28.149740    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1a34bedc-d296-4116-8cfd-b192c94c8dbd-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mlpc8\" (UID: \"1a34bedc-d296-4116-8cfd-b192c94c8dbd\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mlpc8"
Sep 11 17:23:28.149817 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:28.149763    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1a34bedc-d296-4116-8cfd-b192c94c8dbd-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mlpc8\" (UID: \"1a34bedc-d296-4116-8cfd-b192c94c8dbd\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mlpc8"
Sep 11 17:23:28.163168 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:28.163141    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9btlv\" (UniqueName: \"kubernetes.io/projected/1a34bedc-d296-4116-8cfd-b192c94c8dbd-kube-api-access-9btlv\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mlpc8\" (UID: \"1a34bedc-d296-4116-8cfd-b192c94c8dbd\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mlpc8"
Sep 11 17:23:28.259678 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:28.259577    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mlpc8"
Sep 11 17:23:28.541406 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:28.541329    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mlpc8"]
Sep 11 17:23:28.544624 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:23:28.544593    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a34bedc_d296_4116_8cfd_b192c94c8dbd.slice/crio-2f2e3a59d2016190e3d6da693c5e279e0f68ef89c7de71f4ffa7d2efe9a2db4e WatchSource:0}: Error finding container 2f2e3a59d2016190e3d6da693c5e279e0f68ef89c7de71f4ffa7d2efe9a2db4e: Status 404 returned error can't find the container with id 2f2e3a59d2016190e3d6da693c5e279e0f68ef89c7de71f4ffa7d2efe9a2db4e
Sep 11 17:23:28.746583 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:28.746540    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:23:29.320541 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:29.320503    2552 generic.go:356] "Generic (PLEG): container finished" podID="1a34bedc-d296-4116-8cfd-b192c94c8dbd" containerID="41c8f224f8808824471357aa134834c62023dfa1a9457c75cc83ac8a79f99759" exitCode=0
Sep 11 17:23:29.320936 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:29.320591    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mlpc8" event={"ID":"1a34bedc-d296-4116-8cfd-b192c94c8dbd","Type":"ContainerDied","Data":"41c8f224f8808824471357aa134834c62023dfa1a9457c75cc83ac8a79f99759"}
Sep 11 17:23:29.320936 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:29.320654    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mlpc8" event={"ID":"1a34bedc-d296-4116-8cfd-b192c94c8dbd","Type":"ContainerStarted","Data":"2f2e3a59d2016190e3d6da693c5e279e0f68ef89c7de71f4ffa7d2efe9a2db4e"}
Sep 11 17:23:29.321141 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:29.321124    2552 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 17:23:29.978101 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:29.978076    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-845d467d5c-4zthg"]
Sep 11 17:23:29.980681 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:29.980662    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-845d467d5c-4zthg"
Sep 11 17:23:29.983968 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:29.983943    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-9nhbc\""
Sep 11 17:23:29.984069 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:29.983944    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Sep 11 17:23:29.984069 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:29.984063    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Sep 11 17:23:29.984176 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:29.984020    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Sep 11 17:23:30.010310 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:30.010282    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-845d467d5c-4zthg"]
Sep 11 17:23:30.066285 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:30.066249    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/91e20f54-4a44-4f22-b9e6-015c4be287a7-cert\") pod \"lws-controller-manager-845d467d5c-4zthg\" (UID: \"91e20f54-4a44-4f22-b9e6-015c4be287a7\") " pod="openshift-lws-operator/lws-controller-manager-845d467d5c-4zthg"
Sep 11 17:23:30.066460 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:30.066382    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/91e20f54-4a44-4f22-b9e6-015c4be287a7-manager-config\") pod \"lws-controller-manager-845d467d5c-4zthg\" (UID: \"91e20f54-4a44-4f22-b9e6-015c4be287a7\") " pod="openshift-lws-operator/lws-controller-manager-845d467d5c-4zthg"
Sep 11 17:23:30.066460 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:30.066435    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w2v6k\" (UniqueName: \"kubernetes.io/projected/91e20f54-4a44-4f22-b9e6-015c4be287a7-kube-api-access-w2v6k\") pod \"lws-controller-manager-845d467d5c-4zthg\" (UID: \"91e20f54-4a44-4f22-b9e6-015c4be287a7\") " pod="openshift-lws-operator/lws-controller-manager-845d467d5c-4zthg"
Sep 11 17:23:30.066560 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:30.066469    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/91e20f54-4a44-4f22-b9e6-015c4be287a7-metrics-cert\") pod \"lws-controller-manager-845d467d5c-4zthg\" (UID: \"91e20f54-4a44-4f22-b9e6-015c4be287a7\") " pod="openshift-lws-operator/lws-controller-manager-845d467d5c-4zthg"
Sep 11 17:23:30.167412 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:30.167375    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/91e20f54-4a44-4f22-b9e6-015c4be287a7-cert\") pod \"lws-controller-manager-845d467d5c-4zthg\" (UID: \"91e20f54-4a44-4f22-b9e6-015c4be287a7\") " pod="openshift-lws-operator/lws-controller-manager-845d467d5c-4zthg"
Sep 11 17:23:30.167627 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:30.167449    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/91e20f54-4a44-4f22-b9e6-015c4be287a7-manager-config\") pod \"lws-controller-manager-845d467d5c-4zthg\" (UID: \"91e20f54-4a44-4f22-b9e6-015c4be287a7\") " pod="openshift-lws-operator/lws-controller-manager-845d467d5c-4zthg"
Sep 11 17:23:30.167627 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:30.167474    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w2v6k\" (UniqueName: \"kubernetes.io/projected/91e20f54-4a44-4f22-b9e6-015c4be287a7-kube-api-access-w2v6k\") pod \"lws-controller-manager-845d467d5c-4zthg\" (UID: \"91e20f54-4a44-4f22-b9e6-015c4be287a7\") " pod="openshift-lws-operator/lws-controller-manager-845d467d5c-4zthg"
Sep 11 17:23:30.167627 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:30.167502    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/91e20f54-4a44-4f22-b9e6-015c4be287a7-metrics-cert\") pod \"lws-controller-manager-845d467d5c-4zthg\" (UID: \"91e20f54-4a44-4f22-b9e6-015c4be287a7\") " pod="openshift-lws-operator/lws-controller-manager-845d467d5c-4zthg"
Sep 11 17:23:30.168183 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:30.168161    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/91e20f54-4a44-4f22-b9e6-015c4be287a7-manager-config\") pod \"lws-controller-manager-845d467d5c-4zthg\" (UID: \"91e20f54-4a44-4f22-b9e6-015c4be287a7\") " pod="openshift-lws-operator/lws-controller-manager-845d467d5c-4zthg"
Sep 11 17:23:30.169899 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:30.169877    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/91e20f54-4a44-4f22-b9e6-015c4be287a7-metrics-cert\") pod \"lws-controller-manager-845d467d5c-4zthg\" (UID: \"91e20f54-4a44-4f22-b9e6-015c4be287a7\") " pod="openshift-lws-operator/lws-controller-manager-845d467d5c-4zthg"
Sep 11 17:23:30.169996 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:30.169956    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/91e20f54-4a44-4f22-b9e6-015c4be287a7-cert\") pod \"lws-controller-manager-845d467d5c-4zthg\" (UID: \"91e20f54-4a44-4f22-b9e6-015c4be287a7\") " pod="openshift-lws-operator/lws-controller-manager-845d467d5c-4zthg"
Sep 11 17:23:30.224991 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:30.224909    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w2v6k\" (UniqueName: \"kubernetes.io/projected/91e20f54-4a44-4f22-b9e6-015c4be287a7-kube-api-access-w2v6k\") pod \"lws-controller-manager-845d467d5c-4zthg\" (UID: \"91e20f54-4a44-4f22-b9e6-015c4be287a7\") " pod="openshift-lws-operator/lws-controller-manager-845d467d5c-4zthg"
Sep 11 17:23:30.297293 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:30.297256    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-845d467d5c-4zthg"
Sep 11 17:23:30.327242 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:30.327202    2552 generic.go:356] "Generic (PLEG): container finished" podID="1a34bedc-d296-4116-8cfd-b192c94c8dbd" containerID="a1a376ce5da35587cb755de13176d11d61087acd9344a451ebc811362696801f" exitCode=0
Sep 11 17:23:30.327748 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:30.327280    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mlpc8" event={"ID":"1a34bedc-d296-4116-8cfd-b192c94c8dbd","Type":"ContainerDied","Data":"a1a376ce5da35587cb755de13176d11d61087acd9344a451ebc811362696801f"}
Sep 11 17:23:30.426644 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:30.426606    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:23:30.651039 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:30.651003    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-845d467d5c-4zthg"]
Sep 11 17:23:30.653842 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:23:30.653815    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91e20f54_4a44_4f22_b9e6_015c4be287a7.slice/crio-dc8e16101742eb7417b9e1a61e0d8be16b8736997c098dafadd34dfd996ddad2 WatchSource:0}: Error finding container dc8e16101742eb7417b9e1a61e0d8be16b8736997c098dafadd34dfd996ddad2: Status 404 returned error can't find the container with id dc8e16101742eb7417b9e1a61e0d8be16b8736997c098dafadd34dfd996ddad2
Sep 11 17:23:30.860418 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:30.860385    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:23:30.979262 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:30.979226    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-b887fdd9f-5v6pg"]
Sep 11 17:23:30.981565 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:30.981548    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-b887fdd9f-5v6pg"
Sep 11 17:23:31.022858 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:31.022826    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-b887fdd9f-5v6pg"]
Sep 11 17:23:31.031407 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:31.031368    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 11 17:23:31.031407 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:31.031409    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 11 17:23:31.031781 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:31.031761    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 11 17:23:31.032248 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:31.032231    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-vxtdm\""
Sep 11 17:23:31.032359 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:31.032348    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 11 17:23:31.076740 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:31.076709    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b7b66d08-cda4-4fae-85a2-742fb95922c0-manifests\") pod \"opendatahub-operator-controller-manager-b887fdd9f-5v6pg\" (UID: \"b7b66d08-cda4-4fae-85a2-742fb95922c0\") " pod="opendatahub/opendatahub-operator-controller-manager-b887fdd9f-5v6pg"
Sep 11 17:23:31.076956 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:31.076758    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8cjgd\" (UniqueName: \"kubernetes.io/projected/b7b66d08-cda4-4fae-85a2-742fb95922c0-kube-api-access-8cjgd\") pod \"opendatahub-operator-controller-manager-b887fdd9f-5v6pg\" (UID: \"b7b66d08-cda4-4fae-85a2-742fb95922c0\") " pod="opendatahub/opendatahub-operator-controller-manager-b887fdd9f-5v6pg"
Sep 11 17:23:31.076956 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:31.076845    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b7b66d08-cda4-4fae-85a2-742fb95922c0-apiservice-cert\") pod \"opendatahub-operator-controller-manager-b887fdd9f-5v6pg\" (UID: \"b7b66d08-cda4-4fae-85a2-742fb95922c0\") " pod="opendatahub/opendatahub-operator-controller-manager-b887fdd9f-5v6pg"
Sep 11 17:23:31.076956 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:31.076884    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b7b66d08-cda4-4fae-85a2-742fb95922c0-tmp\") pod \"opendatahub-operator-controller-manager-b887fdd9f-5v6pg\" (UID: \"b7b66d08-cda4-4fae-85a2-742fb95922c0\") " pod="opendatahub/opendatahub-operator-controller-manager-b887fdd9f-5v6pg"
Sep 11 17:23:31.076956 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:31.076915    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b7b66d08-cda4-4fae-85a2-742fb95922c0-webhook-cert\") pod \"opendatahub-operator-controller-manager-b887fdd9f-5v6pg\" (UID: \"b7b66d08-cda4-4fae-85a2-742fb95922c0\") " pod="opendatahub/opendatahub-operator-controller-manager-b887fdd9f-5v6pg"
Sep 11 17:23:31.178302 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:31.178219    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b7b66d08-cda4-4fae-85a2-742fb95922c0-tmp\") pod \"opendatahub-operator-controller-manager-b887fdd9f-5v6pg\" (UID: \"b7b66d08-cda4-4fae-85a2-742fb95922c0\") " pod="opendatahub/opendatahub-operator-controller-manager-b887fdd9f-5v6pg"
Sep 11 17:23:31.178302 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:31.178261    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b7b66d08-cda4-4fae-85a2-742fb95922c0-webhook-cert\") pod \"opendatahub-operator-controller-manager-b887fdd9f-5v6pg\" (UID: \"b7b66d08-cda4-4fae-85a2-742fb95922c0\") " pod="opendatahub/opendatahub-operator-controller-manager-b887fdd9f-5v6pg"
Sep 11 17:23:31.178519 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:31.178315    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b7b66d08-cda4-4fae-85a2-742fb95922c0-manifests\") pod \"opendatahub-operator-controller-manager-b887fdd9f-5v6pg\" (UID: \"b7b66d08-cda4-4fae-85a2-742fb95922c0\") " pod="opendatahub/opendatahub-operator-controller-manager-b887fdd9f-5v6pg"
Sep 11 17:23:31.178519 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:31.178344    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8cjgd\" (UniqueName: \"kubernetes.io/projected/b7b66d08-cda4-4fae-85a2-742fb95922c0-kube-api-access-8cjgd\") pod \"opendatahub-operator-controller-manager-b887fdd9f-5v6pg\" (UID: \"b7b66d08-cda4-4fae-85a2-742fb95922c0\") " pod="opendatahub/opendatahub-operator-controller-manager-b887fdd9f-5v6pg"
Sep 11 17:23:31.178519 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:31.178373    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b7b66d08-cda4-4fae-85a2-742fb95922c0-apiservice-cert\") pod \"opendatahub-operator-controller-manager-b887fdd9f-5v6pg\" (UID: \"b7b66d08-cda4-4fae-85a2-742fb95922c0\") " pod="opendatahub/opendatahub-operator-controller-manager-b887fdd9f-5v6pg"
Sep 11 17:23:31.178859 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:31.178829    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b7b66d08-cda4-4fae-85a2-742fb95922c0-manifests\") pod \"opendatahub-operator-controller-manager-b887fdd9f-5v6pg\" (UID: \"b7b66d08-cda4-4fae-85a2-742fb95922c0\") " pod="opendatahub/opendatahub-operator-controller-manager-b887fdd9f-5v6pg"
Sep 11 17:23:31.180535 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:31.180511    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b7b66d08-cda4-4fae-85a2-742fb95922c0-tmp\") pod \"opendatahub-operator-controller-manager-b887fdd9f-5v6pg\" (UID: \"b7b66d08-cda4-4fae-85a2-742fb95922c0\") " pod="opendatahub/opendatahub-operator-controller-manager-b887fdd9f-5v6pg"
Sep 11 17:23:31.180743 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:31.180723    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b7b66d08-cda4-4fae-85a2-742fb95922c0-apiservice-cert\") pod \"opendatahub-operator-controller-manager-b887fdd9f-5v6pg\" (UID: \"b7b66d08-cda4-4fae-85a2-742fb95922c0\") " pod="opendatahub/opendatahub-operator-controller-manager-b887fdd9f-5v6pg"
Sep 11 17:23:31.180920 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:31.180891    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b7b66d08-cda4-4fae-85a2-742fb95922c0-webhook-cert\") pod \"opendatahub-operator-controller-manager-b887fdd9f-5v6pg\" (UID: \"b7b66d08-cda4-4fae-85a2-742fb95922c0\") " pod="opendatahub/opendatahub-operator-controller-manager-b887fdd9f-5v6pg"
Sep 11 17:23:31.202483 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:31.202459    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8cjgd\" (UniqueName: \"kubernetes.io/projected/b7b66d08-cda4-4fae-85a2-742fb95922c0-kube-api-access-8cjgd\") pod \"opendatahub-operator-controller-manager-b887fdd9f-5v6pg\" (UID: \"b7b66d08-cda4-4fae-85a2-742fb95922c0\") " pod="opendatahub/opendatahub-operator-controller-manager-b887fdd9f-5v6pg"
Sep 11 17:23:31.291530 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:31.291483    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-b887fdd9f-5v6pg"
Sep 11 17:23:31.333748 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:31.333702    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:23:31.335072 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:31.335043    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-845d467d5c-4zthg" event={"ID":"91e20f54-4a44-4f22-b9e6-015c4be287a7","Type":"ContainerStarted","Data":"dc8e16101742eb7417b9e1a61e0d8be16b8736997c098dafadd34dfd996ddad2"}
Sep 11 17:23:31.337454 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:31.337418    2552 generic.go:356] "Generic (PLEG): container finished" podID="1a34bedc-d296-4116-8cfd-b192c94c8dbd" containerID="7e3a16d2805ade9d369f9d6ae4d530052339ee6c5dd5e3849a05fbe885eea7da" exitCode=0
Sep 11 17:23:31.337576 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:31.337473    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mlpc8" event={"ID":"1a34bedc-d296-4116-8cfd-b192c94c8dbd","Type":"ContainerDied","Data":"7e3a16d2805ade9d369f9d6ae4d530052339ee6c5dd5e3849a05fbe885eea7da"}
Sep 11 17:23:31.463209 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:31.463089    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:23:31.614779 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:31.614748    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-b887fdd9f-5v6pg"]
Sep 11 17:23:31.617436 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:23:31.617409    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7b66d08_cda4_4fae_85a2_742fb95922c0.slice/crio-873ed60332fb9ecca1e057e8a362e68c4e1531e870274d0f446e64b0ee621182 WatchSource:0}: Error finding container 873ed60332fb9ecca1e057e8a362e68c4e1531e870274d0f446e64b0ee621182: Status 404 returned error can't find the container with id 873ed60332fb9ecca1e057e8a362e68c4e1531e870274d0f446e64b0ee621182
Sep 11 17:23:31.698034 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:31.697977    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:23:31.856778 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:31.856740    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:23:32.009084 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:32.009043    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:23:32.109033 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:32.108950    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:23:32.344737 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:32.344684    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-845d467d5c-4zthg" event={"ID":"91e20f54-4a44-4f22-b9e6-015c4be287a7","Type":"ContainerStarted","Data":"20fb87e8289bbd777a114e5df7d7c689909c7b8dd5be91b9bf59548c016a84db"}
Sep 11 17:23:32.345249 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:32.344780    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-845d467d5c-4zthg"
Sep 11 17:23:32.346990 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:32.346956    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-b887fdd9f-5v6pg" event={"ID":"b7b66d08-cda4-4fae-85a2-742fb95922c0","Type":"ContainerStarted","Data":"873ed60332fb9ecca1e057e8a362e68c4e1531e870274d0f446e64b0ee621182"}
Sep 11 17:23:32.378775 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:32.378464    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-845d467d5c-4zthg" podStartSLOduration=1.782617224 podStartE2EDuration="3.378441405s" totalImagesPullingTime="1.595824181s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:23:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:23:32.377536693 +0000 UTC m=+306.898828217" watchObservedRunningTime="2026-09-11 17:23:32.378441405 +0000 UTC m=+306.899732929"
Sep 11 17:23:32.501991 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:32.501870    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mlpc8"
Sep 11 17:23:32.593173 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:32.593134    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9btlv\" (UniqueName: \"kubernetes.io/projected/1a34bedc-d296-4116-8cfd-b192c94c8dbd-kube-api-access-9btlv\") pod \"1a34bedc-d296-4116-8cfd-b192c94c8dbd\" (UID: \"1a34bedc-d296-4116-8cfd-b192c94c8dbd\") "
Sep 11 17:23:32.593369 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:32.593263    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1a34bedc-d296-4116-8cfd-b192c94c8dbd-util\") pod \"1a34bedc-d296-4116-8cfd-b192c94c8dbd\" (UID: \"1a34bedc-d296-4116-8cfd-b192c94c8dbd\") "
Sep 11 17:23:32.593369 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:32.593299    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1a34bedc-d296-4116-8cfd-b192c94c8dbd-bundle\") pod \"1a34bedc-d296-4116-8cfd-b192c94c8dbd\" (UID: \"1a34bedc-d296-4116-8cfd-b192c94c8dbd\") "
Sep 11 17:23:32.594465 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:32.594433    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a34bedc-d296-4116-8cfd-b192c94c8dbd-bundle" (OuterVolumeSpecName: "bundle") pod "1a34bedc-d296-4116-8cfd-b192c94c8dbd" (UID: "1a34bedc-d296-4116-8cfd-b192c94c8dbd"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:23:32.596163 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:32.596132    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1a34bedc-d296-4116-8cfd-b192c94c8dbd-kube-api-access-9btlv" (OuterVolumeSpecName: "kube-api-access-9btlv") pod "1a34bedc-d296-4116-8cfd-b192c94c8dbd" (UID: "1a34bedc-d296-4116-8cfd-b192c94c8dbd"). InnerVolumeSpecName "kube-api-access-9btlv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:23:32.600939 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:32.600887    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a34bedc-d296-4116-8cfd-b192c94c8dbd-util" (OuterVolumeSpecName: "util") pod "1a34bedc-d296-4116-8cfd-b192c94c8dbd" (UID: "1a34bedc-d296-4116-8cfd-b192c94c8dbd"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:23:32.694497 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:32.694414    2552 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1a34bedc-d296-4116-8cfd-b192c94c8dbd-util\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:23:32.694497 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:32.694447    2552 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1a34bedc-d296-4116-8cfd-b192c94c8dbd-bundle\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:23:32.694497 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:32.694459    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9btlv\" (UniqueName: \"kubernetes.io/projected/1a34bedc-d296-4116-8cfd-b192c94c8dbd-kube-api-access-9btlv\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:23:33.359074 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:33.358961    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mlpc8"
Sep 11 17:23:33.359074 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:33.358994    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mlpc8" event={"ID":"1a34bedc-d296-4116-8cfd-b192c94c8dbd","Type":"ContainerDied","Data":"2f2e3a59d2016190e3d6da693c5e279e0f68ef89c7de71f4ffa7d2efe9a2db4e"}
Sep 11 17:23:33.359074 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:33.359060    2552 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2f2e3a59d2016190e3d6da693c5e279e0f68ef89c7de71f4ffa7d2efe9a2db4e"
Sep 11 17:23:35.371305 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:35.371268    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-b887fdd9f-5v6pg" event={"ID":"b7b66d08-cda4-4fae-85a2-742fb95922c0","Type":"ContainerStarted","Data":"5704f9fe61190e9c96d4fcb5f8257505ac5bcb948c4c4153cc11a6df3144b3f2"}
Sep 11 17:23:36.377990 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:36.377959    2552 generic.go:356] "Generic (PLEG): container finished" podID="b7b66d08-cda4-4fae-85a2-742fb95922c0" containerID="5704f9fe61190e9c96d4fcb5f8257505ac5bcb948c4c4153cc11a6df3144b3f2" exitCode=0
Sep 11 17:23:36.378482 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:36.378051    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-b887fdd9f-5v6pg" event={"ID":"b7b66d08-cda4-4fae-85a2-742fb95922c0","Type":"ContainerDied","Data":"5704f9fe61190e9c96d4fcb5f8257505ac5bcb948c4c4153cc11a6df3144b3f2"}
Sep 11 17:23:37.385016 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:37.384976    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-b887fdd9f-5v6pg" event={"ID":"b7b66d08-cda4-4fae-85a2-742fb95922c0","Type":"ContainerStarted","Data":"1a13ad6e97e8b7b01d1b28dcd4ad6290ea280e59afc99848a4e8dc39c1d8ff95"}
Sep 11 17:23:37.385439 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:37.385147    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-b887fdd9f-5v6pg"
Sep 11 17:23:37.410103 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:37.410042    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-b887fdd9f-5v6pg" podStartSLOduration=2.655740658 podStartE2EDuration="7.410024321s" totalImagesPullingTime="3.682113975s" totalInitContainerRuntime="1.072169688s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:23:30 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:23:37.409071918 +0000 UTC m=+311.930363438" watchObservedRunningTime="2026-09-11 17:23:37.410024321 +0000 UTC m=+311.931315841"
Sep 11 17:23:43.361061 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:43.361023    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-845d467d5c-4zthg"
Sep 11 17:23:48.392211 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:48.392186    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-b887fdd9f-5v6pg"
Sep 11 17:23:51.221634 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:51.221554    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:23:51.386403 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:51.386366    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:23:54.891639 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:54.891598    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:23:55.134968 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:55.134929    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:23:56.876970 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:56.876925    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:23:56.953638 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:56.953606    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:23:58.139627 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:23:58.139589    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:24:01.859243 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:01.859204    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:24:02.020522 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:02.020486    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:24:02.258713 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:02.258623    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-vq8q8"]
Sep 11 17:24:02.258939 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:02.258927    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1a34bedc-d296-4116-8cfd-b192c94c8dbd" containerName="extract"
Sep 11 17:24:02.258982 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:02.258941    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="1a34bedc-d296-4116-8cfd-b192c94c8dbd" containerName="extract"
Sep 11 17:24:02.258982 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:02.258963    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1a34bedc-d296-4116-8cfd-b192c94c8dbd" containerName="pull"
Sep 11 17:24:02.258982 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:02.258968    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="1a34bedc-d296-4116-8cfd-b192c94c8dbd" containerName="pull"
Sep 11 17:24:02.259078 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:02.258983    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1a34bedc-d296-4116-8cfd-b192c94c8dbd" containerName="util"
Sep 11 17:24:02.259078 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:02.258991    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="1a34bedc-d296-4116-8cfd-b192c94c8dbd" containerName="util"
Sep 11 17:24:02.259078 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:02.259042    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="1a34bedc-d296-4116-8cfd-b192c94c8dbd" containerName="extract"
Sep 11 17:24:02.263399 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:02.263378    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-vq8q8"
Sep 11 17:24:02.270595 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:02.270195    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Sep 11 17:24:02.270797 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:02.270771    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-vtt68\""
Sep 11 17:24:02.279781 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:02.279749    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-vq8q8"]
Sep 11 17:24:02.290370 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:02.290334    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:24:02.340306 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:02.340267    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9zfhh\" (UniqueName: \"kubernetes.io/projected/a25b4854-2484-4787-8656-9cba06d6d7c4-kube-api-access-9zfhh\") pod \"odh-model-controller-7998c67dc4-vq8q8\" (UID: \"a25b4854-2484-4787-8656-9cba06d6d7c4\") " pod="opendatahub/odh-model-controller-7998c67dc4-vq8q8"
Sep 11 17:24:02.340306 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:02.340313    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a25b4854-2484-4787-8656-9cba06d6d7c4-cert\") pod \"odh-model-controller-7998c67dc4-vq8q8\" (UID: \"a25b4854-2484-4787-8656-9cba06d6d7c4\") " pod="opendatahub/odh-model-controller-7998c67dc4-vq8q8"
Sep 11 17:24:02.386008 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:02.385972    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:24:02.440934 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:02.440889    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a25b4854-2484-4787-8656-9cba06d6d7c4-cert\") pod \"odh-model-controller-7998c67dc4-vq8q8\" (UID: \"a25b4854-2484-4787-8656-9cba06d6d7c4\") " pod="opendatahub/odh-model-controller-7998c67dc4-vq8q8"
Sep 11 17:24:02.441146 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:02.441004    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9zfhh\" (UniqueName: \"kubernetes.io/projected/a25b4854-2484-4787-8656-9cba06d6d7c4-kube-api-access-9zfhh\") pod \"odh-model-controller-7998c67dc4-vq8q8\" (UID: \"a25b4854-2484-4787-8656-9cba06d6d7c4\") " pod="opendatahub/odh-model-controller-7998c67dc4-vq8q8"
Sep 11 17:24:02.441146 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:24:02.441052    2552 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Sep 11 17:24:02.441252 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:24:02.441145    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a25b4854-2484-4787-8656-9cba06d6d7c4-cert podName:a25b4854-2484-4787-8656-9cba06d6d7c4 nodeName:}" failed. No retries permitted until 2026-09-11 17:24:02.941126847 +0000 UTC m=+337.462418360 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/a25b4854-2484-4787-8656-9cba06d6d7c4-cert") pod "odh-model-controller-7998c67dc4-vq8q8" (UID: "a25b4854-2484-4787-8656-9cba06d6d7c4") : secret "odh-model-controller-webhook-cert" not found
Sep 11 17:24:02.457410 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:02.457375    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9zfhh\" (UniqueName: \"kubernetes.io/projected/a25b4854-2484-4787-8656-9cba06d6d7c4-kube-api-access-9zfhh\") pod \"odh-model-controller-7998c67dc4-vq8q8\" (UID: \"a25b4854-2484-4787-8656-9cba06d6d7c4\") " pod="opendatahub/odh-model-controller-7998c67dc4-vq8q8"
Sep 11 17:24:02.492917 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:02.492885    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:24:02.603244 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:02.603207    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:24:02.912901 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:02.912818    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-dd9vm"]
Sep 11 17:24:02.916377 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:02.916359    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-dd9vm"
Sep 11 17:24:02.918574 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:02.918553    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-452hq\""
Sep 11 17:24:02.918685 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:02.918590    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 11 17:24:02.929947 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:02.929923    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-dd9vm"]
Sep 11 17:24:02.946240 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:02.946202    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/280dfef1-c4c9-487b-bb3e-a1ff2b63d769-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-dd9vm\" (UID: \"280dfef1-c4c9-487b-bb3e-a1ff2b63d769\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-dd9vm"
Sep 11 17:24:02.946240 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:02.946242    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/280dfef1-c4c9-487b-bb3e-a1ff2b63d769-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-dd9vm\" (UID: \"280dfef1-c4c9-487b-bb3e-a1ff2b63d769\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-dd9vm"
Sep 11 17:24:02.946442 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:02.946261    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/280dfef1-c4c9-487b-bb3e-a1ff2b63d769-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-dd9vm\" (UID: \"280dfef1-c4c9-487b-bb3e-a1ff2b63d769\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-dd9vm"
Sep 11 17:24:02.946442 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:02.946278    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/280dfef1-c4c9-487b-bb3e-a1ff2b63d769-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-dd9vm\" (UID: \"280dfef1-c4c9-487b-bb3e-a1ff2b63d769\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-dd9vm"
Sep 11 17:24:02.946442 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:02.946384    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/280dfef1-c4c9-487b-bb3e-a1ff2b63d769-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-dd9vm\" (UID: \"280dfef1-c4c9-487b-bb3e-a1ff2b63d769\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-dd9vm"
Sep 11 17:24:02.946564 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:02.946433    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/280dfef1-c4c9-487b-bb3e-a1ff2b63d769-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-dd9vm\" (UID: \"280dfef1-c4c9-487b-bb3e-a1ff2b63d769\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-dd9vm"
Sep 11 17:24:02.946564 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:02.946471    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/280dfef1-c4c9-487b-bb3e-a1ff2b63d769-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-dd9vm\" (UID: \"280dfef1-c4c9-487b-bb3e-a1ff2b63d769\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-dd9vm"
Sep 11 17:24:02.946564 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:02.946498    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a25b4854-2484-4787-8656-9cba06d6d7c4-cert\") pod \"odh-model-controller-7998c67dc4-vq8q8\" (UID: \"a25b4854-2484-4787-8656-9cba06d6d7c4\") " pod="opendatahub/odh-model-controller-7998c67dc4-vq8q8"
Sep 11 17:24:02.946564 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:02.946530    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/280dfef1-c4c9-487b-bb3e-a1ff2b63d769-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-dd9vm\" (UID: \"280dfef1-c4c9-487b-bb3e-a1ff2b63d769\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-dd9vm"
Sep 11 17:24:02.946701 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:02.946621    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9mtct\" (UniqueName: \"kubernetes.io/projected/280dfef1-c4c9-487b-bb3e-a1ff2b63d769-kube-api-access-9mtct\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-dd9vm\" (UID: \"280dfef1-c4c9-487b-bb3e-a1ff2b63d769\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-dd9vm"
Sep 11 17:24:02.948832 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:02.948801    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a25b4854-2484-4787-8656-9cba06d6d7c4-cert\") pod \"odh-model-controller-7998c67dc4-vq8q8\" (UID: \"a25b4854-2484-4787-8656-9cba06d6d7c4\") " pod="opendatahub/odh-model-controller-7998c67dc4-vq8q8"
Sep 11 17:24:02.952924 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:02.952892    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:24:03.047609 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:03.047578    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/280dfef1-c4c9-487b-bb3e-a1ff2b63d769-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-dd9vm\" (UID: \"280dfef1-c4c9-487b-bb3e-a1ff2b63d769\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-dd9vm"
Sep 11 17:24:03.047609 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:03.047619    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/280dfef1-c4c9-487b-bb3e-a1ff2b63d769-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-dd9vm\" (UID: \"280dfef1-c4c9-487b-bb3e-a1ff2b63d769\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-dd9vm"
Sep 11 17:24:03.047881 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:03.047663    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/280dfef1-c4c9-487b-bb3e-a1ff2b63d769-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-dd9vm\" (UID: \"280dfef1-c4c9-487b-bb3e-a1ff2b63d769\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-dd9vm"
Sep 11 17:24:03.047881 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:03.047687    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9mtct\" (UniqueName: \"kubernetes.io/projected/280dfef1-c4c9-487b-bb3e-a1ff2b63d769-kube-api-access-9mtct\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-dd9vm\" (UID: \"280dfef1-c4c9-487b-bb3e-a1ff2b63d769\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-dd9vm"
Sep 11 17:24:03.047881 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:03.047732    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/280dfef1-c4c9-487b-bb3e-a1ff2b63d769-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-dd9vm\" (UID: \"280dfef1-c4c9-487b-bb3e-a1ff2b63d769\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-dd9vm"
Sep 11 17:24:03.047881 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:03.047804    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/280dfef1-c4c9-487b-bb3e-a1ff2b63d769-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-dd9vm\" (UID: \"280dfef1-c4c9-487b-bb3e-a1ff2b63d769\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-dd9vm"
Sep 11 17:24:03.047881 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:03.047860    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/280dfef1-c4c9-487b-bb3e-a1ff2b63d769-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-dd9vm\" (UID: \"280dfef1-c4c9-487b-bb3e-a1ff2b63d769\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-dd9vm"
Sep 11 17:24:03.048211 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:03.047889    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/280dfef1-c4c9-487b-bb3e-a1ff2b63d769-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-dd9vm\" (UID: \"280dfef1-c4c9-487b-bb3e-a1ff2b63d769\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-dd9vm"
Sep 11 17:24:03.048211 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:03.047974    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/280dfef1-c4c9-487b-bb3e-a1ff2b63d769-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-dd9vm\" (UID: \"280dfef1-c4c9-487b-bb3e-a1ff2b63d769\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-dd9vm"
Sep 11 17:24:03.048316 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:03.048245    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/280dfef1-c4c9-487b-bb3e-a1ff2b63d769-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-dd9vm\" (UID: \"280dfef1-c4c9-487b-bb3e-a1ff2b63d769\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-dd9vm"
Sep 11 17:24:03.048377 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:03.048321    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/280dfef1-c4c9-487b-bb3e-a1ff2b63d769-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-dd9vm\" (UID: \"280dfef1-c4c9-487b-bb3e-a1ff2b63d769\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-dd9vm"
Sep 11 17:24:03.048432 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:03.048393    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/280dfef1-c4c9-487b-bb3e-a1ff2b63d769-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-dd9vm\" (UID: \"280dfef1-c4c9-487b-bb3e-a1ff2b63d769\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-dd9vm"
Sep 11 17:24:03.048477 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:03.048451    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/280dfef1-c4c9-487b-bb3e-a1ff2b63d769-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-dd9vm\" (UID: \"280dfef1-c4c9-487b-bb3e-a1ff2b63d769\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-dd9vm"
Sep 11 17:24:03.048548 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:03.048533    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/280dfef1-c4c9-487b-bb3e-a1ff2b63d769-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-dd9vm\" (UID: \"280dfef1-c4c9-487b-bb3e-a1ff2b63d769\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-dd9vm"
Sep 11 17:24:03.050321 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:03.050298    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/280dfef1-c4c9-487b-bb3e-a1ff2b63d769-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-dd9vm\" (UID: \"280dfef1-c4c9-487b-bb3e-a1ff2b63d769\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-dd9vm"
Sep 11 17:24:03.050412 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:03.050395    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/280dfef1-c4c9-487b-bb3e-a1ff2b63d769-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-dd9vm\" (UID: \"280dfef1-c4c9-487b-bb3e-a1ff2b63d769\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-dd9vm"
Sep 11 17:24:03.057236 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:03.057209    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/280dfef1-c4c9-487b-bb3e-a1ff2b63d769-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-dd9vm\" (UID: \"280dfef1-c4c9-487b-bb3e-a1ff2b63d769\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-dd9vm"
Sep 11 17:24:03.057473 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:03.057439    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9mtct\" (UniqueName: \"kubernetes.io/projected/280dfef1-c4c9-487b-bb3e-a1ff2b63d769-kube-api-access-9mtct\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-dd9vm\" (UID: \"280dfef1-c4c9-487b-bb3e-a1ff2b63d769\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-dd9vm"
Sep 11 17:24:03.061900 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:03.061870    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:24:03.135776 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:03.135733    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-7c92n"]
Sep 11 17:24:03.140193 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:03.140167    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-7c92n"
Sep 11 17:24:03.143216 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:03.143189    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Sep 11 17:24:03.143363 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:03.143315    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-xwgjl\""
Sep 11 17:24:03.149163 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:03.149133    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d550ef84-b465-4c3d-81fb-a0a457d635cb-tls-certs\") pod \"model-serving-api-54d89bd5fc-7c92n\" (UID: \"d550ef84-b465-4c3d-81fb-a0a457d635cb\") " pod="opendatahub/model-serving-api-54d89bd5fc-7c92n"
Sep 11 17:24:03.149294 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:03.149261    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lp6ts\" (UniqueName: \"kubernetes.io/projected/d550ef84-b465-4c3d-81fb-a0a457d635cb-kube-api-access-lp6ts\") pod \"model-serving-api-54d89bd5fc-7c92n\" (UID: \"d550ef84-b465-4c3d-81fb-a0a457d635cb\") " pod="opendatahub/model-serving-api-54d89bd5fc-7c92n"
Sep 11 17:24:03.153097 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:03.153074    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-7c92n"]
Sep 11 17:24:03.174259 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:03.174178    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-vq8q8"
Sep 11 17:24:03.226393 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:03.226365    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-dd9vm"
Sep 11 17:24:03.249840 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:03.249801    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lp6ts\" (UniqueName: \"kubernetes.io/projected/d550ef84-b465-4c3d-81fb-a0a457d635cb-kube-api-access-lp6ts\") pod \"model-serving-api-54d89bd5fc-7c92n\" (UID: \"d550ef84-b465-4c3d-81fb-a0a457d635cb\") " pod="opendatahub/model-serving-api-54d89bd5fc-7c92n"
Sep 11 17:24:03.249963 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:03.249883    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d550ef84-b465-4c3d-81fb-a0a457d635cb-tls-certs\") pod \"model-serving-api-54d89bd5fc-7c92n\" (UID: \"d550ef84-b465-4c3d-81fb-a0a457d635cb\") " pod="opendatahub/model-serving-api-54d89bd5fc-7c92n"
Sep 11 17:24:03.250385 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:24:03.250052    2552 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found
Sep 11 17:24:03.250385 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:24:03.250149    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d550ef84-b465-4c3d-81fb-a0a457d635cb-tls-certs podName:d550ef84-b465-4c3d-81fb-a0a457d635cb nodeName:}" failed. No retries permitted until 2026-09-11 17:24:03.750124317 +0000 UTC m=+338.271415831 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/d550ef84-b465-4c3d-81fb-a0a457d635cb-tls-certs") pod "model-serving-api-54d89bd5fc-7c92n" (UID: "d550ef84-b465-4c3d-81fb-a0a457d635cb") : secret "model-serving-api-tls" not found
Sep 11 17:24:03.293258 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:03.293192    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lp6ts\" (UniqueName: \"kubernetes.io/projected/d550ef84-b465-4c3d-81fb-a0a457d635cb-kube-api-access-lp6ts\") pod \"model-serving-api-54d89bd5fc-7c92n\" (UID: \"d550ef84-b465-4c3d-81fb-a0a457d635cb\") " pod="opendatahub/model-serving-api-54d89bd5fc-7c92n"
Sep 11 17:24:03.337363 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:03.337322    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:24:03.448225 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:03.448137    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:24:03.460700 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:03.460677    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-vq8q8"]
Sep 11 17:24:03.464788 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:24:03.464755    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda25b4854_2484_4787_8656_9cba06d6d7c4.slice/crio-37ae0de52978ac7c7be5ec7ccfdd502bf1070cfa2ee2be77c849b6d164852e60 WatchSource:0}: Error finding container 37ae0de52978ac7c7be5ec7ccfdd502bf1070cfa2ee2be77c849b6d164852e60: Status 404 returned error can't find the container with id 37ae0de52978ac7c7be5ec7ccfdd502bf1070cfa2ee2be77c849b6d164852e60
Sep 11 17:24:03.510100 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:03.510063    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-vq8q8" event={"ID":"a25b4854-2484-4787-8656-9cba06d6d7c4","Type":"ContainerStarted","Data":"37ae0de52978ac7c7be5ec7ccfdd502bf1070cfa2ee2be77c849b6d164852e60"}
Sep 11 17:24:03.517895 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:03.517867    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-dd9vm"]
Sep 11 17:24:03.520447 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:24:03.520416    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod280dfef1_c4c9_487b_bb3e_a1ff2b63d769.slice/crio-359bb571e80e265c4b3aadf3f6d6a737d5621aef14e63f9d2b7a41d428617dd2 WatchSource:0}: Error finding container 359bb571e80e265c4b3aadf3f6d6a737d5621aef14e63f9d2b7a41d428617dd2: Status 404 returned error can't find the container with id 359bb571e80e265c4b3aadf3f6d6a737d5621aef14e63f9d2b7a41d428617dd2
Sep 11 17:24:03.753905 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:03.753818    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d550ef84-b465-4c3d-81fb-a0a457d635cb-tls-certs\") pod \"model-serving-api-54d89bd5fc-7c92n\" (UID: \"d550ef84-b465-4c3d-81fb-a0a457d635cb\") " pod="opendatahub/model-serving-api-54d89bd5fc-7c92n"
Sep 11 17:24:03.756314 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:03.756293    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d550ef84-b465-4c3d-81fb-a0a457d635cb-tls-certs\") pod \"model-serving-api-54d89bd5fc-7c92n\" (UID: \"d550ef84-b465-4c3d-81fb-a0a457d635cb\") " pod="opendatahub/model-serving-api-54d89bd5fc-7c92n"
Sep 11 17:24:04.050856 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:04.050767    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-7c92n"
Sep 11 17:24:04.338726 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:04.338644    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-7c92n"]
Sep 11 17:24:04.342847 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:24:04.342807    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd550ef84_b465_4c3d_81fb_a0a457d635cb.slice/crio-33ce4bc92815552cb3c934916106db7d616d58dc1f27e8005e0440ac169fcecc WatchSource:0}: Error finding container 33ce4bc92815552cb3c934916106db7d616d58dc1f27e8005e0440ac169fcecc: Status 404 returned error can't find the container with id 33ce4bc92815552cb3c934916106db7d616d58dc1f27e8005e0440ac169fcecc
Sep 11 17:24:04.520002 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:04.519907    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-dd9vm" event={"ID":"280dfef1-c4c9-487b-bb3e-a1ff2b63d769","Type":"ContainerStarted","Data":"359bb571e80e265c4b3aadf3f6d6a737d5621aef14e63f9d2b7a41d428617dd2"}
Sep 11 17:24:04.521825 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:04.521794    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-7c92n" event={"ID":"d550ef84-b465-4c3d-81fb-a0a457d635cb","Type":"ContainerStarted","Data":"33ce4bc92815552cb3c934916106db7d616d58dc1f27e8005e0440ac169fcecc"}
Sep 11 17:24:09.065170 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:09.065129    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 17:24:09.065658 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:09.065206    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 17:24:09.065658 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:09.065238    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 17:24:09.559779 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:09.559728    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-vq8q8" event={"ID":"a25b4854-2484-4787-8656-9cba06d6d7c4","Type":"ContainerStarted","Data":"d4d704e8f40496f9ff810d3ec0ac7c2a4d4b30fdbc0485bd667797c40eb7a5fb"}
Sep 11 17:24:09.559985 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:09.559862    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-vq8q8"
Sep 11 17:24:09.561939 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:09.561913    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-dd9vm" event={"ID":"280dfef1-c4c9-487b-bb3e-a1ff2b63d769","Type":"ContainerStarted","Data":"76b3bf0c19d1808393641d87c1e9718cd6991cb0f08811f73873efe8e0a5e76c"}
Sep 11 17:24:09.563708 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:09.563681    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-7c92n" event={"ID":"d550ef84-b465-4c3d-81fb-a0a457d635cb","Type":"ContainerStarted","Data":"6824878161a2aad312cb628b0019b699ed6e2ce8d7688800aea4d4f7d10ccf85"}
Sep 11 17:24:09.563826 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:09.563810    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-7c92n"
Sep 11 17:24:09.599186 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:09.599134    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-vq8q8" podStartSLOduration=2.008472835 podStartE2EDuration="7.599094577s" totalImagesPullingTime="5.590621742s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:24:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:24:09.594419883 +0000 UTC m=+344.115711403" watchObservedRunningTime="2026-09-11 17:24:09.599094577 +0000 UTC m=+344.120386138"
Sep 11 17:24:09.630832 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:09.630775    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-dd9vm" podStartSLOduration=2.088551549 podStartE2EDuration="7.6307582s" totalImagesPullingTime="5.542206651s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:24:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:24:09.629467828 +0000 UTC m=+344.150759349" watchObservedRunningTime="2026-09-11 17:24:09.6307582 +0000 UTC m=+344.152049720"
Sep 11 17:24:09.656346 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:09.656291    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-7c92n" podStartSLOduration=1.892326555 podStartE2EDuration="6.65627094s" totalImagesPullingTime="4.763944385s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:24:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:24:09.654249626 +0000 UTC m=+344.175541164" watchObservedRunningTime="2026-09-11 17:24:09.65627094 +0000 UTC m=+344.177562461"
Sep 11 17:24:10.227146 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:10.227088    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-dd9vm"
Sep 11 17:24:10.231870 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:10.231844    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-dd9vm"
Sep 11 17:24:10.569153 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:10.569047    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-dd9vm"
Sep 11 17:24:10.569856 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:10.569835    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-dd9vm"
Sep 11 17:24:13.022733 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:13.022686    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:24:13.168055 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:13.168014    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-glcc5"]
Sep 11 17:24:13.171535 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:13.171513    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-glcc5"
Sep 11 17:24:13.175136 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:13.175097    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Sep 11 17:24:13.175256 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:13.175187    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-t2lgr\""
Sep 11 17:24:13.178809 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:13.178779    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:24:13.198673 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:13.198638    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-glcc5"]
Sep 11 17:24:13.242232 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:13.242191    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6e0966ed-9453-40cc-9237-5fede02f1d73-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-glcc5\" (UID: \"6e0966ed-9453-40cc-9237-5fede02f1d73\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-glcc5"
Sep 11 17:24:13.242420 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:13.242280    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mrn5j\" (UniqueName: \"kubernetes.io/projected/6e0966ed-9453-40cc-9237-5fede02f1d73-kube-api-access-mrn5j\") pod \"llmisvc-controller-manager-777c6bd7d5-glcc5\" (UID: \"6e0966ed-9453-40cc-9237-5fede02f1d73\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-glcc5"
Sep 11 17:24:13.343403 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:13.343366    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mrn5j\" (UniqueName: \"kubernetes.io/projected/6e0966ed-9453-40cc-9237-5fede02f1d73-kube-api-access-mrn5j\") pod \"llmisvc-controller-manager-777c6bd7d5-glcc5\" (UID: \"6e0966ed-9453-40cc-9237-5fede02f1d73\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-glcc5"
Sep 11 17:24:13.343565 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:13.343461    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6e0966ed-9453-40cc-9237-5fede02f1d73-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-glcc5\" (UID: \"6e0966ed-9453-40cc-9237-5fede02f1d73\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-glcc5"
Sep 11 17:24:13.346065 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:13.346034    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6e0966ed-9453-40cc-9237-5fede02f1d73-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-glcc5\" (UID: \"6e0966ed-9453-40cc-9237-5fede02f1d73\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-glcc5"
Sep 11 17:24:13.355069 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:13.355041    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mrn5j\" (UniqueName: \"kubernetes.io/projected/6e0966ed-9453-40cc-9237-5fede02f1d73-kube-api-access-mrn5j\") pod \"llmisvc-controller-manager-777c6bd7d5-glcc5\" (UID: \"6e0966ed-9453-40cc-9237-5fede02f1d73\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-glcc5"
Sep 11 17:24:13.403200 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:13.403161    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:24:13.481771 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:13.481726    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-glcc5"
Sep 11 17:24:13.559730 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:13.559675    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:24:13.771791 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:13.771764    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-glcc5"]
Sep 11 17:24:13.772239 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:13.772213    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:24:13.773725 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:24:13.773694    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod6e0966ed_9453_40cc_9237_5fede02f1d73.slice/crio-ed4ce462ae738c08080f106b8e5cb106e0e17fa5101049c2422e986f5fc088fa WatchSource:0}: Error finding container ed4ce462ae738c08080f106b8e5cb106e0e17fa5101049c2422e986f5fc088fa: Status 404 returned error can't find the container with id ed4ce462ae738c08080f106b8e5cb106e0e17fa5101049c2422e986f5fc088fa
Sep 11 17:24:14.590803 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:14.590771    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-glcc5" event={"ID":"6e0966ed-9453-40cc-9237-5fede02f1d73","Type":"ContainerStarted","Data":"ed4ce462ae738c08080f106b8e5cb106e0e17fa5101049c2422e986f5fc088fa"}
Sep 11 17:24:17.309665 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:17.309622    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:24:17.427968 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:17.427926    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:24:17.608286 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:17.608248    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-glcc5" event={"ID":"6e0966ed-9453-40cc-9237-5fede02f1d73","Type":"ContainerStarted","Data":"06ef9c8ac8f8b56a68f86f8b8c7bde4a25133cd9f31c95b8ad55492b491d06f7"}
Sep 11 17:24:17.608435 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:17.608338    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-glcc5"
Sep 11 17:24:20.570550 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:20.570472    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-vq8q8"
Sep 11 17:24:20.572597 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:20.572578    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-7c92n"
Sep 11 17:24:20.595394 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:20.595340    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-glcc5" podStartSLOduration=4.654282583 podStartE2EDuration="7.595322583s" totalImagesPullingTime="2.94104s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:24:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:24:17.641481744 +0000 UTC m=+352.162773267" watchObservedRunningTime="2026-09-11 17:24:20.595322583 +0000 UTC m=+355.116614104"
Sep 11 17:24:36.609860 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:36.609825    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:24:36.674474 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:36.674436    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:24:48.123318 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:48.123279    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5b55b4bcf8-kmnkh"]
Sep 11 17:24:48.134232 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:48.134197    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5b55b4bcf8-kmnkh"
Sep 11 17:24:48.144783 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:48.144751    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5b55b4bcf8-kmnkh"]
Sep 11 17:24:48.199234 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:48.199192    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:24:48.256221 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:48.256184    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/280cbc95-d762-4661-bf63-0f5606364d06-console-config\") pod \"console-5b55b4bcf8-kmnkh\" (UID: \"280cbc95-d762-4661-bf63-0f5606364d06\") " pod="openshift-console/console-5b55b4bcf8-kmnkh"
Sep 11 17:24:48.256384 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:48.256233    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/280cbc95-d762-4661-bf63-0f5606364d06-console-serving-cert\") pod \"console-5b55b4bcf8-kmnkh\" (UID: \"280cbc95-d762-4661-bf63-0f5606364d06\") " pod="openshift-console/console-5b55b4bcf8-kmnkh"
Sep 11 17:24:48.256384 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:48.256260    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4z86q\" (UniqueName: \"kubernetes.io/projected/280cbc95-d762-4661-bf63-0f5606364d06-kube-api-access-4z86q\") pod \"console-5b55b4bcf8-kmnkh\" (UID: \"280cbc95-d762-4661-bf63-0f5606364d06\") " pod="openshift-console/console-5b55b4bcf8-kmnkh"
Sep 11 17:24:48.256384 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:48.256362    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/280cbc95-d762-4661-bf63-0f5606364d06-oauth-serving-cert\") pod \"console-5b55b4bcf8-kmnkh\" (UID: \"280cbc95-d762-4661-bf63-0f5606364d06\") " pod="openshift-console/console-5b55b4bcf8-kmnkh"
Sep 11 17:24:48.256499 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:48.256398    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/280cbc95-d762-4661-bf63-0f5606364d06-service-ca\") pod \"console-5b55b4bcf8-kmnkh\" (UID: \"280cbc95-d762-4661-bf63-0f5606364d06\") " pod="openshift-console/console-5b55b4bcf8-kmnkh"
Sep 11 17:24:48.256499 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:48.256421    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/280cbc95-d762-4661-bf63-0f5606364d06-console-oauth-config\") pod \"console-5b55b4bcf8-kmnkh\" (UID: \"280cbc95-d762-4661-bf63-0f5606364d06\") " pod="openshift-console/console-5b55b4bcf8-kmnkh"
Sep 11 17:24:48.256499 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:48.256466    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/280cbc95-d762-4661-bf63-0f5606364d06-trusted-ca-bundle\") pod \"console-5b55b4bcf8-kmnkh\" (UID: \"280cbc95-d762-4661-bf63-0f5606364d06\") " pod="openshift-console/console-5b55b4bcf8-kmnkh"
Sep 11 17:24:48.296721 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:48.296678    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:24:48.356895 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:48.356858    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/280cbc95-d762-4661-bf63-0f5606364d06-oauth-serving-cert\") pod \"console-5b55b4bcf8-kmnkh\" (UID: \"280cbc95-d762-4661-bf63-0f5606364d06\") " pod="openshift-console/console-5b55b4bcf8-kmnkh"
Sep 11 17:24:48.356895 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:48.356903    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/280cbc95-d762-4661-bf63-0f5606364d06-service-ca\") pod \"console-5b55b4bcf8-kmnkh\" (UID: \"280cbc95-d762-4661-bf63-0f5606364d06\") " pod="openshift-console/console-5b55b4bcf8-kmnkh"
Sep 11 17:24:48.357153 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:48.357043    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/280cbc95-d762-4661-bf63-0f5606364d06-console-oauth-config\") pod \"console-5b55b4bcf8-kmnkh\" (UID: \"280cbc95-d762-4661-bf63-0f5606364d06\") " pod="openshift-console/console-5b55b4bcf8-kmnkh"
Sep 11 17:24:48.357153 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:48.357091    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/280cbc95-d762-4661-bf63-0f5606364d06-trusted-ca-bundle\") pod \"console-5b55b4bcf8-kmnkh\" (UID: \"280cbc95-d762-4661-bf63-0f5606364d06\") " pod="openshift-console/console-5b55b4bcf8-kmnkh"
Sep 11 17:24:48.357286 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:48.357156    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/280cbc95-d762-4661-bf63-0f5606364d06-console-config\") pod \"console-5b55b4bcf8-kmnkh\" (UID: \"280cbc95-d762-4661-bf63-0f5606364d06\") " pod="openshift-console/console-5b55b4bcf8-kmnkh"
Sep 11 17:24:48.357286 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:48.357195    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/280cbc95-d762-4661-bf63-0f5606364d06-console-serving-cert\") pod \"console-5b55b4bcf8-kmnkh\" (UID: \"280cbc95-d762-4661-bf63-0f5606364d06\") " pod="openshift-console/console-5b55b4bcf8-kmnkh"
Sep 11 17:24:48.357286 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:48.357227    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4z86q\" (UniqueName: \"kubernetes.io/projected/280cbc95-d762-4661-bf63-0f5606364d06-kube-api-access-4z86q\") pod \"console-5b55b4bcf8-kmnkh\" (UID: \"280cbc95-d762-4661-bf63-0f5606364d06\") " pod="openshift-console/console-5b55b4bcf8-kmnkh"
Sep 11 17:24:48.357625 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:48.357602    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/280cbc95-d762-4661-bf63-0f5606364d06-service-ca\") pod \"console-5b55b4bcf8-kmnkh\" (UID: \"280cbc95-d762-4661-bf63-0f5606364d06\") " pod="openshift-console/console-5b55b4bcf8-kmnkh"
Sep 11 17:24:48.357720 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:48.357693    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/280cbc95-d762-4661-bf63-0f5606364d06-oauth-serving-cert\") pod \"console-5b55b4bcf8-kmnkh\" (UID: \"280cbc95-d762-4661-bf63-0f5606364d06\") " pod="openshift-console/console-5b55b4bcf8-kmnkh"
Sep 11 17:24:48.357834 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:48.357808    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/280cbc95-d762-4661-bf63-0f5606364d06-console-config\") pod \"console-5b55b4bcf8-kmnkh\" (UID: \"280cbc95-d762-4661-bf63-0f5606364d06\") " pod="openshift-console/console-5b55b4bcf8-kmnkh"
Sep 11 17:24:48.357969 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:48.357942    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/280cbc95-d762-4661-bf63-0f5606364d06-trusted-ca-bundle\") pod \"console-5b55b4bcf8-kmnkh\" (UID: \"280cbc95-d762-4661-bf63-0f5606364d06\") " pod="openshift-console/console-5b55b4bcf8-kmnkh"
Sep 11 17:24:48.359952 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:48.359930    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/280cbc95-d762-4661-bf63-0f5606364d06-console-oauth-config\") pod \"console-5b55b4bcf8-kmnkh\" (UID: \"280cbc95-d762-4661-bf63-0f5606364d06\") " pod="openshift-console/console-5b55b4bcf8-kmnkh"
Sep 11 17:24:48.359952 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:48.359940    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/280cbc95-d762-4661-bf63-0f5606364d06-console-serving-cert\") pod \"console-5b55b4bcf8-kmnkh\" (UID: \"280cbc95-d762-4661-bf63-0f5606364d06\") " pod="openshift-console/console-5b55b4bcf8-kmnkh"
Sep 11 17:24:48.366192 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:48.366173    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4z86q\" (UniqueName: \"kubernetes.io/projected/280cbc95-d762-4661-bf63-0f5606364d06-kube-api-access-4z86q\") pod \"console-5b55b4bcf8-kmnkh\" (UID: \"280cbc95-d762-4661-bf63-0f5606364d06\") " pod="openshift-console/console-5b55b4bcf8-kmnkh"
Sep 11 17:24:48.444293 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:48.444203    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5b55b4bcf8-kmnkh"
Sep 11 17:24:48.615292 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:48.615261    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-glcc5"
Sep 11 17:24:48.718779 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:48.718537    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5b55b4bcf8-kmnkh"]
Sep 11 17:24:48.721791 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:24:48.721765    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod280cbc95_d762_4661_bf63_0f5606364d06.slice/crio-b0be60c3844f758ad677bce72d24d438779052f81baa12f5a2391245c23346da WatchSource:0}: Error finding container b0be60c3844f758ad677bce72d24d438779052f81baa12f5a2391245c23346da: Status 404 returned error can't find the container with id b0be60c3844f758ad677bce72d24d438779052f81baa12f5a2391245c23346da
Sep 11 17:24:48.764279 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:48.764249    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5b55b4bcf8-kmnkh" event={"ID":"280cbc95-d762-4661-bf63-0f5606364d06","Type":"ContainerStarted","Data":"b0be60c3844f758ad677bce72d24d438779052f81baa12f5a2391245c23346da"}
Sep 11 17:24:49.771631 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:49.771592    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5b55b4bcf8-kmnkh" event={"ID":"280cbc95-d762-4661-bf63-0f5606364d06","Type":"ContainerStarted","Data":"f43d871bdf25e8b8d29aa7895c9143592fef0be06f8cbb3a4a4cc3252f609451"}
Sep 11 17:24:49.792764 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:49.792706    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5b55b4bcf8-kmnkh" podStartSLOduration=1.7926887969999998 podStartE2EDuration="1.792688797s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:24:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:24:49.791168723 +0000 UTC m=+384.312460245" watchObservedRunningTime="2026-09-11 17:24:49.792688797 +0000 UTC m=+384.313980318"
Sep 11 17:24:58.445020 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:58.444973    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5b55b4bcf8-kmnkh"
Sep 11 17:24:58.445528 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:58.445052    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5b55b4bcf8-kmnkh"
Sep 11 17:24:58.449721 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:58.449696    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5b55b4bcf8-kmnkh"
Sep 11 17:24:58.820388 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:58.820307    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5b55b4bcf8-kmnkh"
Sep 11 17:24:58.875009 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:58.874969    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-55dd4dd97b-pczxw"]
Sep 11 17:24:59.004541 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:59.004502    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:24:59.101711 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:59.101678    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:24:59.173694 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:24:59.173659    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:25:03.259269 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:03.259234    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:25:03.331984 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:03.331949    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:25:08.032456 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:08.032415    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6g4f2f"]
Sep 11 17:25:08.037730 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:08.037709    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6g4f2f"
Sep 11 17:25:08.042540 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:08.042510    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 11 17:25:08.042696 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:08.042555    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 11 17:25:08.043139 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:08.043102    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-cjjsx\""
Sep 11 17:25:08.047578 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:08.047556    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6g4f2f"]
Sep 11 17:25:08.126342 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:08.126300    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cpc4w9"]
Sep 11 17:25:08.127560 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:08.127534    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/833a18d8-8c5b-44ab-994b-d16d987b2d42-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6g4f2f\" (UID: \"833a18d8-8c5b-44ab-994b-d16d987b2d42\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6g4f2f"
Sep 11 17:25:08.127693 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:08.127675    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9pzql\" (UniqueName: \"kubernetes.io/projected/833a18d8-8c5b-44ab-994b-d16d987b2d42-kube-api-access-9pzql\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6g4f2f\" (UID: \"833a18d8-8c5b-44ab-994b-d16d987b2d42\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6g4f2f"
Sep 11 17:25:08.127736 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:08.127716    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/833a18d8-8c5b-44ab-994b-d16d987b2d42-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6g4f2f\" (UID: \"833a18d8-8c5b-44ab-994b-d16d987b2d42\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6g4f2f"
Sep 11 17:25:08.129965 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:08.129951    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cpc4w9"
Sep 11 17:25:08.140212 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:08.140183    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cpc4w9"]
Sep 11 17:25:08.225039 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:08.225002    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ak278x"]
Sep 11 17:25:08.228269 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:08.228249    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jvcxw\" (UniqueName: \"kubernetes.io/projected/7f2913f3-a1ca-4e9e-9e71-503e6f86acbc-kube-api-access-jvcxw\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cpc4w9\" (UID: \"7f2913f3-a1ca-4e9e-9e71-503e6f86acbc\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cpc4w9"
Sep 11 17:25:08.228352 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:08.228304    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7f2913f3-a1ca-4e9e-9e71-503e6f86acbc-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cpc4w9\" (UID: \"7f2913f3-a1ca-4e9e-9e71-503e6f86acbc\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cpc4w9"
Sep 11 17:25:08.228352 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:08.228334    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9pzql\" (UniqueName: \"kubernetes.io/projected/833a18d8-8c5b-44ab-994b-d16d987b2d42-kube-api-access-9pzql\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6g4f2f\" (UID: \"833a18d8-8c5b-44ab-994b-d16d987b2d42\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6g4f2f"
Sep 11 17:25:08.228467 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:08.228395    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/833a18d8-8c5b-44ab-994b-d16d987b2d42-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6g4f2f\" (UID: \"833a18d8-8c5b-44ab-994b-d16d987b2d42\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6g4f2f"
Sep 11 17:25:08.228467 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:08.228450    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/833a18d8-8c5b-44ab-994b-d16d987b2d42-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6g4f2f\" (UID: \"833a18d8-8c5b-44ab-994b-d16d987b2d42\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6g4f2f"
Sep 11 17:25:08.228571 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:08.228499    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ak278x"
Sep 11 17:25:08.228571 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:08.228504    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7f2913f3-a1ca-4e9e-9e71-503e6f86acbc-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cpc4w9\" (UID: \"7f2913f3-a1ca-4e9e-9e71-503e6f86acbc\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cpc4w9"
Sep 11 17:25:08.228736 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:08.228718    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/833a18d8-8c5b-44ab-994b-d16d987b2d42-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6g4f2f\" (UID: \"833a18d8-8c5b-44ab-994b-d16d987b2d42\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6g4f2f"
Sep 11 17:25:08.228836 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:08.228818    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/833a18d8-8c5b-44ab-994b-d16d987b2d42-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6g4f2f\" (UID: \"833a18d8-8c5b-44ab-994b-d16d987b2d42\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6g4f2f"
Sep 11 17:25:08.241026 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:08.240999    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ak278x"]
Sep 11 17:25:08.243527 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:08.243496    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9pzql\" (UniqueName: \"kubernetes.io/projected/833a18d8-8c5b-44ab-994b-d16d987b2d42-kube-api-access-9pzql\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6g4f2f\" (UID: \"833a18d8-8c5b-44ab-994b-d16d987b2d42\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6g4f2f"
Sep 11 17:25:08.329347 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:08.329257    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-99xjx\" (UniqueName: \"kubernetes.io/projected/4fc7d8ab-383b-4def-84dd-b7141fdfa592-kube-api-access-99xjx\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ak278x\" (UID: \"4fc7d8ab-383b-4def-84dd-b7141fdfa592\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ak278x"
Sep 11 17:25:08.329347 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:08.329338    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7f2913f3-a1ca-4e9e-9e71-503e6f86acbc-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cpc4w9\" (UID: \"7f2913f3-a1ca-4e9e-9e71-503e6f86acbc\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cpc4w9"
Sep 11 17:25:08.329560 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:08.329366    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jvcxw\" (UniqueName: \"kubernetes.io/projected/7f2913f3-a1ca-4e9e-9e71-503e6f86acbc-kube-api-access-jvcxw\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cpc4w9\" (UID: \"7f2913f3-a1ca-4e9e-9e71-503e6f86acbc\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cpc4w9"
Sep 11 17:25:08.329560 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:08.329406    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4fc7d8ab-383b-4def-84dd-b7141fdfa592-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ak278x\" (UID: \"4fc7d8ab-383b-4def-84dd-b7141fdfa592\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ak278x"
Sep 11 17:25:08.329560 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:08.329431    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4fc7d8ab-383b-4def-84dd-b7141fdfa592-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ak278x\" (UID: \"4fc7d8ab-383b-4def-84dd-b7141fdfa592\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ak278x"
Sep 11 17:25:08.329560 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:08.329487    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7f2913f3-a1ca-4e9e-9e71-503e6f86acbc-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cpc4w9\" (UID: \"7f2913f3-a1ca-4e9e-9e71-503e6f86acbc\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cpc4w9"
Sep 11 17:25:08.329775 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:08.329737    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7f2913f3-a1ca-4e9e-9e71-503e6f86acbc-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cpc4w9\" (UID: \"7f2913f3-a1ca-4e9e-9e71-503e6f86acbc\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cpc4w9"
Sep 11 17:25:08.329872 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:08.329850    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7f2913f3-a1ca-4e9e-9e71-503e6f86acbc-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cpc4w9\" (UID: \"7f2913f3-a1ca-4e9e-9e71-503e6f86acbc\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cpc4w9"
Sep 11 17:25:08.333729 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:08.333703    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d74hcq8"]
Sep 11 17:25:08.337541 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:08.337522    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d74hcq8"
Sep 11 17:25:08.339252 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:08.339231    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jvcxw\" (UniqueName: \"kubernetes.io/projected/7f2913f3-a1ca-4e9e-9e71-503e6f86acbc-kube-api-access-jvcxw\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cpc4w9\" (UID: \"7f2913f3-a1ca-4e9e-9e71-503e6f86acbc\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cpc4w9"
Sep 11 17:25:08.345995 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:08.345966    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d74hcq8"]
Sep 11 17:25:08.349173 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:08.349153    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6g4f2f"
Sep 11 17:25:08.430758 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:08.430716    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-99xjx\" (UniqueName: \"kubernetes.io/projected/4fc7d8ab-383b-4def-84dd-b7141fdfa592-kube-api-access-99xjx\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ak278x\" (UID: \"4fc7d8ab-383b-4def-84dd-b7141fdfa592\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ak278x"
Sep 11 17:25:08.430913 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:08.430834    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8fd59\" (UniqueName: \"kubernetes.io/projected/f923c407-9a11-4bb8-8b9f-e5672e88a9b5-kube-api-access-8fd59\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d74hcq8\" (UID: \"f923c407-9a11-4bb8-8b9f-e5672e88a9b5\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d74hcq8"
Sep 11 17:25:08.430913 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:08.430899    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4fc7d8ab-383b-4def-84dd-b7141fdfa592-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ak278x\" (UID: \"4fc7d8ab-383b-4def-84dd-b7141fdfa592\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ak278x"
Sep 11 17:25:08.430992 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:08.430930    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4fc7d8ab-383b-4def-84dd-b7141fdfa592-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ak278x\" (UID: \"4fc7d8ab-383b-4def-84dd-b7141fdfa592\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ak278x"
Sep 11 17:25:08.430992 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:08.430967    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f923c407-9a11-4bb8-8b9f-e5672e88a9b5-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d74hcq8\" (UID: \"f923c407-9a11-4bb8-8b9f-e5672e88a9b5\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d74hcq8"
Sep 11 17:25:08.431075 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:08.431007    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f923c407-9a11-4bb8-8b9f-e5672e88a9b5-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d74hcq8\" (UID: \"f923c407-9a11-4bb8-8b9f-e5672e88a9b5\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d74hcq8"
Sep 11 17:25:08.431341 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:08.431321    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4fc7d8ab-383b-4def-84dd-b7141fdfa592-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ak278x\" (UID: \"4fc7d8ab-383b-4def-84dd-b7141fdfa592\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ak278x"
Sep 11 17:25:08.431385 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:08.431340    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4fc7d8ab-383b-4def-84dd-b7141fdfa592-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ak278x\" (UID: \"4fc7d8ab-383b-4def-84dd-b7141fdfa592\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ak278x"
Sep 11 17:25:08.439541 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:08.439501    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cpc4w9"
Sep 11 17:25:08.443407 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:08.443385    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-99xjx\" (UniqueName: \"kubernetes.io/projected/4fc7d8ab-383b-4def-84dd-b7141fdfa592-kube-api-access-99xjx\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ak278x\" (UID: \"4fc7d8ab-383b-4def-84dd-b7141fdfa592\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ak278x"
Sep 11 17:25:08.531845 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:08.531805    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8fd59\" (UniqueName: \"kubernetes.io/projected/f923c407-9a11-4bb8-8b9f-e5672e88a9b5-kube-api-access-8fd59\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d74hcq8\" (UID: \"f923c407-9a11-4bb8-8b9f-e5672e88a9b5\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d74hcq8"
Sep 11 17:25:08.532036 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:08.531873    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f923c407-9a11-4bb8-8b9f-e5672e88a9b5-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d74hcq8\" (UID: \"f923c407-9a11-4bb8-8b9f-e5672e88a9b5\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d74hcq8"
Sep 11 17:25:08.532036 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:08.531900    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f923c407-9a11-4bb8-8b9f-e5672e88a9b5-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d74hcq8\" (UID: \"f923c407-9a11-4bb8-8b9f-e5672e88a9b5\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d74hcq8"
Sep 11 17:25:08.532297 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:08.532278    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f923c407-9a11-4bb8-8b9f-e5672e88a9b5-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d74hcq8\" (UID: \"f923c407-9a11-4bb8-8b9f-e5672e88a9b5\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d74hcq8"
Sep 11 17:25:08.532373 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:08.532333    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f923c407-9a11-4bb8-8b9f-e5672e88a9b5-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d74hcq8\" (UID: \"f923c407-9a11-4bb8-8b9f-e5672e88a9b5\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d74hcq8"
Sep 11 17:25:08.537706 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:08.537686    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ak278x"
Sep 11 17:25:08.542452 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:08.542435    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8fd59\" (UniqueName: \"kubernetes.io/projected/f923c407-9a11-4bb8-8b9f-e5672e88a9b5-kube-api-access-8fd59\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d74hcq8\" (UID: \"f923c407-9a11-4bb8-8b9f-e5672e88a9b5\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d74hcq8"
Sep 11 17:25:08.645157 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:08.645128    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6g4f2f"]
Sep 11 17:25:08.647949 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:25:08.647919    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod833a18d8_8c5b_44ab_994b_d16d987b2d42.slice/crio-88322a44e3daf9ad974e372cd3f8c45b96ce17176c468e3a8ffc9a7bb79cbe15 WatchSource:0}: Error finding container 88322a44e3daf9ad974e372cd3f8c45b96ce17176c468e3a8ffc9a7bb79cbe15: Status 404 returned error can't find the container with id 88322a44e3daf9ad974e372cd3f8c45b96ce17176c468e3a8ffc9a7bb79cbe15
Sep 11 17:25:08.656025 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:08.656005    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d74hcq8"
Sep 11 17:25:08.734584 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:08.734555    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cpc4w9"]
Sep 11 17:25:08.739043 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:25:08.738608    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f2913f3_a1ca_4e9e_9e71_503e6f86acbc.slice/crio-d1da2b2ccd833632e490e78941f41848d6adf5ae91eb620b755623af4a71beab WatchSource:0}: Error finding container d1da2b2ccd833632e490e78941f41848d6adf5ae91eb620b755623af4a71beab: Status 404 returned error can't find the container with id d1da2b2ccd833632e490e78941f41848d6adf5ae91eb620b755623af4a71beab
Sep 11 17:25:08.816646 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:08.816618    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ak278x"]
Sep 11 17:25:08.850575 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:25:08.850542    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4fc7d8ab_383b_4def_84dd_b7141fdfa592.slice/crio-cd014fb4e64272c5cc3e5a66dfd4f3ae981a877dd17095765a64ed45a7b520ac WatchSource:0}: Error finding container cd014fb4e64272c5cc3e5a66dfd4f3ae981a877dd17095765a64ed45a7b520ac: Status 404 returned error can't find the container with id cd014fb4e64272c5cc3e5a66dfd4f3ae981a877dd17095765a64ed45a7b520ac
Sep 11 17:25:08.867527 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:08.867500    2552 generic.go:356] "Generic (PLEG): container finished" podID="833a18d8-8c5b-44ab-994b-d16d987b2d42" containerID="ba80cf3df50851dcd9092af92a429cda165a83d755f7bb2dc82ab668567f2028" exitCode=0
Sep 11 17:25:08.867640 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:08.867577    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6g4f2f" event={"ID":"833a18d8-8c5b-44ab-994b-d16d987b2d42","Type":"ContainerDied","Data":"ba80cf3df50851dcd9092af92a429cda165a83d755f7bb2dc82ab668567f2028"}
Sep 11 17:25:08.867640 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:08.867627    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6g4f2f" event={"ID":"833a18d8-8c5b-44ab-994b-d16d987b2d42","Type":"ContainerStarted","Data":"88322a44e3daf9ad974e372cd3f8c45b96ce17176c468e3a8ffc9a7bb79cbe15"}
Sep 11 17:25:08.869654 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:08.869550    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cpc4w9" event={"ID":"7f2913f3-a1ca-4e9e-9e71-503e6f86acbc","Type":"ContainerStarted","Data":"d1da2b2ccd833632e490e78941f41848d6adf5ae91eb620b755623af4a71beab"}
Sep 11 17:25:08.871642 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:08.871612    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ak278x" event={"ID":"4fc7d8ab-383b-4def-84dd-b7141fdfa592","Type":"ContainerStarted","Data":"cd014fb4e64272c5cc3e5a66dfd4f3ae981a877dd17095765a64ed45a7b520ac"}
Sep 11 17:25:08.936631 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:08.936598    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d74hcq8"]
Sep 11 17:25:09.026979 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:25:09.026942    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf923c407_9a11_4bb8_8b9f_e5672e88a9b5.slice/crio-944602bb8925314001ee830d43f225832f9f9f89e259618f82ac75bd69e71d3c WatchSource:0}: Error finding container 944602bb8925314001ee830d43f225832f9f9f89e259618f82ac75bd69e71d3c: Status 404 returned error can't find the container with id 944602bb8925314001ee830d43f225832f9f9f89e259618f82ac75bd69e71d3c
Sep 11 17:25:09.877764 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:09.877735    2552 generic.go:356] "Generic (PLEG): container finished" podID="4fc7d8ab-383b-4def-84dd-b7141fdfa592" containerID="642e72df0b87ae26f752979e26d4d14970d340a4bc1412b6a11aaff2010ab3f6" exitCode=0
Sep 11 17:25:09.878078 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:09.877813    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ak278x" event={"ID":"4fc7d8ab-383b-4def-84dd-b7141fdfa592","Type":"ContainerDied","Data":"642e72df0b87ae26f752979e26d4d14970d340a4bc1412b6a11aaff2010ab3f6"}
Sep 11 17:25:09.879494 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:09.879473    2552 generic.go:356] "Generic (PLEG): container finished" podID="7f2913f3-a1ca-4e9e-9e71-503e6f86acbc" containerID="cb638069ec1c46d471cc14c14b926c07052bf6604367f940ad5c7a52f72e322b" exitCode=0
Sep 11 17:25:09.879579 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:09.879557    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cpc4w9" event={"ID":"7f2913f3-a1ca-4e9e-9e71-503e6f86acbc","Type":"ContainerDied","Data":"cb638069ec1c46d471cc14c14b926c07052bf6604367f940ad5c7a52f72e322b"}
Sep 11 17:25:09.881176 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:09.881151    2552 generic.go:356] "Generic (PLEG): container finished" podID="f923c407-9a11-4bb8-8b9f-e5672e88a9b5" containerID="164f45d3e3c1b5e3c477055407605fba0b98e4dfcfaecf71c3c9ef00370db742" exitCode=0
Sep 11 17:25:09.881251 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:09.881180    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d74hcq8" event={"ID":"f923c407-9a11-4bb8-8b9f-e5672e88a9b5","Type":"ContainerDied","Data":"164f45d3e3c1b5e3c477055407605fba0b98e4dfcfaecf71c3c9ef00370db742"}
Sep 11 17:25:09.881251 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:09.881227    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d74hcq8" event={"ID":"f923c407-9a11-4bb8-8b9f-e5672e88a9b5","Type":"ContainerStarted","Data":"944602bb8925314001ee830d43f225832f9f9f89e259618f82ac75bd69e71d3c"}
Sep 11 17:25:10.888783 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:10.888757    2552 generic.go:356] "Generic (PLEG): container finished" podID="833a18d8-8c5b-44ab-994b-d16d987b2d42" containerID="7310a97da30fd2e8ca609a0e8e54a61b9e68c1e9cef91097a028bce4332591dc" exitCode=0
Sep 11 17:25:10.889092 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:10.888822    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6g4f2f" event={"ID":"833a18d8-8c5b-44ab-994b-d16d987b2d42","Type":"ContainerDied","Data":"7310a97da30fd2e8ca609a0e8e54a61b9e68c1e9cef91097a028bce4332591dc"}
Sep 11 17:25:11.897245 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:11.897207    2552 generic.go:356] "Generic (PLEG): container finished" podID="833a18d8-8c5b-44ab-994b-d16d987b2d42" containerID="713694c5d3e923c815513bc3dccc432882458a004b7547b41a754915562bb734" exitCode=0
Sep 11 17:25:11.897670 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:11.897304    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6g4f2f" event={"ID":"833a18d8-8c5b-44ab-994b-d16d987b2d42","Type":"ContainerDied","Data":"713694c5d3e923c815513bc3dccc432882458a004b7547b41a754915562bb734"}
Sep 11 17:25:11.899049 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:11.899029    2552 generic.go:356] "Generic (PLEG): container finished" podID="7f2913f3-a1ca-4e9e-9e71-503e6f86acbc" containerID="0e847254b8c76b720504032ccb2c5a1daae0c7e84896ce97fdfa7a446c9a39d6" exitCode=0
Sep 11 17:25:11.899169 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:11.899091    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cpc4w9" event={"ID":"7f2913f3-a1ca-4e9e-9e71-503e6f86acbc","Type":"ContainerDied","Data":"0e847254b8c76b720504032ccb2c5a1daae0c7e84896ce97fdfa7a446c9a39d6"}
Sep 11 17:25:11.901167 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:11.901144    2552 generic.go:356] "Generic (PLEG): container finished" podID="f923c407-9a11-4bb8-8b9f-e5672e88a9b5" containerID="b7b73b2ee76a798bbfe7a48356c77186a7afdffe361cdbe5acca713a64df26a5" exitCode=0
Sep 11 17:25:11.901255 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:11.901237    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d74hcq8" event={"ID":"f923c407-9a11-4bb8-8b9f-e5672e88a9b5","Type":"ContainerDied","Data":"b7b73b2ee76a798bbfe7a48356c77186a7afdffe361cdbe5acca713a64df26a5"}
Sep 11 17:25:11.903213 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:11.903195    2552 generic.go:356] "Generic (PLEG): container finished" podID="4fc7d8ab-383b-4def-84dd-b7141fdfa592" containerID="b717c52bcf13a41b2cb738048306be75121e62ac0faad2fe4accc0a13e3e977b" exitCode=0
Sep 11 17:25:11.903279 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:11.903258    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ak278x" event={"ID":"4fc7d8ab-383b-4def-84dd-b7141fdfa592","Type":"ContainerDied","Data":"b717c52bcf13a41b2cb738048306be75121e62ac0faad2fe4accc0a13e3e977b"}
Sep 11 17:25:12.911351 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:12.911311    2552 generic.go:356] "Generic (PLEG): container finished" podID="4fc7d8ab-383b-4def-84dd-b7141fdfa592" containerID="e5fd4ade3d32263cef68edf53a386fd24a4ccda65e735cba544d843b91dd70c7" exitCode=0
Sep 11 17:25:12.911762 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:12.911394    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ak278x" event={"ID":"4fc7d8ab-383b-4def-84dd-b7141fdfa592","Type":"ContainerDied","Data":"e5fd4ade3d32263cef68edf53a386fd24a4ccda65e735cba544d843b91dd70c7"}
Sep 11 17:25:12.916224 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:12.914206    2552 generic.go:356] "Generic (PLEG): container finished" podID="7f2913f3-a1ca-4e9e-9e71-503e6f86acbc" containerID="d0b4da44a6a75f2de6a97083b043a49491181cd11317f1ff0b90350abe332333" exitCode=0
Sep 11 17:25:12.916224 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:12.914362    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cpc4w9" event={"ID":"7f2913f3-a1ca-4e9e-9e71-503e6f86acbc","Type":"ContainerDied","Data":"d0b4da44a6a75f2de6a97083b043a49491181cd11317f1ff0b90350abe332333"}
Sep 11 17:25:12.923389 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:12.923364    2552 generic.go:356] "Generic (PLEG): container finished" podID="f923c407-9a11-4bb8-8b9f-e5672e88a9b5" containerID="a32da46aeacee6e25ba8ba3f75df95cfffb35ec22fd0ea9b4e7cfef02efb9bd7" exitCode=0
Sep 11 17:25:12.923536 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:12.923452    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d74hcq8" event={"ID":"f923c407-9a11-4bb8-8b9f-e5672e88a9b5","Type":"ContainerDied","Data":"a32da46aeacee6e25ba8ba3f75df95cfffb35ec22fd0ea9b4e7cfef02efb9bd7"}
Sep 11 17:25:13.059691 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:13.059665    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6g4f2f"
Sep 11 17:25:13.176960 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:13.176863    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/833a18d8-8c5b-44ab-994b-d16d987b2d42-bundle\") pod \"833a18d8-8c5b-44ab-994b-d16d987b2d42\" (UID: \"833a18d8-8c5b-44ab-994b-d16d987b2d42\") "
Sep 11 17:25:13.176960 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:13.176944    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9pzql\" (UniqueName: \"kubernetes.io/projected/833a18d8-8c5b-44ab-994b-d16d987b2d42-kube-api-access-9pzql\") pod \"833a18d8-8c5b-44ab-994b-d16d987b2d42\" (UID: \"833a18d8-8c5b-44ab-994b-d16d987b2d42\") "
Sep 11 17:25:13.177240 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:13.177021    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/833a18d8-8c5b-44ab-994b-d16d987b2d42-util\") pod \"833a18d8-8c5b-44ab-994b-d16d987b2d42\" (UID: \"833a18d8-8c5b-44ab-994b-d16d987b2d42\") "
Sep 11 17:25:13.177501 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:13.177449    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/833a18d8-8c5b-44ab-994b-d16d987b2d42-bundle" (OuterVolumeSpecName: "bundle") pod "833a18d8-8c5b-44ab-994b-d16d987b2d42" (UID: "833a18d8-8c5b-44ab-994b-d16d987b2d42"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:25:13.179228 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:13.179199    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/833a18d8-8c5b-44ab-994b-d16d987b2d42-kube-api-access-9pzql" (OuterVolumeSpecName: "kube-api-access-9pzql") pod "833a18d8-8c5b-44ab-994b-d16d987b2d42" (UID: "833a18d8-8c5b-44ab-994b-d16d987b2d42"). InnerVolumeSpecName "kube-api-access-9pzql". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:25:13.185297 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:13.185266    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/833a18d8-8c5b-44ab-994b-d16d987b2d42-util" (OuterVolumeSpecName: "util") pod "833a18d8-8c5b-44ab-994b-d16d987b2d42" (UID: "833a18d8-8c5b-44ab-994b-d16d987b2d42"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:25:13.277847 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:13.277815    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9pzql\" (UniqueName: \"kubernetes.io/projected/833a18d8-8c5b-44ab-994b-d16d987b2d42-kube-api-access-9pzql\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:25:13.277847 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:13.277842    2552 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/833a18d8-8c5b-44ab-994b-d16d987b2d42-util\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:25:13.277847 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:13.277853    2552 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/833a18d8-8c5b-44ab-994b-d16d987b2d42-bundle\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:25:13.931962 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:13.931934    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6g4f2f"
Sep 11 17:25:13.932511 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:13.931963    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6g4f2f" event={"ID":"833a18d8-8c5b-44ab-994b-d16d987b2d42","Type":"ContainerDied","Data":"88322a44e3daf9ad974e372cd3f8c45b96ce17176c468e3a8ffc9a7bb79cbe15"}
Sep 11 17:25:13.932511 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:13.932016    2552 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="88322a44e3daf9ad974e372cd3f8c45b96ce17176c468e3a8ffc9a7bb79cbe15"
Sep 11 17:25:14.089702 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:14.089676    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d74hcq8"
Sep 11 17:25:14.132678 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:14.132653    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cpc4w9"
Sep 11 17:25:14.135811 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:14.135793    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ak278x"
Sep 11 17:25:14.186610 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:14.186523    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7f2913f3-a1ca-4e9e-9e71-503e6f86acbc-bundle\") pod \"7f2913f3-a1ca-4e9e-9e71-503e6f86acbc\" (UID: \"7f2913f3-a1ca-4e9e-9e71-503e6f86acbc\") "
Sep 11 17:25:14.186610 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:14.186555    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f923c407-9a11-4bb8-8b9f-e5672e88a9b5-util\") pod \"f923c407-9a11-4bb8-8b9f-e5672e88a9b5\" (UID: \"f923c407-9a11-4bb8-8b9f-e5672e88a9b5\") "
Sep 11 17:25:14.186610 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:14.186578    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-99xjx\" (UniqueName: \"kubernetes.io/projected/4fc7d8ab-383b-4def-84dd-b7141fdfa592-kube-api-access-99xjx\") pod \"4fc7d8ab-383b-4def-84dd-b7141fdfa592\" (UID: \"4fc7d8ab-383b-4def-84dd-b7141fdfa592\") "
Sep 11 17:25:14.186610 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:14.186604    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jvcxw\" (UniqueName: \"kubernetes.io/projected/7f2913f3-a1ca-4e9e-9e71-503e6f86acbc-kube-api-access-jvcxw\") pod \"7f2913f3-a1ca-4e9e-9e71-503e6f86acbc\" (UID: \"7f2913f3-a1ca-4e9e-9e71-503e6f86acbc\") "
Sep 11 17:25:14.186928 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:14.186664    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4fc7d8ab-383b-4def-84dd-b7141fdfa592-bundle\") pod \"4fc7d8ab-383b-4def-84dd-b7141fdfa592\" (UID: \"4fc7d8ab-383b-4def-84dd-b7141fdfa592\") "
Sep 11 17:25:14.186928 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:14.186691    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7f2913f3-a1ca-4e9e-9e71-503e6f86acbc-util\") pod \"7f2913f3-a1ca-4e9e-9e71-503e6f86acbc\" (UID: \"7f2913f3-a1ca-4e9e-9e71-503e6f86acbc\") "
Sep 11 17:25:14.186928 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:14.186720    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8fd59\" (UniqueName: \"kubernetes.io/projected/f923c407-9a11-4bb8-8b9f-e5672e88a9b5-kube-api-access-8fd59\") pod \"f923c407-9a11-4bb8-8b9f-e5672e88a9b5\" (UID: \"f923c407-9a11-4bb8-8b9f-e5672e88a9b5\") "
Sep 11 17:25:14.186928 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:14.186792    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4fc7d8ab-383b-4def-84dd-b7141fdfa592-util\") pod \"4fc7d8ab-383b-4def-84dd-b7141fdfa592\" (UID: \"4fc7d8ab-383b-4def-84dd-b7141fdfa592\") "
Sep 11 17:25:14.186928 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:14.186817    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f923c407-9a11-4bb8-8b9f-e5672e88a9b5-bundle\") pod \"f923c407-9a11-4bb8-8b9f-e5672e88a9b5\" (UID: \"f923c407-9a11-4bb8-8b9f-e5672e88a9b5\") "
Sep 11 17:25:14.187669 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:14.187439    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f2913f3-a1ca-4e9e-9e71-503e6f86acbc-bundle" (OuterVolumeSpecName: "bundle") pod "7f2913f3-a1ca-4e9e-9e71-503e6f86acbc" (UID: "7f2913f3-a1ca-4e9e-9e71-503e6f86acbc"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:25:14.187815 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:14.187712    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f923c407-9a11-4bb8-8b9f-e5672e88a9b5-bundle" (OuterVolumeSpecName: "bundle") pod "f923c407-9a11-4bb8-8b9f-e5672e88a9b5" (UID: "f923c407-9a11-4bb8-8b9f-e5672e88a9b5"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:25:14.188005 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:14.187960    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4fc7d8ab-383b-4def-84dd-b7141fdfa592-bundle" (OuterVolumeSpecName: "bundle") pod "4fc7d8ab-383b-4def-84dd-b7141fdfa592" (UID: "4fc7d8ab-383b-4def-84dd-b7141fdfa592"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:25:14.189597 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:14.189567    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7f2913f3-a1ca-4e9e-9e71-503e6f86acbc-kube-api-access-jvcxw" (OuterVolumeSpecName: "kube-api-access-jvcxw") pod "7f2913f3-a1ca-4e9e-9e71-503e6f86acbc" (UID: "7f2913f3-a1ca-4e9e-9e71-503e6f86acbc"). InnerVolumeSpecName "kube-api-access-jvcxw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:25:14.189924 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:14.189888    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4fc7d8ab-383b-4def-84dd-b7141fdfa592-kube-api-access-99xjx" (OuterVolumeSpecName: "kube-api-access-99xjx") pod "4fc7d8ab-383b-4def-84dd-b7141fdfa592" (UID: "4fc7d8ab-383b-4def-84dd-b7141fdfa592"). InnerVolumeSpecName "kube-api-access-99xjx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:25:14.190038 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:14.190014    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f923c407-9a11-4bb8-8b9f-e5672e88a9b5-kube-api-access-8fd59" (OuterVolumeSpecName: "kube-api-access-8fd59") pod "f923c407-9a11-4bb8-8b9f-e5672e88a9b5" (UID: "f923c407-9a11-4bb8-8b9f-e5672e88a9b5"). InnerVolumeSpecName "kube-api-access-8fd59". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:25:14.193552 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:14.193526    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4fc7d8ab-383b-4def-84dd-b7141fdfa592-util" (OuterVolumeSpecName: "util") pod "4fc7d8ab-383b-4def-84dd-b7141fdfa592" (UID: "4fc7d8ab-383b-4def-84dd-b7141fdfa592"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:25:14.194015 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:14.193995    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f923c407-9a11-4bb8-8b9f-e5672e88a9b5-util" (OuterVolumeSpecName: "util") pod "f923c407-9a11-4bb8-8b9f-e5672e88a9b5" (UID: "f923c407-9a11-4bb8-8b9f-e5672e88a9b5"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:25:14.194631 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:14.194606    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f2913f3-a1ca-4e9e-9e71-503e6f86acbc-util" (OuterVolumeSpecName: "util") pod "7f2913f3-a1ca-4e9e-9e71-503e6f86acbc" (UID: "7f2913f3-a1ca-4e9e-9e71-503e6f86acbc"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:25:14.287960 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:14.287922    2552 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4fc7d8ab-383b-4def-84dd-b7141fdfa592-util\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:25:14.287960 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:14.287954    2552 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f923c407-9a11-4bb8-8b9f-e5672e88a9b5-bundle\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:25:14.287960 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:14.287962    2552 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7f2913f3-a1ca-4e9e-9e71-503e6f86acbc-bundle\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:25:14.287960 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:14.287969    2552 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f923c407-9a11-4bb8-8b9f-e5672e88a9b5-util\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:25:14.288247 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:14.287977    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-99xjx\" (UniqueName: \"kubernetes.io/projected/4fc7d8ab-383b-4def-84dd-b7141fdfa592-kube-api-access-99xjx\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:25:14.288247 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:14.287988    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jvcxw\" (UniqueName: \"kubernetes.io/projected/7f2913f3-a1ca-4e9e-9e71-503e6f86acbc-kube-api-access-jvcxw\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:25:14.288247 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:14.287997    2552 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4fc7d8ab-383b-4def-84dd-b7141fdfa592-bundle\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:25:14.288247 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:14.288005    2552 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7f2913f3-a1ca-4e9e-9e71-503e6f86acbc-util\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:25:14.288247 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:14.288013    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8fd59\" (UniqueName: \"kubernetes.io/projected/f923c407-9a11-4bb8-8b9f-e5672e88a9b5-kube-api-access-8fd59\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:25:14.940652 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:14.940618    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cpc4w9" event={"ID":"7f2913f3-a1ca-4e9e-9e71-503e6f86acbc","Type":"ContainerDied","Data":"d1da2b2ccd833632e490e78941f41848d6adf5ae91eb620b755623af4a71beab"}
Sep 11 17:25:14.940652 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:14.940639    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cpc4w9"
Sep 11 17:25:14.940652 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:14.940662    2552 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d1da2b2ccd833632e490e78941f41848d6adf5ae91eb620b755623af4a71beab"
Sep 11 17:25:14.942619 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:14.942601    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d74hcq8"
Sep 11 17:25:14.942729 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:14.942634    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d74hcq8" event={"ID":"f923c407-9a11-4bb8-8b9f-e5672e88a9b5","Type":"ContainerDied","Data":"944602bb8925314001ee830d43f225832f9f9f89e259618f82ac75bd69e71d3c"}
Sep 11 17:25:14.942729 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:14.942675    2552 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="944602bb8925314001ee830d43f225832f9f9f89e259618f82ac75bd69e71d3c"
Sep 11 17:25:14.944743 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:14.944722    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ak278x"
Sep 11 17:25:14.944743 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:14.944734    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ak278x" event={"ID":"4fc7d8ab-383b-4def-84dd-b7141fdfa592","Type":"ContainerDied","Data":"cd014fb4e64272c5cc3e5a66dfd4f3ae981a877dd17095765a64ed45a7b520ac"}
Sep 11 17:25:14.944919 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:14.944763    2552 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cd014fb4e64272c5cc3e5a66dfd4f3ae981a877dd17095765a64ed45a7b520ac"
Sep 11 17:25:18.087950 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:18.087912    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:25:20.139413 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:20.139370    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-xrp24"]
Sep 11 17:25:20.139910 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:20.139725    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4fc7d8ab-383b-4def-84dd-b7141fdfa592" containerName="util"
Sep 11 17:25:20.139910 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:20.139736    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="4fc7d8ab-383b-4def-84dd-b7141fdfa592" containerName="util"
Sep 11 17:25:20.139910 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:20.139745    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f923c407-9a11-4bb8-8b9f-e5672e88a9b5" containerName="extract"
Sep 11 17:25:20.139910 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:20.139750    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="f923c407-9a11-4bb8-8b9f-e5672e88a9b5" containerName="extract"
Sep 11 17:25:20.139910 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:20.139756    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7f2913f3-a1ca-4e9e-9e71-503e6f86acbc" containerName="extract"
Sep 11 17:25:20.139910 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:20.139761    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f2913f3-a1ca-4e9e-9e71-503e6f86acbc" containerName="extract"
Sep 11 17:25:20.139910 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:20.139770    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="833a18d8-8c5b-44ab-994b-d16d987b2d42" containerName="util"
Sep 11 17:25:20.139910 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:20.139776    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="833a18d8-8c5b-44ab-994b-d16d987b2d42" containerName="util"
Sep 11 17:25:20.139910 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:20.139785    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4fc7d8ab-383b-4def-84dd-b7141fdfa592" containerName="pull"
Sep 11 17:25:20.139910 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:20.139790    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="4fc7d8ab-383b-4def-84dd-b7141fdfa592" containerName="pull"
Sep 11 17:25:20.139910 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:20.139796    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f923c407-9a11-4bb8-8b9f-e5672e88a9b5" containerName="util"
Sep 11 17:25:20.139910 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:20.139801    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="f923c407-9a11-4bb8-8b9f-e5672e88a9b5" containerName="util"
Sep 11 17:25:20.139910 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:20.139807    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f923c407-9a11-4bb8-8b9f-e5672e88a9b5" containerName="pull"
Sep 11 17:25:20.139910 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:20.139811    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="f923c407-9a11-4bb8-8b9f-e5672e88a9b5" containerName="pull"
Sep 11 17:25:20.139910 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:20.139817    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4fc7d8ab-383b-4def-84dd-b7141fdfa592" containerName="extract"
Sep 11 17:25:20.139910 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:20.139823    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="4fc7d8ab-383b-4def-84dd-b7141fdfa592" containerName="extract"
Sep 11 17:25:20.139910 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:20.139830    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="833a18d8-8c5b-44ab-994b-d16d987b2d42" containerName="pull"
Sep 11 17:25:20.139910 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:20.139834    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="833a18d8-8c5b-44ab-994b-d16d987b2d42" containerName="pull"
Sep 11 17:25:20.139910 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:20.139848    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7f2913f3-a1ca-4e9e-9e71-503e6f86acbc" containerName="pull"
Sep 11 17:25:20.139910 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:20.139854    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f2913f3-a1ca-4e9e-9e71-503e6f86acbc" containerName="pull"
Sep 11 17:25:20.139910 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:20.139860    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7f2913f3-a1ca-4e9e-9e71-503e6f86acbc" containerName="util"
Sep 11 17:25:20.139910 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:20.139871    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f2913f3-a1ca-4e9e-9e71-503e6f86acbc" containerName="util"
Sep 11 17:25:20.139910 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:20.139878    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="833a18d8-8c5b-44ab-994b-d16d987b2d42" containerName="extract"
Sep 11 17:25:20.139910 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:20.139883    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="833a18d8-8c5b-44ab-994b-d16d987b2d42" containerName="extract"
Sep 11 17:25:20.140677 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:20.139935    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="f923c407-9a11-4bb8-8b9f-e5672e88a9b5" containerName="extract"
Sep 11 17:25:20.140677 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:20.139942    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="7f2913f3-a1ca-4e9e-9e71-503e6f86acbc" containerName="extract"
Sep 11 17:25:20.140677 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:20.139949    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="4fc7d8ab-383b-4def-84dd-b7141fdfa592" containerName="extract"
Sep 11 17:25:20.140677 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:20.139956    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="833a18d8-8c5b-44ab-994b-d16d987b2d42" containerName="extract"
Sep 11 17:25:20.142903 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:20.142887    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-xrp24"
Sep 11 17:25:20.145177 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:20.145156    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-zj2hr\""
Sep 11 17:25:20.145757 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:20.145742    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 11 17:25:20.145826 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:20.145790    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 11 17:25:20.155152 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:20.155129    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-xrp24"]
Sep 11 17:25:20.244686 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:20.240574    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nxm2z\" (UniqueName: \"kubernetes.io/projected/da955ccd-7d22-4898-a5c3-2db24a31eef6-kube-api-access-nxm2z\") pod \"limitador-operator-controller-manager-74495c69f8-xrp24\" (UID: \"da955ccd-7d22-4898-a5c3-2db24a31eef6\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-xrp24"
Sep 11 17:25:20.341296 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:20.341264    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nxm2z\" (UniqueName: \"kubernetes.io/projected/da955ccd-7d22-4898-a5c3-2db24a31eef6-kube-api-access-nxm2z\") pod \"limitador-operator-controller-manager-74495c69f8-xrp24\" (UID: \"da955ccd-7d22-4898-a5c3-2db24a31eef6\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-xrp24"
Sep 11 17:25:20.361665 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:20.361630    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nxm2z\" (UniqueName: \"kubernetes.io/projected/da955ccd-7d22-4898-a5c3-2db24a31eef6-kube-api-access-nxm2z\") pod \"limitador-operator-controller-manager-74495c69f8-xrp24\" (UID: \"da955ccd-7d22-4898-a5c3-2db24a31eef6\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-xrp24"
Sep 11 17:25:20.453767 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:20.453664    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-xrp24"
Sep 11 17:25:20.694140 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:20.694080    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:25:20.729049 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:20.729019    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-xrp24"]
Sep 11 17:25:20.730820 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:25:20.730787    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda955ccd_7d22_4898_a5c3_2db24a31eef6.slice/crio-6525dd0c7697c58a83d2db606412093cf35386918146e811dfc4ab5b9638d6ff WatchSource:0}: Error finding container 6525dd0c7697c58a83d2db606412093cf35386918146e811dfc4ab5b9638d6ff: Status 404 returned error can't find the container with id 6525dd0c7697c58a83d2db606412093cf35386918146e811dfc4ab5b9638d6ff
Sep 11 17:25:20.976273 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:20.976234    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-xrp24" event={"ID":"da955ccd-7d22-4898-a5c3-2db24a31eef6","Type":"ContainerStarted","Data":"6525dd0c7697c58a83d2db606412093cf35386918146e811dfc4ab5b9638d6ff"}
Sep 11 17:25:21.093714 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:21.093679    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:25:21.535986 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:21.535896    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:25:23.898729 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:23.898688    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-55dd4dd97b-pczxw" podUID="8737406a-9e72-4560-8331-f791e2d118ca" containerName="console" containerID="cri-o://e2ba79364c83b5234026e718ef2b0fd49f5ff0c3e07ae0d36d93f4d2651082fe" gracePeriod=15
Sep 11 17:25:23.944100 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:23.944061    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:25:23.997242 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:23.997204    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-xrp24" event={"ID":"da955ccd-7d22-4898-a5c3-2db24a31eef6","Type":"ContainerStarted","Data":"5c8227e77fb076d7e92dea2c552964fef42bc48c283050c66d70dec4199c4040"}
Sep 11 17:25:23.997418 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:23.997262    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-xrp24"
Sep 11 17:25:24.019058 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:24.019012    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-xrp24" podStartSLOduration=1.2586567309999999 podStartE2EDuration="4.018996777s" totalImagesPullingTime="2.760340046s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:25:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:25:24.016896555 +0000 UTC m=+418.538188075" watchObservedRunningTime="2026-09-11 17:25:24.018996777 +0000 UTC m=+418.540288298"
Sep 11 17:25:24.157987 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:24.157923    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-55dd4dd97b-pczxw"
Sep 11 17:25:24.277332 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:24.277299    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8737406a-9e72-4560-8331-f791e2d118ca-console-config\") pod \"8737406a-9e72-4560-8331-f791e2d118ca\" (UID: \"8737406a-9e72-4560-8331-f791e2d118ca\") "
Sep 11 17:25:24.277516 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:24.277361    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8737406a-9e72-4560-8331-f791e2d118ca-oauth-serving-cert\") pod \"8737406a-9e72-4560-8331-f791e2d118ca\" (UID: \"8737406a-9e72-4560-8331-f791e2d118ca\") "
Sep 11 17:25:24.277516 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:24.277394    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8737406a-9e72-4560-8331-f791e2d118ca-console-oauth-config\") pod \"8737406a-9e72-4560-8331-f791e2d118ca\" (UID: \"8737406a-9e72-4560-8331-f791e2d118ca\") "
Sep 11 17:25:24.277516 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:24.277411    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8737406a-9e72-4560-8331-f791e2d118ca-service-ca\") pod \"8737406a-9e72-4560-8331-f791e2d118ca\" (UID: \"8737406a-9e72-4560-8331-f791e2d118ca\") "
Sep 11 17:25:24.277516 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:24.277449    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xckwg\" (UniqueName: \"kubernetes.io/projected/8737406a-9e72-4560-8331-f791e2d118ca-kube-api-access-xckwg\") pod \"8737406a-9e72-4560-8331-f791e2d118ca\" (UID: \"8737406a-9e72-4560-8331-f791e2d118ca\") "
Sep 11 17:25:24.277516 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:24.277481    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8737406a-9e72-4560-8331-f791e2d118ca-console-serving-cert\") pod \"8737406a-9e72-4560-8331-f791e2d118ca\" (UID: \"8737406a-9e72-4560-8331-f791e2d118ca\") "
Sep 11 17:25:24.277785 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:24.277517    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8737406a-9e72-4560-8331-f791e2d118ca-trusted-ca-bundle\") pod \"8737406a-9e72-4560-8331-f791e2d118ca\" (UID: \"8737406a-9e72-4560-8331-f791e2d118ca\") "
Sep 11 17:25:24.277872 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:24.277841    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8737406a-9e72-4560-8331-f791e2d118ca-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "8737406a-9e72-4560-8331-f791e2d118ca" (UID: "8737406a-9e72-4560-8331-f791e2d118ca"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 17:25:24.277932 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:24.277878    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8737406a-9e72-4560-8331-f791e2d118ca-console-config" (OuterVolumeSpecName: "console-config") pod "8737406a-9e72-4560-8331-f791e2d118ca" (UID: "8737406a-9e72-4560-8331-f791e2d118ca"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 17:25:24.278040 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:24.278016    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8737406a-9e72-4560-8331-f791e2d118ca-service-ca" (OuterVolumeSpecName: "service-ca") pod "8737406a-9e72-4560-8331-f791e2d118ca" (UID: "8737406a-9e72-4560-8331-f791e2d118ca"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 17:25:24.278092 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:24.278072    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8737406a-9e72-4560-8331-f791e2d118ca-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "8737406a-9e72-4560-8331-f791e2d118ca" (UID: "8737406a-9e72-4560-8331-f791e2d118ca"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 17:25:24.279832 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:24.279795    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8737406a-9e72-4560-8331-f791e2d118ca-kube-api-access-xckwg" (OuterVolumeSpecName: "kube-api-access-xckwg") pod "8737406a-9e72-4560-8331-f791e2d118ca" (UID: "8737406a-9e72-4560-8331-f791e2d118ca"). InnerVolumeSpecName "kube-api-access-xckwg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:25:24.279832 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:24.279804    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8737406a-9e72-4560-8331-f791e2d118ca-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "8737406a-9e72-4560-8331-f791e2d118ca" (UID: "8737406a-9e72-4560-8331-f791e2d118ca"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 17:25:24.279992 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:24.279832    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8737406a-9e72-4560-8331-f791e2d118ca-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "8737406a-9e72-4560-8331-f791e2d118ca" (UID: "8737406a-9e72-4560-8331-f791e2d118ca"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 17:25:24.378564 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:24.378532    2552 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8737406a-9e72-4560-8331-f791e2d118ca-console-config\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:25:24.378564 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:24.378559    2552 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8737406a-9e72-4560-8331-f791e2d118ca-oauth-serving-cert\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:25:24.378564 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:24.378569    2552 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8737406a-9e72-4560-8331-f791e2d118ca-console-oauth-config\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:25:24.378792 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:24.378578    2552 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8737406a-9e72-4560-8331-f791e2d118ca-service-ca\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:25:24.378792 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:24.378587    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xckwg\" (UniqueName: \"kubernetes.io/projected/8737406a-9e72-4560-8331-f791e2d118ca-kube-api-access-xckwg\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:25:24.378792 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:24.378597    2552 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8737406a-9e72-4560-8331-f791e2d118ca-console-serving-cert\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:25:24.378792 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:24.378604    2552 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8737406a-9e72-4560-8331-f791e2d118ca-trusted-ca-bundle\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:25:25.005288 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:25.005258    2552 generic.go:356] "Generic (PLEG): container finished" podID="8737406a-9e72-4560-8331-f791e2d118ca" containerID="e2ba79364c83b5234026e718ef2b0fd49f5ff0c3e07ae0d36d93f4d2651082fe" exitCode=0
Sep 11 17:25:25.005751 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:25.005301    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-55dd4dd97b-pczxw"
Sep 11 17:25:25.005751 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:25.005339    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-55dd4dd97b-pczxw" event={"ID":"8737406a-9e72-4560-8331-f791e2d118ca","Type":"ContainerDied","Data":"e2ba79364c83b5234026e718ef2b0fd49f5ff0c3e07ae0d36d93f4d2651082fe"}
Sep 11 17:25:25.005751 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:25.005387    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-55dd4dd97b-pczxw" event={"ID":"8737406a-9e72-4560-8331-f791e2d118ca","Type":"ContainerDied","Data":"2254b5398a354b4ab1685920cf6ed8f7e11147c6bc8c0f4dd54941e9c95cf0f9"}
Sep 11 17:25:25.005751 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:25.005407    2552 scope.go:117] "RemoveContainer" containerID="e2ba79364c83b5234026e718ef2b0fd49f5ff0c3e07ae0d36d93f4d2651082fe"
Sep 11 17:25:25.018162 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:25.018144    2552 scope.go:117] "RemoveContainer" containerID="e2ba79364c83b5234026e718ef2b0fd49f5ff0c3e07ae0d36d93f4d2651082fe"
Sep 11 17:25:25.018434 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:25:25.018415    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e2ba79364c83b5234026e718ef2b0fd49f5ff0c3e07ae0d36d93f4d2651082fe\": container with ID starting with e2ba79364c83b5234026e718ef2b0fd49f5ff0c3e07ae0d36d93f4d2651082fe not found: ID does not exist" containerID="e2ba79364c83b5234026e718ef2b0fd49f5ff0c3e07ae0d36d93f4d2651082fe"
Sep 11 17:25:25.018506 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:25.018443    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e2ba79364c83b5234026e718ef2b0fd49f5ff0c3e07ae0d36d93f4d2651082fe"} err="failed to get container status \"e2ba79364c83b5234026e718ef2b0fd49f5ff0c3e07ae0d36d93f4d2651082fe\": rpc error: code = NotFound desc = could not find container \"e2ba79364c83b5234026e718ef2b0fd49f5ff0c3e07ae0d36d93f4d2651082fe\": container with ID starting with e2ba79364c83b5234026e718ef2b0fd49f5ff0c3e07ae0d36d93f4d2651082fe not found: ID does not exist"
Sep 11 17:25:25.041335 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:25.041311    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-55dd4dd97b-pczxw"]
Sep 11 17:25:25.052412 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:25.052384    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-55dd4dd97b-pczxw"]
Sep 11 17:25:25.061503 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:25.061468    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:25:25.189486 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:25.189448    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:25:25.955031 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:25.954999    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8737406a-9e72-4560-8331-f791e2d118ca" path="/var/lib/kubelet/pods/8737406a-9e72-4560-8331-f791e2d118ca/volumes"
Sep 11 17:25:26.701402 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:26.701360    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:25:27.068664 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:27.068566    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:25:27.365550 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:27.365519    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:25:29.929523 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:29.929489    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:25:30.034550 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:30.034519    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:25:30.863855 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:30.863820    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:25:31.015619 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:31.015580    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:25:32.045035 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:32.045000    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:25:32.150500 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:32.150468    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:25:34.900585 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:34.900549    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:25:34.983091 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:34.983055    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:25:35.008438 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:35.008411    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-xrp24"
Sep 11 17:25:35.348599 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:35.348571    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:25:35.448273 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:35.448204    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:25:35.736913 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:35.736822    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:25:35.933050 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:35.933011    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:25:36.248635 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:36.248602    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:25:36.348746 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:36.348705    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:25:41.173027 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:41.172994    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:25:41.285794 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:41.285760    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:25:42.983674 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:42.983637    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-xrp24"]
Sep 11 17:25:42.984122 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:42.983904    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-xrp24" podUID="da955ccd-7d22-4898-a5c3-2db24a31eef6" containerName="manager" containerID="cri-o://5c8227e77fb076d7e92dea2c552964fef42bc48c283050c66d70dec4199c4040" gracePeriod=2
Sep 11 17:25:42.995231 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:25:42.995195    2552 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"da955ccd-7d22-4898-a5c3-2db24a31eef6\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"limitador-operator-controller-manager-74495c69f8-xrp24\": pods \"limitador-operator-controller-manager-74495c69f8-xrp24\" not found" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-xrp24"
Sep 11 17:25:42.995373 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:42.995339    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-xrp24"]
Sep 11 17:25:43.017124 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:43.017078    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-xb8h4"]
Sep 11 17:25:43.017479 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:43.017465    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da955ccd-7d22-4898-a5c3-2db24a31eef6" containerName="manager"
Sep 11 17:25:43.017527 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:43.017481    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="da955ccd-7d22-4898-a5c3-2db24a31eef6" containerName="manager"
Sep 11 17:25:43.017527 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:43.017496    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8737406a-9e72-4560-8331-f791e2d118ca" containerName="console"
Sep 11 17:25:43.017527 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:43.017501    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="8737406a-9e72-4560-8331-f791e2d118ca" containerName="console"
Sep 11 17:25:43.017632 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:43.017604    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="8737406a-9e72-4560-8331-f791e2d118ca" containerName="console"
Sep 11 17:25:43.017632 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:43.017617    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="da955ccd-7d22-4898-a5c3-2db24a31eef6" containerName="manager"
Sep 11 17:25:43.029453 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:43.029429    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-xb8h4"
Sep 11 17:25:43.031689 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:25:43.031659    2552 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-xrp24\" is forbidden: User \"system:node:ip-10-0-140-51.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-51.ec2.internal' and this object" podUID="da955ccd-7d22-4898-a5c3-2db24a31eef6" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-xrp24"
Sep 11 17:25:43.038020 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:43.037995    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-xb8h4"]
Sep 11 17:25:43.099650 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:43.099621    2552 generic.go:356] "Generic (PLEG): container finished" podID="da955ccd-7d22-4898-a5c3-2db24a31eef6" containerID="5c8227e77fb076d7e92dea2c552964fef42bc48c283050c66d70dec4199c4040" exitCode=0
Sep 11 17:25:43.138279 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:43.138241    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gkcnh\" (UniqueName: \"kubernetes.io/projected/69263d34-2a29-4704-b91d-f292eb09e9f0-kube-api-access-gkcnh\") pod \"limitador-operator-controller-manager-74495c69f8-xb8h4\" (UID: \"69263d34-2a29-4704-b91d-f292eb09e9f0\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-xb8h4"
Sep 11 17:25:43.238845 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:43.238784    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gkcnh\" (UniqueName: \"kubernetes.io/projected/69263d34-2a29-4704-b91d-f292eb09e9f0-kube-api-access-gkcnh\") pod \"limitador-operator-controller-manager-74495c69f8-xb8h4\" (UID: \"69263d34-2a29-4704-b91d-f292eb09e9f0\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-xb8h4"
Sep 11 17:25:43.247722 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:43.247695    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gkcnh\" (UniqueName: \"kubernetes.io/projected/69263d34-2a29-4704-b91d-f292eb09e9f0-kube-api-access-gkcnh\") pod \"limitador-operator-controller-manager-74495c69f8-xb8h4\" (UID: \"69263d34-2a29-4704-b91d-f292eb09e9f0\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-xb8h4"
Sep 11 17:25:43.258527 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:43.258503    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-xrp24"
Sep 11 17:25:43.260476 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:25:43.260450    2552 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-xrp24\" is forbidden: User \"system:node:ip-10-0-140-51.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-51.ec2.internal' and this object" podUID="da955ccd-7d22-4898-a5c3-2db24a31eef6" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-xrp24"
Sep 11 17:25:43.378256 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:43.378223    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-xb8h4"
Sep 11 17:25:43.440557 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:43.440526    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nxm2z\" (UniqueName: \"kubernetes.io/projected/da955ccd-7d22-4898-a5c3-2db24a31eef6-kube-api-access-nxm2z\") pod \"da955ccd-7d22-4898-a5c3-2db24a31eef6\" (UID: \"da955ccd-7d22-4898-a5c3-2db24a31eef6\") "
Sep 11 17:25:43.443359 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:43.443318    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/da955ccd-7d22-4898-a5c3-2db24a31eef6-kube-api-access-nxm2z" (OuterVolumeSpecName: "kube-api-access-nxm2z") pod "da955ccd-7d22-4898-a5c3-2db24a31eef6" (UID: "da955ccd-7d22-4898-a5c3-2db24a31eef6"). InnerVolumeSpecName "kube-api-access-nxm2z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:25:43.541336 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:43.541243    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nxm2z\" (UniqueName: \"kubernetes.io/projected/da955ccd-7d22-4898-a5c3-2db24a31eef6-kube-api-access-nxm2z\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:25:43.665270 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:43.665236    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-xb8h4"]
Sep 11 17:25:43.667307 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:25:43.667278    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69263d34_2a29_4704_b91d_f292eb09e9f0.slice/crio-2d777b7ef577d2ad7302b8711c926c395814009862fa5fbb02c38128fb4ba982 WatchSource:0}: Error finding container 2d777b7ef577d2ad7302b8711c926c395814009862fa5fbb02c38128fb4ba982: Status 404 returned error can't find the container with id 2d777b7ef577d2ad7302b8711c926c395814009862fa5fbb02c38128fb4ba982
Sep 11 17:25:43.949710 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:43.949675    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="da955ccd-7d22-4898-a5c3-2db24a31eef6" path="/var/lib/kubelet/pods/da955ccd-7d22-4898-a5c3-2db24a31eef6/volumes"
Sep 11 17:25:44.106993 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:44.106955    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-xb8h4" event={"ID":"69263d34-2a29-4704-b91d-f292eb09e9f0","Type":"ContainerStarted","Data":"bf425ff8dc2a764fd10ecdfd20cff224cb87587a29f290d590d595326bb89071"}
Sep 11 17:25:44.107451 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:44.107016    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-xb8h4"
Sep 11 17:25:44.107451 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:44.107039    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-xb8h4" event={"ID":"69263d34-2a29-4704-b91d-f292eb09e9f0","Type":"ContainerStarted","Data":"2d777b7ef577d2ad7302b8711c926c395814009862fa5fbb02c38128fb4ba982"}
Sep 11 17:25:44.108403 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:44.108385    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-xrp24"
Sep 11 17:25:44.108500 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:44.108405    2552 scope.go:117] "RemoveContainer" containerID="5c8227e77fb076d7e92dea2c552964fef42bc48c283050c66d70dec4199c4040"
Sep 11 17:25:44.154152 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:44.154088    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-xb8h4" podStartSLOduration=2.154072958 podStartE2EDuration="2.154072958s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:25:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:25:44.151695022 +0000 UTC m=+438.672986554" watchObservedRunningTime="2026-09-11 17:25:44.154072958 +0000 UTC m=+438.675364478"
Sep 11 17:25:44.566946 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:44.566906    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-fzs45"]
Sep 11 17:25:44.570012 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:44.569989    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-fzs45"
Sep 11 17:25:44.572415 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:44.572388    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-dztsz\""
Sep 11 17:25:44.586038 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:44.586010    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-fzs45"]
Sep 11 17:25:44.648640 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:44.648610    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3ebd01e5-f712-42b8-9e89-deff14ec577a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-fzs45\" (UID: \"3ebd01e5-f712-42b8-9e89-deff14ec577a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-fzs45"
Sep 11 17:25:44.648821 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:44.648695    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hbqnb\" (UniqueName: \"kubernetes.io/projected/3ebd01e5-f712-42b8-9e89-deff14ec577a-kube-api-access-hbqnb\") pod \"kuadrant-operator-controller-manager-5769f99d9f-fzs45\" (UID: \"3ebd01e5-f712-42b8-9e89-deff14ec577a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-fzs45"
Sep 11 17:25:44.749717 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:44.749682    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3ebd01e5-f712-42b8-9e89-deff14ec577a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-fzs45\" (UID: \"3ebd01e5-f712-42b8-9e89-deff14ec577a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-fzs45"
Sep 11 17:25:44.749891 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:44.749727    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hbqnb\" (UniqueName: \"kubernetes.io/projected/3ebd01e5-f712-42b8-9e89-deff14ec577a-kube-api-access-hbqnb\") pod \"kuadrant-operator-controller-manager-5769f99d9f-fzs45\" (UID: \"3ebd01e5-f712-42b8-9e89-deff14ec577a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-fzs45"
Sep 11 17:25:44.750049 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:44.750030    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3ebd01e5-f712-42b8-9e89-deff14ec577a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-fzs45\" (UID: \"3ebd01e5-f712-42b8-9e89-deff14ec577a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-fzs45"
Sep 11 17:25:44.775293 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:44.775263    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hbqnb\" (UniqueName: \"kubernetes.io/projected/3ebd01e5-f712-42b8-9e89-deff14ec577a-kube-api-access-hbqnb\") pod \"kuadrant-operator-controller-manager-5769f99d9f-fzs45\" (UID: \"3ebd01e5-f712-42b8-9e89-deff14ec577a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-fzs45"
Sep 11 17:25:44.879544 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:44.879509    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-fzs45"
Sep 11 17:25:45.162901 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:45.162827    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-fzs45"]
Sep 11 17:25:45.166393 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:25:45.166367    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ebd01e5_f712_42b8_9e89_deff14ec577a.slice/crio-ead3d834ae9e8ba03452fe1208eedc5267773df521d5626d63d0997c6307151c WatchSource:0}: Error finding container ead3d834ae9e8ba03452fe1208eedc5267773df521d5626d63d0997c6307151c: Status 404 returned error can't find the container with id ead3d834ae9e8ba03452fe1208eedc5267773df521d5626d63d0997c6307151c
Sep 11 17:25:46.125367 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:46.125329    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-fzs45" event={"ID":"3ebd01e5-f712-42b8-9e89-deff14ec577a","Type":"ContainerStarted","Data":"ead3d834ae9e8ba03452fe1208eedc5267773df521d5626d63d0997c6307151c"}
Sep 11 17:25:47.753511 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:47.753470    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:25:47.854627 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:47.854585    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:25:48.325689 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:48.325602    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:25:49.147569 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:49.147534    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-fzs45" event={"ID":"3ebd01e5-f712-42b8-9e89-deff14ec577a","Type":"ContainerStarted","Data":"18dcd4d7b55049a0bcd1aea91d80e7b4a1f7409a1631ba55d4369637dab0bbdb"}
Sep 11 17:25:49.147949 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:49.147664    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-fzs45"
Sep 11 17:25:49.200071 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:49.200015    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-fzs45" podStartSLOduration=1.546745737 podStartE2EDuration="5.200000865s" totalImagesPullingTime="3.653255128s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:25:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:25:49.197695795 +0000 UTC m=+443.718987317" watchObservedRunningTime="2026-09-11 17:25:49.200000865 +0000 UTC m=+443.721292386"
Sep 11 17:25:50.316795 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:50.316752    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:25:50.419532 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:50.419496    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:25:53.693871 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:53.693831    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:25:53.797585 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:53.797541    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:25:53.893371 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:53.893334    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:25:55.118136 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:25:55.118086    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-xb8h4"
Sep 11 17:26:00.155176 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:26:00.155148    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-fzs45"
Sep 11 17:26:00.226922 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:26:00.226887    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:26:00.329514 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:26:00.329482    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:26:06.775979 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:26:06.775945    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:26:06.875519 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:26:06.875486    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:26:09.477454 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:26:09.477415    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:26:09.588845 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:26:09.588813    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:26:13.174531 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:26:13.174492    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:26:13.274199 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:26:13.274162    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:26:13.386977 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:26:13.386948    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:26:13.451564 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:26:13.451482    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:26:14.422381 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:26:14.422343    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:26:14.532389 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:26:14.532354    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:26:16.920160 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:26:16.920128    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:26:17.281708 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:26:17.281620    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:26:19.685558 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:26:19.685524    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:26:19.784552 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:26:19.784510    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:26:21.948569 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:26:21.948528    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:26:22.050758 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:26:22.050702    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:26:22.194628 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:26:22.194594    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:26:22.836163 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:26:22.836103    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:26:22.930373 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:26:22.930338    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:26:22.956489 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:26:22.956459    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-59c974bd48-pmch7"]
Sep 11 17:26:22.959151 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:26:22.959136    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-pmch7"
Sep 11 17:26:22.961307 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:26:22.961285    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-nx4ln\""
Sep 11 17:26:22.969099 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:26:22.969075    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c974bd48-pmch7"]
Sep 11 17:26:23.076873 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:26:23.076838    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5kdxb\" (UniqueName: \"kubernetes.io/projected/e3930735-7256-4e4c-904d-8e4a0bf4260f-kube-api-access-5kdxb\") pod \"authorino-59c974bd48-pmch7\" (UID: \"e3930735-7256-4e4c-904d-8e4a0bf4260f\") " pod="kuadrant-system/authorino-59c974bd48-pmch7"
Sep 11 17:26:23.177631 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:26:23.177597    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5kdxb\" (UniqueName: \"kubernetes.io/projected/e3930735-7256-4e4c-904d-8e4a0bf4260f-kube-api-access-5kdxb\") pod \"authorino-59c974bd48-pmch7\" (UID: \"e3930735-7256-4e4c-904d-8e4a0bf4260f\") " pod="kuadrant-system/authorino-59c974bd48-pmch7"
Sep 11 17:26:23.186741 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:26:23.186709    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5kdxb\" (UniqueName: \"kubernetes.io/projected/e3930735-7256-4e4c-904d-8e4a0bf4260f-kube-api-access-5kdxb\") pod \"authorino-59c974bd48-pmch7\" (UID: \"e3930735-7256-4e4c-904d-8e4a0bf4260f\") " pod="kuadrant-system/authorino-59c974bd48-pmch7"
Sep 11 17:26:23.268205 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:26:23.268170    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-pmch7"
Sep 11 17:26:23.542076 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:26:23.542043    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c974bd48-pmch7"]
Sep 11 17:26:23.543549 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:26:23.543520    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode3930735_7256_4e4c_904d_8e4a0bf4260f.slice/crio-b483e653de09d9c4e9e871cfb9790b13cc05b0109a5ebaebd783fdfe5620de2c WatchSource:0}: Error finding container b483e653de09d9c4e9e871cfb9790b13cc05b0109a5ebaebd783fdfe5620de2c: Status 404 returned error can't find the container with id b483e653de09d9c4e9e871cfb9790b13cc05b0109a5ebaebd783fdfe5620de2c
Sep 11 17:26:24.338879 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:26:24.338844    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-pmch7" event={"ID":"e3930735-7256-4e4c-904d-8e4a0bf4260f","Type":"ContainerStarted","Data":"b483e653de09d9c4e9e871cfb9790b13cc05b0109a5ebaebd783fdfe5620de2c"}
Sep 11 17:26:26.243951 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:26:26.243854    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:26:26.343030 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:26:26.342992    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:26:26.353012 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:26:26.352972    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-pmch7" event={"ID":"e3930735-7256-4e4c-904d-8e4a0bf4260f","Type":"ContainerStarted","Data":"9443581d09b85350f292c888f9dc9277c72149bc609ff5b43c59e7b9fd050693"}
Sep 11 17:26:26.368729 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:26:26.368683    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-59c974bd48-pmch7" podStartSLOduration=2.49849045 podStartE2EDuration="4.368668612s" totalImagesPullingTime="1.870178162s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:26:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:26:26.367923573 +0000 UTC m=+480.889215095" watchObservedRunningTime="2026-09-11 17:26:26.368668612 +0000 UTC m=+480.889960132"
Sep 11 17:26:43.278174 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:26:43.278135    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:26:43.392611 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:26:43.392578    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:26:43.773523 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:26:43.773487    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:26:56.332123 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:26:56.332068    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:26:57.020075 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:26:57.020042    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:26:58.397797 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:26:58.397767    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59c974bd48-pmch7"]
Sep 11 17:26:58.398192 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:26:58.397933    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-59c974bd48-pmch7" podUID="e3930735-7256-4e4c-904d-8e4a0bf4260f" containerName="authorino" containerID="cri-o://9443581d09b85350f292c888f9dc9277c72149bc609ff5b43c59e7b9fd050693" gracePeriod=30
Sep 11 17:26:58.477239 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:26:58.477206    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-66f9b6cfdd-mhkv5"]
Sep 11 17:26:58.480608 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:26:58.480589    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-66f9b6cfdd-mhkv5"
Sep 11 17:26:58.482802 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:26:58.482779    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Sep 11 17:26:58.490198 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:26:58.490166    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-66f9b6cfdd-mhkv5"]
Sep 11 17:26:58.530023 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:26:58.529988    2552 generic.go:356] "Generic (PLEG): container finished" podID="e3930735-7256-4e4c-904d-8e4a0bf4260f" containerID="9443581d09b85350f292c888f9dc9277c72149bc609ff5b43c59e7b9fd050693" exitCode=0
Sep 11 17:26:58.530178 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:26:58.530051    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-pmch7" event={"ID":"e3930735-7256-4e4c-904d-8e4a0bf4260f","Type":"ContainerDied","Data":"9443581d09b85350f292c888f9dc9277c72149bc609ff5b43c59e7b9fd050693"}
Sep 11 17:26:58.560915 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:26:58.560876    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1d5ccc6b-56b9-4a65-967c-46fa32e262cf-tls-cert\") pod \"authorino-66f9b6cfdd-mhkv5\" (UID: \"1d5ccc6b-56b9-4a65-967c-46fa32e262cf\") " pod="kuadrant-system/authorino-66f9b6cfdd-mhkv5"
Sep 11 17:26:58.561081 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:26:58.560929    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fjm66\" (UniqueName: \"kubernetes.io/projected/1d5ccc6b-56b9-4a65-967c-46fa32e262cf-kube-api-access-fjm66\") pod \"authorino-66f9b6cfdd-mhkv5\" (UID: \"1d5ccc6b-56b9-4a65-967c-46fa32e262cf\") " pod="kuadrant-system/authorino-66f9b6cfdd-mhkv5"
Sep 11 17:26:58.658332 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:26:58.658271    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-pmch7"
Sep 11 17:26:58.661348 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:26:58.661330    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1d5ccc6b-56b9-4a65-967c-46fa32e262cf-tls-cert\") pod \"authorino-66f9b6cfdd-mhkv5\" (UID: \"1d5ccc6b-56b9-4a65-967c-46fa32e262cf\") " pod="kuadrant-system/authorino-66f9b6cfdd-mhkv5"
Sep 11 17:26:58.661410 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:26:58.661369    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fjm66\" (UniqueName: \"kubernetes.io/projected/1d5ccc6b-56b9-4a65-967c-46fa32e262cf-kube-api-access-fjm66\") pod \"authorino-66f9b6cfdd-mhkv5\" (UID: \"1d5ccc6b-56b9-4a65-967c-46fa32e262cf\") " pod="kuadrant-system/authorino-66f9b6cfdd-mhkv5"
Sep 11 17:26:58.663992 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:26:58.663970    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1d5ccc6b-56b9-4a65-967c-46fa32e262cf-tls-cert\") pod \"authorino-66f9b6cfdd-mhkv5\" (UID: \"1d5ccc6b-56b9-4a65-967c-46fa32e262cf\") " pod="kuadrant-system/authorino-66f9b6cfdd-mhkv5"
Sep 11 17:26:58.668946 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:26:58.668924    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fjm66\" (UniqueName: \"kubernetes.io/projected/1d5ccc6b-56b9-4a65-967c-46fa32e262cf-kube-api-access-fjm66\") pod \"authorino-66f9b6cfdd-mhkv5\" (UID: \"1d5ccc6b-56b9-4a65-967c-46fa32e262cf\") " pod="kuadrant-system/authorino-66f9b6cfdd-mhkv5"
Sep 11 17:26:58.762375 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:26:58.762345    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5kdxb\" (UniqueName: \"kubernetes.io/projected/e3930735-7256-4e4c-904d-8e4a0bf4260f-kube-api-access-5kdxb\") pod \"e3930735-7256-4e4c-904d-8e4a0bf4260f\" (UID: \"e3930735-7256-4e4c-904d-8e4a0bf4260f\") "
Sep 11 17:26:58.764438 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:26:58.764407    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e3930735-7256-4e4c-904d-8e4a0bf4260f-kube-api-access-5kdxb" (OuterVolumeSpecName: "kube-api-access-5kdxb") pod "e3930735-7256-4e4c-904d-8e4a0bf4260f" (UID: "e3930735-7256-4e4c-904d-8e4a0bf4260f"). InnerVolumeSpecName "kube-api-access-5kdxb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:26:58.791357 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:26:58.791323    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-66f9b6cfdd-mhkv5"
Sep 11 17:26:58.863136 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:26:58.863082    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5kdxb\" (UniqueName: \"kubernetes.io/projected/e3930735-7256-4e4c-904d-8e4a0bf4260f-kube-api-access-5kdxb\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:26:59.063021 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:26:59.062995    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-66f9b6cfdd-mhkv5"]
Sep 11 17:26:59.064723 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:26:59.064693    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1d5ccc6b_56b9_4a65_967c_46fa32e262cf.slice/crio-14da6d866ce53fbc02e26d4bad1175a3174bbbea801cc7a4ebe5c670cf4c8a3b WatchSource:0}: Error finding container 14da6d866ce53fbc02e26d4bad1175a3174bbbea801cc7a4ebe5c670cf4c8a3b: Status 404 returned error can't find the container with id 14da6d866ce53fbc02e26d4bad1175a3174bbbea801cc7a4ebe5c670cf4c8a3b
Sep 11 17:26:59.536423 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:26:59.536387    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-66f9b6cfdd-mhkv5" event={"ID":"1d5ccc6b-56b9-4a65-967c-46fa32e262cf","Type":"ContainerStarted","Data":"14da6d866ce53fbc02e26d4bad1175a3174bbbea801cc7a4ebe5c670cf4c8a3b"}
Sep 11 17:26:59.537676 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:26:59.537654    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-pmch7" event={"ID":"e3930735-7256-4e4c-904d-8e4a0bf4260f","Type":"ContainerDied","Data":"b483e653de09d9c4e9e871cfb9790b13cc05b0109a5ebaebd783fdfe5620de2c"}
Sep 11 17:26:59.537676 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:26:59.537673    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-pmch7"
Sep 11 17:26:59.537784 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:26:59.537695    2552 scope.go:117] "RemoveContainer" containerID="9443581d09b85350f292c888f9dc9277c72149bc609ff5b43c59e7b9fd050693"
Sep 11 17:26:59.558039 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:26:59.558013    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59c974bd48-pmch7"]
Sep 11 17:26:59.561445 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:26:59.561422    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-59c974bd48-pmch7"]
Sep 11 17:26:59.950067 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:26:59.950040    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e3930735-7256-4e4c-904d-8e4a0bf4260f" path="/var/lib/kubelet/pods/e3930735-7256-4e4c-904d-8e4a0bf4260f/volumes"
Sep 11 17:27:00.545433 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:27:00.545398    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-66f9b6cfdd-mhkv5" event={"ID":"1d5ccc6b-56b9-4a65-967c-46fa32e262cf","Type":"ContainerStarted","Data":"98184740890ac3e140ab05575c9175cfc2c05d182b8b6a81cba4322723426d7f"}
Sep 11 17:27:00.562995 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:27:00.562951    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-66f9b6cfdd-mhkv5" podStartSLOduration=1.699518967 podStartE2EDuration="2.562934495s" totalImagesPullingTime="863.415528ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:26:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:27:00.562920472 +0000 UTC m=+515.084211993" watchObservedRunningTime="2026-09-11 17:27:00.562934495 +0000 UTC m=+515.084226141"
Sep 11 17:27:03.216928 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:27:03.216893    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:27:03.287681 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:27:03.287648    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:27:03.344326 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:27:03.344290    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:27:03.650078 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:27:03.650049    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:27:03.718340 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:27:03.718306    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:27:06.720562 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:27:06.720525    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:27:06.817394 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:27:06.817350    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:27:08.243531 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:27:08.243487    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:27:08.342258 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:27:08.342221    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:27:17.680620 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:27:17.680575    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:27:17.751019 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:27:17.750985    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:27:23.363883 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:27:23.363794    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:27:38.496474 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:27:38.496426    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:27:38.594329 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:27:38.594293    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:27:39.914840 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:27:39.914801    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:27:40.015539 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:27:40.015503    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:27:40.198763 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:27:40.198656    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:27:40.455488 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:27:40.455394    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:27:40.553897 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:27:40.553859    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:27:41.159966 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:27:41.159933    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:27:52.239642 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:27:52.239608    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:27:52.327352 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:27:52.327308    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:27:57.249410 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:27:57.249372    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:28:03.707417 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:28:03.707378    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:28:16.106095 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:28:16.106059    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:28:16.559479 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:28:16.559392    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:28:21.902700 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:28:21.902657    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-lv2n9"]
Sep 11 17:28:21.903085 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:28:21.903037    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e3930735-7256-4e4c-904d-8e4a0bf4260f" containerName="authorino"
Sep 11 17:28:21.903085 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:28:21.903048    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="e3930735-7256-4e4c-904d-8e4a0bf4260f" containerName="authorino"
Sep 11 17:28:21.903184 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:28:21.903142    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="e3930735-7256-4e4c-904d-8e4a0bf4260f" containerName="authorino"
Sep 11 17:28:21.906067 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:28:21.906051    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-lv2n9"
Sep 11 17:28:21.908561 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:28:21.908540    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 11 17:28:21.909275 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:28:21.909251    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-embedding-simulated-kserve-self-signed-certs\""
Sep 11 17:28:21.909405 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:28:21.909312    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 11 17:28:21.909405 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:28:21.909323    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-d66tb\""
Sep 11 17:28:21.929245 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:28:21.929219    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-lv2n9"]
Sep 11 17:28:21.958864 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:28:21.958825    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:28:21.990758 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:28:21.990725    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c6d2302e-3deb-44cb-a6e0-ada382faac64-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-lv2n9\" (UID: \"c6d2302e-3deb-44cb-a6e0-ada382faac64\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-lv2n9"
Sep 11 17:28:21.990945 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:28:21.990793    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c6d2302e-3deb-44cb-a6e0-ada382faac64-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-lv2n9\" (UID: \"c6d2302e-3deb-44cb-a6e0-ada382faac64\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-lv2n9"
Sep 11 17:28:21.990945 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:28:21.990824    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c6d2302e-3deb-44cb-a6e0-ada382faac64-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-lv2n9\" (UID: \"c6d2302e-3deb-44cb-a6e0-ada382faac64\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-lv2n9"
Sep 11 17:28:21.990945 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:28:21.990876    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c6d2302e-3deb-44cb-a6e0-ada382faac64-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-lv2n9\" (UID: \"c6d2302e-3deb-44cb-a6e0-ada382faac64\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-lv2n9"
Sep 11 17:28:21.990945 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:28:21.990907    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jggrz\" (UniqueName: \"kubernetes.io/projected/c6d2302e-3deb-44cb-a6e0-ada382faac64-kube-api-access-jggrz\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-lv2n9\" (UID: \"c6d2302e-3deb-44cb-a6e0-ada382faac64\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-lv2n9"
Sep 11 17:28:21.991152 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:28:21.990976    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c6d2302e-3deb-44cb-a6e0-ada382faac64-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-lv2n9\" (UID: \"c6d2302e-3deb-44cb-a6e0-ada382faac64\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-lv2n9"
Sep 11 17:28:22.092226 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:28:22.092178    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c6d2302e-3deb-44cb-a6e0-ada382faac64-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-lv2n9\" (UID: \"c6d2302e-3deb-44cb-a6e0-ada382faac64\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-lv2n9"
Sep 11 17:28:22.092226 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:28:22.092230    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c6d2302e-3deb-44cb-a6e0-ada382faac64-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-lv2n9\" (UID: \"c6d2302e-3deb-44cb-a6e0-ada382faac64\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-lv2n9"
Sep 11 17:28:22.092520 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:28:22.092264    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c6d2302e-3deb-44cb-a6e0-ada382faac64-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-lv2n9\" (UID: \"c6d2302e-3deb-44cb-a6e0-ada382faac64\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-lv2n9"
Sep 11 17:28:22.092520 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:28:22.092285    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jggrz\" (UniqueName: \"kubernetes.io/projected/c6d2302e-3deb-44cb-a6e0-ada382faac64-kube-api-access-jggrz\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-lv2n9\" (UID: \"c6d2302e-3deb-44cb-a6e0-ada382faac64\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-lv2n9"
Sep 11 17:28:22.092520 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:28:22.092316    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c6d2302e-3deb-44cb-a6e0-ada382faac64-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-lv2n9\" (UID: \"c6d2302e-3deb-44cb-a6e0-ada382faac64\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-lv2n9"
Sep 11 17:28:22.092520 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:28:22.092351    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c6d2302e-3deb-44cb-a6e0-ada382faac64-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-lv2n9\" (UID: \"c6d2302e-3deb-44cb-a6e0-ada382faac64\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-lv2n9"
Sep 11 17:28:22.092750 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:28:22.092646    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c6d2302e-3deb-44cb-a6e0-ada382faac64-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-lv2n9\" (UID: \"c6d2302e-3deb-44cb-a6e0-ada382faac64\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-lv2n9"
Sep 11 17:28:22.092750 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:28:22.092702    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c6d2302e-3deb-44cb-a6e0-ada382faac64-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-lv2n9\" (UID: \"c6d2302e-3deb-44cb-a6e0-ada382faac64\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-lv2n9"
Sep 11 17:28:22.092836 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:28:22.092775    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c6d2302e-3deb-44cb-a6e0-ada382faac64-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-lv2n9\" (UID: \"c6d2302e-3deb-44cb-a6e0-ada382faac64\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-lv2n9"
Sep 11 17:28:22.094608 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:28:22.094590    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c6d2302e-3deb-44cb-a6e0-ada382faac64-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-lv2n9\" (UID: \"c6d2302e-3deb-44cb-a6e0-ada382faac64\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-lv2n9"
Sep 11 17:28:22.094785 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:28:22.094769    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c6d2302e-3deb-44cb-a6e0-ada382faac64-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-lv2n9\" (UID: \"c6d2302e-3deb-44cb-a6e0-ada382faac64\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-lv2n9"
Sep 11 17:28:22.100173 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:28:22.100154    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jggrz\" (UniqueName: \"kubernetes.io/projected/c6d2302e-3deb-44cb-a6e0-ada382faac64-kube-api-access-jggrz\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-lv2n9\" (UID: \"c6d2302e-3deb-44cb-a6e0-ada382faac64\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-lv2n9"
Sep 11 17:28:22.217350 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:28:22.217260    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-lv2n9"
Sep 11 17:28:22.497959 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:28:22.497876    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-lv2n9"]
Sep 11 17:28:22.501175 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:28:22.501142    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6d2302e_3deb_44cb_a6e0_ada382faac64.slice/crio-31020b8b573d8687c2b0202cba69bda1b7d793611b72563539f922cccf9b6b3c WatchSource:0}: Error finding container 31020b8b573d8687c2b0202cba69bda1b7d793611b72563539f922cccf9b6b3c: Status 404 returned error can't find the container with id 31020b8b573d8687c2b0202cba69bda1b7d793611b72563539f922cccf9b6b3c
Sep 11 17:28:23.009990 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:28:23.009940    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-lv2n9" event={"ID":"c6d2302e-3deb-44cb-a6e0-ada382faac64","Type":"ContainerStarted","Data":"31020b8b573d8687c2b0202cba69bda1b7d793611b72563539f922cccf9b6b3c"}
Sep 11 17:28:24.016782 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:28:24.016747    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-lv2n9" event={"ID":"c6d2302e-3deb-44cb-a6e0-ada382faac64","Type":"ContainerStarted","Data":"a05d9902de20a1dd0195f45ccd9ed99b073e370017d3e44f2c39e911e9e68f05"}
Sep 11 17:28:24.038333 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:28:24.038256    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-lv2n9" podStartSLOduration=2.005103133 podStartE2EDuration="3.0382349s" totalImagesPullingTime="1.033131767s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:28:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:28:24.037664317 +0000 UTC m=+598.558955876" watchObservedRunningTime="2026-09-11 17:28:24.0382349 +0000 UTC m=+598.559526421"
Sep 11 17:28:25.902764 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:28:25.902732    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-hg7p8_eb5f3be1-2e36-40ba-b3a0-3bec413056e8/console-operator/1.log"
Sep 11 17:28:25.904405 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:28:25.904384    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-hg7p8_eb5f3be1-2e36-40ba-b3a0-3bec413056e8/console-operator/1.log"
Sep 11 17:28:25.906793 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:28:25.906773    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6k9tr_0620c0a0-8070-4307-86d9-c69a76ea14a9/ovn-acl-logging/0.log"
Sep 11 17:28:25.908046 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:28:25.908029    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6k9tr_0620c0a0-8070-4307-86d9-c69a76ea14a9/ovn-acl-logging/0.log"
Sep 11 17:28:32.217425 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:28:32.217378    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-lv2n9"
Sep 11 17:28:32.219954 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:28:32.217459    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-lv2n9"
Sep 11 17:28:32.229855 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:28:32.229830    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-lv2n9"
Sep 11 17:28:33.077600 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:28:33.077566    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-lv2n9"
Sep 11 17:28:34.811097 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:28:34.811059    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:28:43.823969 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:28:43.823937    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:28:45.927728 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:28:45.927693    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:28:52.461898 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:28:52.461856    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:28:58.546408 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:28:58.546359    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:29:49.321878 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:29:49.321844    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:30:00.185581 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:30:00.185542    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29819130-txfjr"]
Sep 11 17:30:00.188852 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:30:00.188834    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29819130-txfjr"
Sep 11 17:30:00.192100 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:30:00.192080    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9n8gf\""
Sep 11 17:30:00.198929 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:30:00.198910    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29819130-txfjr"
Sep 11 17:30:00.207230 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:30:00.207207    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29819130-txfjr"]
Sep 11 17:30:00.472502 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:30:00.472477    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29819130-txfjr"]
Sep 11 17:30:00.474293 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:30:00.474260    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbee1e8bc_89ab_492d_a578_be2125b7d47a.slice/crio-bb448bef18a34df79f79b9e268dbfe069fdd772f2ecba3c117076774a3e85d45 WatchSource:0}: Error finding container bb448bef18a34df79f79b9e268dbfe069fdd772f2ecba3c117076774a3e85d45: Status 404 returned error can't find the container with id bb448bef18a34df79f79b9e268dbfe069fdd772f2ecba3c117076774a3e85d45
Sep 11 17:30:00.476057 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:30:00.476041    2552 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 17:30:00.559305 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:30:00.559271    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29819130-txfjr" event={"ID":"bee1e8bc-89ab-492d-a578-be2125b7d47a","Type":"ContainerStarted","Data":"bb448bef18a34df79f79b9e268dbfe069fdd772f2ecba3c117076774a3e85d45"}
Sep 11 17:30:04.583667 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:30:04.583631    2552 generic.go:356] "Generic (PLEG): container finished" podID="bee1e8bc-89ab-492d-a578-be2125b7d47a" containerID="b2a07fa5f98e540db37ba4b94614b2eb47de45a4aca8d3dbeaf11f953823902c" exitCode=0
Sep 11 17:30:04.584069 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:30:04.583725    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29819130-txfjr" event={"ID":"bee1e8bc-89ab-492d-a578-be2125b7d47a","Type":"ContainerDied","Data":"b2a07fa5f98e540db37ba4b94614b2eb47de45a4aca8d3dbeaf11f953823902c"}
Sep 11 17:30:05.722906 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:30:05.722879    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29819130-txfjr"
Sep 11 17:30:06.595775 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:30:06.595741    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29819130-txfjr"
Sep 11 17:30:06.595989 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:30:06.595779    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29819130-txfjr" event={"ID":"bee1e8bc-89ab-492d-a578-be2125b7d47a","Type":"ContainerDied","Data":"bb448bef18a34df79f79b9e268dbfe069fdd772f2ecba3c117076774a3e85d45"}
Sep 11 17:30:06.595989 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:30:06.595822    2552 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bb448bef18a34df79f79b9e268dbfe069fdd772f2ecba3c117076774a3e85d45"
Sep 11 17:30:37.521344 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:30:37.521254    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:30:47.519789 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:30:47.519753    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5cdcddb749-szqt9"]
Sep 11 17:30:47.520267 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:30:47.520185    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bee1e8bc-89ab-492d-a578-be2125b7d47a" containerName="cleanup"
Sep 11 17:30:47.520267 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:30:47.520199    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="bee1e8bc-89ab-492d-a578-be2125b7d47a" containerName="cleanup"
Sep 11 17:30:47.520348 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:30:47.520279    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="bee1e8bc-89ab-492d-a578-be2125b7d47a" containerName="cleanup"
Sep 11 17:30:47.525933 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:30:47.525910    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5cdcddb749-szqt9"
Sep 11 17:30:47.532381 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:30:47.532351    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5cdcddb749-szqt9"]
Sep 11 17:30:47.682700 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:30:47.682662    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/48df3a88-833b-480f-9e21-def162c86d75-tls-cert\") pod \"authorino-5cdcddb749-szqt9\" (UID: \"48df3a88-833b-480f-9e21-def162c86d75\") " pod="kuadrant-system/authorino-5cdcddb749-szqt9"
Sep 11 17:30:47.682867 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:30:47.682754    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7ztn5\" (UniqueName: \"kubernetes.io/projected/48df3a88-833b-480f-9e21-def162c86d75-kube-api-access-7ztn5\") pod \"authorino-5cdcddb749-szqt9\" (UID: \"48df3a88-833b-480f-9e21-def162c86d75\") " pod="kuadrant-system/authorino-5cdcddb749-szqt9"
Sep 11 17:30:47.783664 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:30:47.783568    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7ztn5\" (UniqueName: \"kubernetes.io/projected/48df3a88-833b-480f-9e21-def162c86d75-kube-api-access-7ztn5\") pod \"authorino-5cdcddb749-szqt9\" (UID: \"48df3a88-833b-480f-9e21-def162c86d75\") " pod="kuadrant-system/authorino-5cdcddb749-szqt9"
Sep 11 17:30:47.783664 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:30:47.783631    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/48df3a88-833b-480f-9e21-def162c86d75-tls-cert\") pod \"authorino-5cdcddb749-szqt9\" (UID: \"48df3a88-833b-480f-9e21-def162c86d75\") " pod="kuadrant-system/authorino-5cdcddb749-szqt9"
Sep 11 17:30:47.786190 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:30:47.786164    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/48df3a88-833b-480f-9e21-def162c86d75-tls-cert\") pod \"authorino-5cdcddb749-szqt9\" (UID: \"48df3a88-833b-480f-9e21-def162c86d75\") " pod="kuadrant-system/authorino-5cdcddb749-szqt9"
Sep 11 17:30:47.790797 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:30:47.790775    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7ztn5\" (UniqueName: \"kubernetes.io/projected/48df3a88-833b-480f-9e21-def162c86d75-kube-api-access-7ztn5\") pod \"authorino-5cdcddb749-szqt9\" (UID: \"48df3a88-833b-480f-9e21-def162c86d75\") " pod="kuadrant-system/authorino-5cdcddb749-szqt9"
Sep 11 17:30:47.836245 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:30:47.836204    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5cdcddb749-szqt9"
Sep 11 17:30:48.106705 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:30:48.106676    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5cdcddb749-szqt9"]
Sep 11 17:30:48.109335 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:30:48.109304    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod48df3a88_833b_480f_9e21_def162c86d75.slice/crio-95b93f54e302c8e06fc1f26db302782b0c117eeb2e6d371688d7aff28f6bf7c5 WatchSource:0}: Error finding container 95b93f54e302c8e06fc1f26db302782b0c117eeb2e6d371688d7aff28f6bf7c5: Status 404 returned error can't find the container with id 95b93f54e302c8e06fc1f26db302782b0c117eeb2e6d371688d7aff28f6bf7c5
Sep 11 17:30:48.834254 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:30:48.834211    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5cdcddb749-szqt9" event={"ID":"48df3a88-833b-480f-9e21-def162c86d75","Type":"ContainerStarted","Data":"95b93f54e302c8e06fc1f26db302782b0c117eeb2e6d371688d7aff28f6bf7c5"}
Sep 11 17:30:50.849002 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:30:50.848955    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5cdcddb749-szqt9" event={"ID":"48df3a88-833b-480f-9e21-def162c86d75","Type":"ContainerStarted","Data":"98ca1b0d59086527974bd59e229bbc08f36cf83a65ed93888c3004b60fa9f704"}
Sep 11 17:30:50.869800 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:30:50.869751    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5cdcddb749-szqt9" podStartSLOduration=1.80017235 podStartE2EDuration="3.869735568s" totalImagesPullingTime="2.069563218s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:30:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:30:50.866563358 +0000 UTC m=+745.387854885" watchObservedRunningTime="2026-09-11 17:30:50.869735568 +0000 UTC m=+745.391027089"
Sep 11 17:30:50.899133 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:30:50.897145    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-66f9b6cfdd-mhkv5"]
Sep 11 17:30:50.899319 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:30:50.899290    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-66f9b6cfdd-mhkv5" podUID="1d5ccc6b-56b9-4a65-967c-46fa32e262cf" containerName="authorino" containerID="cri-o://98184740890ac3e140ab05575c9175cfc2c05d182b8b6a81cba4322723426d7f" gracePeriod=30
Sep 11 17:30:51.157020 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:30:51.156993    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-66f9b6cfdd-mhkv5"
Sep 11 17:30:51.317043 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:30:51.317003    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fjm66\" (UniqueName: \"kubernetes.io/projected/1d5ccc6b-56b9-4a65-967c-46fa32e262cf-kube-api-access-fjm66\") pod \"1d5ccc6b-56b9-4a65-967c-46fa32e262cf\" (UID: \"1d5ccc6b-56b9-4a65-967c-46fa32e262cf\") "
Sep 11 17:30:51.317274 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:30:51.317091    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1d5ccc6b-56b9-4a65-967c-46fa32e262cf-tls-cert\") pod \"1d5ccc6b-56b9-4a65-967c-46fa32e262cf\" (UID: \"1d5ccc6b-56b9-4a65-967c-46fa32e262cf\") "
Sep 11 17:30:51.319465 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:30:51.319433    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1d5ccc6b-56b9-4a65-967c-46fa32e262cf-kube-api-access-fjm66" (OuterVolumeSpecName: "kube-api-access-fjm66") pod "1d5ccc6b-56b9-4a65-967c-46fa32e262cf" (UID: "1d5ccc6b-56b9-4a65-967c-46fa32e262cf"). InnerVolumeSpecName "kube-api-access-fjm66". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:30:51.329035 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:30:51.328999    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1d5ccc6b-56b9-4a65-967c-46fa32e262cf-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "1d5ccc6b-56b9-4a65-967c-46fa32e262cf" (UID: "1d5ccc6b-56b9-4a65-967c-46fa32e262cf"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 17:30:51.418358 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:30:51.418276    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fjm66\" (UniqueName: \"kubernetes.io/projected/1d5ccc6b-56b9-4a65-967c-46fa32e262cf-kube-api-access-fjm66\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:30:51.418358 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:30:51.418310    2552 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1d5ccc6b-56b9-4a65-967c-46fa32e262cf-tls-cert\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:30:51.436213 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:30:51.436170    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:30:51.528630 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:30:51.528581    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:30:51.855367 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:30:51.855329    2552 generic.go:356] "Generic (PLEG): container finished" podID="1d5ccc6b-56b9-4a65-967c-46fa32e262cf" containerID="98184740890ac3e140ab05575c9175cfc2c05d182b8b6a81cba4322723426d7f" exitCode=0
Sep 11 17:30:51.855367 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:30:51.855369    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-66f9b6cfdd-mhkv5"
Sep 11 17:30:51.855838 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:30:51.855410    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-66f9b6cfdd-mhkv5" event={"ID":"1d5ccc6b-56b9-4a65-967c-46fa32e262cf","Type":"ContainerDied","Data":"98184740890ac3e140ab05575c9175cfc2c05d182b8b6a81cba4322723426d7f"}
Sep 11 17:30:51.855838 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:30:51.855453    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-66f9b6cfdd-mhkv5" event={"ID":"1d5ccc6b-56b9-4a65-967c-46fa32e262cf","Type":"ContainerDied","Data":"14da6d866ce53fbc02e26d4bad1175a3174bbbea801cc7a4ebe5c670cf4c8a3b"}
Sep 11 17:30:51.855838 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:30:51.855475    2552 scope.go:117] "RemoveContainer" containerID="98184740890ac3e140ab05575c9175cfc2c05d182b8b6a81cba4322723426d7f"
Sep 11 17:30:51.866357 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:30:51.866340    2552 scope.go:117] "RemoveContainer" containerID="98184740890ac3e140ab05575c9175cfc2c05d182b8b6a81cba4322723426d7f"
Sep 11 17:30:51.866665 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:30:51.866645    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"98184740890ac3e140ab05575c9175cfc2c05d182b8b6a81cba4322723426d7f\": container with ID starting with 98184740890ac3e140ab05575c9175cfc2c05d182b8b6a81cba4322723426d7f not found: ID does not exist" containerID="98184740890ac3e140ab05575c9175cfc2c05d182b8b6a81cba4322723426d7f"
Sep 11 17:30:51.866746 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:30:51.866676    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"98184740890ac3e140ab05575c9175cfc2c05d182b8b6a81cba4322723426d7f"} err="failed to get container status \"98184740890ac3e140ab05575c9175cfc2c05d182b8b6a81cba4322723426d7f\": rpc error: code = NotFound desc = could not find container \"98184740890ac3e140ab05575c9175cfc2c05d182b8b6a81cba4322723426d7f\": container with ID starting with 98184740890ac3e140ab05575c9175cfc2c05d182b8b6a81cba4322723426d7f not found: ID does not exist"
Sep 11 17:30:51.878565 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:30:51.878537    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-66f9b6cfdd-mhkv5"]
Sep 11 17:30:51.883815 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:30:51.883789    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-66f9b6cfdd-mhkv5"]
Sep 11 17:30:51.950529 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:30:51.950494    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1d5ccc6b-56b9-4a65-967c-46fa32e262cf" path="/var/lib/kubelet/pods/1d5ccc6b-56b9-4a65-967c-46fa32e262cf/volumes"
Sep 11 17:30:52.547785 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:30:52.547747    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:30:52.645412 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:30:52.645375    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:31:09.015838 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:09.015799    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:31:09.334482 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:09.334400    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-08c6b2-openshift-default-5d4477884-jms7z"]
Sep 11 17:31:09.334801 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:09.334789    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1d5ccc6b-56b9-4a65-967c-46fa32e262cf" containerName="authorino"
Sep 11 17:31:09.334849 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:09.334803    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="1d5ccc6b-56b9-4a65-967c-46fa32e262cf" containerName="authorino"
Sep 11 17:31:09.334889 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:09.334864    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="1d5ccc6b-56b9-4a65-967c-46fa32e262cf" containerName="authorino"
Sep 11 17:31:09.339677 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:09.339559    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-08c6b2-openshift-default-5d4477884-jms7z"
Sep 11 17:31:09.342577 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:09.342559    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-08c6b2-openshift-default-dockercfg-4j2h2\""
Sep 11 17:31:09.362819 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:09.362792    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-08c6b2-openshift-default-5d4477884-jms7z"]
Sep 11 17:31:09.366519 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:09.366493    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d886651d-540e-47c7-8df0-7234dd3adca1-istio-token\") pod \"e2e-inf-a-08c6b2-openshift-default-5d4477884-jms7z\" (UID: \"d886651d-540e-47c7-8df0-7234dd3adca1\") " pod="openshift-ingress/e2e-inf-a-08c6b2-openshift-default-5d4477884-jms7z"
Sep 11 17:31:09.366636 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:09.366532    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d886651d-540e-47c7-8df0-7234dd3adca1-istio-podinfo\") pod \"e2e-inf-a-08c6b2-openshift-default-5d4477884-jms7z\" (UID: \"d886651d-540e-47c7-8df0-7234dd3adca1\") " pod="openshift-ingress/e2e-inf-a-08c6b2-openshift-default-5d4477884-jms7z"
Sep 11 17:31:09.366636 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:09.366554    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d886651d-540e-47c7-8df0-7234dd3adca1-workload-socket\") pod \"e2e-inf-a-08c6b2-openshift-default-5d4477884-jms7z\" (UID: \"d886651d-540e-47c7-8df0-7234dd3adca1\") " pod="openshift-ingress/e2e-inf-a-08c6b2-openshift-default-5d4477884-jms7z"
Sep 11 17:31:09.366636 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:09.366623    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8z2gf\" (UniqueName: \"kubernetes.io/projected/d886651d-540e-47c7-8df0-7234dd3adca1-kube-api-access-8z2gf\") pod \"e2e-inf-a-08c6b2-openshift-default-5d4477884-jms7z\" (UID: \"d886651d-540e-47c7-8df0-7234dd3adca1\") " pod="openshift-ingress/e2e-inf-a-08c6b2-openshift-default-5d4477884-jms7z"
Sep 11 17:31:09.366773 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:09.366655    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d886651d-540e-47c7-8df0-7234dd3adca1-istio-envoy\") pod \"e2e-inf-a-08c6b2-openshift-default-5d4477884-jms7z\" (UID: \"d886651d-540e-47c7-8df0-7234dd3adca1\") " pod="openshift-ingress/e2e-inf-a-08c6b2-openshift-default-5d4477884-jms7z"
Sep 11 17:31:09.366773 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:09.366673    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d886651d-540e-47c7-8df0-7234dd3adca1-credential-socket\") pod \"e2e-inf-a-08c6b2-openshift-default-5d4477884-jms7z\" (UID: \"d886651d-540e-47c7-8df0-7234dd3adca1\") " pod="openshift-ingress/e2e-inf-a-08c6b2-openshift-default-5d4477884-jms7z"
Sep 11 17:31:09.366773 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:09.366699    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d886651d-540e-47c7-8df0-7234dd3adca1-workload-certs\") pod \"e2e-inf-a-08c6b2-openshift-default-5d4477884-jms7z\" (UID: \"d886651d-540e-47c7-8df0-7234dd3adca1\") " pod="openshift-ingress/e2e-inf-a-08c6b2-openshift-default-5d4477884-jms7z"
Sep 11 17:31:09.366773 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:09.366758    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d886651d-540e-47c7-8df0-7234dd3adca1-istiod-ca-cert\") pod \"e2e-inf-a-08c6b2-openshift-default-5d4477884-jms7z\" (UID: \"d886651d-540e-47c7-8df0-7234dd3adca1\") " pod="openshift-ingress/e2e-inf-a-08c6b2-openshift-default-5d4477884-jms7z"
Sep 11 17:31:09.366986 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:09.366833    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d886651d-540e-47c7-8df0-7234dd3adca1-istio-data\") pod \"e2e-inf-a-08c6b2-openshift-default-5d4477884-jms7z\" (UID: \"d886651d-540e-47c7-8df0-7234dd3adca1\") " pod="openshift-ingress/e2e-inf-a-08c6b2-openshift-default-5d4477884-jms7z"
Sep 11 17:31:09.455371 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:09.455336    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-144bfa-openshift-default-5dc996f7dd-ws7l7"]
Sep 11 17:31:09.459102 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:09.459080    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-144bfa-openshift-default-5dc996f7dd-ws7l7"
Sep 11 17:31:09.462456 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:09.462436    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w4-144bfa-openshift-default-dockercfg-d4vv8\""
Sep 11 17:31:09.467567 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:09.467547    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d886651d-540e-47c7-8df0-7234dd3adca1-workload-certs\") pod \"e2e-inf-a-08c6b2-openshift-default-5d4477884-jms7z\" (UID: \"d886651d-540e-47c7-8df0-7234dd3adca1\") " pod="openshift-ingress/e2e-inf-a-08c6b2-openshift-default-5d4477884-jms7z"
Sep 11 17:31:09.467665 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:09.467572    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d886651d-540e-47c7-8df0-7234dd3adca1-istiod-ca-cert\") pod \"e2e-inf-a-08c6b2-openshift-default-5d4477884-jms7z\" (UID: \"d886651d-540e-47c7-8df0-7234dd3adca1\") " pod="openshift-ingress/e2e-inf-a-08c6b2-openshift-default-5d4477884-jms7z"
Sep 11 17:31:09.467665 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:09.467605    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ae170805-c6a4-469f-b280-8e7629749557-workload-certs\") pod \"e2e-worker-w4-144bfa-openshift-default-5dc996f7dd-ws7l7\" (UID: \"ae170805-c6a4-469f-b280-8e7629749557\") " pod="openshift-ingress/e2e-worker-w4-144bfa-openshift-default-5dc996f7dd-ws7l7"
Sep 11 17:31:09.467665 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:09.467629    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d886651d-540e-47c7-8df0-7234dd3adca1-istio-data\") pod \"e2e-inf-a-08c6b2-openshift-default-5d4477884-jms7z\" (UID: \"d886651d-540e-47c7-8df0-7234dd3adca1\") " pod="openshift-ingress/e2e-inf-a-08c6b2-openshift-default-5d4477884-jms7z"
Sep 11 17:31:09.467828 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:09.467673    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ae170805-c6a4-469f-b280-8e7629749557-istio-podinfo\") pod \"e2e-worker-w4-144bfa-openshift-default-5dc996f7dd-ws7l7\" (UID: \"ae170805-c6a4-469f-b280-8e7629749557\") " pod="openshift-ingress/e2e-worker-w4-144bfa-openshift-default-5dc996f7dd-ws7l7"
Sep 11 17:31:09.467828 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:09.467774    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d886651d-540e-47c7-8df0-7234dd3adca1-istio-token\") pod \"e2e-inf-a-08c6b2-openshift-default-5d4477884-jms7z\" (UID: \"d886651d-540e-47c7-8df0-7234dd3adca1\") " pod="openshift-ingress/e2e-inf-a-08c6b2-openshift-default-5d4477884-jms7z"
Sep 11 17:31:09.467828 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:09.467801    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae170805-c6a4-469f-b280-8e7629749557-workload-socket\") pod \"e2e-worker-w4-144bfa-openshift-default-5dc996f7dd-ws7l7\" (UID: \"ae170805-c6a4-469f-b280-8e7629749557\") " pod="openshift-ingress/e2e-worker-w4-144bfa-openshift-default-5dc996f7dd-ws7l7"
Sep 11 17:31:09.468008 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:09.467829    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae170805-c6a4-469f-b280-8e7629749557-credential-socket\") pod \"e2e-worker-w4-144bfa-openshift-default-5dc996f7dd-ws7l7\" (UID: \"ae170805-c6a4-469f-b280-8e7629749557\") " pod="openshift-ingress/e2e-worker-w4-144bfa-openshift-default-5dc996f7dd-ws7l7"
Sep 11 17:31:09.468008 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:09.467894    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d886651d-540e-47c7-8df0-7234dd3adca1-istio-podinfo\") pod \"e2e-inf-a-08c6b2-openshift-default-5d4477884-jms7z\" (UID: \"d886651d-540e-47c7-8df0-7234dd3adca1\") " pod="openshift-ingress/e2e-inf-a-08c6b2-openshift-default-5d4477884-jms7z"
Sep 11 17:31:09.468008 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:09.467959    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d886651d-540e-47c7-8df0-7234dd3adca1-istio-data\") pod \"e2e-inf-a-08c6b2-openshift-default-5d4477884-jms7z\" (UID: \"d886651d-540e-47c7-8df0-7234dd3adca1\") " pod="openshift-ingress/e2e-inf-a-08c6b2-openshift-default-5d4477884-jms7z"
Sep 11 17:31:09.468210 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:09.468022    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d886651d-540e-47c7-8df0-7234dd3adca1-workload-certs\") pod \"e2e-inf-a-08c6b2-openshift-default-5d4477884-jms7z\" (UID: \"d886651d-540e-47c7-8df0-7234dd3adca1\") " pod="openshift-ingress/e2e-inf-a-08c6b2-openshift-default-5d4477884-jms7z"
Sep 11 17:31:09.468210 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:09.468056    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d886651d-540e-47c7-8df0-7234dd3adca1-workload-socket\") pod \"e2e-inf-a-08c6b2-openshift-default-5d4477884-jms7z\" (UID: \"d886651d-540e-47c7-8df0-7234dd3adca1\") " pod="openshift-ingress/e2e-inf-a-08c6b2-openshift-default-5d4477884-jms7z"
Sep 11 17:31:09.468210 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:09.468103    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ae170805-c6a4-469f-b280-8e7629749557-istio-token\") pod \"e2e-worker-w4-144bfa-openshift-default-5dc996f7dd-ws7l7\" (UID: \"ae170805-c6a4-469f-b280-8e7629749557\") " pod="openshift-ingress/e2e-worker-w4-144bfa-openshift-default-5dc996f7dd-ws7l7"
Sep 11 17:31:09.468210 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:09.468160    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ae170805-c6a4-469f-b280-8e7629749557-istio-envoy\") pod \"e2e-worker-w4-144bfa-openshift-default-5dc996f7dd-ws7l7\" (UID: \"ae170805-c6a4-469f-b280-8e7629749557\") " pod="openshift-ingress/e2e-worker-w4-144bfa-openshift-default-5dc996f7dd-ws7l7"
Sep 11 17:31:09.468398 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:09.468218    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8z2gf\" (UniqueName: \"kubernetes.io/projected/d886651d-540e-47c7-8df0-7234dd3adca1-kube-api-access-8z2gf\") pod \"e2e-inf-a-08c6b2-openshift-default-5d4477884-jms7z\" (UID: \"d886651d-540e-47c7-8df0-7234dd3adca1\") " pod="openshift-ingress/e2e-inf-a-08c6b2-openshift-default-5d4477884-jms7z"
Sep 11 17:31:09.468398 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:09.468253    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rkzcg\" (UniqueName: \"kubernetes.io/projected/ae170805-c6a4-469f-b280-8e7629749557-kube-api-access-rkzcg\") pod \"e2e-worker-w4-144bfa-openshift-default-5dc996f7dd-ws7l7\" (UID: \"ae170805-c6a4-469f-b280-8e7629749557\") " pod="openshift-ingress/e2e-worker-w4-144bfa-openshift-default-5dc996f7dd-ws7l7"
Sep 11 17:31:09.468398 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:09.468293    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d886651d-540e-47c7-8df0-7234dd3adca1-istiod-ca-cert\") pod \"e2e-inf-a-08c6b2-openshift-default-5d4477884-jms7z\" (UID: \"d886651d-540e-47c7-8df0-7234dd3adca1\") " pod="openshift-ingress/e2e-inf-a-08c6b2-openshift-default-5d4477884-jms7z"
Sep 11 17:31:09.468398 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:09.468296    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ae170805-c6a4-469f-b280-8e7629749557-istio-data\") pod \"e2e-worker-w4-144bfa-openshift-default-5dc996f7dd-ws7l7\" (UID: \"ae170805-c6a4-469f-b280-8e7629749557\") " pod="openshift-ingress/e2e-worker-w4-144bfa-openshift-default-5dc996f7dd-ws7l7"
Sep 11 17:31:09.468398 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:09.468337    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ae170805-c6a4-469f-b280-8e7629749557-istiod-ca-cert\") pod \"e2e-worker-w4-144bfa-openshift-default-5dc996f7dd-ws7l7\" (UID: \"ae170805-c6a4-469f-b280-8e7629749557\") " pod="openshift-ingress/e2e-worker-w4-144bfa-openshift-default-5dc996f7dd-ws7l7"
Sep 11 17:31:09.468398 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:09.468370    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d886651d-540e-47c7-8df0-7234dd3adca1-istio-envoy\") pod \"e2e-inf-a-08c6b2-openshift-default-5d4477884-jms7z\" (UID: \"d886651d-540e-47c7-8df0-7234dd3adca1\") " pod="openshift-ingress/e2e-inf-a-08c6b2-openshift-default-5d4477884-jms7z"
Sep 11 17:31:09.468398 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:09.468398    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d886651d-540e-47c7-8df0-7234dd3adca1-credential-socket\") pod \"e2e-inf-a-08c6b2-openshift-default-5d4477884-jms7z\" (UID: \"d886651d-540e-47c7-8df0-7234dd3adca1\") " pod="openshift-ingress/e2e-inf-a-08c6b2-openshift-default-5d4477884-jms7z"
Sep 11 17:31:09.468681 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:09.468445    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d886651d-540e-47c7-8df0-7234dd3adca1-workload-socket\") pod \"e2e-inf-a-08c6b2-openshift-default-5d4477884-jms7z\" (UID: \"d886651d-540e-47c7-8df0-7234dd3adca1\") " pod="openshift-ingress/e2e-inf-a-08c6b2-openshift-default-5d4477884-jms7z"
Sep 11 17:31:09.468751 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:09.468686    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d886651d-540e-47c7-8df0-7234dd3adca1-credential-socket\") pod \"e2e-inf-a-08c6b2-openshift-default-5d4477884-jms7z\" (UID: \"d886651d-540e-47c7-8df0-7234dd3adca1\") " pod="openshift-ingress/e2e-inf-a-08c6b2-openshift-default-5d4477884-jms7z"
Sep 11 17:31:09.470506 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:09.470485    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d886651d-540e-47c7-8df0-7234dd3adca1-istio-envoy\") pod \"e2e-inf-a-08c6b2-openshift-default-5d4477884-jms7z\" (UID: \"d886651d-540e-47c7-8df0-7234dd3adca1\") " pod="openshift-ingress/e2e-inf-a-08c6b2-openshift-default-5d4477884-jms7z"
Sep 11 17:31:09.470579 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:09.470527    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d886651d-540e-47c7-8df0-7234dd3adca1-istio-podinfo\") pod \"e2e-inf-a-08c6b2-openshift-default-5d4477884-jms7z\" (UID: \"d886651d-540e-47c7-8df0-7234dd3adca1\") " pod="openshift-ingress/e2e-inf-a-08c6b2-openshift-default-5d4477884-jms7z"
Sep 11 17:31:09.489380 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:09.489353    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d886651d-540e-47c7-8df0-7234dd3adca1-istio-token\") pod \"e2e-inf-a-08c6b2-openshift-default-5d4477884-jms7z\" (UID: \"d886651d-540e-47c7-8df0-7234dd3adca1\") " pod="openshift-ingress/e2e-inf-a-08c6b2-openshift-default-5d4477884-jms7z"
Sep 11 17:31:09.490096 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:09.490072    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8z2gf\" (UniqueName: \"kubernetes.io/projected/d886651d-540e-47c7-8df0-7234dd3adca1-kube-api-access-8z2gf\") pod \"e2e-inf-a-08c6b2-openshift-default-5d4477884-jms7z\" (UID: \"d886651d-540e-47c7-8df0-7234dd3adca1\") " pod="openshift-ingress/e2e-inf-a-08c6b2-openshift-default-5d4477884-jms7z"
Sep 11 17:31:09.490372 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:09.490355    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-144bfa-openshift-default-5dc996f7dd-ws7l7"]
Sep 11 17:31:09.569037 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:09.569005    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ae170805-c6a4-469f-b280-8e7629749557-workload-certs\") pod \"e2e-worker-w4-144bfa-openshift-default-5dc996f7dd-ws7l7\" (UID: \"ae170805-c6a4-469f-b280-8e7629749557\") " pod="openshift-ingress/e2e-worker-w4-144bfa-openshift-default-5dc996f7dd-ws7l7"
Sep 11 17:31:09.569259 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:09.569045    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ae170805-c6a4-469f-b280-8e7629749557-istio-podinfo\") pod \"e2e-worker-w4-144bfa-openshift-default-5dc996f7dd-ws7l7\" (UID: \"ae170805-c6a4-469f-b280-8e7629749557\") " pod="openshift-ingress/e2e-worker-w4-144bfa-openshift-default-5dc996f7dd-ws7l7"
Sep 11 17:31:09.569259 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:09.569093    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae170805-c6a4-469f-b280-8e7629749557-workload-socket\") pod \"e2e-worker-w4-144bfa-openshift-default-5dc996f7dd-ws7l7\" (UID: \"ae170805-c6a4-469f-b280-8e7629749557\") " pod="openshift-ingress/e2e-worker-w4-144bfa-openshift-default-5dc996f7dd-ws7l7"
Sep 11 17:31:09.569259 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:09.569131    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae170805-c6a4-469f-b280-8e7629749557-credential-socket\") pod \"e2e-worker-w4-144bfa-openshift-default-5dc996f7dd-ws7l7\" (UID: \"ae170805-c6a4-469f-b280-8e7629749557\") " pod="openshift-ingress/e2e-worker-w4-144bfa-openshift-default-5dc996f7dd-ws7l7"
Sep 11 17:31:09.569259 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:09.569189    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ae170805-c6a4-469f-b280-8e7629749557-istio-token\") pod \"e2e-worker-w4-144bfa-openshift-default-5dc996f7dd-ws7l7\" (UID: \"ae170805-c6a4-469f-b280-8e7629749557\") " pod="openshift-ingress/e2e-worker-w4-144bfa-openshift-default-5dc996f7dd-ws7l7"
Sep 11 17:31:09.569259 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:09.569221    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ae170805-c6a4-469f-b280-8e7629749557-istio-envoy\") pod \"e2e-worker-w4-144bfa-openshift-default-5dc996f7dd-ws7l7\" (UID: \"ae170805-c6a4-469f-b280-8e7629749557\") " pod="openshift-ingress/e2e-worker-w4-144bfa-openshift-default-5dc996f7dd-ws7l7"
Sep 11 17:31:09.569259 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:09.569250    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rkzcg\" (UniqueName: \"kubernetes.io/projected/ae170805-c6a4-469f-b280-8e7629749557-kube-api-access-rkzcg\") pod \"e2e-worker-w4-144bfa-openshift-default-5dc996f7dd-ws7l7\" (UID: \"ae170805-c6a4-469f-b280-8e7629749557\") " pod="openshift-ingress/e2e-worker-w4-144bfa-openshift-default-5dc996f7dd-ws7l7"
Sep 11 17:31:09.569589 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:09.569285    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ae170805-c6a4-469f-b280-8e7629749557-istio-data\") pod \"e2e-worker-w4-144bfa-openshift-default-5dc996f7dd-ws7l7\" (UID: \"ae170805-c6a4-469f-b280-8e7629749557\") " pod="openshift-ingress/e2e-worker-w4-144bfa-openshift-default-5dc996f7dd-ws7l7"
Sep 11 17:31:09.569589 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:09.569311    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ae170805-c6a4-469f-b280-8e7629749557-istiod-ca-cert\") pod \"e2e-worker-w4-144bfa-openshift-default-5dc996f7dd-ws7l7\" (UID: \"ae170805-c6a4-469f-b280-8e7629749557\") " pod="openshift-ingress/e2e-worker-w4-144bfa-openshift-default-5dc996f7dd-ws7l7"
Sep 11 17:31:09.569589 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:09.569466    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ae170805-c6a4-469f-b280-8e7629749557-workload-certs\") pod \"e2e-worker-w4-144bfa-openshift-default-5dc996f7dd-ws7l7\" (UID: \"ae170805-c6a4-469f-b280-8e7629749557\") " pod="openshift-ingress/e2e-worker-w4-144bfa-openshift-default-5dc996f7dd-ws7l7"
Sep 11 17:31:09.569589 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:09.569481    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae170805-c6a4-469f-b280-8e7629749557-workload-socket\") pod \"e2e-worker-w4-144bfa-openshift-default-5dc996f7dd-ws7l7\" (UID: \"ae170805-c6a4-469f-b280-8e7629749557\") " pod="openshift-ingress/e2e-worker-w4-144bfa-openshift-default-5dc996f7dd-ws7l7"
Sep 11 17:31:09.569812 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:09.569593    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae170805-c6a4-469f-b280-8e7629749557-credential-socket\") pod \"e2e-worker-w4-144bfa-openshift-default-5dc996f7dd-ws7l7\" (UID: \"ae170805-c6a4-469f-b280-8e7629749557\") " pod="openshift-ingress/e2e-worker-w4-144bfa-openshift-default-5dc996f7dd-ws7l7"
Sep 11 17:31:09.569812 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:09.569748    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ae170805-c6a4-469f-b280-8e7629749557-istio-data\") pod \"e2e-worker-w4-144bfa-openshift-default-5dc996f7dd-ws7l7\" (UID: \"ae170805-c6a4-469f-b280-8e7629749557\") " pod="openshift-ingress/e2e-worker-w4-144bfa-openshift-default-5dc996f7dd-ws7l7"
Sep 11 17:31:09.570037 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:09.570016    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ae170805-c6a4-469f-b280-8e7629749557-istiod-ca-cert\") pod \"e2e-worker-w4-144bfa-openshift-default-5dc996f7dd-ws7l7\" (UID: \"ae170805-c6a4-469f-b280-8e7629749557\") " pod="openshift-ingress/e2e-worker-w4-144bfa-openshift-default-5dc996f7dd-ws7l7"
Sep 11 17:31:09.571712 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:09.571691    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ae170805-c6a4-469f-b280-8e7629749557-istio-podinfo\") pod \"e2e-worker-w4-144bfa-openshift-default-5dc996f7dd-ws7l7\" (UID: \"ae170805-c6a4-469f-b280-8e7629749557\") " pod="openshift-ingress/e2e-worker-w4-144bfa-openshift-default-5dc996f7dd-ws7l7"
Sep 11 17:31:09.571920 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:09.571896    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ae170805-c6a4-469f-b280-8e7629749557-istio-envoy\") pod \"e2e-worker-w4-144bfa-openshift-default-5dc996f7dd-ws7l7\" (UID: \"ae170805-c6a4-469f-b280-8e7629749557\") " pod="openshift-ingress/e2e-worker-w4-144bfa-openshift-default-5dc996f7dd-ws7l7"
Sep 11 17:31:09.587542 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:09.587487    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ae170805-c6a4-469f-b280-8e7629749557-istio-token\") pod \"e2e-worker-w4-144bfa-openshift-default-5dc996f7dd-ws7l7\" (UID: \"ae170805-c6a4-469f-b280-8e7629749557\") " pod="openshift-ingress/e2e-worker-w4-144bfa-openshift-default-5dc996f7dd-ws7l7"
Sep 11 17:31:09.587723 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:09.587697    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rkzcg\" (UniqueName: \"kubernetes.io/projected/ae170805-c6a4-469f-b280-8e7629749557-kube-api-access-rkzcg\") pod \"e2e-worker-w4-144bfa-openshift-default-5dc996f7dd-ws7l7\" (UID: \"ae170805-c6a4-469f-b280-8e7629749557\") " pod="openshift-ingress/e2e-worker-w4-144bfa-openshift-default-5dc996f7dd-ws7l7"
Sep 11 17:31:09.650207 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:09.650175    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-08c6b2-openshift-default-5d4477884-jms7z"
Sep 11 17:31:09.769407 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:09.769374    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-144bfa-openshift-default-5dc996f7dd-ws7l7"
Sep 11 17:31:09.932134 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:09.932092    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-08c6b2-openshift-default-5d4477884-jms7z"]
Sep 11 17:31:09.933673 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:31:09.933648    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd886651d_540e_47c7_8df0_7234dd3adca1.slice/crio-99a0685ca3c87f45c9d79f2b9309f2a7bab7f2fb7840b5eff7ea319e6590982e WatchSource:0}: Error finding container 99a0685ca3c87f45c9d79f2b9309f2a7bab7f2fb7840b5eff7ea319e6590982e: Status 404 returned error can't find the container with id 99a0685ca3c87f45c9d79f2b9309f2a7bab7f2fb7840b5eff7ea319e6590982e
Sep 11 17:31:09.935958 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:09.935924    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 17:31:09.936073 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:09.936004    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 17:31:09.936073 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:09.936049    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 17:31:09.957623 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:09.957596    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-08c6b2-openshift-default-5d4477884-jms7z" event={"ID":"d886651d-540e-47c7-8df0-7234dd3adca1","Type":"ContainerStarted","Data":"99a0685ca3c87f45c9d79f2b9309f2a7bab7f2fb7840b5eff7ea319e6590982e"}
Sep 11 17:31:10.048275 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:10.048192    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-144bfa-openshift-default-5dc996f7dd-ws7l7"]
Sep 11 17:31:10.051137 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:31:10.051096    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae170805_c6a4_469f_b280_8e7629749557.slice/crio-ac60dedbc966c4fe7d95c93ccbab657ff50f4cc0839423f40f4c072ee981130d WatchSource:0}: Error finding container ac60dedbc966c4fe7d95c93ccbab657ff50f4cc0839423f40f4c072ee981130d: Status 404 returned error can't find the container with id ac60dedbc966c4fe7d95c93ccbab657ff50f4cc0839423f40f4c072ee981130d
Sep 11 17:31:10.054173 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:10.054135    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 17:31:10.054272 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:10.054221    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 17:31:10.054272 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:10.054264    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 17:31:10.239433 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:10.239345    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:31:10.337347 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:10.337309    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:31:10.443703 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:10.443671    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:31:10.544655 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:10.544563    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:31:10.964197 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:10.964165    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-08c6b2-openshift-default-5d4477884-jms7z" event={"ID":"d886651d-540e-47c7-8df0-7234dd3adca1","Type":"ContainerStarted","Data":"7c5c9b36711108494c07db005ca918e97b0b49aeb777d1ff5ccce3bbcd24ec54"}
Sep 11 17:31:10.965888 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:10.965861    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-144bfa-openshift-default-5dc996f7dd-ws7l7" event={"ID":"ae170805-c6a4-469f-b280-8e7629749557","Type":"ContainerStarted","Data":"99222f6f532e8093632159f0abb4c3c2f4ebac614a270467e0e7e71cea3735c8"}
Sep 11 17:31:10.966052 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:10.965900    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-144bfa-openshift-default-5dc996f7dd-ws7l7" event={"ID":"ae170805-c6a4-469f-b280-8e7629749557","Type":"ContainerStarted","Data":"ac60dedbc966c4fe7d95c93ccbab657ff50f4cc0839423f40f4c072ee981130d"}
Sep 11 17:31:10.983813 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:10.983764    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-08c6b2-openshift-default-5d4477884-jms7z" podStartSLOduration=1.9837480730000001 podStartE2EDuration="1.983748073s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:31:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:31:10.983139794 +0000 UTC m=+765.504431312" watchObservedRunningTime="2026-09-11 17:31:10.983748073 +0000 UTC m=+765.505039595"
Sep 11 17:31:11.008515 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:11.008463    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-144bfa-openshift-default-5dc996f7dd-ws7l7" podStartSLOduration=2.00844384 podStartE2EDuration="2.00844384s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:31:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:31:11.006133702 +0000 UTC m=+765.527425214" watchObservedRunningTime="2026-09-11 17:31:11.00844384 +0000 UTC m=+765.529735361"
Sep 11 17:31:11.027145 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:11.027103    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7698-600b08"]
Sep 11 17:31:11.030869 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:11.030853    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7698-600b08"
Sep 11 17:31:11.033172 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:11.033142    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-6ptn5\""
Sep 11 17:31:11.039211 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:11.039179    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7698-600b08"]
Sep 11 17:31:11.083217 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:11.083183    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q94t4\" (UniqueName: \"kubernetes.io/projected/e29d4d86-fa04-43c5-94e4-84dc28a3d19d-kube-api-access-q94t4\") pod \"test-curl-7698-600b08\" (UID: \"e29d4d86-fa04-43c5-94e4-84dc28a3d19d\") " pod="opendatahub/test-curl-7698-600b08"
Sep 11 17:31:11.184340 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:11.184307    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q94t4\" (UniqueName: \"kubernetes.io/projected/e29d4d86-fa04-43c5-94e4-84dc28a3d19d-kube-api-access-q94t4\") pod \"test-curl-7698-600b08\" (UID: \"e29d4d86-fa04-43c5-94e4-84dc28a3d19d\") " pod="opendatahub/test-curl-7698-600b08"
Sep 11 17:31:11.191737 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:11.191706    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q94t4\" (UniqueName: \"kubernetes.io/projected/e29d4d86-fa04-43c5-94e4-84dc28a3d19d-kube-api-access-q94t4\") pod \"test-curl-7698-600b08\" (UID: \"e29d4d86-fa04-43c5-94e4-84dc28a3d19d\") " pod="opendatahub/test-curl-7698-600b08"
Sep 11 17:31:11.342869 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:11.342839    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7698-600b08"
Sep 11 17:31:11.616761 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:11.616732    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7698-600b08"]
Sep 11 17:31:11.618683 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:31:11.618652    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode29d4d86_fa04_43c5_94e4_84dc28a3d19d.slice/crio-ba1535804f36f9db7832cf9686829ebf44a38bdba61aa0f5f22115386e12a82a WatchSource:0}: Error finding container ba1535804f36f9db7832cf9686829ebf44a38bdba61aa0f5f22115386e12a82a: Status 404 returned error can't find the container with id ba1535804f36f9db7832cf9686829ebf44a38bdba61aa0f5f22115386e12a82a
Sep 11 17:31:11.650637 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:11.650605    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-08c6b2-openshift-default-5d4477884-jms7z"
Sep 11 17:31:11.655935 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:11.655911    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-08c6b2-openshift-default-5d4477884-jms7z"
Sep 11 17:31:11.770067 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:11.770027    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-144bfa-openshift-default-5dc996f7dd-ws7l7"
Sep 11 17:31:11.775887 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:11.775860    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-144bfa-openshift-default-5dc996f7dd-ws7l7"
Sep 11 17:31:11.972414 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:11.972322    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7698-600b08" event={"ID":"e29d4d86-fa04-43c5-94e4-84dc28a3d19d","Type":"ContainerStarted","Data":"ba1535804f36f9db7832cf9686829ebf44a38bdba61aa0f5f22115386e12a82a"}
Sep 11 17:31:11.972776 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:11.972750    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-08c6b2-openshift-default-5d4477884-jms7z"
Sep 11 17:31:11.972904 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:11.972808    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-144bfa-openshift-default-5dc996f7dd-ws7l7"
Sep 11 17:31:11.973544 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:11.973521    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-08c6b2-openshift-default-5d4477884-jms7z"
Sep 11 17:31:11.973657 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:11.973574    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-144bfa-openshift-default-5dc996f7dd-ws7l7"
Sep 11 17:31:11.988453 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:11.988432    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:31:12.979487 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:12.979445    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7698-600b08" event={"ID":"e29d4d86-fa04-43c5-94e4-84dc28a3d19d","Type":"ContainerStarted","Data":"4b406e16df32079d440f2e1c690ea0a172110868b86ee325ad1395040d6d07d6"}
Sep 11 17:31:12.990507 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:12.990453    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:31:13.136234 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:13.136200    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:31:13.234998 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:13.234909    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:31:13.347332 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:13.347298    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:31:13.441857 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:13.441817    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:31:13.579354 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:13.579258    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7698-600b08" podStartSLOduration=1.705733998 podStartE2EDuration="2.579238946s" totalImagesPullingTime="873.504948ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:31:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:31:13.009027186 +0000 UTC m=+767.530318707" watchObservedRunningTime="2026-09-11 17:31:13.579238946 +0000 UTC m=+768.100530467"
Sep 11 17:31:13.581393 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:13.581374    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7698-600b08"]
Sep 11 17:31:13.594043 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:13.594017    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7698-600b08"]
Sep 11 17:31:13.708038 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:13.708003    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7698-9b3132"]
Sep 11 17:31:13.708399 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:13.708386    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e29d4d86-fa04-43c5-94e4-84dc28a3d19d" containerName="test-curl-7698-600b08"
Sep 11 17:31:13.708460 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:13.708400    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="e29d4d86-fa04-43c5-94e4-84dc28a3d19d" containerName="test-curl-7698-600b08"
Sep 11 17:31:13.708500 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:13.708465    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="e29d4d86-fa04-43c5-94e4-84dc28a3d19d" containerName="test-curl-7698-600b08"
Sep 11 17:31:13.711709 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:13.711693    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7698-9b3132"
Sep 11 17:31:13.718442 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:13.718420    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7698-9b3132"]
Sep 11 17:31:13.808446 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:13.808409    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6cjxr\" (UniqueName: \"kubernetes.io/projected/05237a85-f901-4f22-bb61-56d4ef6c222a-kube-api-access-6cjxr\") pod \"test-curl-7698-9b3132\" (UID: \"05237a85-f901-4f22-bb61-56d4ef6c222a\") " pod="opendatahub/test-curl-7698-9b3132"
Sep 11 17:31:13.909705 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:13.909671    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6cjxr\" (UniqueName: \"kubernetes.io/projected/05237a85-f901-4f22-bb61-56d4ef6c222a-kube-api-access-6cjxr\") pod \"test-curl-7698-9b3132\" (UID: \"05237a85-f901-4f22-bb61-56d4ef6c222a\") " pod="opendatahub/test-curl-7698-9b3132"
Sep 11 17:31:13.920852 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:13.920821    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6cjxr\" (UniqueName: \"kubernetes.io/projected/05237a85-f901-4f22-bb61-56d4ef6c222a-kube-api-access-6cjxr\") pod \"test-curl-7698-9b3132\" (UID: \"05237a85-f901-4f22-bb61-56d4ef6c222a\") " pod="opendatahub/test-curl-7698-9b3132"
Sep 11 17:31:13.985541 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:31:13.985507    2552 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7698-600b08" container="test-curl-7698-600b08"
Sep 11 17:31:14.021206 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:14.021169    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7698-9b3132"
Sep 11 17:31:14.108317 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:14.108257    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:31:14.215225 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:14.215141    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:31:14.300814 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:14.300787    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7698-9b3132"]
Sep 11 17:31:14.302453 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:31:14.302425    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod05237a85_f901_4f22_bb61_56d4ef6c222a.slice/crio-05333b848abc0aae3be1599588fd1924ce57622fdcae24e6f1f2ae5938c97451 WatchSource:0}: Error finding container 05333b848abc0aae3be1599588fd1924ce57622fdcae24e6f1f2ae5938c97451: Status 404 returned error can't find the container with id 05333b848abc0aae3be1599588fd1924ce57622fdcae24e6f1f2ae5938c97451
Sep 11 17:31:14.486481 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:14.486392    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:31:14.673955 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:14.673918    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:31:14.776189 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:14.776080    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:31:14.993254 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:14.993210    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7698-9b3132" event={"ID":"05237a85-f901-4f22-bb61-56d4ef6c222a","Type":"ContainerStarted","Data":"76d115efd0512855f934344639b8fd2e72c6b3544a5158e11de630d1b2eff59f"}
Sep 11 17:31:14.993254 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:14.993260    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7698-9b3132" event={"ID":"05237a85-f901-4f22-bb61-56d4ef6c222a","Type":"ContainerStarted","Data":"05333b848abc0aae3be1599588fd1924ce57622fdcae24e6f1f2ae5938c97451"}
Sep 11 17:31:14.993734 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:14.993345    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7698-600b08" podUID="e29d4d86-fa04-43c5-94e4-84dc28a3d19d" containerName="test-curl-7698-600b08" containerID="cri-o://4b406e16df32079d440f2e1c690ea0a172110868b86ee325ad1395040d6d07d6" gracePeriod=2
Sep 11 17:31:15.010230 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:15.010183    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7698-9b3132" podStartSLOduration=2.010166509 podStartE2EDuration="2.010166509s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:31:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:31:15.007167445 +0000 UTC m=+769.528458966" watchObservedRunningTime="2026-09-11 17:31:15.010166509 +0000 UTC m=+769.531458030"
Sep 11 17:31:15.302591 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:15.302559    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:31:15.413005 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:15.412902    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:31:15.512719 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:15.512663    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:31:15.606381 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:15.606345    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7698-9b3132"]
Sep 11 17:31:15.610674 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:15.610641    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7698-9b3132"]
Sep 11 17:31:15.978051 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:15.977985    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:31:15.999155 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:31:15.999101    2552 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7698-9b3132" container="test-curl-7698-9b3132"
Sep 11 17:31:16.067829 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:16.067788    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:31:16.144840 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:16.144804    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:31:16.414814 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:16.414778    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-autores-49e398-openshift-default-66cbbd759d-nf4cx"]
Sep 11 17:31:16.415215 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:16.415193    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="05237a85-f901-4f22-bb61-56d4ef6c222a" containerName="test-curl-7698-9b3132"
Sep 11 17:31:16.415215 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:16.415214    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="05237a85-f901-4f22-bb61-56d4ef6c222a" containerName="test-curl-7698-9b3132"
Sep 11 17:31:16.415396 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:16.415336    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="05237a85-f901-4f22-bb61-56d4ef6c222a" containerName="test-curl-7698-9b3132"
Sep 11 17:31:16.418681 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:16.418660    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-49e398-openshift-default-66cbbd759d-nf4cx"
Sep 11 17:31:16.421046 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:16.421028    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-autores-49e398-openshift-default-dockercfg-29725\""
Sep 11 17:31:16.430087 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:16.430063    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-autores-49e398-openshift-default-66cbbd759d-nf4cx"]
Sep 11 17:31:16.452062 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:16.452033    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:31:16.540080 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:16.540048    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8d4763e0-c0ce-4b9a-8a05-ff4f718b96f0-istio-token\") pod \"e2e-autores-49e398-openshift-default-66cbbd759d-nf4cx\" (UID: \"8d4763e0-c0ce-4b9a-8a05-ff4f718b96f0\") " pod="openshift-ingress/e2e-autores-49e398-openshift-default-66cbbd759d-nf4cx"
Sep 11 17:31:16.540298 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:16.540153    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8d4763e0-c0ce-4b9a-8a05-ff4f718b96f0-credential-socket\") pod \"e2e-autores-49e398-openshift-default-66cbbd759d-nf4cx\" (UID: \"8d4763e0-c0ce-4b9a-8a05-ff4f718b96f0\") " pod="openshift-ingress/e2e-autores-49e398-openshift-default-66cbbd759d-nf4cx"
Sep 11 17:31:16.540298 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:16.540196    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8d4763e0-c0ce-4b9a-8a05-ff4f718b96f0-workload-certs\") pod \"e2e-autores-49e398-openshift-default-66cbbd759d-nf4cx\" (UID: \"8d4763e0-c0ce-4b9a-8a05-ff4f718b96f0\") " pod="openshift-ingress/e2e-autores-49e398-openshift-default-66cbbd759d-nf4cx"
Sep 11 17:31:16.540298 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:16.540269    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8d4763e0-c0ce-4b9a-8a05-ff4f718b96f0-istio-envoy\") pod \"e2e-autores-49e398-openshift-default-66cbbd759d-nf4cx\" (UID: \"8d4763e0-c0ce-4b9a-8a05-ff4f718b96f0\") " pod="openshift-ingress/e2e-autores-49e398-openshift-default-66cbbd759d-nf4cx"
Sep 11 17:31:16.540450 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:16.540382    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8d4763e0-c0ce-4b9a-8a05-ff4f718b96f0-istio-data\") pod \"e2e-autores-49e398-openshift-default-66cbbd759d-nf4cx\" (UID: \"8d4763e0-c0ce-4b9a-8a05-ff4f718b96f0\") " pod="openshift-ingress/e2e-autores-49e398-openshift-default-66cbbd759d-nf4cx"
Sep 11 17:31:16.540450 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:16.540425    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8d4763e0-c0ce-4b9a-8a05-ff4f718b96f0-istio-podinfo\") pod \"e2e-autores-49e398-openshift-default-66cbbd759d-nf4cx\" (UID: \"8d4763e0-c0ce-4b9a-8a05-ff4f718b96f0\") " pod="openshift-ingress/e2e-autores-49e398-openshift-default-66cbbd759d-nf4cx"
Sep 11 17:31:16.540552 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:16.540451    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8d4763e0-c0ce-4b9a-8a05-ff4f718b96f0-istiod-ca-cert\") pod \"e2e-autores-49e398-openshift-default-66cbbd759d-nf4cx\" (UID: \"8d4763e0-c0ce-4b9a-8a05-ff4f718b96f0\") " pod="openshift-ingress/e2e-autores-49e398-openshift-default-66cbbd759d-nf4cx"
Sep 11 17:31:16.540552 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:16.540523    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mp255\" (UniqueName: \"kubernetes.io/projected/8d4763e0-c0ce-4b9a-8a05-ff4f718b96f0-kube-api-access-mp255\") pod \"e2e-autores-49e398-openshift-default-66cbbd759d-nf4cx\" (UID: \"8d4763e0-c0ce-4b9a-8a05-ff4f718b96f0\") " pod="openshift-ingress/e2e-autores-49e398-openshift-default-66cbbd759d-nf4cx"
Sep 11 17:31:16.540653 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:16.540568    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8d4763e0-c0ce-4b9a-8a05-ff4f718b96f0-workload-socket\") pod \"e2e-autores-49e398-openshift-default-66cbbd759d-nf4cx\" (UID: \"8d4763e0-c0ce-4b9a-8a05-ff4f718b96f0\") " pod="openshift-ingress/e2e-autores-49e398-openshift-default-66cbbd759d-nf4cx"
Sep 11 17:31:16.641387 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:16.641347    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8d4763e0-c0ce-4b9a-8a05-ff4f718b96f0-istio-token\") pod \"e2e-autores-49e398-openshift-default-66cbbd759d-nf4cx\" (UID: \"8d4763e0-c0ce-4b9a-8a05-ff4f718b96f0\") " pod="openshift-ingress/e2e-autores-49e398-openshift-default-66cbbd759d-nf4cx"
Sep 11 17:31:16.641584 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:16.641402    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8d4763e0-c0ce-4b9a-8a05-ff4f718b96f0-credential-socket\") pod \"e2e-autores-49e398-openshift-default-66cbbd759d-nf4cx\" (UID: \"8d4763e0-c0ce-4b9a-8a05-ff4f718b96f0\") " pod="openshift-ingress/e2e-autores-49e398-openshift-default-66cbbd759d-nf4cx"
Sep 11 17:31:16.641584 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:16.641532    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8d4763e0-c0ce-4b9a-8a05-ff4f718b96f0-workload-certs\") pod \"e2e-autores-49e398-openshift-default-66cbbd759d-nf4cx\" (UID: \"8d4763e0-c0ce-4b9a-8a05-ff4f718b96f0\") " pod="openshift-ingress/e2e-autores-49e398-openshift-default-66cbbd759d-nf4cx"
Sep 11 17:31:16.641706 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:16.641587    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8d4763e0-c0ce-4b9a-8a05-ff4f718b96f0-istio-envoy\") pod \"e2e-autores-49e398-openshift-default-66cbbd759d-nf4cx\" (UID: \"8d4763e0-c0ce-4b9a-8a05-ff4f718b96f0\") " pod="openshift-ingress/e2e-autores-49e398-openshift-default-66cbbd759d-nf4cx"
Sep 11 17:31:16.641706 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:16.641664    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8d4763e0-c0ce-4b9a-8a05-ff4f718b96f0-istio-data\") pod \"e2e-autores-49e398-openshift-default-66cbbd759d-nf4cx\" (UID: \"8d4763e0-c0ce-4b9a-8a05-ff4f718b96f0\") " pod="openshift-ingress/e2e-autores-49e398-openshift-default-66cbbd759d-nf4cx"
Sep 11 17:31:16.641811 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:16.641710    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8d4763e0-c0ce-4b9a-8a05-ff4f718b96f0-istio-podinfo\") pod \"e2e-autores-49e398-openshift-default-66cbbd759d-nf4cx\" (UID: \"8d4763e0-c0ce-4b9a-8a05-ff4f718b96f0\") " pod="openshift-ingress/e2e-autores-49e398-openshift-default-66cbbd759d-nf4cx"
Sep 11 17:31:16.641811 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:16.641735    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8d4763e0-c0ce-4b9a-8a05-ff4f718b96f0-istiod-ca-cert\") pod \"e2e-autores-49e398-openshift-default-66cbbd759d-nf4cx\" (UID: \"8d4763e0-c0ce-4b9a-8a05-ff4f718b96f0\") " pod="openshift-ingress/e2e-autores-49e398-openshift-default-66cbbd759d-nf4cx"
Sep 11 17:31:16.641811 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:16.641770    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mp255\" (UniqueName: \"kubernetes.io/projected/8d4763e0-c0ce-4b9a-8a05-ff4f718b96f0-kube-api-access-mp255\") pod \"e2e-autores-49e398-openshift-default-66cbbd759d-nf4cx\" (UID: \"8d4763e0-c0ce-4b9a-8a05-ff4f718b96f0\") " pod="openshift-ingress/e2e-autores-49e398-openshift-default-66cbbd759d-nf4cx"
Sep 11 17:31:16.641983 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:16.641815    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8d4763e0-c0ce-4b9a-8a05-ff4f718b96f0-credential-socket\") pod \"e2e-autores-49e398-openshift-default-66cbbd759d-nf4cx\" (UID: \"8d4763e0-c0ce-4b9a-8a05-ff4f718b96f0\") " pod="openshift-ingress/e2e-autores-49e398-openshift-default-66cbbd759d-nf4cx"
Sep 11 17:31:16.641983 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:16.641866    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8d4763e0-c0ce-4b9a-8a05-ff4f718b96f0-workload-socket\") pod \"e2e-autores-49e398-openshift-default-66cbbd759d-nf4cx\" (UID: \"8d4763e0-c0ce-4b9a-8a05-ff4f718b96f0\") " pod="openshift-ingress/e2e-autores-49e398-openshift-default-66cbbd759d-nf4cx"
Sep 11 17:31:16.642559 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:16.642461    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8d4763e0-c0ce-4b9a-8a05-ff4f718b96f0-istio-data\") pod \"e2e-autores-49e398-openshift-default-66cbbd759d-nf4cx\" (UID: \"8d4763e0-c0ce-4b9a-8a05-ff4f718b96f0\") " pod="openshift-ingress/e2e-autores-49e398-openshift-default-66cbbd759d-nf4cx"
Sep 11 17:31:16.642559 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:16.642471    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8d4763e0-c0ce-4b9a-8a05-ff4f718b96f0-workload-socket\") pod \"e2e-autores-49e398-openshift-default-66cbbd759d-nf4cx\" (UID: \"8d4763e0-c0ce-4b9a-8a05-ff4f718b96f0\") " pod="openshift-ingress/e2e-autores-49e398-openshift-default-66cbbd759d-nf4cx"
Sep 11 17:31:16.642559 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:16.642501    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8d4763e0-c0ce-4b9a-8a05-ff4f718b96f0-istiod-ca-cert\") pod \"e2e-autores-49e398-openshift-default-66cbbd759d-nf4cx\" (UID: \"8d4763e0-c0ce-4b9a-8a05-ff4f718b96f0\") " pod="openshift-ingress/e2e-autores-49e398-openshift-default-66cbbd759d-nf4cx"
Sep 11 17:31:16.642816 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:16.642611    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8d4763e0-c0ce-4b9a-8a05-ff4f718b96f0-workload-certs\") pod \"e2e-autores-49e398-openshift-default-66cbbd759d-nf4cx\" (UID: \"8d4763e0-c0ce-4b9a-8a05-ff4f718b96f0\") " pod="openshift-ingress/e2e-autores-49e398-openshift-default-66cbbd759d-nf4cx"
Sep 11 17:31:16.643950 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:16.643930    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8d4763e0-c0ce-4b9a-8a05-ff4f718b96f0-istio-envoy\") pod \"e2e-autores-49e398-openshift-default-66cbbd759d-nf4cx\" (UID: \"8d4763e0-c0ce-4b9a-8a05-ff4f718b96f0\") " pod="openshift-ingress/e2e-autores-49e398-openshift-default-66cbbd759d-nf4cx"
Sep 11 17:31:16.644534 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:16.644517    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8d4763e0-c0ce-4b9a-8a05-ff4f718b96f0-istio-podinfo\") pod \"e2e-autores-49e398-openshift-default-66cbbd759d-nf4cx\" (UID: \"8d4763e0-c0ce-4b9a-8a05-ff4f718b96f0\") " pod="openshift-ingress/e2e-autores-49e398-openshift-default-66cbbd759d-nf4cx"
Sep 11 17:31:16.649181 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:16.649150    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:31:16.649906 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:16.649886    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8d4763e0-c0ce-4b9a-8a05-ff4f718b96f0-istio-token\") pod \"e2e-autores-49e398-openshift-default-66cbbd759d-nf4cx\" (UID: \"8d4763e0-c0ce-4b9a-8a05-ff4f718b96f0\") " pod="openshift-ingress/e2e-autores-49e398-openshift-default-66cbbd759d-nf4cx"
Sep 11 17:31:16.650334 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:16.650314    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mp255\" (UniqueName: \"kubernetes.io/projected/8d4763e0-c0ce-4b9a-8a05-ff4f718b96f0-kube-api-access-mp255\") pod \"e2e-autores-49e398-openshift-default-66cbbd759d-nf4cx\" (UID: \"8d4763e0-c0ce-4b9a-8a05-ff4f718b96f0\") " pod="openshift-ingress/e2e-autores-49e398-openshift-default-66cbbd759d-nf4cx"
Sep 11 17:31:16.730790 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:16.730710    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-49e398-openshift-default-66cbbd759d-nf4cx"
Sep 11 17:31:16.750651 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:16.750618    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:31:16.986866 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:16.986781    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:31:17.005160 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:17.005104    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7698-9b3132" podUID="05237a85-f901-4f22-bb61-56d4ef6c222a" containerName="test-curl-7698-9b3132" containerID="cri-o://76d115efd0512855f934344639b8fd2e72c6b3544a5158e11de630d1b2eff59f" gracePeriod=2
Sep 11 17:31:17.024491 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:17.024467    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-autores-49e398-openshift-default-66cbbd759d-nf4cx"]
Sep 11 17:31:17.038273 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:31:17.038242    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d4763e0_c0ce_4b9a_8a05_ff4f718b96f0.slice/crio-6601544c482ec1f6067c95d060d87ea4e31fc527cbce3b1031e95e691b0d4c4d WatchSource:0}: Error finding container 6601544c482ec1f6067c95d060d87ea4e31fc527cbce3b1031e95e691b0d4c4d: Status 404 returned error can't find the container with id 6601544c482ec1f6067c95d060d87ea4e31fc527cbce3b1031e95e691b0d4c4d
Sep 11 17:31:17.040965 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:17.040926    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 17:31:17.041062 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:17.041007    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 17:31:17.041062 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:17.041035    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 17:31:17.164300 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:17.164274    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7698-600b08"
Sep 11 17:31:17.248696 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:17.248609    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q94t4\" (UniqueName: \"kubernetes.io/projected/e29d4d86-fa04-43c5-94e4-84dc28a3d19d-kube-api-access-q94t4\") pod \"e29d4d86-fa04-43c5-94e4-84dc28a3d19d\" (UID: \"e29d4d86-fa04-43c5-94e4-84dc28a3d19d\") "
Sep 11 17:31:17.251134 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:17.251083    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e29d4d86-fa04-43c5-94e4-84dc28a3d19d-kube-api-access-q94t4" (OuterVolumeSpecName: "kube-api-access-q94t4") pod "e29d4d86-fa04-43c5-94e4-84dc28a3d19d" (UID: "e29d4d86-fa04-43c5-94e4-84dc28a3d19d"). InnerVolumeSpecName "kube-api-access-q94t4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:31:17.300165 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:17.300131    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:31:17.349691 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:17.349660    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q94t4\" (UniqueName: \"kubernetes.io/projected/e29d4d86-fa04-43c5-94e4-84dc28a3d19d-kube-api-access-q94t4\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:31:17.949613 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:17.949522    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e29d4d86-fa04-43c5-94e4-84dc28a3d19d" path="/var/lib/kubelet/pods/e29d4d86-fa04-43c5-94e4-84dc28a3d19d/volumes"
Sep 11 17:31:18.012392 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:18.012358    2552 generic.go:356] "Generic (PLEG): container finished" podID="e29d4d86-fa04-43c5-94e4-84dc28a3d19d" containerID="4b406e16df32079d440f2e1c690ea0a172110868b86ee325ad1395040d6d07d6" exitCode=137
Sep 11 17:31:18.012836 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:18.012409    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7698-600b08"
Sep 11 17:31:18.012836 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:18.012459    2552 scope.go:117] "RemoveContainer" containerID="4b406e16df32079d440f2e1c690ea0a172110868b86ee325ad1395040d6d07d6"
Sep 11 17:31:18.014616 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:18.014583    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-49e398-openshift-default-66cbbd759d-nf4cx" event={"ID":"8d4763e0-c0ce-4b9a-8a05-ff4f718b96f0","Type":"ContainerStarted","Data":"646c4823f006ad259a058824a1c60f5aab0dddea84724b709ab153aec5eeb7c7"}
Sep 11 17:31:18.014741 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:18.014647    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-49e398-openshift-default-66cbbd759d-nf4cx" event={"ID":"8d4763e0-c0ce-4b9a-8a05-ff4f718b96f0","Type":"ContainerStarted","Data":"6601544c482ec1f6067c95d060d87ea4e31fc527cbce3b1031e95e691b0d4c4d"}
Sep 11 17:31:18.021824 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:18.021808    2552 scope.go:117] "RemoveContainer" containerID="4b406e16df32079d440f2e1c690ea0a172110868b86ee325ad1395040d6d07d6"
Sep 11 17:31:18.022091 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:31:18.022069    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4b406e16df32079d440f2e1c690ea0a172110868b86ee325ad1395040d6d07d6\": container with ID starting with 4b406e16df32079d440f2e1c690ea0a172110868b86ee325ad1395040d6d07d6 not found: ID does not exist" containerID="4b406e16df32079d440f2e1c690ea0a172110868b86ee325ad1395040d6d07d6"
Sep 11 17:31:18.022179 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:18.022102    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4b406e16df32079d440f2e1c690ea0a172110868b86ee325ad1395040d6d07d6"} err="failed to get container status \"4b406e16df32079d440f2e1c690ea0a172110868b86ee325ad1395040d6d07d6\": rpc error: code = NotFound desc = could not find container \"4b406e16df32079d440f2e1c690ea0a172110868b86ee325ad1395040d6d07d6\": container with ID starting with 4b406e16df32079d440f2e1c690ea0a172110868b86ee325ad1395040d6d07d6 not found: ID does not exist"
Sep 11 17:31:18.035819 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:18.035770    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-autores-49e398-openshift-default-66cbbd759d-nf4cx" podStartSLOduration=2.035752509 podStartE2EDuration="2.035752509s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:31:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:31:18.032973623 +0000 UTC m=+772.554265146" watchObservedRunningTime="2026-09-11 17:31:18.035752509 +0000 UTC m=+772.557044030"
Sep 11 17:31:18.533956 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:18.533915    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:31:18.627372 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:18.627341    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:31:18.731908 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:18.731871    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-autores-49e398-openshift-default-66cbbd759d-nf4cx"
Sep 11 17:31:18.739134 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:18.738536    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-autores-49e398-openshift-default-66cbbd759d-nf4cx"
Sep 11 17:31:18.759565 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:18.759536    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-23a9cc4a-openshift-default-775b7dcc48-2dxkx"]
Sep 11 17:31:18.764480 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:18.764452    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-23a9cc4a-openshift-default-775b7dcc48-2dxkx"
Sep 11 17:31:18.766587 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:18.766563    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-23a9cc4a-openshift-default-dockercfg-s47tz\""
Sep 11 17:31:18.774054 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:18.774025    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-23a9cc4a-openshift-default-775b7dcc48-2dxkx"]
Sep 11 17:31:18.781164 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:18.781093    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:31:18.864212 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:18.864160    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/145c6962-f155-4a4d-bce7-b030b7955a6e-workload-certs\") pod \"e2e-ait-23a9cc4a-openshift-default-775b7dcc48-2dxkx\" (UID: \"145c6962-f155-4a4d-bce7-b030b7955a6e\") " pod="openshift-ingress/e2e-ait-23a9cc4a-openshift-default-775b7dcc48-2dxkx"
Sep 11 17:31:18.864395 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:18.864226    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/145c6962-f155-4a4d-bce7-b030b7955a6e-workload-socket\") pod \"e2e-ait-23a9cc4a-openshift-default-775b7dcc48-2dxkx\" (UID: \"145c6962-f155-4a4d-bce7-b030b7955a6e\") " pod="openshift-ingress/e2e-ait-23a9cc4a-openshift-default-775b7dcc48-2dxkx"
Sep 11 17:31:18.864395 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:18.864263    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/145c6962-f155-4a4d-bce7-b030b7955a6e-istio-podinfo\") pod \"e2e-ait-23a9cc4a-openshift-default-775b7dcc48-2dxkx\" (UID: \"145c6962-f155-4a4d-bce7-b030b7955a6e\") " pod="openshift-ingress/e2e-ait-23a9cc4a-openshift-default-775b7dcc48-2dxkx"
Sep 11 17:31:18.864395 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:18.864314    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/145c6962-f155-4a4d-bce7-b030b7955a6e-credential-socket\") pod \"e2e-ait-23a9cc4a-openshift-default-775b7dcc48-2dxkx\" (UID: \"145c6962-f155-4a4d-bce7-b030b7955a6e\") " pod="openshift-ingress/e2e-ait-23a9cc4a-openshift-default-775b7dcc48-2dxkx"
Sep 11 17:31:18.864395 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:18.864349    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/145c6962-f155-4a4d-bce7-b030b7955a6e-istiod-ca-cert\") pod \"e2e-ait-23a9cc4a-openshift-default-775b7dcc48-2dxkx\" (UID: \"145c6962-f155-4a4d-bce7-b030b7955a6e\") " pod="openshift-ingress/e2e-ait-23a9cc4a-openshift-default-775b7dcc48-2dxkx"
Sep 11 17:31:18.864616 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:18.864407    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/145c6962-f155-4a4d-bce7-b030b7955a6e-istio-token\") pod \"e2e-ait-23a9cc4a-openshift-default-775b7dcc48-2dxkx\" (UID: \"145c6962-f155-4a4d-bce7-b030b7955a6e\") " pod="openshift-ingress/e2e-ait-23a9cc4a-openshift-default-775b7dcc48-2dxkx"
Sep 11 17:31:18.864616 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:18.864454    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/145c6962-f155-4a4d-bce7-b030b7955a6e-istio-envoy\") pod \"e2e-ait-23a9cc4a-openshift-default-775b7dcc48-2dxkx\" (UID: \"145c6962-f155-4a4d-bce7-b030b7955a6e\") " pod="openshift-ingress/e2e-ait-23a9cc4a-openshift-default-775b7dcc48-2dxkx"
Sep 11 17:31:18.864616 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:18.864483    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/145c6962-f155-4a4d-bce7-b030b7955a6e-istio-data\") pod \"e2e-ait-23a9cc4a-openshift-default-775b7dcc48-2dxkx\" (UID: \"145c6962-f155-4a4d-bce7-b030b7955a6e\") " pod="openshift-ingress/e2e-ait-23a9cc4a-openshift-default-775b7dcc48-2dxkx"
Sep 11 17:31:18.864616 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:18.864542    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-txxpd\" (UniqueName: \"kubernetes.io/projected/145c6962-f155-4a4d-bce7-b030b7955a6e-kube-api-access-txxpd\") pod \"e2e-ait-23a9cc4a-openshift-default-775b7dcc48-2dxkx\" (UID: \"145c6962-f155-4a4d-bce7-b030b7955a6e\") " pod="openshift-ingress/e2e-ait-23a9cc4a-openshift-default-775b7dcc48-2dxkx"
Sep 11 17:31:18.887446 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:18.887414    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:31:18.965613 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:18.965570    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/145c6962-f155-4a4d-bce7-b030b7955a6e-credential-socket\") pod \"e2e-ait-23a9cc4a-openshift-default-775b7dcc48-2dxkx\" (UID: \"145c6962-f155-4a4d-bce7-b030b7955a6e\") " pod="openshift-ingress/e2e-ait-23a9cc4a-openshift-default-775b7dcc48-2dxkx"
Sep 11 17:31:18.965613 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:18.965618    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/145c6962-f155-4a4d-bce7-b030b7955a6e-istiod-ca-cert\") pod \"e2e-ait-23a9cc4a-openshift-default-775b7dcc48-2dxkx\" (UID: \"145c6962-f155-4a4d-bce7-b030b7955a6e\") " pod="openshift-ingress/e2e-ait-23a9cc4a-openshift-default-775b7dcc48-2dxkx"
Sep 11 17:31:18.965824 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:18.965668    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/145c6962-f155-4a4d-bce7-b030b7955a6e-istio-token\") pod \"e2e-ait-23a9cc4a-openshift-default-775b7dcc48-2dxkx\" (UID: \"145c6962-f155-4a4d-bce7-b030b7955a6e\") " pod="openshift-ingress/e2e-ait-23a9cc4a-openshift-default-775b7dcc48-2dxkx"
Sep 11 17:31:18.965824 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:18.965709    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/145c6962-f155-4a4d-bce7-b030b7955a6e-istio-envoy\") pod \"e2e-ait-23a9cc4a-openshift-default-775b7dcc48-2dxkx\" (UID: \"145c6962-f155-4a4d-bce7-b030b7955a6e\") " pod="openshift-ingress/e2e-ait-23a9cc4a-openshift-default-775b7dcc48-2dxkx"
Sep 11 17:31:18.965824 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:18.965736    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/145c6962-f155-4a4d-bce7-b030b7955a6e-istio-data\") pod \"e2e-ait-23a9cc4a-openshift-default-775b7dcc48-2dxkx\" (UID: \"145c6962-f155-4a4d-bce7-b030b7955a6e\") " pod="openshift-ingress/e2e-ait-23a9cc4a-openshift-default-775b7dcc48-2dxkx"
Sep 11 17:31:18.965824 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:18.965785    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-txxpd\" (UniqueName: \"kubernetes.io/projected/145c6962-f155-4a4d-bce7-b030b7955a6e-kube-api-access-txxpd\") pod \"e2e-ait-23a9cc4a-openshift-default-775b7dcc48-2dxkx\" (UID: \"145c6962-f155-4a4d-bce7-b030b7955a6e\") " pod="openshift-ingress/e2e-ait-23a9cc4a-openshift-default-775b7dcc48-2dxkx"
Sep 11 17:31:18.966025 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:18.965838    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/145c6962-f155-4a4d-bce7-b030b7955a6e-workload-certs\") pod \"e2e-ait-23a9cc4a-openshift-default-775b7dcc48-2dxkx\" (UID: \"145c6962-f155-4a4d-bce7-b030b7955a6e\") " pod="openshift-ingress/e2e-ait-23a9cc4a-openshift-default-775b7dcc48-2dxkx"
Sep 11 17:31:18.966025 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:18.965895    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/145c6962-f155-4a4d-bce7-b030b7955a6e-workload-socket\") pod \"e2e-ait-23a9cc4a-openshift-default-775b7dcc48-2dxkx\" (UID: \"145c6962-f155-4a4d-bce7-b030b7955a6e\") " pod="openshift-ingress/e2e-ait-23a9cc4a-openshift-default-775b7dcc48-2dxkx"
Sep 11 17:31:18.966025 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:18.965947    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/145c6962-f155-4a4d-bce7-b030b7955a6e-istio-podinfo\") pod \"e2e-ait-23a9cc4a-openshift-default-775b7dcc48-2dxkx\" (UID: \"145c6962-f155-4a4d-bce7-b030b7955a6e\") " pod="openshift-ingress/e2e-ait-23a9cc4a-openshift-default-775b7dcc48-2dxkx"
Sep 11 17:31:18.966230 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:18.966038    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/145c6962-f155-4a4d-bce7-b030b7955a6e-credential-socket\") pod \"e2e-ait-23a9cc4a-openshift-default-775b7dcc48-2dxkx\" (UID: \"145c6962-f155-4a4d-bce7-b030b7955a6e\") " pod="openshift-ingress/e2e-ait-23a9cc4a-openshift-default-775b7dcc48-2dxkx"
Sep 11 17:31:18.966230 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:18.966171    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/145c6962-f155-4a4d-bce7-b030b7955a6e-istio-data\") pod \"e2e-ait-23a9cc4a-openshift-default-775b7dcc48-2dxkx\" (UID: \"145c6962-f155-4a4d-bce7-b030b7955a6e\") " pod="openshift-ingress/e2e-ait-23a9cc4a-openshift-default-775b7dcc48-2dxkx"
Sep 11 17:31:18.966336 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:18.966287    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/145c6962-f155-4a4d-bce7-b030b7955a6e-workload-certs\") pod \"e2e-ait-23a9cc4a-openshift-default-775b7dcc48-2dxkx\" (UID: \"145c6962-f155-4a4d-bce7-b030b7955a6e\") " pod="openshift-ingress/e2e-ait-23a9cc4a-openshift-default-775b7dcc48-2dxkx"
Sep 11 17:31:18.966398 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:18.966374    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/145c6962-f155-4a4d-bce7-b030b7955a6e-workload-socket\") pod \"e2e-ait-23a9cc4a-openshift-default-775b7dcc48-2dxkx\" (UID: \"145c6962-f155-4a4d-bce7-b030b7955a6e\") " pod="openshift-ingress/e2e-ait-23a9cc4a-openshift-default-775b7dcc48-2dxkx"
Sep 11 17:31:18.966542 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:18.966510    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/145c6962-f155-4a4d-bce7-b030b7955a6e-istiod-ca-cert\") pod \"e2e-ait-23a9cc4a-openshift-default-775b7dcc48-2dxkx\" (UID: \"145c6962-f155-4a4d-bce7-b030b7955a6e\") " pod="openshift-ingress/e2e-ait-23a9cc4a-openshift-default-775b7dcc48-2dxkx"
Sep 11 17:31:18.968144 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:18.968107    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/145c6962-f155-4a4d-bce7-b030b7955a6e-istio-envoy\") pod \"e2e-ait-23a9cc4a-openshift-default-775b7dcc48-2dxkx\" (UID: \"145c6962-f155-4a4d-bce7-b030b7955a6e\") " pod="openshift-ingress/e2e-ait-23a9cc4a-openshift-default-775b7dcc48-2dxkx"
Sep 11 17:31:18.968449 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:18.968429    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/145c6962-f155-4a4d-bce7-b030b7955a6e-istio-podinfo\") pod \"e2e-ait-23a9cc4a-openshift-default-775b7dcc48-2dxkx\" (UID: \"145c6962-f155-4a4d-bce7-b030b7955a6e\") " pod="openshift-ingress/e2e-ait-23a9cc4a-openshift-default-775b7dcc48-2dxkx"
Sep 11 17:31:18.977459 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:18.977435    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-txxpd\" (UniqueName: \"kubernetes.io/projected/145c6962-f155-4a4d-bce7-b030b7955a6e-kube-api-access-txxpd\") pod \"e2e-ait-23a9cc4a-openshift-default-775b7dcc48-2dxkx\" (UID: \"145c6962-f155-4a4d-bce7-b030b7955a6e\") " pod="openshift-ingress/e2e-ait-23a9cc4a-openshift-default-775b7dcc48-2dxkx"
Sep 11 17:31:18.978316 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:18.978298    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/145c6962-f155-4a4d-bce7-b030b7955a6e-istio-token\") pod \"e2e-ait-23a9cc4a-openshift-default-775b7dcc48-2dxkx\" (UID: \"145c6962-f155-4a4d-bce7-b030b7955a6e\") " pod="openshift-ingress/e2e-ait-23a9cc4a-openshift-default-775b7dcc48-2dxkx"
Sep 11 17:31:19.022557 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:19.022536    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-autores-49e398-openshift-default-66cbbd759d-nf4cx"
Sep 11 17:31:19.023564 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:19.023543    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-autores-49e398-openshift-default-66cbbd759d-nf4cx"
Sep 11 17:31:19.076040 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:19.076010    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-23a9cc4a-openshift-default-775b7dcc48-2dxkx"
Sep 11 17:31:19.083324 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:19.083290    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:31:19.193766 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:19.193735    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7698-9b3132"
Sep 11 17:31:19.268891 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:19.268860    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6cjxr\" (UniqueName: \"kubernetes.io/projected/05237a85-f901-4f22-bb61-56d4ef6c222a-kube-api-access-6cjxr\") pod \"05237a85-f901-4f22-bb61-56d4ef6c222a\" (UID: \"05237a85-f901-4f22-bb61-56d4ef6c222a\") "
Sep 11 17:31:19.270930 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:19.270906    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/05237a85-f901-4f22-bb61-56d4ef6c222a-kube-api-access-6cjxr" (OuterVolumeSpecName: "kube-api-access-6cjxr") pod "05237a85-f901-4f22-bb61-56d4ef6c222a" (UID: "05237a85-f901-4f22-bb61-56d4ef6c222a"). InnerVolumeSpecName "kube-api-access-6cjxr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:31:19.369855 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:19.369765    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6cjxr\" (UniqueName: \"kubernetes.io/projected/05237a85-f901-4f22-bb61-56d4ef6c222a-kube-api-access-6cjxr\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:31:19.372329 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:19.372307    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-23a9cc4a-openshift-default-775b7dcc48-2dxkx"]
Sep 11 17:31:19.374009 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:31:19.373985    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod145c6962_f155_4a4d_bce7_b030b7955a6e.slice/crio-ce1ce68f84ffc3721527e6d2f5dad2f04a4e446eab3a8723dc28a40b002c8a26 WatchSource:0}: Error finding container ce1ce68f84ffc3721527e6d2f5dad2f04a4e446eab3a8723dc28a40b002c8a26: Status 404 returned error can't find the container with id ce1ce68f84ffc3721527e6d2f5dad2f04a4e446eab3a8723dc28a40b002c8a26
Sep 11 17:31:19.376300 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:19.376272    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 17:31:19.376356 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:19.376336    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 17:31:19.376392 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:19.376365    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 17:31:19.639494 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:19.639396    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:31:19.954030 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:19.953951    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="05237a85-f901-4f22-bb61-56d4ef6c222a" path="/var/lib/kubelet/pods/05237a85-f901-4f22-bb61-56d4ef6c222a/volumes"
Sep 11 17:31:20.029571 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:20.029542    2552 generic.go:356] "Generic (PLEG): container finished" podID="05237a85-f901-4f22-bb61-56d4ef6c222a" containerID="76d115efd0512855f934344639b8fd2e72c6b3544a5158e11de630d1b2eff59f" exitCode=137
Sep 11 17:31:20.030012 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:20.029581    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7698-9b3132"
Sep 11 17:31:20.030012 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:20.029644    2552 scope.go:117] "RemoveContainer" containerID="76d115efd0512855f934344639b8fd2e72c6b3544a5158e11de630d1b2eff59f"
Sep 11 17:31:20.031780 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:20.031741    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-23a9cc4a-openshift-default-775b7dcc48-2dxkx" event={"ID":"145c6962-f155-4a4d-bce7-b030b7955a6e","Type":"ContainerStarted","Data":"1b5547fdc0c0c73c0ee37e44ebbaf432f8ed65de08f18c84580ed743be84fbdd"}
Sep 11 17:31:20.031934 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:20.031834    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-23a9cc4a-openshift-default-775b7dcc48-2dxkx" event={"ID":"145c6962-f155-4a4d-bce7-b030b7955a6e","Type":"ContainerStarted","Data":"ce1ce68f84ffc3721527e6d2f5dad2f04a4e446eab3a8723dc28a40b002c8a26"}
Sep 11 17:31:20.040056 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:20.039874    2552 scope.go:117] "RemoveContainer" containerID="76d115efd0512855f934344639b8fd2e72c6b3544a5158e11de630d1b2eff59f"
Sep 11 17:31:20.040304 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:31:20.040280    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"76d115efd0512855f934344639b8fd2e72c6b3544a5158e11de630d1b2eff59f\": container with ID starting with 76d115efd0512855f934344639b8fd2e72c6b3544a5158e11de630d1b2eff59f not found: ID does not exist" containerID="76d115efd0512855f934344639b8fd2e72c6b3544a5158e11de630d1b2eff59f"
Sep 11 17:31:20.040387 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:20.040314    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"76d115efd0512855f934344639b8fd2e72c6b3544a5158e11de630d1b2eff59f"} err="failed to get container status \"76d115efd0512855f934344639b8fd2e72c6b3544a5158e11de630d1b2eff59f\": rpc error: code = NotFound desc = could not find container \"76d115efd0512855f934344639b8fd2e72c6b3544a5158e11de630d1b2eff59f\": container with ID starting with 76d115efd0512855f934344639b8fd2e72c6b3544a5158e11de630d1b2eff59f not found: ID does not exist"
Sep 11 17:31:20.053259 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:20.053229    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:31:20.074305 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:20.074261    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-23a9cc4a-openshift-default-775b7dcc48-2dxkx" podStartSLOduration=2.074246634 podStartE2EDuration="2.074246634s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:31:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:31:20.071976991 +0000 UTC m=+774.593268512" watchObservedRunningTime="2026-09-11 17:31:20.074246634 +0000 UTC m=+774.595538224"
Sep 11 17:31:20.077354 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:20.077339    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-23a9cc4a-openshift-default-775b7dcc48-2dxkx"
Sep 11 17:31:20.082667 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:20.082637    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-23a9cc4a-openshift-default-775b7dcc48-2dxkx"
Sep 11 17:31:20.153658 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:20.153623    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:31:20.727186 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:20.726969    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:31:21.040251 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:21.040147    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-23a9cc4a-openshift-default-775b7dcc48-2dxkx"
Sep 11 17:31:21.041287 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:21.041265    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-23a9cc4a-openshift-default-775b7dcc48-2dxkx"
Sep 11 17:31:21.582015 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:21.581969    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:31:21.665868 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:21.665826    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:31:21.850247 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:21.850208    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:31:23.114501 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:23.114461    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:31:24.088136 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:24.086310    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:31:24.217609 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:24.217565    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:31:24.432801 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:24.432760    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:31:24.691611 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:24.691506    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:31:24.789566 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:24.789351    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:31:25.509357 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:25.509322    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:31:26.361852 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:26.361820    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-49e398-86698d695f-f447p"]
Sep 11 17:31:26.378648 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:26.378620    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-49e398-86698d695f-f447p"
Sep 11 17:31:26.387381 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:26.385665    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-autores-49e398-serving-cert\""
Sep 11 17:31:26.387381 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:26.385933    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 17:31:26.387381 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:26.386135    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-zp7dq\""
Sep 11 17:31:26.387381 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:26.386326    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 11 17:31:26.387381 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:26.387124    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 11 17:31:26.408784 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:26.408201    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-49e398-86698d695f-f447p"]
Sep 11 17:31:26.410327 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:26.410258    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:31:26.454825 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:26.454784    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/107916e5-934a-406e-b26c-b668814b6285-openshift-service-ca\") pod \"maas-api-e2e-autores-49e398-86698d695f-f447p\" (UID: \"107916e5-934a-406e-b26c-b668814b6285\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-49e398-86698d695f-f447p"
Sep 11 17:31:26.455269 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:26.455242    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qhpxz\" (UniqueName: \"kubernetes.io/projected/107916e5-934a-406e-b26c-b668814b6285-kube-api-access-qhpxz\") pod \"maas-api-e2e-autores-49e398-86698d695f-f447p\" (UID: \"107916e5-934a-406e-b26c-b668814b6285\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-49e398-86698d695f-f447p"
Sep 11 17:31:26.455442 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:26.455424    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/107916e5-934a-406e-b26c-b668814b6285-maas-api-tls\") pod \"maas-api-e2e-autores-49e398-86698d695f-f447p\" (UID: \"107916e5-934a-406e-b26c-b668814b6285\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-49e398-86698d695f-f447p"
Sep 11 17:31:26.455599 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:26.455584    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/107916e5-934a-406e-b26c-b668814b6285-metrics-certs\") pod \"maas-api-e2e-autores-49e398-86698d695f-f447p\" (UID: \"107916e5-934a-406e-b26c-b668814b6285\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-49e398-86698d695f-f447p"
Sep 11 17:31:26.556851 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:26.556799    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qhpxz\" (UniqueName: \"kubernetes.io/projected/107916e5-934a-406e-b26c-b668814b6285-kube-api-access-qhpxz\") pod \"maas-api-e2e-autores-49e398-86698d695f-f447p\" (UID: \"107916e5-934a-406e-b26c-b668814b6285\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-49e398-86698d695f-f447p"
Sep 11 17:31:26.557449 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:26.557408    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/107916e5-934a-406e-b26c-b668814b6285-maas-api-tls\") pod \"maas-api-e2e-autores-49e398-86698d695f-f447p\" (UID: \"107916e5-934a-406e-b26c-b668814b6285\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-49e398-86698d695f-f447p"
Sep 11 17:31:26.558103 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:26.558067    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/107916e5-934a-406e-b26c-b668814b6285-metrics-certs\") pod \"maas-api-e2e-autores-49e398-86698d695f-f447p\" (UID: \"107916e5-934a-406e-b26c-b668814b6285\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-49e398-86698d695f-f447p"
Sep 11 17:31:26.558382 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:26.558363    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/107916e5-934a-406e-b26c-b668814b6285-openshift-service-ca\") pod \"maas-api-e2e-autores-49e398-86698d695f-f447p\" (UID: \"107916e5-934a-406e-b26c-b668814b6285\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-49e398-86698d695f-f447p"
Sep 11 17:31:26.558919 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:31:26.557722    2552 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-autores-49e398-serving-cert: secret "maas-api-e2e-autores-49e398-serving-cert" not found
Sep 11 17:31:26.559207 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:31:26.559194    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/107916e5-934a-406e-b26c-b668814b6285-maas-api-tls podName:107916e5-934a-406e-b26c-b668814b6285 nodeName:}" failed. No retries permitted until 2026-09-11 17:31:27.059081072 +0000 UTC m=+781.580372590 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/107916e5-934a-406e-b26c-b668814b6285-maas-api-tls") pod "maas-api-e2e-autores-49e398-86698d695f-f447p" (UID: "107916e5-934a-406e-b26c-b668814b6285") : secret "maas-api-e2e-autores-49e398-serving-cert" not found
Sep 11 17:31:26.559412 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:26.559291    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/107916e5-934a-406e-b26c-b668814b6285-openshift-service-ca\") pod \"maas-api-e2e-autores-49e398-86698d695f-f447p\" (UID: \"107916e5-934a-406e-b26c-b668814b6285\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-49e398-86698d695f-f447p"
Sep 11 17:31:26.562090 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:26.562050    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/107916e5-934a-406e-b26c-b668814b6285-metrics-certs\") pod \"maas-api-e2e-autores-49e398-86698d695f-f447p\" (UID: \"107916e5-934a-406e-b26c-b668814b6285\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-49e398-86698d695f-f447p"
Sep 11 17:31:26.581838 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:26.581785    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qhpxz\" (UniqueName: \"kubernetes.io/projected/107916e5-934a-406e-b26c-b668814b6285-kube-api-access-qhpxz\") pod \"maas-api-e2e-autores-49e398-86698d695f-f447p\" (UID: \"107916e5-934a-406e-b26c-b668814b6285\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-49e398-86698d695f-f447p"
Sep 11 17:31:27.066665 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:27.066618    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/107916e5-934a-406e-b26c-b668814b6285-maas-api-tls\") pod \"maas-api-e2e-autores-49e398-86698d695f-f447p\" (UID: \"107916e5-934a-406e-b26c-b668814b6285\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-49e398-86698d695f-f447p"
Sep 11 17:31:27.069944 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:27.069911    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/107916e5-934a-406e-b26c-b668814b6285-maas-api-tls\") pod \"maas-api-e2e-autores-49e398-86698d695f-f447p\" (UID: \"107916e5-934a-406e-b26c-b668814b6285\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-49e398-86698d695f-f447p"
Sep 11 17:31:27.210443 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:27.210408    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:31:27.305099 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:27.305064    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-49e398-86698d695f-f447p"
Sep 11 17:31:27.312628 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:27.312594    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:31:27.677613 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:27.677573    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-49e398-86698d695f-f447p"]
Sep 11 17:31:27.709569 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:27.709535    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:31:28.113130 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:28.112954    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-49e398-86698d695f-f447p" event={"ID":"107916e5-934a-406e-b26c-b668814b6285","Type":"ContainerStarted","Data":"b73b683c6751331b9c5bf4dc88c1da6e8211a5bf8224ac5ba5d88c17c40c410f"}
Sep 11 17:31:28.547938 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:28.547842    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:31:29.495207 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:29.495169    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:31:30.179964 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:30.179928    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:31:30.642414 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:30.642370    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:31:30.812920 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:30.812885    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:31:30.902451 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:30.902357    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:31:31.139134 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:31.138206    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-49e398-86698d695f-f447p" event={"ID":"107916e5-934a-406e-b26c-b668814b6285","Type":"ContainerStarted","Data":"d87ef3cdc4d2be6fda9b8bae588afb10791b604a4916e963f9bfea01b74cf8a6"}
Sep 11 17:31:31.139134 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:31.139073    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-autores-49e398-86698d695f-f447p"
Sep 11 17:31:31.160400 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:31.160290    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-autores-49e398-86698d695f-f447p" podStartSLOduration=2.603623348 podStartE2EDuration="5.160267445s" totalImagesPullingTime="2.556644097s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:31:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:31:31.155902915 +0000 UTC m=+785.677194437" watchObservedRunningTime="2026-09-11 17:31:31.160267445 +0000 UTC m=+785.681558967"
Sep 11 17:31:31.731079 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:31.731047    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:31:31.831457 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:31.831422    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:31:32.041326 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:32.041247    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:31:32.145749 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:32.145717    2552 generic.go:356] "Generic (PLEG): container finished" podID="ae170805-c6a4-469f-b280-8e7629749557" containerID="99222f6f532e8093632159f0abb4c3c2f4ebac614a270467e0e7e71cea3735c8" exitCode=0
Sep 11 17:31:32.145929 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:32.145789    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-144bfa-openshift-default-5dc996f7dd-ws7l7" event={"ID":"ae170805-c6a4-469f-b280-8e7629749557","Type":"ContainerDied","Data":"99222f6f532e8093632159f0abb4c3c2f4ebac614a270467e0e7e71cea3735c8"}
Sep 11 17:31:32.146320 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:32.146304    2552 scope.go:117] "RemoveContainer" containerID="99222f6f532e8093632159f0abb4c3c2f4ebac614a270467e0e7e71cea3735c8"
Sep 11 17:31:32.147709 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:32.147673    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 17:31:32.147803 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:32.147765    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 17:31:32.147803 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:32.147792    2552 generic.go:356] "Generic (PLEG): container finished" podID="d886651d-540e-47c7-8df0-7234dd3adca1" containerID="7c5c9b36711108494c07db005ca918e97b0b49aeb777d1ff5ccce3bbcd24ec54" exitCode=0
Sep 11 17:31:32.147889 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:32.147821    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 17:31:32.147889 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:32.147868    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-08c6b2-openshift-default-5d4477884-jms7z" event={"ID":"d886651d-540e-47c7-8df0-7234dd3adca1","Type":"ContainerDied","Data":"7c5c9b36711108494c07db005ca918e97b0b49aeb777d1ff5ccce3bbcd24ec54"}
Sep 11 17:31:32.148358 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:32.148341    2552 scope.go:117] "RemoveContainer" containerID="7c5c9b36711108494c07db005ca918e97b0b49aeb777d1ff5ccce3bbcd24ec54"
Sep 11 17:31:32.149352 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:32.149326    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 17:31:32.149460 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:32.149400    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 17:31:32.149460 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:32.149452    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 17:31:32.755442 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:32.755395    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:31:32.855219 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:32.855089    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:31:33.161876 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:33.160356    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-08c6b2-openshift-default-5d4477884-jms7z" event={"ID":"d886651d-540e-47c7-8df0-7234dd3adca1","Type":"ContainerStarted","Data":"afc6a587b4f264083ff0f9fba1c6d4f2efc5b6ebd79138260fbeed94de49233e"}
Sep 11 17:31:33.166035 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:33.164659    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-144bfa-openshift-default-5dc996f7dd-ws7l7" event={"ID":"ae170805-c6a4-469f-b280-8e7629749557","Type":"ContainerStarted","Data":"47b6c8b94c1c604285b5d8c2992077d7960c23b7a0b7905f9560b34ab9521935"}
Sep 11 17:31:33.206546 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:33.206514    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:31:33.304586 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:33.304547    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:31:33.406818 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:33.406780    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:31:33.651411 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:33.651185    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-08c6b2-openshift-default-5d4477884-jms7z"
Sep 11 17:31:33.651787 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:33.651739    2552 patch_prober.go:28] interesting pod/e2e-inf-a-08c6b2-openshift-default-5d4477884-jms7z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body=
Sep 11 17:31:33.651908 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:33.651852    2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-08c6b2-openshift-default-5d4477884-jms7z" podUID="d886651d-540e-47c7-8df0-7234dd3adca1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused"
Sep 11 17:31:33.665837 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:33.665784    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:31:33.771343 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:33.771261    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-144bfa-openshift-default-5dc996f7dd-ws7l7"
Sep 11 17:31:33.772259 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:33.771920    2552 patch_prober.go:28] interesting pod/e2e-worker-w4-144bfa-openshift-default-5dc996f7dd-ws7l7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body=
Sep 11 17:31:33.772259 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:33.771978    2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-144bfa-openshift-default-5dc996f7dd-ws7l7" podUID="ae170805-c6a4-469f-b280-8e7629749557" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused"
Sep 11 17:31:33.862009 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:33.861919    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:31:34.651323 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:34.651281    2552 patch_prober.go:28] interesting pod/e2e-inf-a-08c6b2-openshift-default-5d4477884-jms7z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body=
Sep 11 17:31:34.651556 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:34.651362    2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-08c6b2-openshift-default-5d4477884-jms7z" podUID="d886651d-540e-47c7-8df0-7234dd3adca1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused"
Sep 11 17:31:34.770756 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:34.770706    2552 patch_prober.go:28] interesting pod/e2e-worker-w4-144bfa-openshift-default-5dc996f7dd-ws7l7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body=
Sep 11 17:31:34.770971 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:34.770776    2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-144bfa-openshift-default-5dc996f7dd-ws7l7" podUID="ae170805-c6a4-469f-b280-8e7629749557" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused"
Sep 11 17:31:34.910166 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:34.910073    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:31:35.453260 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:35.453221    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:31:35.651440 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:35.651401    2552 patch_prober.go:28] interesting pod/e2e-inf-a-08c6b2-openshift-default-5d4477884-jms7z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body=
Sep 11 17:31:35.651771 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:35.651731    2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-08c6b2-openshift-default-5d4477884-jms7z" podUID="d886651d-540e-47c7-8df0-7234dd3adca1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused"
Sep 11 17:31:35.770770 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:35.770657    2552 patch_prober.go:28] interesting pod/e2e-worker-w4-144bfa-openshift-default-5dc996f7dd-ws7l7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body=
Sep 11 17:31:35.770961 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:35.770765    2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-144bfa-openshift-default-5dc996f7dd-ws7l7" podUID="ae170805-c6a4-469f-b280-8e7629749557" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused"
Sep 11 17:31:35.863255 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:35.863214    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:31:36.651621 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:36.651571    2552 patch_prober.go:28] interesting pod/e2e-inf-a-08c6b2-openshift-default-5d4477884-jms7z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body=
Sep 11 17:31:36.652170 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:36.651644    2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-08c6b2-openshift-default-5d4477884-jms7z" podUID="d886651d-540e-47c7-8df0-7234dd3adca1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused"
Sep 11 17:31:36.771376 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:36.771337    2552 patch_prober.go:28] interesting pod/e2e-worker-w4-144bfa-openshift-default-5dc996f7dd-ws7l7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body=
Sep 11 17:31:36.773464 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:36.773248    2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-144bfa-openshift-default-5dc996f7dd-ws7l7" podUID="ae170805-c6a4-469f-b280-8e7629749557" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused"
Sep 11 17:31:36.774004 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:36.773037    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:31:37.647851 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:37.647807    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:31:37.651518 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:37.651479    2552 patch_prober.go:28] interesting pod/e2e-inf-a-08c6b2-openshift-default-5d4477884-jms7z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body=
Sep 11 17:31:37.651693 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:37.651540    2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-08c6b2-openshift-default-5d4477884-jms7z" podUID="d886651d-540e-47c7-8df0-7234dd3adca1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused"
Sep 11 17:31:37.770548 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:37.770511    2552 patch_prober.go:28] interesting pod/e2e-worker-w4-144bfa-openshift-default-5dc996f7dd-ws7l7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body=
Sep 11 17:31:37.770854 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:37.770816    2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-144bfa-openshift-default-5dc996f7dd-ws7l7" podUID="ae170805-c6a4-469f-b280-8e7629749557" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused"
Sep 11 17:31:37.970541 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:37.970439    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:31:38.175359 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:38.175328    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-autores-49e398-86698d695f-f447p"
Sep 11 17:31:38.212643 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:38.212609    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:31:38.309203 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:38.299554    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:31:38.528255 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:38.528216    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:31:38.570581 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:38.570462    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-cbbb09-openshift-default-7657d596f6-c5cbc"]
Sep 11 17:31:38.598388 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:38.598358    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-cbbb09-openshift-default-7657d596f6-c5cbc"]
Sep 11 17:31:38.598734 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:38.598706    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-cbbb09-openshift-default-7657d596f6-c5cbc"
Sep 11 17:31:38.604594 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:38.604571    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-cbbb09-openshift-default-dockercfg-8lfdc\""
Sep 11 17:31:38.613705 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:38.610162    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:31:38.628728 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:38.628693    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/979c7e0c-c7e6-4e8a-bdbb-b1e1820989e1-istiod-ca-cert\") pod \"e2e-inf-b-cbbb09-openshift-default-7657d596f6-c5cbc\" (UID: \"979c7e0c-c7e6-4e8a-bdbb-b1e1820989e1\") " pod="openshift-ingress/e2e-inf-b-cbbb09-openshift-default-7657d596f6-c5cbc"
Sep 11 17:31:38.628873 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:38.628762    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/979c7e0c-c7e6-4e8a-bdbb-b1e1820989e1-istio-token\") pod \"e2e-inf-b-cbbb09-openshift-default-7657d596f6-c5cbc\" (UID: \"979c7e0c-c7e6-4e8a-bdbb-b1e1820989e1\") " pod="openshift-ingress/e2e-inf-b-cbbb09-openshift-default-7657d596f6-c5cbc"
Sep 11 17:31:38.628873 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:38.628805    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/979c7e0c-c7e6-4e8a-bdbb-b1e1820989e1-workload-certs\") pod \"e2e-inf-b-cbbb09-openshift-default-7657d596f6-c5cbc\" (UID: \"979c7e0c-c7e6-4e8a-bdbb-b1e1820989e1\") " pod="openshift-ingress/e2e-inf-b-cbbb09-openshift-default-7657d596f6-c5cbc"
Sep 11 17:31:38.628873 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:38.628866    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/979c7e0c-c7e6-4e8a-bdbb-b1e1820989e1-workload-socket\") pod \"e2e-inf-b-cbbb09-openshift-default-7657d596f6-c5cbc\" (UID: \"979c7e0c-c7e6-4e8a-bdbb-b1e1820989e1\") " pod="openshift-ingress/e2e-inf-b-cbbb09-openshift-default-7657d596f6-c5cbc"
Sep 11 17:31:38.629045 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:38.628897    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/979c7e0c-c7e6-4e8a-bdbb-b1e1820989e1-istio-podinfo\") pod \"e2e-inf-b-cbbb09-openshift-default-7657d596f6-c5cbc\" (UID: \"979c7e0c-c7e6-4e8a-bdbb-b1e1820989e1\") " pod="openshift-ingress/e2e-inf-b-cbbb09-openshift-default-7657d596f6-c5cbc"
Sep 11 17:31:38.629045 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:38.628949    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/979c7e0c-c7e6-4e8a-bdbb-b1e1820989e1-istio-envoy\") pod \"e2e-inf-b-cbbb09-openshift-default-7657d596f6-c5cbc\" (UID: \"979c7e0c-c7e6-4e8a-bdbb-b1e1820989e1\") " pod="openshift-ingress/e2e-inf-b-cbbb09-openshift-default-7657d596f6-c5cbc"
Sep 11 17:31:38.629045 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:38.629016    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zrthp\" (UniqueName: \"kubernetes.io/projected/979c7e0c-c7e6-4e8a-bdbb-b1e1820989e1-kube-api-access-zrthp\") pod \"e2e-inf-b-cbbb09-openshift-default-7657d596f6-c5cbc\" (UID: \"979c7e0c-c7e6-4e8a-bdbb-b1e1820989e1\") " pod="openshift-ingress/e2e-inf-b-cbbb09-openshift-default-7657d596f6-c5cbc"
Sep 11 17:31:38.629203 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:38.629066    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/979c7e0c-c7e6-4e8a-bdbb-b1e1820989e1-istio-data\") pod \"e2e-inf-b-cbbb09-openshift-default-7657d596f6-c5cbc\" (UID: \"979c7e0c-c7e6-4e8a-bdbb-b1e1820989e1\") " pod="openshift-ingress/e2e-inf-b-cbbb09-openshift-default-7657d596f6-c5cbc"
Sep 11 17:31:38.629203 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:38.629096    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/979c7e0c-c7e6-4e8a-bdbb-b1e1820989e1-credential-socket\") pod \"e2e-inf-b-cbbb09-openshift-default-7657d596f6-c5cbc\" (UID: \"979c7e0c-c7e6-4e8a-bdbb-b1e1820989e1\") " pod="openshift-ingress/e2e-inf-b-cbbb09-openshift-default-7657d596f6-c5cbc"
Sep 11 17:31:38.702497 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:38.702466    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-08c6b2-openshift-default-5d4477884-jms7z"
Sep 11 17:31:38.703695 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:38.703666    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-08c6b2-openshift-default-5d4477884-jms7z"
Sep 11 17:31:38.704421 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:38.704382    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-08c6b2-openshift-default-5d4477884-jms7z"
Sep 11 17:31:38.729844 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:38.729819    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/979c7e0c-c7e6-4e8a-bdbb-b1e1820989e1-istio-token\") pod \"e2e-inf-b-cbbb09-openshift-default-7657d596f6-c5cbc\" (UID: \"979c7e0c-c7e6-4e8a-bdbb-b1e1820989e1\") " pod="openshift-ingress/e2e-inf-b-cbbb09-openshift-default-7657d596f6-c5cbc"
Sep 11 17:31:38.729954 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:38.729854    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/979c7e0c-c7e6-4e8a-bdbb-b1e1820989e1-workload-certs\") pod \"e2e-inf-b-cbbb09-openshift-default-7657d596f6-c5cbc\" (UID: \"979c7e0c-c7e6-4e8a-bdbb-b1e1820989e1\") " pod="openshift-ingress/e2e-inf-b-cbbb09-openshift-default-7657d596f6-c5cbc"
Sep 11 17:31:38.730359 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:38.730338    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/979c7e0c-c7e6-4e8a-bdbb-b1e1820989e1-workload-certs\") pod \"e2e-inf-b-cbbb09-openshift-default-7657d596f6-c5cbc\" (UID: \"979c7e0c-c7e6-4e8a-bdbb-b1e1820989e1\") " pod="openshift-ingress/e2e-inf-b-cbbb09-openshift-default-7657d596f6-c5cbc"
Sep 11 17:31:38.730474 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:38.730172    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/979c7e0c-c7e6-4e8a-bdbb-b1e1820989e1-workload-socket\") pod \"e2e-inf-b-cbbb09-openshift-default-7657d596f6-c5cbc\" (UID: \"979c7e0c-c7e6-4e8a-bdbb-b1e1820989e1\") " pod="openshift-ingress/e2e-inf-b-cbbb09-openshift-default-7657d596f6-c5cbc"
Sep 11 17:31:38.730513 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:38.730476    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/979c7e0c-c7e6-4e8a-bdbb-b1e1820989e1-workload-socket\") pod \"e2e-inf-b-cbbb09-openshift-default-7657d596f6-c5cbc\" (UID: \"979c7e0c-c7e6-4e8a-bdbb-b1e1820989e1\") " pod="openshift-ingress/e2e-inf-b-cbbb09-openshift-default-7657d596f6-c5cbc"
Sep 11 17:31:38.730513 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:38.730496    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/979c7e0c-c7e6-4e8a-bdbb-b1e1820989e1-istio-podinfo\") pod \"e2e-inf-b-cbbb09-openshift-default-7657d596f6-c5cbc\" (UID: \"979c7e0c-c7e6-4e8a-bdbb-b1e1820989e1\") " pod="openshift-ingress/e2e-inf-b-cbbb09-openshift-default-7657d596f6-c5cbc"
Sep 11 17:31:38.730581 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:38.730523    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/979c7e0c-c7e6-4e8a-bdbb-b1e1820989e1-istio-envoy\") pod \"e2e-inf-b-cbbb09-openshift-default-7657d596f6-c5cbc\" (UID: \"979c7e0c-c7e6-4e8a-bdbb-b1e1820989e1\") " pod="openshift-ingress/e2e-inf-b-cbbb09-openshift-default-7657d596f6-c5cbc"
Sep 11 17:31:38.730616 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:38.730589    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zrthp\" (UniqueName: \"kubernetes.io/projected/979c7e0c-c7e6-4e8a-bdbb-b1e1820989e1-kube-api-access-zrthp\") pod \"e2e-inf-b-cbbb09-openshift-default-7657d596f6-c5cbc\" (UID: \"979c7e0c-c7e6-4e8a-bdbb-b1e1820989e1\") " pod="openshift-ingress/e2e-inf-b-cbbb09-openshift-default-7657d596f6-c5cbc"
Sep 11 17:31:38.730651 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:38.730624    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/979c7e0c-c7e6-4e8a-bdbb-b1e1820989e1-istio-data\") pod \"e2e-inf-b-cbbb09-openshift-default-7657d596f6-c5cbc\" (UID: \"979c7e0c-c7e6-4e8a-bdbb-b1e1820989e1\") " pod="openshift-ingress/e2e-inf-b-cbbb09-openshift-default-7657d596f6-c5cbc"
Sep 11 17:31:38.730685 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:38.730662    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/979c7e0c-c7e6-4e8a-bdbb-b1e1820989e1-credential-socket\") pod \"e2e-inf-b-cbbb09-openshift-default-7657d596f6-c5cbc\" (UID: \"979c7e0c-c7e6-4e8a-bdbb-b1e1820989e1\") " pod="openshift-ingress/e2e-inf-b-cbbb09-openshift-default-7657d596f6-c5cbc"
Sep 11 17:31:38.730736 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:38.730725    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/979c7e0c-c7e6-4e8a-bdbb-b1e1820989e1-istiod-ca-cert\") pod \"e2e-inf-b-cbbb09-openshift-default-7657d596f6-c5cbc\" (UID: \"979c7e0c-c7e6-4e8a-bdbb-b1e1820989e1\") " pod="openshift-ingress/e2e-inf-b-cbbb09-openshift-default-7657d596f6-c5cbc"
Sep 11 17:31:38.731059 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:38.731033    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/979c7e0c-c7e6-4e8a-bdbb-b1e1820989e1-istio-data\") pod \"e2e-inf-b-cbbb09-openshift-default-7657d596f6-c5cbc\" (UID: \"979c7e0c-c7e6-4e8a-bdbb-b1e1820989e1\") " pod="openshift-ingress/e2e-inf-b-cbbb09-openshift-default-7657d596f6-c5cbc"
Sep 11 17:31:38.731401 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:38.731377    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/979c7e0c-c7e6-4e8a-bdbb-b1e1820989e1-credential-socket\") pod \"e2e-inf-b-cbbb09-openshift-default-7657d596f6-c5cbc\" (UID: \"979c7e0c-c7e6-4e8a-bdbb-b1e1820989e1\") " pod="openshift-ingress/e2e-inf-b-cbbb09-openshift-default-7657d596f6-c5cbc"
Sep 11 17:31:38.731658 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:38.731640    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/979c7e0c-c7e6-4e8a-bdbb-b1e1820989e1-istiod-ca-cert\") pod \"e2e-inf-b-cbbb09-openshift-default-7657d596f6-c5cbc\" (UID: \"979c7e0c-c7e6-4e8a-bdbb-b1e1820989e1\") " pod="openshift-ingress/e2e-inf-b-cbbb09-openshift-default-7657d596f6-c5cbc"
Sep 11 17:31:38.732857 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:38.732840    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/979c7e0c-c7e6-4e8a-bdbb-b1e1820989e1-istio-podinfo\") pod \"e2e-inf-b-cbbb09-openshift-default-7657d596f6-c5cbc\" (UID: \"979c7e0c-c7e6-4e8a-bdbb-b1e1820989e1\") " pod="openshift-ingress/e2e-inf-b-cbbb09-openshift-default-7657d596f6-c5cbc"
Sep 11 17:31:38.733515 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:38.733495    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/979c7e0c-c7e6-4e8a-bdbb-b1e1820989e1-istio-envoy\") pod \"e2e-inf-b-cbbb09-openshift-default-7657d596f6-c5cbc\" (UID: \"979c7e0c-c7e6-4e8a-bdbb-b1e1820989e1\") " pod="openshift-ingress/e2e-inf-b-cbbb09-openshift-default-7657d596f6-c5cbc"
Sep 11 17:31:38.735395 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:38.735373    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:31:38.758796 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:38.758739    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/979c7e0c-c7e6-4e8a-bdbb-b1e1820989e1-istio-token\") pod \"e2e-inf-b-cbbb09-openshift-default-7657d596f6-c5cbc\" (UID: \"979c7e0c-c7e6-4e8a-bdbb-b1e1820989e1\") " pod="openshift-ingress/e2e-inf-b-cbbb09-openshift-default-7657d596f6-c5cbc"
Sep 11 17:31:38.762370 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:38.762344    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zrthp\" (UniqueName: \"kubernetes.io/projected/979c7e0c-c7e6-4e8a-bdbb-b1e1820989e1-kube-api-access-zrthp\") pod \"e2e-inf-b-cbbb09-openshift-default-7657d596f6-c5cbc\" (UID: \"979c7e0c-c7e6-4e8a-bdbb-b1e1820989e1\") " pod="openshift-ingress/e2e-inf-b-cbbb09-openshift-default-7657d596f6-c5cbc"
Sep 11 17:31:38.770129 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:38.769945    2552 patch_prober.go:28] interesting pod/e2e-worker-w4-144bfa-openshift-default-5dc996f7dd-ws7l7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body=
Sep 11 17:31:38.770129 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:38.769999    2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-144bfa-openshift-default-5dc996f7dd-ws7l7" podUID="ae170805-c6a4-469f-b280-8e7629749557" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused"
Sep 11 17:31:38.913371 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:38.913319    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:31:38.930194 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:38.930165    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-cbbb09-openshift-default-7657d596f6-c5cbc"
Sep 11 17:31:39.343302 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:39.339991    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-cbbb09-openshift-default-7657d596f6-c5cbc"]
Sep 11 17:31:39.352132 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:39.351188    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 17:31:39.352132 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:39.351273    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 17:31:39.352132 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:39.351325    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 17:31:39.641768 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:39.641727    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:31:39.876538 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:39.876399    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-144bfa-openshift-default-5dc996f7dd-ws7l7"
Sep 11 17:31:39.877013 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:39.876836    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-144bfa-openshift-default-5dc996f7dd-ws7l7"
Sep 11 17:31:39.878317 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:39.878280    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-144bfa-openshift-default-5dc996f7dd-ws7l7"
Sep 11 17:31:40.226824 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:40.226789    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-cbbb09-openshift-default-7657d596f6-c5cbc" event={"ID":"979c7e0c-c7e6-4e8a-bdbb-b1e1820989e1","Type":"ContainerStarted","Data":"561a56db8923b8fa05fba1f57925e479917969a0619ce3b8a72157f5f69a6942"}
Sep 11 17:31:40.227037 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:40.226839    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-cbbb09-openshift-default-7657d596f6-c5cbc" event={"ID":"979c7e0c-c7e6-4e8a-bdbb-b1e1820989e1","Type":"ContainerStarted","Data":"ba38e1ea14e554b55bd241b0e4cb7590799974db8d345ec0e71d38fcf910bce9"}
Sep 11 17:31:40.245644 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:40.245577    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-cbbb09-openshift-default-7657d596f6-c5cbc" podStartSLOduration=2.245541826 podStartE2EDuration="2.245541826s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:31:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:31:40.243980126 +0000 UTC m=+794.765271648" watchObservedRunningTime="2026-09-11 17:31:40.245541826 +0000 UTC m=+794.766833348"
Sep 11 17:31:40.348943 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:40.348904    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:31:40.931209 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:40.931177    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-cbbb09-openshift-default-7657d596f6-c5cbc"
Sep 11 17:31:40.974464 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:40.974432    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-cbbb09-openshift-default-7657d596f6-c5cbc"
Sep 11 17:31:41.061855 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:41.061812    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:31:41.235056 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:41.234967    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-cbbb09-openshift-default-7657d596f6-c5cbc"
Sep 11 17:31:41.242532 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:41.242492    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-cbbb09-openshift-default-7657d596f6-c5cbc"
Sep 11 17:31:41.830868 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:41.830825    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:31:42.193084 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:42.193003    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:31:42.287326 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:42.287288    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:31:42.667099 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:42.667067    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:31:43.706446 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:43.706398    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:31:44.453205 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:44.453149    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:31:44.873939 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:44.873906    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:31:45.215157 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:45.215045    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:31:45.320515 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:45.320475    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:31:45.421067 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:45.421034    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:31:45.770908 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:45.770868    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:31:45.865309 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:45.865266    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:31:46.166934 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:46.166886    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:31:46.995534 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:46.995448    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:31:47.287203 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:47.287170    2552 generic.go:356] "Generic (PLEG): container finished" podID="8d4763e0-c0ce-4b9a-8a05-ff4f718b96f0" containerID="646c4823f006ad259a058824a1c60f5aab0dddea84724b709ab153aec5eeb7c7" exitCode=0
Sep 11 17:31:47.287635 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:47.287251    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-49e398-openshift-default-66cbbd759d-nf4cx" event={"ID":"8d4763e0-c0ce-4b9a-8a05-ff4f718b96f0","Type":"ContainerDied","Data":"646c4823f006ad259a058824a1c60f5aab0dddea84724b709ab153aec5eeb7c7"}
Sep 11 17:31:47.287758 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:47.287742    2552 scope.go:117] "RemoveContainer" containerID="646c4823f006ad259a058824a1c60f5aab0dddea84724b709ab153aec5eeb7c7"
Sep 11 17:31:47.288971 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:47.288939    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 17:31:47.289082 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:47.289025    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 17:31:47.289082 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:47.289074    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 17:31:47.682242 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:47.682209    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:31:48.294313 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:48.294136    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:31:48.298900 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:48.298533    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-49e398-openshift-default-66cbbd759d-nf4cx" event={"ID":"8d4763e0-c0ce-4b9a-8a05-ff4f718b96f0","Type":"ContainerStarted","Data":"1a3257d07557fccd8f9c1155dcbb8e3e87106db299bf99a2df5f37bebe89299d"}
Sep 11 17:31:48.390796 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:48.390763    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:31:48.491168 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:48.491135    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:31:48.557944 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:48.557818    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:31:48.710892 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:48.710851    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-23a9cc4a-openshift-default-775b7dcc48-2dxkx"]
Sep 11 17:31:48.711177 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:48.711145    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-23a9cc4a-openshift-default-775b7dcc48-2dxkx" podUID="145c6962-f155-4a4d-bce7-b030b7955a6e" containerName="istio-proxy" containerID="cri-o://1b5547fdc0c0c73c0ee37e44ebbaf432f8ed65de08f18c84580ed743be84fbdd" gracePeriod=30
Sep 11 17:31:48.731908 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:48.731872    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-autores-49e398-openshift-default-66cbbd759d-nf4cx"
Sep 11 17:31:48.733021 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:48.732994    2552 patch_prober.go:28] interesting pod/e2e-autores-49e398-openshift-default-66cbbd759d-nf4cx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body=
Sep 11 17:31:48.733174 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:48.733038    2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-49e398-openshift-default-66cbbd759d-nf4cx" podUID="8d4763e0-c0ce-4b9a-8a05-ff4f718b96f0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused"
Sep 11 17:31:48.750136 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:48.745653    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:31:48.879062 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:48.879018    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:31:49.022598 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:49.022563    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-autores-49e398-openshift-default-66cbbd759d-nf4cx"
Sep 11 17:31:49.167937 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:49.167836    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:31:49.273663 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:49.273627    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:31:49.373899 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:49.373859    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:31:49.680156 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:49.679976    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-d19de295-gw-openshift-default-6845bbcd48-9j7gs"]
Sep 11 17:31:49.681764 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:49.680684    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:31:49.684069 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:49.684048    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-d19de295-gw-openshift-default-6845bbcd48-9j7gs"
Sep 11 17:31:49.684254 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:49.684232    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-d19de295-gw-openshift-default-6845bbcd48-9j7gs"]
Sep 11 17:31:49.686424 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:49.686402    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-d19de295-gw-openshift-default-dockercfg-wx9l2\""
Sep 11 17:31:49.731850 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:49.731815    2552 patch_prober.go:28] interesting pod/e2e-autores-49e398-openshift-default-66cbbd759d-nf4cx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body=
Sep 11 17:31:49.732034 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:49.731879    2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-49e398-openshift-default-66cbbd759d-nf4cx" podUID="8d4763e0-c0ce-4b9a-8a05-ff4f718b96f0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused"
Sep 11 17:31:49.793884 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:49.793837    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8wns2\" (UniqueName: \"kubernetes.io/projected/833f77b2-5c53-490b-babb-03d2e6f6d4b8-kube-api-access-8wns2\") pod \"e2e-migrate-d19de295-gw-openshift-default-6845bbcd48-9j7gs\" (UID: \"833f77b2-5c53-490b-babb-03d2e6f6d4b8\") " pod="openshift-ingress/e2e-migrate-d19de295-gw-openshift-default-6845bbcd48-9j7gs"
Sep 11 17:31:49.794060 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:49.793933    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/833f77b2-5c53-490b-babb-03d2e6f6d4b8-istio-data\") pod \"e2e-migrate-d19de295-gw-openshift-default-6845bbcd48-9j7gs\" (UID: \"833f77b2-5c53-490b-babb-03d2e6f6d4b8\") " pod="openshift-ingress/e2e-migrate-d19de295-gw-openshift-default-6845bbcd48-9j7gs"
Sep 11 17:31:49.794060 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:49.793991    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/833f77b2-5c53-490b-babb-03d2e6f6d4b8-istiod-ca-cert\") pod \"e2e-migrate-d19de295-gw-openshift-default-6845bbcd48-9j7gs\" (UID: \"833f77b2-5c53-490b-babb-03d2e6f6d4b8\") " pod="openshift-ingress/e2e-migrate-d19de295-gw-openshift-default-6845bbcd48-9j7gs"
Sep 11 17:31:49.794178 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:49.794089    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/833f77b2-5c53-490b-babb-03d2e6f6d4b8-workload-socket\") pod \"e2e-migrate-d19de295-gw-openshift-default-6845bbcd48-9j7gs\" (UID: \"833f77b2-5c53-490b-babb-03d2e6f6d4b8\") " pod="openshift-ingress/e2e-migrate-d19de295-gw-openshift-default-6845bbcd48-9j7gs"
Sep 11 17:31:49.794178 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:49.794137    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/833f77b2-5c53-490b-babb-03d2e6f6d4b8-workload-certs\") pod \"e2e-migrate-d19de295-gw-openshift-default-6845bbcd48-9j7gs\" (UID: \"833f77b2-5c53-490b-babb-03d2e6f6d4b8\") " pod="openshift-ingress/e2e-migrate-d19de295-gw-openshift-default-6845bbcd48-9j7gs"
Sep 11 17:31:49.794178 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:49.794162    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/833f77b2-5c53-490b-babb-03d2e6f6d4b8-istio-envoy\") pod \"e2e-migrate-d19de295-gw-openshift-default-6845bbcd48-9j7gs\" (UID: \"833f77b2-5c53-490b-babb-03d2e6f6d4b8\") " pod="openshift-ingress/e2e-migrate-d19de295-gw-openshift-default-6845bbcd48-9j7gs"
Sep 11 17:31:49.794331 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:49.794225    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/833f77b2-5c53-490b-babb-03d2e6f6d4b8-istio-token\") pod \"e2e-migrate-d19de295-gw-openshift-default-6845bbcd48-9j7gs\" (UID: \"833f77b2-5c53-490b-babb-03d2e6f6d4b8\") " pod="openshift-ingress/e2e-migrate-d19de295-gw-openshift-default-6845bbcd48-9j7gs"
Sep 11 17:31:49.794331 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:49.794292    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/833f77b2-5c53-490b-babb-03d2e6f6d4b8-istio-podinfo\") pod \"e2e-migrate-d19de295-gw-openshift-default-6845bbcd48-9j7gs\" (UID: \"833f77b2-5c53-490b-babb-03d2e6f6d4b8\") " pod="openshift-ingress/e2e-migrate-d19de295-gw-openshift-default-6845bbcd48-9j7gs"
Sep 11 17:31:49.794435 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:49.794346    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/833f77b2-5c53-490b-babb-03d2e6f6d4b8-credential-socket\") pod \"e2e-migrate-d19de295-gw-openshift-default-6845bbcd48-9j7gs\" (UID: \"833f77b2-5c53-490b-babb-03d2e6f6d4b8\") " pod="openshift-ingress/e2e-migrate-d19de295-gw-openshift-default-6845bbcd48-9j7gs"
Sep 11 17:31:49.844930 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:49.844883    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:31:49.895453 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:49.895419    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/833f77b2-5c53-490b-babb-03d2e6f6d4b8-workload-certs\") pod \"e2e-migrate-d19de295-gw-openshift-default-6845bbcd48-9j7gs\" (UID: \"833f77b2-5c53-490b-babb-03d2e6f6d4b8\") " pod="openshift-ingress/e2e-migrate-d19de295-gw-openshift-default-6845bbcd48-9j7gs"
Sep 11 17:31:49.895453 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:49.895455    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/833f77b2-5c53-490b-babb-03d2e6f6d4b8-istio-envoy\") pod \"e2e-migrate-d19de295-gw-openshift-default-6845bbcd48-9j7gs\" (UID: \"833f77b2-5c53-490b-babb-03d2e6f6d4b8\") " pod="openshift-ingress/e2e-migrate-d19de295-gw-openshift-default-6845bbcd48-9j7gs"
Sep 11 17:31:49.895730 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:49.895484    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/833f77b2-5c53-490b-babb-03d2e6f6d4b8-istio-token\") pod \"e2e-migrate-d19de295-gw-openshift-default-6845bbcd48-9j7gs\" (UID: \"833f77b2-5c53-490b-babb-03d2e6f6d4b8\") " pod="openshift-ingress/e2e-migrate-d19de295-gw-openshift-default-6845bbcd48-9j7gs"
Sep 11 17:31:49.895730 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:49.895509    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/833f77b2-5c53-490b-babb-03d2e6f6d4b8-istio-podinfo\") pod \"e2e-migrate-d19de295-gw-openshift-default-6845bbcd48-9j7gs\" (UID: \"833f77b2-5c53-490b-babb-03d2e6f6d4b8\") " pod="openshift-ingress/e2e-migrate-d19de295-gw-openshift-default-6845bbcd48-9j7gs"
Sep 11 17:31:49.895730 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:49.895544    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/833f77b2-5c53-490b-babb-03d2e6f6d4b8-credential-socket\") pod \"e2e-migrate-d19de295-gw-openshift-default-6845bbcd48-9j7gs\" (UID: \"833f77b2-5c53-490b-babb-03d2e6f6d4b8\") " pod="openshift-ingress/e2e-migrate-d19de295-gw-openshift-default-6845bbcd48-9j7gs"
Sep 11 17:31:49.895730 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:49.895589    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8wns2\" (UniqueName: \"kubernetes.io/projected/833f77b2-5c53-490b-babb-03d2e6f6d4b8-kube-api-access-8wns2\") pod \"e2e-migrate-d19de295-gw-openshift-default-6845bbcd48-9j7gs\" (UID: \"833f77b2-5c53-490b-babb-03d2e6f6d4b8\") " pod="openshift-ingress/e2e-migrate-d19de295-gw-openshift-default-6845bbcd48-9j7gs"
Sep 11 17:31:49.895730 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:49.895645    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/833f77b2-5c53-490b-babb-03d2e6f6d4b8-istio-data\") pod \"e2e-migrate-d19de295-gw-openshift-default-6845bbcd48-9j7gs\" (UID: \"833f77b2-5c53-490b-babb-03d2e6f6d4b8\") " pod="openshift-ingress/e2e-migrate-d19de295-gw-openshift-default-6845bbcd48-9j7gs"
Sep 11 17:31:49.895730 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:49.895697    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/833f77b2-5c53-490b-babb-03d2e6f6d4b8-istiod-ca-cert\") pod \"e2e-migrate-d19de295-gw-openshift-default-6845bbcd48-9j7gs\" (UID: \"833f77b2-5c53-490b-babb-03d2e6f6d4b8\") " pod="openshift-ingress/e2e-migrate-d19de295-gw-openshift-default-6845bbcd48-9j7gs"
Sep 11 17:31:49.896097 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:49.895794    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/833f77b2-5c53-490b-babb-03d2e6f6d4b8-workload-socket\") pod \"e2e-migrate-d19de295-gw-openshift-default-6845bbcd48-9j7gs\" (UID: \"833f77b2-5c53-490b-babb-03d2e6f6d4b8\") " pod="openshift-ingress/e2e-migrate-d19de295-gw-openshift-default-6845bbcd48-9j7gs"
Sep 11 17:31:49.896097 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:49.895914    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/833f77b2-5c53-490b-babb-03d2e6f6d4b8-workload-certs\") pod \"e2e-migrate-d19de295-gw-openshift-default-6845bbcd48-9j7gs\" (UID: \"833f77b2-5c53-490b-babb-03d2e6f6d4b8\") " pod="openshift-ingress/e2e-migrate-d19de295-gw-openshift-default-6845bbcd48-9j7gs"
Sep 11 17:31:49.896235 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:49.896156    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/833f77b2-5c53-490b-babb-03d2e6f6d4b8-credential-socket\") pod \"e2e-migrate-d19de295-gw-openshift-default-6845bbcd48-9j7gs\" (UID: \"833f77b2-5c53-490b-babb-03d2e6f6d4b8\") " pod="openshift-ingress/e2e-migrate-d19de295-gw-openshift-default-6845bbcd48-9j7gs"
Sep 11 17:31:49.896435 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:49.896412    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/833f77b2-5c53-490b-babb-03d2e6f6d4b8-istio-data\") pod \"e2e-migrate-d19de295-gw-openshift-default-6845bbcd48-9j7gs\" (UID: \"833f77b2-5c53-490b-babb-03d2e6f6d4b8\") " pod="openshift-ingress/e2e-migrate-d19de295-gw-openshift-default-6845bbcd48-9j7gs"
Sep 11 17:31:49.896523 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:49.896494    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/833f77b2-5c53-490b-babb-03d2e6f6d4b8-workload-socket\") pod \"e2e-migrate-d19de295-gw-openshift-default-6845bbcd48-9j7gs\" (UID: \"833f77b2-5c53-490b-babb-03d2e6f6d4b8\") " pod="openshift-ingress/e2e-migrate-d19de295-gw-openshift-default-6845bbcd48-9j7gs"
Sep 11 17:31:49.896613 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:49.896593    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/833f77b2-5c53-490b-babb-03d2e6f6d4b8-istiod-ca-cert\") pod \"e2e-migrate-d19de295-gw-openshift-default-6845bbcd48-9j7gs\" (UID: \"833f77b2-5c53-490b-babb-03d2e6f6d4b8\") " pod="openshift-ingress/e2e-migrate-d19de295-gw-openshift-default-6845bbcd48-9j7gs"
Sep 11 17:31:49.898061 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:49.898039    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/833f77b2-5c53-490b-babb-03d2e6f6d4b8-istio-envoy\") pod \"e2e-migrate-d19de295-gw-openshift-default-6845bbcd48-9j7gs\" (UID: \"833f77b2-5c53-490b-babb-03d2e6f6d4b8\") " pod="openshift-ingress/e2e-migrate-d19de295-gw-openshift-default-6845bbcd48-9j7gs"
Sep 11 17:31:49.898407 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:49.898388    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/833f77b2-5c53-490b-babb-03d2e6f6d4b8-istio-podinfo\") pod \"e2e-migrate-d19de295-gw-openshift-default-6845bbcd48-9j7gs\" (UID: \"833f77b2-5c53-490b-babb-03d2e6f6d4b8\") " pod="openshift-ingress/e2e-migrate-d19de295-gw-openshift-default-6845bbcd48-9j7gs"
Sep 11 17:31:49.904906 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:49.904868    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/833f77b2-5c53-490b-babb-03d2e6f6d4b8-istio-token\") pod \"e2e-migrate-d19de295-gw-openshift-default-6845bbcd48-9j7gs\" (UID: \"833f77b2-5c53-490b-babb-03d2e6f6d4b8\") " pod="openshift-ingress/e2e-migrate-d19de295-gw-openshift-default-6845bbcd48-9j7gs"
Sep 11 17:31:49.904906 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:49.904893    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8wns2\" (UniqueName: \"kubernetes.io/projected/833f77b2-5c53-490b-babb-03d2e6f6d4b8-kube-api-access-8wns2\") pod \"e2e-migrate-d19de295-gw-openshift-default-6845bbcd48-9j7gs\" (UID: \"833f77b2-5c53-490b-babb-03d2e6f6d4b8\") " pod="openshift-ingress/e2e-migrate-d19de295-gw-openshift-default-6845bbcd48-9j7gs"
Sep 11 17:31:49.996460 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:49.996358    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-d19de295-gw-openshift-default-6845bbcd48-9j7gs"
Sep 11 17:31:50.028236 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:50.028205    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:31:50.337917 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:50.337709    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-d19de295-gw-openshift-default-6845bbcd48-9j7gs"]
Sep 11 17:31:50.346283 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:50.346247    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 17:31:50.346391 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:50.346335    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 17:31:50.346391 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:50.346377    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 17:31:50.677733 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:50.677684    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:31:50.731993 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:50.731645    2552 patch_prober.go:28] interesting pod/e2e-autores-49e398-openshift-default-66cbbd759d-nf4cx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body=
Sep 11 17:31:50.731993 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:50.731707    2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-49e398-openshift-default-66cbbd759d-nf4cx" podUID="8d4763e0-c0ce-4b9a-8a05-ff4f718b96f0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused"
Sep 11 17:31:51.041663 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:51.041560    2552 patch_prober.go:28] interesting pod/e2e-ait-23a9cc4a-openshift-default-775b7dcc48-2dxkx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Sep 11 17:31:51.041663 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:51.041631    2552 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-23a9cc4a-openshift-default-775b7dcc48-2dxkx" podUID="145c6962-f155-4a4d-bce7-b030b7955a6e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Sep 11 17:31:51.320995 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:51.320893    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-d19de295-gw-openshift-default-6845bbcd48-9j7gs" event={"ID":"833f77b2-5c53-490b-babb-03d2e6f6d4b8","Type":"ContainerStarted","Data":"14bbee24d43322f8c763705e49ad4fea5ceb821f4196c84113f958ab5086e558"}
Sep 11 17:31:51.320995 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:51.320964    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-d19de295-gw-openshift-default-6845bbcd48-9j7gs" event={"ID":"833f77b2-5c53-490b-babb-03d2e6f6d4b8","Type":"ContainerStarted","Data":"04d11a7b8c88a853f26b6334a30701174582c738a7b5c1168aab0175fb7edf3a"}
Sep 11 17:31:51.322778 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:51.322751    2552 generic.go:356] "Generic (PLEG): container finished" podID="145c6962-f155-4a4d-bce7-b030b7955a6e" containerID="1b5547fdc0c0c73c0ee37e44ebbaf432f8ed65de08f18c84580ed743be84fbdd" exitCode=0
Sep 11 17:31:51.322900 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:51.322819    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-23a9cc4a-openshift-default-775b7dcc48-2dxkx" event={"ID":"145c6962-f155-4a4d-bce7-b030b7955a6e","Type":"ContainerDied","Data":"1b5547fdc0c0c73c0ee37e44ebbaf432f8ed65de08f18c84580ed743be84fbdd"}
Sep 11 17:31:51.341951 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:51.341876    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-d19de295-gw-openshift-default-6845bbcd48-9j7gs" podStartSLOduration=2.341854638 podStartE2EDuration="2.341854638s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:31:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:31:51.338655926 +0000 UTC m=+805.859947448" watchObservedRunningTime="2026-09-11 17:31:51.341854638 +0000 UTC m=+805.863146163"
Sep 11 17:31:51.358950 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:51.358902    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:31:51.617822 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:51.617797    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-23a9cc4a-openshift-default-775b7dcc48-2dxkx"
Sep 11 17:31:51.715567 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:51.715535    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/145c6962-f155-4a4d-bce7-b030b7955a6e-istio-token\") pod \"145c6962-f155-4a4d-bce7-b030b7955a6e\" (UID: \"145c6962-f155-4a4d-bce7-b030b7955a6e\") "
Sep 11 17:31:51.716044 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:51.715577    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/145c6962-f155-4a4d-bce7-b030b7955a6e-workload-certs\") pod \"145c6962-f155-4a4d-bce7-b030b7955a6e\" (UID: \"145c6962-f155-4a4d-bce7-b030b7955a6e\") "
Sep 11 17:31:51.716044 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:51.715600    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/145c6962-f155-4a4d-bce7-b030b7955a6e-istio-envoy\") pod \"145c6962-f155-4a4d-bce7-b030b7955a6e\" (UID: \"145c6962-f155-4a4d-bce7-b030b7955a6e\") "
Sep 11 17:31:51.716044 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:51.715625    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-txxpd\" (UniqueName: \"kubernetes.io/projected/145c6962-f155-4a4d-bce7-b030b7955a6e-kube-api-access-txxpd\") pod \"145c6962-f155-4a4d-bce7-b030b7955a6e\" (UID: \"145c6962-f155-4a4d-bce7-b030b7955a6e\") "
Sep 11 17:31:51.716044 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:51.715660    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/145c6962-f155-4a4d-bce7-b030b7955a6e-istio-podinfo\") pod \"145c6962-f155-4a4d-bce7-b030b7955a6e\" (UID: \"145c6962-f155-4a4d-bce7-b030b7955a6e\") "
Sep 11 17:31:51.716044 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:51.715689    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/145c6962-f155-4a4d-bce7-b030b7955a6e-credential-socket\") pod \"145c6962-f155-4a4d-bce7-b030b7955a6e\" (UID: \"145c6962-f155-4a4d-bce7-b030b7955a6e\") "
Sep 11 17:31:51.716044 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:51.715720    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/145c6962-f155-4a4d-bce7-b030b7955a6e-workload-socket\") pod \"145c6962-f155-4a4d-bce7-b030b7955a6e\" (UID: \"145c6962-f155-4a4d-bce7-b030b7955a6e\") "
Sep 11 17:31:51.716044 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:51.715761    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/145c6962-f155-4a4d-bce7-b030b7955a6e-istiod-ca-cert\") pod \"145c6962-f155-4a4d-bce7-b030b7955a6e\" (UID: \"145c6962-f155-4a4d-bce7-b030b7955a6e\") "
Sep 11 17:31:51.716044 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:51.715790    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/145c6962-f155-4a4d-bce7-b030b7955a6e-istio-data\") pod \"145c6962-f155-4a4d-bce7-b030b7955a6e\" (UID: \"145c6962-f155-4a4d-bce7-b030b7955a6e\") "
Sep 11 17:31:51.716488 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:51.716214    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/145c6962-f155-4a4d-bce7-b030b7955a6e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "145c6962-f155-4a4d-bce7-b030b7955a6e" (UID: "145c6962-f155-4a4d-bce7-b030b7955a6e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:31:51.716488 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:51.716302    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/145c6962-f155-4a4d-bce7-b030b7955a6e-istio-data" (OuterVolumeSpecName: "istio-data") pod "145c6962-f155-4a4d-bce7-b030b7955a6e" (UID: "145c6962-f155-4a4d-bce7-b030b7955a6e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:31:51.716488 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:51.716294    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/145c6962-f155-4a4d-bce7-b030b7955a6e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "145c6962-f155-4a4d-bce7-b030b7955a6e" (UID: "145c6962-f155-4a4d-bce7-b030b7955a6e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:31:51.716488 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:51.716372    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/145c6962-f155-4a4d-bce7-b030b7955a6e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "145c6962-f155-4a4d-bce7-b030b7955a6e" (UID: "145c6962-f155-4a4d-bce7-b030b7955a6e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:31:51.716488 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:51.716375    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/145c6962-f155-4a4d-bce7-b030b7955a6e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "145c6962-f155-4a4d-bce7-b030b7955a6e" (UID: "145c6962-f155-4a4d-bce7-b030b7955a6e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 17:31:51.718306 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:51.718280    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/145c6962-f155-4a4d-bce7-b030b7955a6e-kube-api-access-txxpd" (OuterVolumeSpecName: "kube-api-access-txxpd") pod "145c6962-f155-4a4d-bce7-b030b7955a6e" (UID: "145c6962-f155-4a4d-bce7-b030b7955a6e"). InnerVolumeSpecName "kube-api-access-txxpd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:31:51.718439 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:51.718419    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/145c6962-f155-4a4d-bce7-b030b7955a6e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "145c6962-f155-4a4d-bce7-b030b7955a6e" (UID: "145c6962-f155-4a4d-bce7-b030b7955a6e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 17:31:51.718982 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:51.718962    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/145c6962-f155-4a4d-bce7-b030b7955a6e-istio-token" (OuterVolumeSpecName: "istio-token") pod "145c6962-f155-4a4d-bce7-b030b7955a6e" (UID: "145c6962-f155-4a4d-bce7-b030b7955a6e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:31:51.719073 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:51.719057    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/145c6962-f155-4a4d-bce7-b030b7955a6e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "145c6962-f155-4a4d-bce7-b030b7955a6e" (UID: "145c6962-f155-4a4d-bce7-b030b7955a6e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:31:51.731486 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:51.731456    2552 patch_prober.go:28] interesting pod/e2e-autores-49e398-openshift-default-66cbbd759d-nf4cx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body=
Sep 11 17:31:51.731598 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:51.731514    2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-49e398-openshift-default-66cbbd759d-nf4cx" podUID="8d4763e0-c0ce-4b9a-8a05-ff4f718b96f0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused"
Sep 11 17:31:51.817463 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:51.817432    2552 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/145c6962-f155-4a4d-bce7-b030b7955a6e-workload-socket\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:31:51.817463 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:51.817459    2552 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/145c6962-f155-4a4d-bce7-b030b7955a6e-istiod-ca-cert\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:31:51.817463 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:51.817468    2552 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/145c6962-f155-4a4d-bce7-b030b7955a6e-istio-data\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:31:51.817698 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:51.817476    2552 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/145c6962-f155-4a4d-bce7-b030b7955a6e-istio-token\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:31:51.817698 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:51.817484    2552 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/145c6962-f155-4a4d-bce7-b030b7955a6e-workload-certs\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:31:51.817698 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:51.817492    2552 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/145c6962-f155-4a4d-bce7-b030b7955a6e-istio-envoy\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:31:51.817698 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:51.817499    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-txxpd\" (UniqueName: \"kubernetes.io/projected/145c6962-f155-4a4d-bce7-b030b7955a6e-kube-api-access-txxpd\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:31:51.817698 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:51.817507    2552 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/145c6962-f155-4a4d-bce7-b030b7955a6e-istio-podinfo\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:31:51.817698 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:51.817518    2552 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/145c6962-f155-4a4d-bce7-b030b7955a6e-credential-socket\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:31:51.997347 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:51.997296    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-d19de295-gw-openshift-default-6845bbcd48-9j7gs"
Sep 11 17:31:52.002725 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:52.002697    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-d19de295-gw-openshift-default-6845bbcd48-9j7gs"
Sep 11 17:31:52.231381 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:52.231284    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:31:52.330783 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:52.330750    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-23a9cc4a-openshift-default-775b7dcc48-2dxkx"
Sep 11 17:31:52.330783 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:52.330776    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-23a9cc4a-openshift-default-775b7dcc48-2dxkx" event={"ID":"145c6962-f155-4a4d-bce7-b030b7955a6e","Type":"ContainerDied","Data":"ce1ce68f84ffc3721527e6d2f5dad2f04a4e446eab3a8723dc28a40b002c8a26"}
Sep 11 17:31:52.330993 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:52.330843    2552 scope.go:117] "RemoveContainer" containerID="1b5547fdc0c0c73c0ee37e44ebbaf432f8ed65de08f18c84580ed743be84fbdd"
Sep 11 17:31:52.331222 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:52.331195    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-d19de295-gw-openshift-default-6845bbcd48-9j7gs"
Sep 11 17:31:52.332473 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:52.332452    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-d19de295-gw-openshift-default-6845bbcd48-9j7gs"
Sep 11 17:31:52.370578 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:52.370547    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-23a9cc4a-openshift-default-775b7dcc48-2dxkx"]
Sep 11 17:31:52.378452 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:52.378416    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-23a9cc4a-openshift-default-775b7dcc48-2dxkx"]
Sep 11 17:31:52.732247 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:52.732206    2552 patch_prober.go:28] interesting pod/e2e-autores-49e398-openshift-default-66cbbd759d-nf4cx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body=
Sep 11 17:31:52.732790 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:52.732286    2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-49e398-openshift-default-66cbbd759d-nf4cx" podUID="8d4763e0-c0ce-4b9a-8a05-ff4f718b96f0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused"
Sep 11 17:31:53.073774 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:53.073611    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:31:53.194852 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:53.194816    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:31:53.735374 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:53.735342    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-autores-49e398-openshift-default-66cbbd759d-nf4cx"
Sep 11 17:31:53.736710 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:53.736685    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-autores-49e398-openshift-default-66cbbd759d-nf4cx"
Sep 11 17:31:53.948136 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:53.948083    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:31:53.950922 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:53.950890    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="145c6962-f155-4a4d-bce7-b030b7955a6e" path="/var/lib/kubelet/pods/145c6962-f155-4a4d-bce7-b030b7955a6e/volumes"
Sep 11 17:31:55.009902 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:55.009861    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:31:55.110097 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:55.110061    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:31:55.366657 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:55.360361    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:31:55.460167 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:55.460132    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:31:55.591631 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:55.591588    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:31:55.700497 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:55.700401    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:31:55.809768 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:55.809715    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:31:55.909534 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:55.909500    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:31:56.728034 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:56.727987    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:31:56.900205 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:56.900168    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:31:57.096973 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:57.096947    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:31:57.441446 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:57.441341    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:31:58.260462 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:58.260430    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:31:58.925338 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:58.925308    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:31:59.695883 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:31:59.695842    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:32:01.671314 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:01.671278    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:32:01.771857 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:01.771818    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:32:01.871635 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:01.871602    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:32:02.012162 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:02.012058    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:32:03.421252 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:03.421175    2552 generic.go:356] "Generic (PLEG): container finished" podID="979c7e0c-c7e6-4e8a-bdbb-b1e1820989e1" containerID="561a56db8923b8fa05fba1f57925e479917969a0619ce3b8a72157f5f69a6942" exitCode=0
Sep 11 17:32:03.421252 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:03.421212    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-cbbb09-openshift-default-7657d596f6-c5cbc" event={"ID":"979c7e0c-c7e6-4e8a-bdbb-b1e1820989e1","Type":"ContainerDied","Data":"561a56db8923b8fa05fba1f57925e479917969a0619ce3b8a72157f5f69a6942"}
Sep 11 17:32:03.421692 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:03.421582    2552 scope.go:117] "RemoveContainer" containerID="561a56db8923b8fa05fba1f57925e479917969a0619ce3b8a72157f5f69a6942"
Sep 11 17:32:03.422734 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:03.422706    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 17:32:03.422836 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:03.422771    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 17:32:03.422836 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:03.422803    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 17:32:04.429077 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:04.429040    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-cbbb09-openshift-default-7657d596f6-c5cbc" event={"ID":"979c7e0c-c7e6-4e8a-bdbb-b1e1820989e1","Type":"ContainerStarted","Data":"d4aa3c470f2ec9edac7d30f766e1feb0ae7fa0cea33d49063d0da23b97c28cbf"}
Sep 11 17:32:04.931304 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:04.931269    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-cbbb09-openshift-default-7657d596f6-c5cbc"
Sep 11 17:32:04.931496 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:04.931433    2552 patch_prober.go:28] interesting pod/e2e-inf-b-cbbb09-openshift-default-7657d596f6-c5cbc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body=
Sep 11 17:32:04.931567 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:04.931486    2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-cbbb09-openshift-default-7657d596f6-c5cbc" podUID="979c7e0c-c7e6-4e8a-bdbb-b1e1820989e1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused"
Sep 11 17:32:05.930505 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:05.930473    2552 patch_prober.go:28] interesting pod/e2e-inf-b-cbbb09-openshift-default-7657d596f6-c5cbc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body=
Sep 11 17:32:05.930906 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:05.930525    2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-cbbb09-openshift-default-7657d596f6-c5cbc" podUID="979c7e0c-c7e6-4e8a-bdbb-b1e1820989e1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused"
Sep 11 17:32:06.169057 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:06.169019    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:32:06.930603 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:06.930567    2552 patch_prober.go:28] interesting pod/e2e-inf-b-cbbb09-openshift-default-7657d596f6-c5cbc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body=
Sep 11 17:32:06.931200 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:06.931170    2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-cbbb09-openshift-default-7657d596f6-c5cbc" podUID="979c7e0c-c7e6-4e8a-bdbb-b1e1820989e1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused"
Sep 11 17:32:07.931237 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:07.931187    2552 patch_prober.go:28] interesting pod/e2e-inf-b-cbbb09-openshift-default-7657d596f6-c5cbc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body=
Sep 11 17:32:07.931706 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:07.931314    2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-cbbb09-openshift-default-7657d596f6-c5cbc" podUID="979c7e0c-c7e6-4e8a-bdbb-b1e1820989e1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused"
Sep 11 17:32:07.985809 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:07.985775    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:32:08.083396 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:08.083348    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:32:08.930641 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:08.930605    2552 patch_prober.go:28] interesting pod/e2e-inf-b-cbbb09-openshift-default-7657d596f6-c5cbc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body=
Sep 11 17:32:08.930851 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:08.930667    2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-cbbb09-openshift-default-7657d596f6-c5cbc" podUID="979c7e0c-c7e6-4e8a-bdbb-b1e1820989e1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused"
Sep 11 17:32:09.930616 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:09.930579    2552 patch_prober.go:28] interesting pod/e2e-inf-b-cbbb09-openshift-default-7657d596f6-c5cbc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body=
Sep 11 17:32:09.931144 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:09.931082    2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-cbbb09-openshift-default-7657d596f6-c5cbc" podUID="979c7e0c-c7e6-4e8a-bdbb-b1e1820989e1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused"
Sep 11 17:32:10.934616 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:10.934584    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-cbbb09-openshift-default-7657d596f6-c5cbc"
Sep 11 17:32:10.935246 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:10.935017    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-cbbb09-openshift-default-7657d596f6-c5cbc"
Sep 11 17:32:10.945661 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:10.945627    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-cbbb09-openshift-default-7657d596f6-c5cbc"
Sep 11 17:32:15.518711 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:15.518623    2552 generic.go:356] "Generic (PLEG): container finished" podID="833f77b2-5c53-490b-babb-03d2e6f6d4b8" containerID="14bbee24d43322f8c763705e49ad4fea5ceb821f4196c84113f958ab5086e558" exitCode=0
Sep 11 17:32:15.518711 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:15.518697    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-d19de295-gw-openshift-default-6845bbcd48-9j7gs" event={"ID":"833f77b2-5c53-490b-babb-03d2e6f6d4b8","Type":"ContainerDied","Data":"14bbee24d43322f8c763705e49ad4fea5ceb821f4196c84113f958ab5086e558"}
Sep 11 17:32:15.519174 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:15.519159    2552 scope.go:117] "RemoveContainer" containerID="14bbee24d43322f8c763705e49ad4fea5ceb821f4196c84113f958ab5086e558"
Sep 11 17:32:15.520359 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:15.520329    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 17:32:15.520472 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:15.520395    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 17:32:15.520472 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:15.520426    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 17:32:15.795140 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:15.795032    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:32:16.526585 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:16.526553    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-d19de295-gw-openshift-default-6845bbcd48-9j7gs" event={"ID":"833f77b2-5c53-490b-babb-03d2e6f6d4b8","Type":"ContainerStarted","Data":"4235b746f0de469fb0f00850cd952a463375dd1c460c8362644e66527307ce69"}
Sep 11 17:32:16.996879 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:16.996837    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-d19de295-gw-openshift-default-6845bbcd48-9j7gs"
Sep 11 17:32:16.997142 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:16.997081    2552 patch_prober.go:28] interesting pod/e2e-migrate-d19de295-gw-openshift-default-6845bbcd48-9j7gs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body=
Sep 11 17:32:16.997252 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:16.997162    2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-d19de295-gw-openshift-default-6845bbcd48-9j7gs" podUID="833f77b2-5c53-490b-babb-03d2e6f6d4b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused"
Sep 11 17:32:17.232786 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:17.232755    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:32:17.971562 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:17.971518    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:32:17.996879 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:17.996844    2552 patch_prober.go:28] interesting pod/e2e-migrate-d19de295-gw-openshift-default-6845bbcd48-9j7gs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body=
Sep 11 17:32:17.997052 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:17.996913    2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-d19de295-gw-openshift-default-6845bbcd48-9j7gs" podUID="833f77b2-5c53-490b-babb-03d2e6f6d4b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused"
Sep 11 17:32:18.729578 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:18.729535    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:32:18.997874 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:18.997786    2552 patch_prober.go:28] interesting pod/e2e-migrate-d19de295-gw-openshift-default-6845bbcd48-9j7gs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body=
Sep 11 17:32:18.997874 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:18.997849    2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-d19de295-gw-openshift-default-6845bbcd48-9j7gs" podUID="833f77b2-5c53-490b-babb-03d2e6f6d4b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused"
Sep 11 17:32:19.528199 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:19.528137    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:32:19.997520 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:19.997484    2552 patch_prober.go:28] interesting pod/e2e-migrate-d19de295-gw-openshift-default-6845bbcd48-9j7gs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body=
Sep 11 17:32:19.997703 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:19.997536    2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-d19de295-gw-openshift-default-6845bbcd48-9j7gs" podUID="833f77b2-5c53-490b-babb-03d2e6f6d4b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused"
Sep 11 17:32:20.433999 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:20.433943    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:32:20.997123 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:20.997081    2552 patch_prober.go:28] interesting pod/e2e-migrate-d19de295-gw-openshift-default-6845bbcd48-9j7gs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body=
Sep 11 17:32:20.997317 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:20.997192    2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-d19de295-gw-openshift-default-6845bbcd48-9j7gs" podUID="833f77b2-5c53-490b-babb-03d2e6f6d4b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused"
Sep 11 17:32:21.247830 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:21.247707    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:32:21.997859 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:21.997822    2552 patch_prober.go:28] interesting pod/e2e-migrate-d19de295-gw-openshift-default-6845bbcd48-9j7gs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body=
Sep 11 17:32:21.998359 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:21.997875    2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-d19de295-gw-openshift-default-6845bbcd48-9j7gs" podUID="833f77b2-5c53-490b-babb-03d2e6f6d4b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused"
Sep 11 17:32:22.055659 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:22.055625    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:32:22.331818 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:22.331731    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-d19de295-gw-openshift-default-6845bbcd48-9j7gs"
Sep 11 17:32:22.717854 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:22.717817    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:32:23.000858 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:23.000774    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-d19de295-gw-openshift-default-6845bbcd48-9j7gs"
Sep 11 17:32:23.001800 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:23.001779    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-d19de295-gw-openshift-default-6845bbcd48-9j7gs"
Sep 11 17:32:23.570701 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:23.570667    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:32:24.222828 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:24.222792    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:32:24.323375 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:24.323343    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:32:24.426061 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:24.426028    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:32:24.534245 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:24.532063    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:32:24.590989 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:24.590953    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-d19de295-gw-openshift-default-6845bbcd48-9j7gs"]
Sep 11 17:32:24.591347 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:24.591311    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-d19de295-gw-openshift-default-6845bbcd48-9j7gs" podUID="833f77b2-5c53-490b-babb-03d2e6f6d4b8" containerName="istio-proxy" containerID="cri-o://4235b746f0de469fb0f00850cd952a463375dd1c460c8362644e66527307ce69" gracePeriod=30
Sep 11 17:32:24.708462 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:24.708427    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:32:25.018461 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:25.018410    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:32:25.324323 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:25.324226    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:32:25.424102 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:25.424062    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:32:25.523583 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:25.523553    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:32:26.232695 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:26.232658    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:32:27.033957 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:27.033924    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:32:27.757217 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:27.757181    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:32:27.956022 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:27.955964    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:32:28.037693 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:28.037588    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:32:28.759635 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:28.759600    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:32:29.497878 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:29.497837    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-0ad997be-7978bfd4c5-j2whz"]
Sep 11 17:32:29.498515 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:29.498491    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="145c6962-f155-4a4d-bce7-b030b7955a6e" containerName="istio-proxy"
Sep 11 17:32:29.498515 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:29.498513    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="145c6962-f155-4a4d-bce7-b030b7955a6e" containerName="istio-proxy"
Sep 11 17:32:29.498666 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:29.498587    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="145c6962-f155-4a4d-bce7-b030b7955a6e" containerName="istio-proxy"
Sep 11 17:32:29.501361 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:29.501344    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0ad997be-7978bfd4c5-j2whz"
Sep 11 17:32:29.503547 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:29.503522    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-0ad997be-serving-cert\""
Sep 11 17:32:29.514725 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:29.514699    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-0ad997be-7978bfd4c5-j2whz"]
Sep 11 17:32:29.538803 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:29.538728    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:32:29.604421 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:29.604377    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:32:29.635551 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:29.635514    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5b41c1f9-bb4a-4a5c-b7f8-0747ea205953-openshift-service-ca\") pod \"maas-api-e2e-ait-0ad997be-7978bfd4c5-j2whz\" (UID: \"5b41c1f9-bb4a-4a5c-b7f8-0747ea205953\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0ad997be-7978bfd4c5-j2whz"
Sep 11 17:32:29.635833 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:29.635804    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5b41c1f9-bb4a-4a5c-b7f8-0747ea205953-metrics-certs\") pod \"maas-api-e2e-ait-0ad997be-7978bfd4c5-j2whz\" (UID: \"5b41c1f9-bb4a-4a5c-b7f8-0747ea205953\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0ad997be-7978bfd4c5-j2whz"
Sep 11 17:32:29.635981 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:29.635962    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5b41c1f9-bb4a-4a5c-b7f8-0747ea205953-maas-api-tls\") pod \"maas-api-e2e-ait-0ad997be-7978bfd4c5-j2whz\" (UID: \"5b41c1f9-bb4a-4a5c-b7f8-0747ea205953\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0ad997be-7978bfd4c5-j2whz"
Sep 11 17:32:29.636050 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:29.636009    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hhh65\" (UniqueName: \"kubernetes.io/projected/5b41c1f9-bb4a-4a5c-b7f8-0747ea205953-kube-api-access-hhh65\") pod \"maas-api-e2e-ait-0ad997be-7978bfd4c5-j2whz\" (UID: \"5b41c1f9-bb4a-4a5c-b7f8-0747ea205953\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0ad997be-7978bfd4c5-j2whz"
Sep 11 17:32:29.737532 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:29.737498    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5b41c1f9-bb4a-4a5c-b7f8-0747ea205953-openshift-service-ca\") pod \"maas-api-e2e-ait-0ad997be-7978bfd4c5-j2whz\" (UID: \"5b41c1f9-bb4a-4a5c-b7f8-0747ea205953\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0ad997be-7978bfd4c5-j2whz"
Sep 11 17:32:29.737665 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:29.737545    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5b41c1f9-bb4a-4a5c-b7f8-0747ea205953-metrics-certs\") pod \"maas-api-e2e-ait-0ad997be-7978bfd4c5-j2whz\" (UID: \"5b41c1f9-bb4a-4a5c-b7f8-0747ea205953\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0ad997be-7978bfd4c5-j2whz"
Sep 11 17:32:29.737665 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:29.737651    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5b41c1f9-bb4a-4a5c-b7f8-0747ea205953-maas-api-tls\") pod \"maas-api-e2e-ait-0ad997be-7978bfd4c5-j2whz\" (UID: \"5b41c1f9-bb4a-4a5c-b7f8-0747ea205953\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0ad997be-7978bfd4c5-j2whz"
Sep 11 17:32:29.737796 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:29.737691    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hhh65\" (UniqueName: \"kubernetes.io/projected/5b41c1f9-bb4a-4a5c-b7f8-0747ea205953-kube-api-access-hhh65\") pod \"maas-api-e2e-ait-0ad997be-7978bfd4c5-j2whz\" (UID: \"5b41c1f9-bb4a-4a5c-b7f8-0747ea205953\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0ad997be-7978bfd4c5-j2whz"
Sep 11 17:32:29.738004 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:32:29.737975    2552 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-0ad997be-serving-cert: secret "maas-api-e2e-ait-0ad997be-serving-cert" not found
Sep 11 17:32:29.738170 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:32:29.738068    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5b41c1f9-bb4a-4a5c-b7f8-0747ea205953-maas-api-tls podName:5b41c1f9-bb4a-4a5c-b7f8-0747ea205953 nodeName:}" failed. No retries permitted until 2026-09-11 17:32:30.238037113 +0000 UTC m=+844.759328613 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/5b41c1f9-bb4a-4a5c-b7f8-0747ea205953-maas-api-tls") pod "maas-api-e2e-ait-0ad997be-7978bfd4c5-j2whz" (UID: "5b41c1f9-bb4a-4a5c-b7f8-0747ea205953") : secret "maas-api-e2e-ait-0ad997be-serving-cert" not found
Sep 11 17:32:29.738266 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:29.738238    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5b41c1f9-bb4a-4a5c-b7f8-0747ea205953-openshift-service-ca\") pod \"maas-api-e2e-ait-0ad997be-7978bfd4c5-j2whz\" (UID: \"5b41c1f9-bb4a-4a5c-b7f8-0747ea205953\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0ad997be-7978bfd4c5-j2whz"
Sep 11 17:32:29.740764 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:29.740743    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5b41c1f9-bb4a-4a5c-b7f8-0747ea205953-metrics-certs\") pod \"maas-api-e2e-ait-0ad997be-7978bfd4c5-j2whz\" (UID: \"5b41c1f9-bb4a-4a5c-b7f8-0747ea205953\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0ad997be-7978bfd4c5-j2whz"
Sep 11 17:32:29.748094 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:29.748035    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hhh65\" (UniqueName: \"kubernetes.io/projected/5b41c1f9-bb4a-4a5c-b7f8-0747ea205953-kube-api-access-hhh65\") pod \"maas-api-e2e-ait-0ad997be-7978bfd4c5-j2whz\" (UID: \"5b41c1f9-bb4a-4a5c-b7f8-0747ea205953\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0ad997be-7978bfd4c5-j2whz"
Sep 11 17:32:29.900239 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:29.900204    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:32:29.950169 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:29.950146    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-d19de295-gw-openshift-default-6845bbcd48-9j7gs"
Sep 11 17:32:30.005913 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:30.005827    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:32:30.097438 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:30.097406    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:32:30.142205 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:30.142155    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/833f77b2-5c53-490b-babb-03d2e6f6d4b8-workload-certs\") pod \"833f77b2-5c53-490b-babb-03d2e6f6d4b8\" (UID: \"833f77b2-5c53-490b-babb-03d2e6f6d4b8\") "
Sep 11 17:32:30.142205 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:30.142197    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/833f77b2-5c53-490b-babb-03d2e6f6d4b8-credential-socket\") pod \"833f77b2-5c53-490b-babb-03d2e6f6d4b8\" (UID: \"833f77b2-5c53-490b-babb-03d2e6f6d4b8\") "
Sep 11 17:32:30.142468 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:30.142233    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/833f77b2-5c53-490b-babb-03d2e6f6d4b8-istio-data\") pod \"833f77b2-5c53-490b-babb-03d2e6f6d4b8\" (UID: \"833f77b2-5c53-490b-babb-03d2e6f6d4b8\") "
Sep 11 17:32:30.142468 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:30.142293    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/833f77b2-5c53-490b-babb-03d2e6f6d4b8-istio-envoy\") pod \"833f77b2-5c53-490b-babb-03d2e6f6d4b8\" (UID: \"833f77b2-5c53-490b-babb-03d2e6f6d4b8\") "
Sep 11 17:32:30.142468 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:30.142327    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/833f77b2-5c53-490b-babb-03d2e6f6d4b8-istiod-ca-cert\") pod \"833f77b2-5c53-490b-babb-03d2e6f6d4b8\" (UID: \"833f77b2-5c53-490b-babb-03d2e6f6d4b8\") "
Sep 11 17:32:30.142468 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:30.142366    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/833f77b2-5c53-490b-babb-03d2e6f6d4b8-istio-podinfo\") pod \"833f77b2-5c53-490b-babb-03d2e6f6d4b8\" (UID: \"833f77b2-5c53-490b-babb-03d2e6f6d4b8\") "
Sep 11 17:32:30.142468 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:30.142433    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8wns2\" (UniqueName: \"kubernetes.io/projected/833f77b2-5c53-490b-babb-03d2e6f6d4b8-kube-api-access-8wns2\") pod \"833f77b2-5c53-490b-babb-03d2e6f6d4b8\" (UID: \"833f77b2-5c53-490b-babb-03d2e6f6d4b8\") "
Sep 11 17:32:30.142468 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:30.142470    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/833f77b2-5c53-490b-babb-03d2e6f6d4b8-istio-token\") pod \"833f77b2-5c53-490b-babb-03d2e6f6d4b8\" (UID: \"833f77b2-5c53-490b-babb-03d2e6f6d4b8\") "
Sep 11 17:32:30.142790 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:30.142521    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/833f77b2-5c53-490b-babb-03d2e6f6d4b8-workload-socket\") pod \"833f77b2-5c53-490b-babb-03d2e6f6d4b8\" (UID: \"833f77b2-5c53-490b-babb-03d2e6f6d4b8\") "
Sep 11 17:32:30.142790 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:30.142523    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/833f77b2-5c53-490b-babb-03d2e6f6d4b8-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "833f77b2-5c53-490b-babb-03d2e6f6d4b8" (UID: "833f77b2-5c53-490b-babb-03d2e6f6d4b8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:32:30.142790 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:30.142754    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/833f77b2-5c53-490b-babb-03d2e6f6d4b8-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "833f77b2-5c53-490b-babb-03d2e6f6d4b8" (UID: "833f77b2-5c53-490b-babb-03d2e6f6d4b8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:32:30.142942 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:30.142858    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/833f77b2-5c53-490b-babb-03d2e6f6d4b8-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "833f77b2-5c53-490b-babb-03d2e6f6d4b8" (UID: "833f77b2-5c53-490b-babb-03d2e6f6d4b8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 17:32:30.142942 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:30.142890    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/833f77b2-5c53-490b-babb-03d2e6f6d4b8-istio-data" (OuterVolumeSpecName: "istio-data") pod "833f77b2-5c53-490b-babb-03d2e6f6d4b8" (UID: "833f77b2-5c53-490b-babb-03d2e6f6d4b8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:32:30.143048 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:30.142937    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/833f77b2-5c53-490b-babb-03d2e6f6d4b8-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "833f77b2-5c53-490b-babb-03d2e6f6d4b8" (UID: "833f77b2-5c53-490b-babb-03d2e6f6d4b8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:32:30.143048 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:30.142985    2552 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/833f77b2-5c53-490b-babb-03d2e6f6d4b8-workload-certs\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:32:30.143048 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:30.143003    2552 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/833f77b2-5c53-490b-babb-03d2e6f6d4b8-credential-socket\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:32:30.143048 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:30.143013    2552 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/833f77b2-5c53-490b-babb-03d2e6f6d4b8-istio-data\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:32:30.143048 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:30.143022    2552 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/833f77b2-5c53-490b-babb-03d2e6f6d4b8-istiod-ca-cert\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:32:30.145023 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:30.144998    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/833f77b2-5c53-490b-babb-03d2e6f6d4b8-istio-token" (OuterVolumeSpecName: "istio-token") pod "833f77b2-5c53-490b-babb-03d2e6f6d4b8" (UID: "833f77b2-5c53-490b-babb-03d2e6f6d4b8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:32:30.145343 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:30.145318    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/833f77b2-5c53-490b-babb-03d2e6f6d4b8-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "833f77b2-5c53-490b-babb-03d2e6f6d4b8" (UID: "833f77b2-5c53-490b-babb-03d2e6f6d4b8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 17:32:30.145446 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:30.145321    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/833f77b2-5c53-490b-babb-03d2e6f6d4b8-kube-api-access-8wns2" (OuterVolumeSpecName: "kube-api-access-8wns2") pod "833f77b2-5c53-490b-babb-03d2e6f6d4b8" (UID: "833f77b2-5c53-490b-babb-03d2e6f6d4b8"). InnerVolumeSpecName "kube-api-access-8wns2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:32:30.145446 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:30.145411    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/833f77b2-5c53-490b-babb-03d2e6f6d4b8-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "833f77b2-5c53-490b-babb-03d2e6f6d4b8" (UID: "833f77b2-5c53-490b-babb-03d2e6f6d4b8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:32:30.244610 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:30.244562    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5b41c1f9-bb4a-4a5c-b7f8-0747ea205953-maas-api-tls\") pod \"maas-api-e2e-ait-0ad997be-7978bfd4c5-j2whz\" (UID: \"5b41c1f9-bb4a-4a5c-b7f8-0747ea205953\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0ad997be-7978bfd4c5-j2whz"
Sep 11 17:32:30.244813 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:30.244704    2552 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/833f77b2-5c53-490b-babb-03d2e6f6d4b8-istio-envoy\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:32:30.244813 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:30.244720    2552 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/833f77b2-5c53-490b-babb-03d2e6f6d4b8-istio-podinfo\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:32:30.244813 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:30.244734    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8wns2\" (UniqueName: \"kubernetes.io/projected/833f77b2-5c53-490b-babb-03d2e6f6d4b8-kube-api-access-8wns2\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:32:30.244813 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:30.244748    2552 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/833f77b2-5c53-490b-babb-03d2e6f6d4b8-istio-token\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:32:30.244813 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:30.244762    2552 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/833f77b2-5c53-490b-babb-03d2e6f6d4b8-workload-socket\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:32:30.248629 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:30.248061    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5b41c1f9-bb4a-4a5c-b7f8-0747ea205953-maas-api-tls\") pod \"maas-api-e2e-ait-0ad997be-7978bfd4c5-j2whz\" (UID: \"5b41c1f9-bb4a-4a5c-b7f8-0747ea205953\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0ad997be-7978bfd4c5-j2whz"
Sep 11 17:32:30.413888 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:30.413849    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0ad997be-7978bfd4c5-j2whz"
Sep 11 17:32:30.448872 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:30.448835    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:32:30.633595 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:30.633563    2552 generic.go:356] "Generic (PLEG): container finished" podID="833f77b2-5c53-490b-babb-03d2e6f6d4b8" containerID="4235b746f0de469fb0f00850cd952a463375dd1c460c8362644e66527307ce69" exitCode=0
Sep 11 17:32:30.634010 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:30.633614    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-d19de295-gw-openshift-default-6845bbcd48-9j7gs"
Sep 11 17:32:30.634010 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:30.633654    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-d19de295-gw-openshift-default-6845bbcd48-9j7gs" event={"ID":"833f77b2-5c53-490b-babb-03d2e6f6d4b8","Type":"ContainerDied","Data":"4235b746f0de469fb0f00850cd952a463375dd1c460c8362644e66527307ce69"}
Sep 11 17:32:30.634010 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:30.633710    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-d19de295-gw-openshift-default-6845bbcd48-9j7gs" event={"ID":"833f77b2-5c53-490b-babb-03d2e6f6d4b8","Type":"ContainerDied","Data":"04d11a7b8c88a853f26b6334a30701174582c738a7b5c1168aab0175fb7edf3a"}
Sep 11 17:32:30.634010 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:30.633736    2552 scope.go:117] "RemoveContainer" containerID="4235b746f0de469fb0f00850cd952a463375dd1c460c8362644e66527307ce69"
Sep 11 17:32:30.644893 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:30.644869    2552 scope.go:117] "RemoveContainer" containerID="14bbee24d43322f8c763705e49ad4fea5ceb821f4196c84113f958ab5086e558"
Sep 11 17:32:30.654058 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:30.654035    2552 scope.go:117] "RemoveContainer" containerID="4235b746f0de469fb0f00850cd952a463375dd1c460c8362644e66527307ce69"
Sep 11 17:32:30.654435 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:32:30.654415    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4235b746f0de469fb0f00850cd952a463375dd1c460c8362644e66527307ce69\": container with ID starting with 4235b746f0de469fb0f00850cd952a463375dd1c460c8362644e66527307ce69 not found: ID does not exist" containerID="4235b746f0de469fb0f00850cd952a463375dd1c460c8362644e66527307ce69"
Sep 11 17:32:30.654487 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:30.654445    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4235b746f0de469fb0f00850cd952a463375dd1c460c8362644e66527307ce69"} err="failed to get container status \"4235b746f0de469fb0f00850cd952a463375dd1c460c8362644e66527307ce69\": rpc error: code = NotFound desc = could not find container \"4235b746f0de469fb0f00850cd952a463375dd1c460c8362644e66527307ce69\": container with ID starting with 4235b746f0de469fb0f00850cd952a463375dd1c460c8362644e66527307ce69 not found: ID does not exist"
Sep 11 17:32:30.654487 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:30.654461    2552 scope.go:117] "RemoveContainer" containerID="14bbee24d43322f8c763705e49ad4fea5ceb821f4196c84113f958ab5086e558"
Sep 11 17:32:30.654684 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:32:30.654671    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"14bbee24d43322f8c763705e49ad4fea5ceb821f4196c84113f958ab5086e558\": container with ID starting with 14bbee24d43322f8c763705e49ad4fea5ceb821f4196c84113f958ab5086e558 not found: ID does not exist" containerID="14bbee24d43322f8c763705e49ad4fea5ceb821f4196c84113f958ab5086e558"
Sep 11 17:32:30.654723 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:30.654688    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"14bbee24d43322f8c763705e49ad4fea5ceb821f4196c84113f958ab5086e558"} err="failed to get container status \"14bbee24d43322f8c763705e49ad4fea5ceb821f4196c84113f958ab5086e558\": rpc error: code = NotFound desc = could not find container \"14bbee24d43322f8c763705e49ad4fea5ceb821f4196c84113f958ab5086e558\": container with ID starting with 14bbee24d43322f8c763705e49ad4fea5ceb821f4196c84113f958ab5086e558 not found: ID does not exist"
Sep 11 17:32:30.659201 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:30.659177    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-d19de295-gw-openshift-default-6845bbcd48-9j7gs"]
Sep 11 17:32:30.673546 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:30.673477    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-d19de295-gw-openshift-default-6845bbcd48-9j7gs"]
Sep 11 17:32:30.691628 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:30.691605    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-0ad997be-7978bfd4c5-j2whz"]
Sep 11 17:32:30.692950 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:32:30.692922    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b41c1f9_bb4a_4a5c_b7f8_0747ea205953.slice/crio-06e578fe1a523f662cc3799d0d2b5644acb57de69b90c1a6614dab9ba6ecb19b WatchSource:0}: Error finding container 06e578fe1a523f662cc3799d0d2b5644acb57de69b90c1a6614dab9ba6ecb19b: Status 404 returned error can't find the container with id 06e578fe1a523f662cc3799d0d2b5644acb57de69b90c1a6614dab9ba6ecb19b
Sep 11 17:32:31.344214 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:31.344169    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:32:31.644063 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:31.643977    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0ad997be-7978bfd4c5-j2whz" event={"ID":"5b41c1f9-bb4a-4a5c-b7f8-0747ea205953","Type":"ContainerStarted","Data":"879e90e90a93521f594a94865fa76df3bc7202cbec4b5124616d32f26bb0713a"}
Sep 11 17:32:31.644063 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:31.644035    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0ad997be-7978bfd4c5-j2whz" event={"ID":"5b41c1f9-bb4a-4a5c-b7f8-0747ea205953","Type":"ContainerStarted","Data":"06e578fe1a523f662cc3799d0d2b5644acb57de69b90c1a6614dab9ba6ecb19b"}
Sep 11 17:32:31.644488 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:31.644139    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0ad997be-7978bfd4c5-j2whz"
Sep 11 17:32:31.660717 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:31.660667    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0ad997be-7978bfd4c5-j2whz" podStartSLOduration=2.302064397 podStartE2EDuration="2.660651633s" totalImagesPullingTime="358.587236ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:32:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:32:31.658042397 +0000 UTC m=+846.179333918" watchObservedRunningTime="2026-09-11 17:32:31.660651633 +0000 UTC m=+846.181943153"
Sep 11 17:32:31.951908 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:31.951822    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="833f77b2-5c53-490b-babb-03d2e6f6d4b8" path="/var/lib/kubelet/pods/833f77b2-5c53-490b-babb-03d2e6f6d4b8/volumes"
Sep 11 17:32:32.051231 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:32.051185    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:32:32.285608 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:32.285522    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:32:32.729168 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:32.729129    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:32:33.543499 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:33.543468    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:32:37.654905 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:37.654872    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0ad997be-7978bfd4c5-j2whz"
Sep 11 17:32:51.513385 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:51.513352    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:32:52.336928 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:52.336882    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:32:53.044719 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:53.044685    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:32:54.226518 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:54.226486    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:32:55.105501 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:55.105467    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:32:55.947433 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:55.947393    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:32:56.665219 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:56.665173    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:32:57.691062 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:57.691028    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:32:58.445481 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:58.445427    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:32:59.310336 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:59.310302    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:32:59.967047 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:32:59.967010    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:33:00.534076 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:00.534043    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-0ad997be-7978bfd4c5-j2whz"]
Sep 11 17:33:00.534519 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:00.534311    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0ad997be-7978bfd4c5-j2whz" podUID="5b41c1f9-bb4a-4a5c-b7f8-0747ea205953" containerName="maas-api" containerID="cri-o://879e90e90a93521f594a94865fa76df3bc7202cbec4b5124616d32f26bb0713a" gracePeriod=30
Sep 11 17:33:00.565704 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:00.565668    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:33:00.642876 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:00.642849    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:33:00.719397 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:00.719366    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:33:00.785965 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:00.785898    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0ad997be-7978bfd4c5-j2whz"
Sep 11 17:33:00.789875 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:00.789848    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:33:00.823474 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:00.823444    2552 generic.go:356] "Generic (PLEG): container finished" podID="5b41c1f9-bb4a-4a5c-b7f8-0747ea205953" containerID="879e90e90a93521f594a94865fa76df3bc7202cbec4b5124616d32f26bb0713a" exitCode=0
Sep 11 17:33:00.823676 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:00.823507    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0ad997be-7978bfd4c5-j2whz"
Sep 11 17:33:00.823676 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:00.823522    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0ad997be-7978bfd4c5-j2whz" event={"ID":"5b41c1f9-bb4a-4a5c-b7f8-0747ea205953","Type":"ContainerDied","Data":"879e90e90a93521f594a94865fa76df3bc7202cbec4b5124616d32f26bb0713a"}
Sep 11 17:33:00.823676 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:00.823572    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0ad997be-7978bfd4c5-j2whz" event={"ID":"5b41c1f9-bb4a-4a5c-b7f8-0747ea205953","Type":"ContainerDied","Data":"06e578fe1a523f662cc3799d0d2b5644acb57de69b90c1a6614dab9ba6ecb19b"}
Sep 11 17:33:00.823676 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:00.823596    2552 scope.go:117] "RemoveContainer" containerID="879e90e90a93521f594a94865fa76df3bc7202cbec4b5124616d32f26bb0713a"
Sep 11 17:33:00.833031 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:00.833016    2552 scope.go:117] "RemoveContainer" containerID="879e90e90a93521f594a94865fa76df3bc7202cbec4b5124616d32f26bb0713a"
Sep 11 17:33:00.833320 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:33:00.833303    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"879e90e90a93521f594a94865fa76df3bc7202cbec4b5124616d32f26bb0713a\": container with ID starting with 879e90e90a93521f594a94865fa76df3bc7202cbec4b5124616d32f26bb0713a not found: ID does not exist" containerID="879e90e90a93521f594a94865fa76df3bc7202cbec4b5124616d32f26bb0713a"
Sep 11 17:33:00.833375 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:00.833327    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"879e90e90a93521f594a94865fa76df3bc7202cbec4b5124616d32f26bb0713a"} err="failed to get container status \"879e90e90a93521f594a94865fa76df3bc7202cbec4b5124616d32f26bb0713a\": rpc error: code = NotFound desc = could not find container \"879e90e90a93521f594a94865fa76df3bc7202cbec4b5124616d32f26bb0713a\": container with ID starting with 879e90e90a93521f594a94865fa76df3bc7202cbec4b5124616d32f26bb0713a not found: ID does not exist"
Sep 11 17:33:00.948134 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:00.947183    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5b41c1f9-bb4a-4a5c-b7f8-0747ea205953-openshift-service-ca\") pod \"5b41c1f9-bb4a-4a5c-b7f8-0747ea205953\" (UID: \"5b41c1f9-bb4a-4a5c-b7f8-0747ea205953\") "
Sep 11 17:33:00.948134 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:00.947228    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5b41c1f9-bb4a-4a5c-b7f8-0747ea205953-maas-api-tls\") pod \"5b41c1f9-bb4a-4a5c-b7f8-0747ea205953\" (UID: \"5b41c1f9-bb4a-4a5c-b7f8-0747ea205953\") "
Sep 11 17:33:00.948134 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:00.947308    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5b41c1f9-bb4a-4a5c-b7f8-0747ea205953-metrics-certs\") pod \"5b41c1f9-bb4a-4a5c-b7f8-0747ea205953\" (UID: \"5b41c1f9-bb4a-4a5c-b7f8-0747ea205953\") "
Sep 11 17:33:00.948134 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:00.947335    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hhh65\" (UniqueName: \"kubernetes.io/projected/5b41c1f9-bb4a-4a5c-b7f8-0747ea205953-kube-api-access-hhh65\") pod \"5b41c1f9-bb4a-4a5c-b7f8-0747ea205953\" (UID: \"5b41c1f9-bb4a-4a5c-b7f8-0747ea205953\") "
Sep 11 17:33:00.950155 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:00.948654    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5b41c1f9-bb4a-4a5c-b7f8-0747ea205953-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "5b41c1f9-bb4a-4a5c-b7f8-0747ea205953" (UID: "5b41c1f9-bb4a-4a5c-b7f8-0747ea205953"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 17:33:00.950905 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:00.950859    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5b41c1f9-bb4a-4a5c-b7f8-0747ea205953-kube-api-access-hhh65" (OuterVolumeSpecName: "kube-api-access-hhh65") pod "5b41c1f9-bb4a-4a5c-b7f8-0747ea205953" (UID: "5b41c1f9-bb4a-4a5c-b7f8-0747ea205953"). InnerVolumeSpecName "kube-api-access-hhh65". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:33:00.951438 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:00.951417    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5b41c1f9-bb4a-4a5c-b7f8-0747ea205953-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "5b41c1f9-bb4a-4a5c-b7f8-0747ea205953" (UID: "5b41c1f9-bb4a-4a5c-b7f8-0747ea205953"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 17:33:00.951580 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:00.951556    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5b41c1f9-bb4a-4a5c-b7f8-0747ea205953-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "5b41c1f9-bb4a-4a5c-b7f8-0747ea205953" (UID: "5b41c1f9-bb4a-4a5c-b7f8-0747ea205953"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 17:33:01.048651 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:01.048569    2552 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5b41c1f9-bb4a-4a5c-b7f8-0747ea205953-openshift-service-ca\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:33:01.048651 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:01.048599    2552 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5b41c1f9-bb4a-4a5c-b7f8-0747ea205953-maas-api-tls\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:33:01.048651 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:01.048608    2552 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5b41c1f9-bb4a-4a5c-b7f8-0747ea205953-metrics-certs\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:33:01.048651 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:01.048616    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hhh65\" (UniqueName: \"kubernetes.io/projected/5b41c1f9-bb4a-4a5c-b7f8-0747ea205953-kube-api-access-hhh65\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:33:01.146268 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:01.146231    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-0ad997be-7978bfd4c5-j2whz"]
Sep 11 17:33:01.148690 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:01.148670    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-0ad997be-7978bfd4c5-j2whz"]
Sep 11 17:33:01.247289 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:01.247259    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:33:01.931043 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:01.931005    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:33:01.951377 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:01.951345    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5b41c1f9-bb4a-4a5c-b7f8-0747ea205953" path="/var/lib/kubelet/pods/5b41c1f9-bb4a-4a5c-b7f8-0747ea205953/volumes"
Sep 11 17:33:02.006810 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:02.006769    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:33:02.380838 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:02.380795    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:33:02.647710 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:02.647625    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-357aed65-openshift-default-65599f675b-srqvl"]
Sep 11 17:33:02.648078 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:02.648064    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="833f77b2-5c53-490b-babb-03d2e6f6d4b8" containerName="istio-proxy"
Sep 11 17:33:02.648152 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:02.648080    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="833f77b2-5c53-490b-babb-03d2e6f6d4b8" containerName="istio-proxy"
Sep 11 17:33:02.648152 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:02.648092    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="833f77b2-5c53-490b-babb-03d2e6f6d4b8" containerName="istio-proxy"
Sep 11 17:33:02.648152 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:02.648098    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="833f77b2-5c53-490b-babb-03d2e6f6d4b8" containerName="istio-proxy"
Sep 11 17:33:02.648152 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:02.648121    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b41c1f9-bb4a-4a5c-b7f8-0747ea205953" containerName="maas-api"
Sep 11 17:33:02.648152 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:02.648131    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b41c1f9-bb4a-4a5c-b7f8-0747ea205953" containerName="maas-api"
Sep 11 17:33:02.648315 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:02.648198    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="833f77b2-5c53-490b-babb-03d2e6f6d4b8" containerName="istio-proxy"
Sep 11 17:33:02.648315 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:02.648206    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="833f77b2-5c53-490b-babb-03d2e6f6d4b8" containerName="istio-proxy"
Sep 11 17:33:02.648315 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:02.648215    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="5b41c1f9-bb4a-4a5c-b7f8-0747ea205953" containerName="maas-api"
Sep 11 17:33:02.651402 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:02.651378    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-357aed65-openshift-default-65599f675b-srqvl"
Sep 11 17:33:02.653450 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:02.653428    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-357aed65-openshift-default-dockercfg-xv2d5\""
Sep 11 17:33:02.664082 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:02.664051    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-357aed65-openshift-default-65599f675b-srqvl"]
Sep 11 17:33:02.685413 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:02.685385    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:33:02.749214 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:02.749154    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:33:02.767379 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:02.767348    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b39a4c0-6d65-48fb-8d38-9a9523b9c154-credential-socket\") pod \"e2e-derive-357aed65-openshift-default-65599f675b-srqvl\" (UID: \"1b39a4c0-6d65-48fb-8d38-9a9523b9c154\") " pod="openshift-ingress/e2e-derive-357aed65-openshift-default-65599f675b-srqvl"
Sep 11 17:33:02.767379 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:02.767379    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1b39a4c0-6d65-48fb-8d38-9a9523b9c154-workload-certs\") pod \"e2e-derive-357aed65-openshift-default-65599f675b-srqvl\" (UID: \"1b39a4c0-6d65-48fb-8d38-9a9523b9c154\") " pod="openshift-ingress/e2e-derive-357aed65-openshift-default-65599f675b-srqvl"
Sep 11 17:33:02.767586 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:02.767411    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1b39a4c0-6d65-48fb-8d38-9a9523b9c154-istio-token\") pod \"e2e-derive-357aed65-openshift-default-65599f675b-srqvl\" (UID: \"1b39a4c0-6d65-48fb-8d38-9a9523b9c154\") " pod="openshift-ingress/e2e-derive-357aed65-openshift-default-65599f675b-srqvl"
Sep 11 17:33:02.767586 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:02.767472    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ql6ft\" (UniqueName: \"kubernetes.io/projected/1b39a4c0-6d65-48fb-8d38-9a9523b9c154-kube-api-access-ql6ft\") pod \"e2e-derive-357aed65-openshift-default-65599f675b-srqvl\" (UID: \"1b39a4c0-6d65-48fb-8d38-9a9523b9c154\") " pod="openshift-ingress/e2e-derive-357aed65-openshift-default-65599f675b-srqvl"
Sep 11 17:33:02.767586 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:02.767531    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1b39a4c0-6d65-48fb-8d38-9a9523b9c154-istio-data\") pod \"e2e-derive-357aed65-openshift-default-65599f675b-srqvl\" (UID: \"1b39a4c0-6d65-48fb-8d38-9a9523b9c154\") " pod="openshift-ingress/e2e-derive-357aed65-openshift-default-65599f675b-srqvl"
Sep 11 17:33:02.767586 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:02.767575    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1b39a4c0-6d65-48fb-8d38-9a9523b9c154-istio-podinfo\") pod \"e2e-derive-357aed65-openshift-default-65599f675b-srqvl\" (UID: \"1b39a4c0-6d65-48fb-8d38-9a9523b9c154\") " pod="openshift-ingress/e2e-derive-357aed65-openshift-default-65599f675b-srqvl"
Sep 11 17:33:02.767753 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:02.767602    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b39a4c0-6d65-48fb-8d38-9a9523b9c154-workload-socket\") pod \"e2e-derive-357aed65-openshift-default-65599f675b-srqvl\" (UID: \"1b39a4c0-6d65-48fb-8d38-9a9523b9c154\") " pod="openshift-ingress/e2e-derive-357aed65-openshift-default-65599f675b-srqvl"
Sep 11 17:33:02.767753 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:02.767620    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1b39a4c0-6d65-48fb-8d38-9a9523b9c154-istiod-ca-cert\") pod \"e2e-derive-357aed65-openshift-default-65599f675b-srqvl\" (UID: \"1b39a4c0-6d65-48fb-8d38-9a9523b9c154\") " pod="openshift-ingress/e2e-derive-357aed65-openshift-default-65599f675b-srqvl"
Sep 11 17:33:02.767753 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:02.767664    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1b39a4c0-6d65-48fb-8d38-9a9523b9c154-istio-envoy\") pod \"e2e-derive-357aed65-openshift-default-65599f675b-srqvl\" (UID: \"1b39a4c0-6d65-48fb-8d38-9a9523b9c154\") " pod="openshift-ingress/e2e-derive-357aed65-openshift-default-65599f675b-srqvl"
Sep 11 17:33:02.868588 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:02.868553    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1b39a4c0-6d65-48fb-8d38-9a9523b9c154-istio-data\") pod \"e2e-derive-357aed65-openshift-default-65599f675b-srqvl\" (UID: \"1b39a4c0-6d65-48fb-8d38-9a9523b9c154\") " pod="openshift-ingress/e2e-derive-357aed65-openshift-default-65599f675b-srqvl"
Sep 11 17:33:02.868764 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:02.868607    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1b39a4c0-6d65-48fb-8d38-9a9523b9c154-istio-podinfo\") pod \"e2e-derive-357aed65-openshift-default-65599f675b-srqvl\" (UID: \"1b39a4c0-6d65-48fb-8d38-9a9523b9c154\") " pod="openshift-ingress/e2e-derive-357aed65-openshift-default-65599f675b-srqvl"
Sep 11 17:33:02.868764 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:02.868730    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b39a4c0-6d65-48fb-8d38-9a9523b9c154-workload-socket\") pod \"e2e-derive-357aed65-openshift-default-65599f675b-srqvl\" (UID: \"1b39a4c0-6d65-48fb-8d38-9a9523b9c154\") " pod="openshift-ingress/e2e-derive-357aed65-openshift-default-65599f675b-srqvl"
Sep 11 17:33:02.868865 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:02.868767    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1b39a4c0-6d65-48fb-8d38-9a9523b9c154-istiod-ca-cert\") pod \"e2e-derive-357aed65-openshift-default-65599f675b-srqvl\" (UID: \"1b39a4c0-6d65-48fb-8d38-9a9523b9c154\") " pod="openshift-ingress/e2e-derive-357aed65-openshift-default-65599f675b-srqvl"
Sep 11 17:33:02.868865 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:02.868844    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1b39a4c0-6d65-48fb-8d38-9a9523b9c154-istio-envoy\") pod \"e2e-derive-357aed65-openshift-default-65599f675b-srqvl\" (UID: \"1b39a4c0-6d65-48fb-8d38-9a9523b9c154\") " pod="openshift-ingress/e2e-derive-357aed65-openshift-default-65599f675b-srqvl"
Sep 11 17:33:02.868974 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:02.868950    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b39a4c0-6d65-48fb-8d38-9a9523b9c154-credential-socket\") pod \"e2e-derive-357aed65-openshift-default-65599f675b-srqvl\" (UID: \"1b39a4c0-6d65-48fb-8d38-9a9523b9c154\") " pod="openshift-ingress/e2e-derive-357aed65-openshift-default-65599f675b-srqvl"
Sep 11 17:33:02.869032 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:02.868972    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1b39a4c0-6d65-48fb-8d38-9a9523b9c154-istio-data\") pod \"e2e-derive-357aed65-openshift-default-65599f675b-srqvl\" (UID: \"1b39a4c0-6d65-48fb-8d38-9a9523b9c154\") " pod="openshift-ingress/e2e-derive-357aed65-openshift-default-65599f675b-srqvl"
Sep 11 17:33:02.869032 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:02.868991    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1b39a4c0-6d65-48fb-8d38-9a9523b9c154-workload-certs\") pod \"e2e-derive-357aed65-openshift-default-65599f675b-srqvl\" (UID: \"1b39a4c0-6d65-48fb-8d38-9a9523b9c154\") " pod="openshift-ingress/e2e-derive-357aed65-openshift-default-65599f675b-srqvl"
Sep 11 17:33:02.869032 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:02.869029    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1b39a4c0-6d65-48fb-8d38-9a9523b9c154-istio-token\") pod \"e2e-derive-357aed65-openshift-default-65599f675b-srqvl\" (UID: \"1b39a4c0-6d65-48fb-8d38-9a9523b9c154\") " pod="openshift-ingress/e2e-derive-357aed65-openshift-default-65599f675b-srqvl"
Sep 11 17:33:02.869226 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:02.869060    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ql6ft\" (UniqueName: \"kubernetes.io/projected/1b39a4c0-6d65-48fb-8d38-9a9523b9c154-kube-api-access-ql6ft\") pod \"e2e-derive-357aed65-openshift-default-65599f675b-srqvl\" (UID: \"1b39a4c0-6d65-48fb-8d38-9a9523b9c154\") " pod="openshift-ingress/e2e-derive-357aed65-openshift-default-65599f675b-srqvl"
Sep 11 17:33:02.869226 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:02.869072    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b39a4c0-6d65-48fb-8d38-9a9523b9c154-workload-socket\") pod \"e2e-derive-357aed65-openshift-default-65599f675b-srqvl\" (UID: \"1b39a4c0-6d65-48fb-8d38-9a9523b9c154\") " pod="openshift-ingress/e2e-derive-357aed65-openshift-default-65599f675b-srqvl"
Sep 11 17:33:02.869338 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:02.869298    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b39a4c0-6d65-48fb-8d38-9a9523b9c154-credential-socket\") pod \"e2e-derive-357aed65-openshift-default-65599f675b-srqvl\" (UID: \"1b39a4c0-6d65-48fb-8d38-9a9523b9c154\") " pod="openshift-ingress/e2e-derive-357aed65-openshift-default-65599f675b-srqvl"
Sep 11 17:33:02.869400 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:02.869358    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1b39a4c0-6d65-48fb-8d38-9a9523b9c154-workload-certs\") pod \"e2e-derive-357aed65-openshift-default-65599f675b-srqvl\" (UID: \"1b39a4c0-6d65-48fb-8d38-9a9523b9c154\") " pod="openshift-ingress/e2e-derive-357aed65-openshift-default-65599f675b-srqvl"
Sep 11 17:33:02.869651 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:02.869627    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1b39a4c0-6d65-48fb-8d38-9a9523b9c154-istiod-ca-cert\") pod \"e2e-derive-357aed65-openshift-default-65599f675b-srqvl\" (UID: \"1b39a4c0-6d65-48fb-8d38-9a9523b9c154\") " pod="openshift-ingress/e2e-derive-357aed65-openshift-default-65599f675b-srqvl"
Sep 11 17:33:02.870975 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:02.870955    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1b39a4c0-6d65-48fb-8d38-9a9523b9c154-istio-envoy\") pod \"e2e-derive-357aed65-openshift-default-65599f675b-srqvl\" (UID: \"1b39a4c0-6d65-48fb-8d38-9a9523b9c154\") " pod="openshift-ingress/e2e-derive-357aed65-openshift-default-65599f675b-srqvl"
Sep 11 17:33:02.871204 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:02.871187    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1b39a4c0-6d65-48fb-8d38-9a9523b9c154-istio-podinfo\") pod \"e2e-derive-357aed65-openshift-default-65599f675b-srqvl\" (UID: \"1b39a4c0-6d65-48fb-8d38-9a9523b9c154\") " pod="openshift-ingress/e2e-derive-357aed65-openshift-default-65599f675b-srqvl"
Sep 11 17:33:02.876199 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:02.876179    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ql6ft\" (UniqueName: \"kubernetes.io/projected/1b39a4c0-6d65-48fb-8d38-9a9523b9c154-kube-api-access-ql6ft\") pod \"e2e-derive-357aed65-openshift-default-65599f675b-srqvl\" (UID: \"1b39a4c0-6d65-48fb-8d38-9a9523b9c154\") " pod="openshift-ingress/e2e-derive-357aed65-openshift-default-65599f675b-srqvl"
Sep 11 17:33:02.876308 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:02.876206    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1b39a4c0-6d65-48fb-8d38-9a9523b9c154-istio-token\") pod \"e2e-derive-357aed65-openshift-default-65599f675b-srqvl\" (UID: \"1b39a4c0-6d65-48fb-8d38-9a9523b9c154\") " pod="openshift-ingress/e2e-derive-357aed65-openshift-default-65599f675b-srqvl"
Sep 11 17:33:02.963720 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:02.963635    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-357aed65-openshift-default-65599f675b-srqvl"
Sep 11 17:33:03.034483 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:03.034443    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:33:03.241889 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:03.241865    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-357aed65-openshift-default-65599f675b-srqvl"]
Sep 11 17:33:03.243798 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:33:03.243762    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b39a4c0_6d65_48fb_8d38_9a9523b9c154.slice/crio-b39e5d94583e412344aeb8a5391947649a5985d2c63a24551fddaf19840eb38d WatchSource:0}: Error finding container b39e5d94583e412344aeb8a5391947649a5985d2c63a24551fddaf19840eb38d: Status 404 returned error can't find the container with id b39e5d94583e412344aeb8a5391947649a5985d2c63a24551fddaf19840eb38d
Sep 11 17:33:03.246268 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:03.246228    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 17:33:03.246379 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:03.246309    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 17:33:03.246379 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:03.246351    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 17:33:03.541641 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:03.541539    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:33:03.848404 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:03.848370    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-357aed65-openshift-default-65599f675b-srqvl" event={"ID":"1b39a4c0-6d65-48fb-8d38-9a9523b9c154","Type":"ContainerStarted","Data":"0bc49c9c0b458b5f9346728d67af2489c4c02a2797f66ede80b29fd2da5ee43f"}
Sep 11 17:33:03.848592 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:03.848428    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-357aed65-openshift-default-65599f675b-srqvl" event={"ID":"1b39a4c0-6d65-48fb-8d38-9a9523b9c154","Type":"ContainerStarted","Data":"b39e5d94583e412344aeb8a5391947649a5985d2c63a24551fddaf19840eb38d"}
Sep 11 17:33:03.869398 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:03.869343    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-357aed65-openshift-default-65599f675b-srqvl" podStartSLOduration=1.8693251100000001 podStartE2EDuration="1.86932511s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:33:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:33:03.866390447 +0000 UTC m=+878.387681969" watchObservedRunningTime="2026-09-11 17:33:03.86932511 +0000 UTC m=+878.390616630"
Sep 11 17:33:03.964717 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:03.964681    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-357aed65-openshift-default-65599f675b-srqvl"
Sep 11 17:33:04.253380 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:04.253294    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:33:04.969661 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:04.969631    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-357aed65-openshift-default-65599f675b-srqvl"
Sep 11 17:33:05.031034 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:05.031003    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:33:05.217019 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:05.216983    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:33:05.325441 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:05.325353    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:33:05.866135 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:05.863148    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-357aed65-openshift-default-65599f675b-srqvl"
Sep 11 17:33:05.868280 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:05.868250    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-357aed65-openshift-default-65599f675b-srqvl"
Sep 11 17:33:05.951812 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:05.951775    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:33:06.679667 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:06.679622    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:33:07.349214 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:07.349178    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:33:08.032385 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:08.032349    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:33:08.760553 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:08.760516    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:33:09.608028 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:09.607997    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-357aed65-6d56675cd7-tvhjk"]
Sep 11 17:33:09.611891 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:09.611867    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-357aed65-6d56675cd7-tvhjk"
Sep 11 17:33:09.613812 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:09.613793    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-357aed65-serving-cert\""
Sep 11 17:33:09.622994 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:09.622969    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-357aed65-6d56675cd7-tvhjk"]
Sep 11 17:33:09.635968 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:09.635930    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/33a15c58-dee4-44d2-9ebb-7071420d82d8-maas-api-tls\") pod \"maas-api-e2e-derive-357aed65-6d56675cd7-tvhjk\" (UID: \"33a15c58-dee4-44d2-9ebb-7071420d82d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-357aed65-6d56675cd7-tvhjk"
Sep 11 17:33:09.636192 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:09.636008    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/33a15c58-dee4-44d2-9ebb-7071420d82d8-metrics-certs\") pod \"maas-api-e2e-derive-357aed65-6d56675cd7-tvhjk\" (UID: \"33a15c58-dee4-44d2-9ebb-7071420d82d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-357aed65-6d56675cd7-tvhjk"
Sep 11 17:33:09.636192 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:09.636039    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8x9lq\" (UniqueName: \"kubernetes.io/projected/33a15c58-dee4-44d2-9ebb-7071420d82d8-kube-api-access-8x9lq\") pod \"maas-api-e2e-derive-357aed65-6d56675cd7-tvhjk\" (UID: \"33a15c58-dee4-44d2-9ebb-7071420d82d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-357aed65-6d56675cd7-tvhjk"
Sep 11 17:33:09.636192 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:09.636100    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/33a15c58-dee4-44d2-9ebb-7071420d82d8-openshift-service-ca\") pod \"maas-api-e2e-derive-357aed65-6d56675cd7-tvhjk\" (UID: \"33a15c58-dee4-44d2-9ebb-7071420d82d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-357aed65-6d56675cd7-tvhjk"
Sep 11 17:33:09.648088 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:09.648054    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:33:09.736747 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:09.736714    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/33a15c58-dee4-44d2-9ebb-7071420d82d8-maas-api-tls\") pod \"maas-api-e2e-derive-357aed65-6d56675cd7-tvhjk\" (UID: \"33a15c58-dee4-44d2-9ebb-7071420d82d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-357aed65-6d56675cd7-tvhjk"
Sep 11 17:33:09.736968 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:09.736781    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/33a15c58-dee4-44d2-9ebb-7071420d82d8-metrics-certs\") pod \"maas-api-e2e-derive-357aed65-6d56675cd7-tvhjk\" (UID: \"33a15c58-dee4-44d2-9ebb-7071420d82d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-357aed65-6d56675cd7-tvhjk"
Sep 11 17:33:09.736968 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:09.736809    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8x9lq\" (UniqueName: \"kubernetes.io/projected/33a15c58-dee4-44d2-9ebb-7071420d82d8-kube-api-access-8x9lq\") pod \"maas-api-e2e-derive-357aed65-6d56675cd7-tvhjk\" (UID: \"33a15c58-dee4-44d2-9ebb-7071420d82d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-357aed65-6d56675cd7-tvhjk"
Sep 11 17:33:09.736968 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:09.736864    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/33a15c58-dee4-44d2-9ebb-7071420d82d8-openshift-service-ca\") pod \"maas-api-e2e-derive-357aed65-6d56675cd7-tvhjk\" (UID: \"33a15c58-dee4-44d2-9ebb-7071420d82d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-357aed65-6d56675cd7-tvhjk"
Sep 11 17:33:09.737718 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:09.737686    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/33a15c58-dee4-44d2-9ebb-7071420d82d8-openshift-service-ca\") pod \"maas-api-e2e-derive-357aed65-6d56675cd7-tvhjk\" (UID: \"33a15c58-dee4-44d2-9ebb-7071420d82d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-357aed65-6d56675cd7-tvhjk"
Sep 11 17:33:09.739479 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:09.739459    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/33a15c58-dee4-44d2-9ebb-7071420d82d8-metrics-certs\") pod \"maas-api-e2e-derive-357aed65-6d56675cd7-tvhjk\" (UID: \"33a15c58-dee4-44d2-9ebb-7071420d82d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-357aed65-6d56675cd7-tvhjk"
Sep 11 17:33:09.739821 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:09.739802    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/33a15c58-dee4-44d2-9ebb-7071420d82d8-maas-api-tls\") pod \"maas-api-e2e-derive-357aed65-6d56675cd7-tvhjk\" (UID: \"33a15c58-dee4-44d2-9ebb-7071420d82d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-357aed65-6d56675cd7-tvhjk"
Sep 11 17:33:09.745331 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:09.745304    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8x9lq\" (UniqueName: \"kubernetes.io/projected/33a15c58-dee4-44d2-9ebb-7071420d82d8-kube-api-access-8x9lq\") pod \"maas-api-e2e-derive-357aed65-6d56675cd7-tvhjk\" (UID: \"33a15c58-dee4-44d2-9ebb-7071420d82d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-357aed65-6d56675cd7-tvhjk"
Sep 11 17:33:09.750057 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:09.750028    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:33:09.805068 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:09.805035    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:33:09.926986 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:09.926902    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-357aed65-6d56675cd7-tvhjk"
Sep 11 17:33:10.111729 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:10.111691    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:33:10.207387 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:10.207360    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-357aed65-6d56675cd7-tvhjk"]
Sep 11 17:33:10.208989 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:10.208961    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:33:10.209738 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:33:10.209707    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33a15c58_dee4_44d2_9ebb_7071420d82d8.slice/crio-c39923e1033eba11272a14262d188b85149b9105ec7610ff9a73ad3179a97d9f WatchSource:0}: Error finding container c39923e1033eba11272a14262d188b85149b9105ec7610ff9a73ad3179a97d9f: Status 404 returned error can't find the container with id c39923e1033eba11272a14262d188b85149b9105ec7610ff9a73ad3179a97d9f
Sep 11 17:33:10.894961 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:10.894926    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-357aed65-6d56675cd7-tvhjk" event={"ID":"33a15c58-dee4-44d2-9ebb-7071420d82d8","Type":"ContainerStarted","Data":"2533e6130b7693c1ab8f9e16e99f847b995477bd26098eba0f2bd93a994a6e0d"}
Sep 11 17:33:10.895478 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:10.895171    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-357aed65-6d56675cd7-tvhjk"
Sep 11 17:33:10.895478 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:10.895211    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-357aed65-6d56675cd7-tvhjk" event={"ID":"33a15c58-dee4-44d2-9ebb-7071420d82d8","Type":"ContainerStarted","Data":"c39923e1033eba11272a14262d188b85149b9105ec7610ff9a73ad3179a97d9f"}
Sep 11 17:33:10.911273 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:10.911220    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-357aed65-6d56675cd7-tvhjk" podStartSLOduration=1.562957194 podStartE2EDuration="1.91120374s" totalImagesPullingTime="348.246546ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:33:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:33:10.909196775 +0000 UTC m=+885.430488297" watchObservedRunningTime="2026-09-11 17:33:10.91120374 +0000 UTC m=+885.432495260"
Sep 11 17:33:12.029799 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:12.029763    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:33:12.131696 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:12.131641    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:33:16.907270 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:16.907239    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-357aed65-6d56675cd7-tvhjk"
Sep 11 17:33:25.950241 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:25.950160    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-hg7p8_eb5f3be1-2e36-40ba-b3a0-3bec413056e8/console-operator/1.log"
Sep 11 17:33:25.953826 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:25.953801    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-hg7p8_eb5f3be1-2e36-40ba-b3a0-3bec413056e8/console-operator/1.log"
Sep 11 17:33:25.954148 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:25.954132    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6k9tr_0620c0a0-8070-4307-86d9-c69a76ea14a9/ovn-acl-logging/0.log"
Sep 11 17:33:25.957624 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:25.957603    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6k9tr_0620c0a0-8070-4307-86d9-c69a76ea14a9/ovn-acl-logging/0.log"
Sep 11 17:33:29.016431 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:29.016395    2552 generic.go:356] "Generic (PLEG): container finished" podID="1b39a4c0-6d65-48fb-8d38-9a9523b9c154" containerID="0bc49c9c0b458b5f9346728d67af2489c4c02a2797f66ede80b29fd2da5ee43f" exitCode=0
Sep 11 17:33:29.016963 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:29.016472    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-357aed65-openshift-default-65599f675b-srqvl" event={"ID":"1b39a4c0-6d65-48fb-8d38-9a9523b9c154","Type":"ContainerDied","Data":"0bc49c9c0b458b5f9346728d67af2489c4c02a2797f66ede80b29fd2da5ee43f"}
Sep 11 17:33:29.017027 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:29.016997    2552 scope.go:117] "RemoveContainer" containerID="0bc49c9c0b458b5f9346728d67af2489c4c02a2797f66ede80b29fd2da5ee43f"
Sep 11 17:33:29.018164 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:29.018139    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 17:33:29.018230 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:29.018204    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 17:33:29.018268 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:29.018235    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 17:33:30.024174 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:30.024141    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-357aed65-openshift-default-65599f675b-srqvl" event={"ID":"1b39a4c0-6d65-48fb-8d38-9a9523b9c154","Type":"ContainerStarted","Data":"cc236e2588899bc4a0dc11aee9eb6f0bdced8f668776814a4d3ff55c388cdfc7"}
Sep 11 17:33:30.964244 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:30.964204    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-357aed65-openshift-default-65599f675b-srqvl"
Sep 11 17:33:30.964548 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:30.964231    2552 patch_prober.go:28] interesting pod/e2e-derive-357aed65-openshift-default-65599f675b-srqvl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body=
Sep 11 17:33:30.964719 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:30.964696    2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-357aed65-openshift-default-65599f675b-srqvl" podUID="1b39a4c0-6d65-48fb-8d38-9a9523b9c154" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused"
Sep 11 17:33:31.645406 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:31.645350    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:33:31.964782 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:31.964686    2552 patch_prober.go:28] interesting pod/e2e-derive-357aed65-openshift-default-65599f675b-srqvl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body=
Sep 11 17:33:31.965070 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:31.965042    2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-357aed65-openshift-default-65599f675b-srqvl" podUID="1b39a4c0-6d65-48fb-8d38-9a9523b9c154" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused"
Sep 11 17:33:32.386522 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:32.386483    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:33:32.965152 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:32.965087    2552 patch_prober.go:28] interesting pod/e2e-derive-357aed65-openshift-default-65599f675b-srqvl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body=
Sep 11 17:33:32.965559 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:32.965180    2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-357aed65-openshift-default-65599f675b-srqvl" podUID="1b39a4c0-6d65-48fb-8d38-9a9523b9c154" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused"
Sep 11 17:33:33.141900 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:33.141867    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:33:33.925925 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:33.925895    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:33:33.964284 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:33.964235    2552 patch_prober.go:28] interesting pod/e2e-derive-357aed65-openshift-default-65599f675b-srqvl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body=
Sep 11 17:33:33.964492 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:33.964310    2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-357aed65-openshift-default-65599f675b-srqvl" podUID="1b39a4c0-6d65-48fb-8d38-9a9523b9c154" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused"
Sep 11 17:33:34.740870 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:34.740838    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:33:34.964737 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:34.964693    2552 patch_prober.go:28] interesting pod/e2e-derive-357aed65-openshift-default-65599f675b-srqvl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body=
Sep 11 17:33:34.964943 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:34.964754    2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-357aed65-openshift-default-65599f675b-srqvl" podUID="1b39a4c0-6d65-48fb-8d38-9a9523b9c154" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused"
Sep 11 17:33:35.616060 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:35.616027    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:33:35.863386 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:35.863350    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-357aed65-openshift-default-65599f675b-srqvl"
Sep 11 17:33:35.968561 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:35.968479    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-357aed65-openshift-default-65599f675b-srqvl"
Sep 11 17:33:36.085051 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:36.085021    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-357aed65-openshift-default-65599f675b-srqvl"
Sep 11 17:33:36.344787 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:36.344726    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:33:37.701411 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:37.701382    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:33:38.162455 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:38.162420    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:33:38.913723 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:38.913673    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:33:39.562523 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:39.562486    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:33:40.273796 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:40.273759    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:33:43.905071 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:43.905037    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:33:46.668060 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:46.668014    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:33:47.246257 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:47.246215    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-357aed65-6d56675cd7-tvhjk"]
Sep 11 17:33:47.247014 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:47.246721    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-357aed65-6d56675cd7-tvhjk" podUID="33a15c58-dee4-44d2-9ebb-7071420d82d8" containerName="maas-api" containerID="cri-o://2533e6130b7693c1ab8f9e16e99f847b995477bd26098eba0f2bd93a994a6e0d" gracePeriod=30
Sep 11 17:33:47.278058 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:47.278028    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:33:47.350697 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:47.350653    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:33:47.411867 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:47.411811    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:33:47.481475 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:47.481448    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:33:47.501448 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:47.501387    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-357aed65-6d56675cd7-tvhjk"
Sep 11 17:33:47.548913 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:47.548884    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:33:47.629801 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:47.629770    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8x9lq\" (UniqueName: \"kubernetes.io/projected/33a15c58-dee4-44d2-9ebb-7071420d82d8-kube-api-access-8x9lq\") pod \"33a15c58-dee4-44d2-9ebb-7071420d82d8\" (UID: \"33a15c58-dee4-44d2-9ebb-7071420d82d8\") "
Sep 11 17:33:47.629801 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:47.629806    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/33a15c58-dee4-44d2-9ebb-7071420d82d8-maas-api-tls\") pod \"33a15c58-dee4-44d2-9ebb-7071420d82d8\" (UID: \"33a15c58-dee4-44d2-9ebb-7071420d82d8\") "
Sep 11 17:33:47.630014 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:47.629853    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/33a15c58-dee4-44d2-9ebb-7071420d82d8-openshift-service-ca\") pod \"33a15c58-dee4-44d2-9ebb-7071420d82d8\" (UID: \"33a15c58-dee4-44d2-9ebb-7071420d82d8\") "
Sep 11 17:33:47.630014 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:47.629892    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/33a15c58-dee4-44d2-9ebb-7071420d82d8-metrics-certs\") pod \"33a15c58-dee4-44d2-9ebb-7071420d82d8\" (UID: \"33a15c58-dee4-44d2-9ebb-7071420d82d8\") "
Sep 11 17:33:47.630293 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:47.630268    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/33a15c58-dee4-44d2-9ebb-7071420d82d8-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "33a15c58-dee4-44d2-9ebb-7071420d82d8" (UID: "33a15c58-dee4-44d2-9ebb-7071420d82d8"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 17:33:47.632038 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:47.632016    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/33a15c58-dee4-44d2-9ebb-7071420d82d8-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "33a15c58-dee4-44d2-9ebb-7071420d82d8" (UID: "33a15c58-dee4-44d2-9ebb-7071420d82d8"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 17:33:47.632143 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:47.632060    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/33a15c58-dee4-44d2-9ebb-7071420d82d8-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "33a15c58-dee4-44d2-9ebb-7071420d82d8" (UID: "33a15c58-dee4-44d2-9ebb-7071420d82d8"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 17:33:47.632143 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:47.632079    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/33a15c58-dee4-44d2-9ebb-7071420d82d8-kube-api-access-8x9lq" (OuterVolumeSpecName: "kube-api-access-8x9lq") pod "33a15c58-dee4-44d2-9ebb-7071420d82d8" (UID: "33a15c58-dee4-44d2-9ebb-7071420d82d8"). InnerVolumeSpecName "kube-api-access-8x9lq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:33:47.730840 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:47.730803    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8x9lq\" (UniqueName: \"kubernetes.io/projected/33a15c58-dee4-44d2-9ebb-7071420d82d8-kube-api-access-8x9lq\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:33:47.730840 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:47.730840    2552 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/33a15c58-dee4-44d2-9ebb-7071420d82d8-maas-api-tls\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:33:47.731385 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:47.730854    2552 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/33a15c58-dee4-44d2-9ebb-7071420d82d8-openshift-service-ca\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:33:47.731385 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:47.730866    2552 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/33a15c58-dee4-44d2-9ebb-7071420d82d8-metrics-certs\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:33:47.733743 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:47.733717    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-357aed65-openshift-default-65599f675b-srqvl"]
Sep 11 17:33:47.733938 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:47.733917    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-357aed65-openshift-default-65599f675b-srqvl" podUID="1b39a4c0-6d65-48fb-8d38-9a9523b9c154" containerName="istio-proxy" containerID="cri-o://cc236e2588899bc4a0dc11aee9eb6f0bdced8f668776814a4d3ff55c388cdfc7" gracePeriod=30
Sep 11 17:33:47.794513 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:47.794424    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:33:47.908193 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:47.908161    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:33:48.160663 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:48.160630    2552 generic.go:356] "Generic (PLEG): container finished" podID="33a15c58-dee4-44d2-9ebb-7071420d82d8" containerID="2533e6130b7693c1ab8f9e16e99f847b995477bd26098eba0f2bd93a994a6e0d" exitCode=0
Sep 11 17:33:48.160837 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:48.160676    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-357aed65-6d56675cd7-tvhjk"
Sep 11 17:33:48.160837 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:48.160710    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-357aed65-6d56675cd7-tvhjk" event={"ID":"33a15c58-dee4-44d2-9ebb-7071420d82d8","Type":"ContainerDied","Data":"2533e6130b7693c1ab8f9e16e99f847b995477bd26098eba0f2bd93a994a6e0d"}
Sep 11 17:33:48.160837 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:48.160771    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-357aed65-6d56675cd7-tvhjk" event={"ID":"33a15c58-dee4-44d2-9ebb-7071420d82d8","Type":"ContainerDied","Data":"c39923e1033eba11272a14262d188b85149b9105ec7610ff9a73ad3179a97d9f"}
Sep 11 17:33:48.160837 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:48.160797    2552 scope.go:117] "RemoveContainer" containerID="2533e6130b7693c1ab8f9e16e99f847b995477bd26098eba0f2bd93a994a6e0d"
Sep 11 17:33:48.170403 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:48.170384    2552 scope.go:117] "RemoveContainer" containerID="2533e6130b7693c1ab8f9e16e99f847b995477bd26098eba0f2bd93a994a6e0d"
Sep 11 17:33:48.170693 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:33:48.170674    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2533e6130b7693c1ab8f9e16e99f847b995477bd26098eba0f2bd93a994a6e0d\": container with ID starting with 2533e6130b7693c1ab8f9e16e99f847b995477bd26098eba0f2bd93a994a6e0d not found: ID does not exist" containerID="2533e6130b7693c1ab8f9e16e99f847b995477bd26098eba0f2bd93a994a6e0d"
Sep 11 17:33:48.170751 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:48.170702    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2533e6130b7693c1ab8f9e16e99f847b995477bd26098eba0f2bd93a994a6e0d"} err="failed to get container status \"2533e6130b7693c1ab8f9e16e99f847b995477bd26098eba0f2bd93a994a6e0d\": rpc error: code = NotFound desc = could not find container \"2533e6130b7693c1ab8f9e16e99f847b995477bd26098eba0f2bd93a994a6e0d\": container with ID starting with 2533e6130b7693c1ab8f9e16e99f847b995477bd26098eba0f2bd93a994a6e0d not found: ID does not exist"
Sep 11 17:33:48.179829 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:48.179805    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-357aed65-6d56675cd7-tvhjk"]
Sep 11 17:33:48.183324 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:48.183301    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-357aed65-6d56675cd7-tvhjk"]
Sep 11 17:33:48.392304 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:48.392266    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:33:48.485446 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:48.485331    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:33:48.585800 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:48.585765    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:33:48.764661 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:48.764580    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:33:49.642716 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:49.642678    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:33:49.951348 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:49.951250    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="33a15c58-dee4-44d2-9ebb-7071420d82d8" path="/var/lib/kubelet/pods/33a15c58-dee4-44d2-9ebb-7071420d82d8/volumes"
Sep 11 17:33:49.971579 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:49.971550    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:33:50.556650 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:50.556598    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:33:51.086419 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:51.086386    2552 patch_prober.go:28] interesting pod/e2e-derive-357aed65-openshift-default-65599f675b-srqvl container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 11 17:33:51.086791 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:51.086454    2552 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-derive-357aed65-openshift-default-65599f675b-srqvl" podUID="1b39a4c0-6d65-48fb-8d38-9a9523b9c154" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 11 17:33:51.372253 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:51.372215    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:33:52.164128 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:52.164090    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:33:52.997203 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:52.997140    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-357aed65-openshift-default-65599f675b-srqvl"
Sep 11 17:33:53.084454 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:53.084421    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b39a4c0-6d65-48fb-8d38-9a9523b9c154-workload-socket\") pod \"1b39a4c0-6d65-48fb-8d38-9a9523b9c154\" (UID: \"1b39a4c0-6d65-48fb-8d38-9a9523b9c154\") "
Sep 11 17:33:53.084454 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:53.084459    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1b39a4c0-6d65-48fb-8d38-9a9523b9c154-istio-data\") pod \"1b39a4c0-6d65-48fb-8d38-9a9523b9c154\" (UID: \"1b39a4c0-6d65-48fb-8d38-9a9523b9c154\") "
Sep 11 17:33:53.084697 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:53.084502    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1b39a4c0-6d65-48fb-8d38-9a9523b9c154-istio-podinfo\") pod \"1b39a4c0-6d65-48fb-8d38-9a9523b9c154\" (UID: \"1b39a4c0-6d65-48fb-8d38-9a9523b9c154\") "
Sep 11 17:33:53.084697 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:53.084522    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1b39a4c0-6d65-48fb-8d38-9a9523b9c154-istiod-ca-cert\") pod \"1b39a4c0-6d65-48fb-8d38-9a9523b9c154\" (UID: \"1b39a4c0-6d65-48fb-8d38-9a9523b9c154\") "
Sep 11 17:33:53.084697 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:53.084544    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1b39a4c0-6d65-48fb-8d38-9a9523b9c154-istio-token\") pod \"1b39a4c0-6d65-48fb-8d38-9a9523b9c154\" (UID: \"1b39a4c0-6d65-48fb-8d38-9a9523b9c154\") "
Sep 11 17:33:53.084697 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:53.084571    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ql6ft\" (UniqueName: \"kubernetes.io/projected/1b39a4c0-6d65-48fb-8d38-9a9523b9c154-kube-api-access-ql6ft\") pod \"1b39a4c0-6d65-48fb-8d38-9a9523b9c154\" (UID: \"1b39a4c0-6d65-48fb-8d38-9a9523b9c154\") "
Sep 11 17:33:53.084697 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:53.084633    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1b39a4c0-6d65-48fb-8d38-9a9523b9c154-istio-envoy\") pod \"1b39a4c0-6d65-48fb-8d38-9a9523b9c154\" (UID: \"1b39a4c0-6d65-48fb-8d38-9a9523b9c154\") "
Sep 11 17:33:53.085004 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:53.084703    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b39a4c0-6d65-48fb-8d38-9a9523b9c154-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1b39a4c0-6d65-48fb-8d38-9a9523b9c154" (UID: "1b39a4c0-6d65-48fb-8d38-9a9523b9c154"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:33:53.085004 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:53.084723    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b39a4c0-6d65-48fb-8d38-9a9523b9c154-credential-socket\") pod \"1b39a4c0-6d65-48fb-8d38-9a9523b9c154\" (UID: \"1b39a4c0-6d65-48fb-8d38-9a9523b9c154\") "
Sep 11 17:33:53.085004 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:53.084747    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b39a4c0-6d65-48fb-8d38-9a9523b9c154-istio-data" (OuterVolumeSpecName: "istio-data") pod "1b39a4c0-6d65-48fb-8d38-9a9523b9c154" (UID: "1b39a4c0-6d65-48fb-8d38-9a9523b9c154"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:33:53.085004 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:53.084923    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b39a4c0-6d65-48fb-8d38-9a9523b9c154-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1b39a4c0-6d65-48fb-8d38-9a9523b9c154" (UID: "1b39a4c0-6d65-48fb-8d38-9a9523b9c154"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:33:53.085004 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:53.084963    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1b39a4c0-6d65-48fb-8d38-9a9523b9c154-workload-certs\") pod \"1b39a4c0-6d65-48fb-8d38-9a9523b9c154\" (UID: \"1b39a4c0-6d65-48fb-8d38-9a9523b9c154\") "
Sep 11 17:33:53.085300 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:53.085214    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b39a4c0-6d65-48fb-8d38-9a9523b9c154-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1b39a4c0-6d65-48fb-8d38-9a9523b9c154" (UID: "1b39a4c0-6d65-48fb-8d38-9a9523b9c154"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:33:53.085601 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:53.085520    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1b39a4c0-6d65-48fb-8d38-9a9523b9c154-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1b39a4c0-6d65-48fb-8d38-9a9523b9c154" (UID: "1b39a4c0-6d65-48fb-8d38-9a9523b9c154"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 17:33:53.085711 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:53.085664    2552 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b39a4c0-6d65-48fb-8d38-9a9523b9c154-credential-socket\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:33:53.085711 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:53.085685    2552 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1b39a4c0-6d65-48fb-8d38-9a9523b9c154-workload-certs\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:33:53.085711 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:53.085700    2552 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b39a4c0-6d65-48fb-8d38-9a9523b9c154-workload-socket\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:33:53.085852 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:53.085713    2552 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1b39a4c0-6d65-48fb-8d38-9a9523b9c154-istio-data\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:33:53.085852 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:53.085727    2552 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1b39a4c0-6d65-48fb-8d38-9a9523b9c154-istiod-ca-cert\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:33:53.086917 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:53.086895    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b39a4c0-6d65-48fb-8d38-9a9523b9c154-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1b39a4c0-6d65-48fb-8d38-9a9523b9c154" (UID: "1b39a4c0-6d65-48fb-8d38-9a9523b9c154"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:33:53.087026 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:53.086983    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b39a4c0-6d65-48fb-8d38-9a9523b9c154-kube-api-access-ql6ft" (OuterVolumeSpecName: "kube-api-access-ql6ft") pod "1b39a4c0-6d65-48fb-8d38-9a9523b9c154" (UID: "1b39a4c0-6d65-48fb-8d38-9a9523b9c154"). InnerVolumeSpecName "kube-api-access-ql6ft". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:33:53.087091 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:53.087070    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1b39a4c0-6d65-48fb-8d38-9a9523b9c154-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1b39a4c0-6d65-48fb-8d38-9a9523b9c154" (UID: "1b39a4c0-6d65-48fb-8d38-9a9523b9c154"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 17:33:53.087280 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:53.087263    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b39a4c0-6d65-48fb-8d38-9a9523b9c154-istio-token" (OuterVolumeSpecName: "istio-token") pod "1b39a4c0-6d65-48fb-8d38-9a9523b9c154" (UID: "1b39a4c0-6d65-48fb-8d38-9a9523b9c154"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:33:53.187037 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:53.187008    2552 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1b39a4c0-6d65-48fb-8d38-9a9523b9c154-istio-envoy\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:33:53.187037 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:53.187031    2552 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1b39a4c0-6d65-48fb-8d38-9a9523b9c154-istio-podinfo\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:33:53.187037 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:53.187040    2552 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1b39a4c0-6d65-48fb-8d38-9a9523b9c154-istio-token\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:33:53.187478 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:53.187049    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ql6ft\" (UniqueName: \"kubernetes.io/projected/1b39a4c0-6d65-48fb-8d38-9a9523b9c154-kube-api-access-ql6ft\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:33:53.207612 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:53.207580    2552 generic.go:356] "Generic (PLEG): container finished" podID="1b39a4c0-6d65-48fb-8d38-9a9523b9c154" containerID="cc236e2588899bc4a0dc11aee9eb6f0bdced8f668776814a4d3ff55c388cdfc7" exitCode=0
Sep 11 17:33:53.207765 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:53.207628    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-357aed65-openshift-default-65599f675b-srqvl"
Sep 11 17:33:53.207765 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:53.207657    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-357aed65-openshift-default-65599f675b-srqvl" event={"ID":"1b39a4c0-6d65-48fb-8d38-9a9523b9c154","Type":"ContainerDied","Data":"cc236e2588899bc4a0dc11aee9eb6f0bdced8f668776814a4d3ff55c388cdfc7"}
Sep 11 17:33:53.207765 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:53.207714    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-357aed65-openshift-default-65599f675b-srqvl" event={"ID":"1b39a4c0-6d65-48fb-8d38-9a9523b9c154","Type":"ContainerDied","Data":"b39e5d94583e412344aeb8a5391947649a5985d2c63a24551fddaf19840eb38d"}
Sep 11 17:33:53.207765 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:53.207737    2552 scope.go:117] "RemoveContainer" containerID="cc236e2588899bc4a0dc11aee9eb6f0bdced8f668776814a4d3ff55c388cdfc7"
Sep 11 17:33:53.218084 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:53.218064    2552 scope.go:117] "RemoveContainer" containerID="0bc49c9c0b458b5f9346728d67af2489c4c02a2797f66ede80b29fd2da5ee43f"
Sep 11 17:33:53.227024 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:53.227005    2552 scope.go:117] "RemoveContainer" containerID="cc236e2588899bc4a0dc11aee9eb6f0bdced8f668776814a4d3ff55c388cdfc7"
Sep 11 17:33:53.227330 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:33:53.227306    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cc236e2588899bc4a0dc11aee9eb6f0bdced8f668776814a4d3ff55c388cdfc7\": container with ID starting with cc236e2588899bc4a0dc11aee9eb6f0bdced8f668776814a4d3ff55c388cdfc7 not found: ID does not exist" containerID="cc236e2588899bc4a0dc11aee9eb6f0bdced8f668776814a4d3ff55c388cdfc7"
Sep 11 17:33:53.227439 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:53.227337    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cc236e2588899bc4a0dc11aee9eb6f0bdced8f668776814a4d3ff55c388cdfc7"} err="failed to get container status \"cc236e2588899bc4a0dc11aee9eb6f0bdced8f668776814a4d3ff55c388cdfc7\": rpc error: code = NotFound desc = could not find container \"cc236e2588899bc4a0dc11aee9eb6f0bdced8f668776814a4d3ff55c388cdfc7\": container with ID starting with cc236e2588899bc4a0dc11aee9eb6f0bdced8f668776814a4d3ff55c388cdfc7 not found: ID does not exist"
Sep 11 17:33:53.227439 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:53.227356    2552 scope.go:117] "RemoveContainer" containerID="0bc49c9c0b458b5f9346728d67af2489c4c02a2797f66ede80b29fd2da5ee43f"
Sep 11 17:33:53.227620 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:33:53.227604    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0bc49c9c0b458b5f9346728d67af2489c4c02a2797f66ede80b29fd2da5ee43f\": container with ID starting with 0bc49c9c0b458b5f9346728d67af2489c4c02a2797f66ede80b29fd2da5ee43f not found: ID does not exist" containerID="0bc49c9c0b458b5f9346728d67af2489c4c02a2797f66ede80b29fd2da5ee43f"
Sep 11 17:33:53.227677 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:53.227629    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0bc49c9c0b458b5f9346728d67af2489c4c02a2797f66ede80b29fd2da5ee43f"} err="failed to get container status \"0bc49c9c0b458b5f9346728d67af2489c4c02a2797f66ede80b29fd2da5ee43f\": rpc error: code = NotFound desc = could not find container \"0bc49c9c0b458b5f9346728d67af2489c4c02a2797f66ede80b29fd2da5ee43f\": container with ID starting with 0bc49c9c0b458b5f9346728d67af2489c4c02a2797f66ede80b29fd2da5ee43f not found: ID does not exist"
Sep 11 17:33:53.231281 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:53.231261    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-357aed65-openshift-default-65599f675b-srqvl"]
Sep 11 17:33:53.239128 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:53.239088    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-357aed65-openshift-default-65599f675b-srqvl"]
Sep 11 17:33:53.951062 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:53.951030    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1b39a4c0-6d65-48fb-8d38-9a9523b9c154" path="/var/lib/kubelet/pods/1b39a4c0-6d65-48fb-8d38-9a9523b9c154/volumes"
Sep 11 17:33:55.624929 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:55.624898    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:33:55.733734 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:55.733705    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:33:55.825790 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:55.825756    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:33:55.924821 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:55.924734    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:33:56.324013 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:56.323922    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:33:59.980774 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:59.980742    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-08c6b2/test-model-08c6b2-kserve-78ff75c84c-fkhts"]
Sep 11 17:33:59.981342 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:59.981324    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b39a4c0-6d65-48fb-8d38-9a9523b9c154" containerName="istio-proxy"
Sep 11 17:33:59.981342 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:59.981343    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b39a4c0-6d65-48fb-8d38-9a9523b9c154" containerName="istio-proxy"
Sep 11 17:33:59.981508 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:59.981353    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33a15c58-dee4-44d2-9ebb-7071420d82d8" containerName="maas-api"
Sep 11 17:33:59.981508 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:59.981360    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="33a15c58-dee4-44d2-9ebb-7071420d82d8" containerName="maas-api"
Sep 11 17:33:59.981508 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:59.981389    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b39a4c0-6d65-48fb-8d38-9a9523b9c154" containerName="istio-proxy"
Sep 11 17:33:59.981508 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:59.981398    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b39a4c0-6d65-48fb-8d38-9a9523b9c154" containerName="istio-proxy"
Sep 11 17:33:59.981508 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:59.981509    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="1b39a4c0-6d65-48fb-8d38-9a9523b9c154" containerName="istio-proxy"
Sep 11 17:33:59.981779 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:59.981521    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="33a15c58-dee4-44d2-9ebb-7071420d82d8" containerName="maas-api"
Sep 11 17:33:59.986125 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:59.986095    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-08c6b2/test-model-08c6b2-kserve-78ff75c84c-fkhts"
Sep 11 17:33:59.989374 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:59.989343    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-08c6b2\"/\"kube-root-ca.crt\""
Sep 11 17:33:59.989491 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:59.989381    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-08c6b2\"/\"openshift-service-ca.crt\""
Sep 11 17:33:59.989491 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:59.989387    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-08c6b2\"/\"default-dockercfg-584fs\""
Sep 11 17:33:59.989491 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:59.989343    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-08c6b2\"/\"test-model-08c6b2-kserve-self-signed-certs\""
Sep 11 17:33:59.995206 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:33:59.995181    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-08c6b2/test-model-08c6b2-kserve-78ff75c84c-fkhts"]
Sep 11 17:34:00.020020 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:34:00.019990    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:34:00.056033 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:34:00.056001    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3f65c572-1df7-45a1-bbf1-4e42e9373fa5-tmp-dir\") pod \"test-model-08c6b2-kserve-78ff75c84c-fkhts\" (UID: \"3f65c572-1df7-45a1-bbf1-4e42e9373fa5\") " pod="ai-tenant-e2e-inf-a-08c6b2/test-model-08c6b2-kserve-78ff75c84c-fkhts"
Sep 11 17:34:00.056033 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:34:00.056036    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3f65c572-1df7-45a1-bbf1-4e42e9373fa5-model-cache\") pod \"test-model-08c6b2-kserve-78ff75c84c-fkhts\" (UID: \"3f65c572-1df7-45a1-bbf1-4e42e9373fa5\") " pod="ai-tenant-e2e-inf-a-08c6b2/test-model-08c6b2-kserve-78ff75c84c-fkhts"
Sep 11 17:34:00.056278 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:34:00.056061    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3f65c572-1df7-45a1-bbf1-4e42e9373fa5-dshm\") pod \"test-model-08c6b2-kserve-78ff75c84c-fkhts\" (UID: \"3f65c572-1df7-45a1-bbf1-4e42e9373fa5\") " pod="ai-tenant-e2e-inf-a-08c6b2/test-model-08c6b2-kserve-78ff75c84c-fkhts"
Sep 11 17:34:00.056278 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:34:00.056185    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g6xz8\" (UniqueName: \"kubernetes.io/projected/3f65c572-1df7-45a1-bbf1-4e42e9373fa5-kube-api-access-g6xz8\") pod \"test-model-08c6b2-kserve-78ff75c84c-fkhts\" (UID: \"3f65c572-1df7-45a1-bbf1-4e42e9373fa5\") " pod="ai-tenant-e2e-inf-a-08c6b2/test-model-08c6b2-kserve-78ff75c84c-fkhts"
Sep 11 17:34:00.056278 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:34:00.056266    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3f65c572-1df7-45a1-bbf1-4e42e9373fa5-tls-certs\") pod \"test-model-08c6b2-kserve-78ff75c84c-fkhts\" (UID: \"3f65c572-1df7-45a1-bbf1-4e42e9373fa5\") " pod="ai-tenant-e2e-inf-a-08c6b2/test-model-08c6b2-kserve-78ff75c84c-fkhts"
Sep 11 17:34:00.056383 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:34:00.056299    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3f65c572-1df7-45a1-bbf1-4e42e9373fa5-home\") pod \"test-model-08c6b2-kserve-78ff75c84c-fkhts\" (UID: \"3f65c572-1df7-45a1-bbf1-4e42e9373fa5\") " pod="ai-tenant-e2e-inf-a-08c6b2/test-model-08c6b2-kserve-78ff75c84c-fkhts"
Sep 11 17:34:00.157375 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:34:00.157339    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3f65c572-1df7-45a1-bbf1-4e42e9373fa5-tmp-dir\") pod \"test-model-08c6b2-kserve-78ff75c84c-fkhts\" (UID: \"3f65c572-1df7-45a1-bbf1-4e42e9373fa5\") " pod="ai-tenant-e2e-inf-a-08c6b2/test-model-08c6b2-kserve-78ff75c84c-fkhts"
Sep 11 17:34:00.157375 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:34:00.157380    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3f65c572-1df7-45a1-bbf1-4e42e9373fa5-model-cache\") pod \"test-model-08c6b2-kserve-78ff75c84c-fkhts\" (UID: \"3f65c572-1df7-45a1-bbf1-4e42e9373fa5\") " pod="ai-tenant-e2e-inf-a-08c6b2/test-model-08c6b2-kserve-78ff75c84c-fkhts"
Sep 11 17:34:00.157614 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:34:00.157409    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3f65c572-1df7-45a1-bbf1-4e42e9373fa5-dshm\") pod \"test-model-08c6b2-kserve-78ff75c84c-fkhts\" (UID: \"3f65c572-1df7-45a1-bbf1-4e42e9373fa5\") " pod="ai-tenant-e2e-inf-a-08c6b2/test-model-08c6b2-kserve-78ff75c84c-fkhts"
Sep 11 17:34:00.157614 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:34:00.157457    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g6xz8\" (UniqueName: \"kubernetes.io/projected/3f65c572-1df7-45a1-bbf1-4e42e9373fa5-kube-api-access-g6xz8\") pod \"test-model-08c6b2-kserve-78ff75c84c-fkhts\" (UID: \"3f65c572-1df7-45a1-bbf1-4e42e9373fa5\") " pod="ai-tenant-e2e-inf-a-08c6b2/test-model-08c6b2-kserve-78ff75c84c-fkhts"
Sep 11 17:34:00.157614 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:34:00.157540    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3f65c572-1df7-45a1-bbf1-4e42e9373fa5-tls-certs\") pod \"test-model-08c6b2-kserve-78ff75c84c-fkhts\" (UID: \"3f65c572-1df7-45a1-bbf1-4e42e9373fa5\") " pod="ai-tenant-e2e-inf-a-08c6b2/test-model-08c6b2-kserve-78ff75c84c-fkhts"
Sep 11 17:34:00.157614 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:34:00.157578    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3f65c572-1df7-45a1-bbf1-4e42e9373fa5-home\") pod \"test-model-08c6b2-kserve-78ff75c84c-fkhts\" (UID: \"3f65c572-1df7-45a1-bbf1-4e42e9373fa5\") " pod="ai-tenant-e2e-inf-a-08c6b2/test-model-08c6b2-kserve-78ff75c84c-fkhts"
Sep 11 17:34:00.157828 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:34:00.157776    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3f65c572-1df7-45a1-bbf1-4e42e9373fa5-model-cache\") pod \"test-model-08c6b2-kserve-78ff75c84c-fkhts\" (UID: \"3f65c572-1df7-45a1-bbf1-4e42e9373fa5\") " pod="ai-tenant-e2e-inf-a-08c6b2/test-model-08c6b2-kserve-78ff75c84c-fkhts"
Sep 11 17:34:00.157886 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:34:00.157825    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3f65c572-1df7-45a1-bbf1-4e42e9373fa5-tmp-dir\") pod \"test-model-08c6b2-kserve-78ff75c84c-fkhts\" (UID: \"3f65c572-1df7-45a1-bbf1-4e42e9373fa5\") " pod="ai-tenant-e2e-inf-a-08c6b2/test-model-08c6b2-kserve-78ff75c84c-fkhts"
Sep 11 17:34:00.157964 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:34:00.157944    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3f65c572-1df7-45a1-bbf1-4e42e9373fa5-home\") pod \"test-model-08c6b2-kserve-78ff75c84c-fkhts\" (UID: \"3f65c572-1df7-45a1-bbf1-4e42e9373fa5\") " pod="ai-tenant-e2e-inf-a-08c6b2/test-model-08c6b2-kserve-78ff75c84c-fkhts"
Sep 11 17:34:00.159861 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:34:00.159835    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3f65c572-1df7-45a1-bbf1-4e42e9373fa5-dshm\") pod \"test-model-08c6b2-kserve-78ff75c84c-fkhts\" (UID: \"3f65c572-1df7-45a1-bbf1-4e42e9373fa5\") " pod="ai-tenant-e2e-inf-a-08c6b2/test-model-08c6b2-kserve-78ff75c84c-fkhts"
Sep 11 17:34:00.159978 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:34:00.159959    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3f65c572-1df7-45a1-bbf1-4e42e9373fa5-tls-certs\") pod \"test-model-08c6b2-kserve-78ff75c84c-fkhts\" (UID: \"3f65c572-1df7-45a1-bbf1-4e42e9373fa5\") " pod="ai-tenant-e2e-inf-a-08c6b2/test-model-08c6b2-kserve-78ff75c84c-fkhts"
Sep 11 17:34:00.165856 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:34:00.165834    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g6xz8\" (UniqueName: \"kubernetes.io/projected/3f65c572-1df7-45a1-bbf1-4e42e9373fa5-kube-api-access-g6xz8\") pod \"test-model-08c6b2-kserve-78ff75c84c-fkhts\" (UID: \"3f65c572-1df7-45a1-bbf1-4e42e9373fa5\") " pod="ai-tenant-e2e-inf-a-08c6b2/test-model-08c6b2-kserve-78ff75c84c-fkhts"
Sep 11 17:34:00.297412 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:34:00.297312    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-08c6b2/test-model-08c6b2-kserve-78ff75c84c-fkhts"
Sep 11 17:34:00.576487 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:34:00.576464    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-08c6b2/test-model-08c6b2-kserve-78ff75c84c-fkhts"]
Sep 11 17:34:00.578740 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:34:00.578711    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3f65c572_1df7_45a1_bbf1_4e42e9373fa5.slice/crio-1717ca8079bd419e66e76b39c0c8ab31e26592c9e6ae794289711c070f6f3852 WatchSource:0}: Error finding container 1717ca8079bd419e66e76b39c0c8ab31e26592c9e6ae794289711c070f6f3852: Status 404 returned error can't find the container with id 1717ca8079bd419e66e76b39c0c8ab31e26592c9e6ae794289711c070f6f3852
Sep 11 17:34:01.259595 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:34:01.259561    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-08c6b2/test-model-08c6b2-kserve-78ff75c84c-fkhts" event={"ID":"3f65c572-1df7-45a1-bbf1-4e42e9373fa5","Type":"ContainerStarted","Data":"bb2087a15121b0bf4a77c3281aac86e380a72cf8544609d3209830c472db4da6"}
Sep 11 17:34:01.259991 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:34:01.259610    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-08c6b2/test-model-08c6b2-kserve-78ff75c84c-fkhts" event={"ID":"3f65c572-1df7-45a1-bbf1-4e42e9373fa5","Type":"ContainerStarted","Data":"1717ca8079bd419e66e76b39c0c8ab31e26592c9e6ae794289711c070f6f3852"}
Sep 11 17:34:01.276273 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:34:01.276204    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-08c6b2/test-model-08c6b2-kserve-78ff75c84c-fkhts" podStartSLOduration=2.27618974 podStartE2EDuration="2.27618974s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:33:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:34:01.27504684 +0000 UTC m=+935.796338367" watchObservedRunningTime="2026-09-11 17:34:01.27618974 +0000 UTC m=+935.797481268"
Sep 11 17:34:10.297572 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:34:10.297535    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-08c6b2/test-model-08c6b2-kserve-78ff75c84c-fkhts"
Sep 11 17:34:10.297572 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:34:10.297581    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-08c6b2/test-model-08c6b2-kserve-78ff75c84c-fkhts"
Sep 11 17:34:10.310494 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:34:10.310468    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-08c6b2/test-model-08c6b2-kserve-78ff75c84c-fkhts"
Sep 11 17:34:10.326675 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:34:10.326647    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-08c6b2/test-model-08c6b2-kserve-78ff75c84c-fkhts"
Sep 11 17:34:11.131314 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:34:11.131261    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:34:11.197543 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:34:11.197493    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:34:19.608029 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:34:19.607997    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:34:20.320297 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:34:20.320264    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:34:21.130277 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:34:21.130239    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:34:21.803728 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:34:21.803689    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:34:22.231486 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:34:22.231454    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:34:22.529556 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:34:22.529470    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:34:23.355357 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:34:23.355325    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:34:24.069290 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:34:24.069255    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:34:26.975315 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:34:26.975278    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:34:28.192352 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:34:28.192321    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:34:28.262918 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:34:28.262882    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:34:28.399389 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:34:28.399352    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:34:29.951049 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:34:29.951017    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-49e398-86698d695f-f447p"]
Sep 11 17:34:29.951538 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:34:29.951313    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-autores-49e398-86698d695f-f447p" podUID="107916e5-934a-406e-b26c-b668814b6285" containerName="maas-api" containerID="cri-o://d87ef3cdc4d2be6fda9b8bae588afb10791b604a4916e963f9bfea01b74cf8a6" gracePeriod=30
Sep 11 17:34:29.984012 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:34:29.983985    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:34:30.062836 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:34:30.062803    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:34:30.204274 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:34:30.204201    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-49e398-86698d695f-f447p"
Sep 11 17:34:30.260495 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:34:30.260458    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/107916e5-934a-406e-b26c-b668814b6285-openshift-service-ca\") pod \"107916e5-934a-406e-b26c-b668814b6285\" (UID: \"107916e5-934a-406e-b26c-b668814b6285\") "
Sep 11 17:34:30.260665 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:34:30.260530    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/107916e5-934a-406e-b26c-b668814b6285-maas-api-tls\") pod \"107916e5-934a-406e-b26c-b668814b6285\" (UID: \"107916e5-934a-406e-b26c-b668814b6285\") "
Sep 11 17:34:30.260665 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:34:30.260575    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qhpxz\" (UniqueName: \"kubernetes.io/projected/107916e5-934a-406e-b26c-b668814b6285-kube-api-access-qhpxz\") pod \"107916e5-934a-406e-b26c-b668814b6285\" (UID: \"107916e5-934a-406e-b26c-b668814b6285\") "
Sep 11 17:34:30.260665 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:34:30.260605    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/107916e5-934a-406e-b26c-b668814b6285-metrics-certs\") pod \"107916e5-934a-406e-b26c-b668814b6285\" (UID: \"107916e5-934a-406e-b26c-b668814b6285\") "
Sep 11 17:34:30.260892 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:34:30.260869    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/107916e5-934a-406e-b26c-b668814b6285-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "107916e5-934a-406e-b26c-b668814b6285" (UID: "107916e5-934a-406e-b26c-b668814b6285"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 17:34:30.262774 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:34:30.262741    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/107916e5-934a-406e-b26c-b668814b6285-kube-api-access-qhpxz" (OuterVolumeSpecName: "kube-api-access-qhpxz") pod "107916e5-934a-406e-b26c-b668814b6285" (UID: "107916e5-934a-406e-b26c-b668814b6285"). InnerVolumeSpecName "kube-api-access-qhpxz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:34:30.262895 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:34:30.262801    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/107916e5-934a-406e-b26c-b668814b6285-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "107916e5-934a-406e-b26c-b668814b6285" (UID: "107916e5-934a-406e-b26c-b668814b6285"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 17:34:30.262895 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:34:30.262856    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/107916e5-934a-406e-b26c-b668814b6285-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "107916e5-934a-406e-b26c-b668814b6285" (UID: "107916e5-934a-406e-b26c-b668814b6285"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 17:34:30.264719 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:34:30.264678    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:34:30.363306 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:34:30.362478    2552 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/107916e5-934a-406e-b26c-b668814b6285-maas-api-tls\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:34:30.363306 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:34:30.362511    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qhpxz\" (UniqueName: \"kubernetes.io/projected/107916e5-934a-406e-b26c-b668814b6285-kube-api-access-qhpxz\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:34:30.363306 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:34:30.362528    2552 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/107916e5-934a-406e-b26c-b668814b6285-metrics-certs\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:34:30.363306 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:34:30.362543    2552 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/107916e5-934a-406e-b26c-b668814b6285-openshift-service-ca\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:34:30.370684 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:34:30.370647    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:34:30.437316 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:34:30.437280    2552 generic.go:356] "Generic (PLEG): container finished" podID="107916e5-934a-406e-b26c-b668814b6285" containerID="d87ef3cdc4d2be6fda9b8bae588afb10791b604a4916e963f9bfea01b74cf8a6" exitCode=0
Sep 11 17:34:30.437503 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:34:30.437336    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-49e398-86698d695f-f447p"
Sep 11 17:34:30.437503 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:34:30.437367    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-49e398-86698d695f-f447p" event={"ID":"107916e5-934a-406e-b26c-b668814b6285","Type":"ContainerDied","Data":"d87ef3cdc4d2be6fda9b8bae588afb10791b604a4916e963f9bfea01b74cf8a6"}
Sep 11 17:34:30.437503 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:34:30.437439    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-49e398-86698d695f-f447p" event={"ID":"107916e5-934a-406e-b26c-b668814b6285","Type":"ContainerDied","Data":"b73b683c6751331b9c5bf4dc88c1da6e8211a5bf8224ac5ba5d88c17c40c410f"}
Sep 11 17:34:30.437503 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:34:30.437465    2552 scope.go:117] "RemoveContainer" containerID="d87ef3cdc4d2be6fda9b8bae588afb10791b604a4916e963f9bfea01b74cf8a6"
Sep 11 17:34:30.448511 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:34:30.448494    2552 scope.go:117] "RemoveContainer" containerID="d87ef3cdc4d2be6fda9b8bae588afb10791b604a4916e963f9bfea01b74cf8a6"
Sep 11 17:34:30.448835 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:34:30.448814    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d87ef3cdc4d2be6fda9b8bae588afb10791b604a4916e963f9bfea01b74cf8a6\": container with ID starting with d87ef3cdc4d2be6fda9b8bae588afb10791b604a4916e963f9bfea01b74cf8a6 not found: ID does not exist" containerID="d87ef3cdc4d2be6fda9b8bae588afb10791b604a4916e963f9bfea01b74cf8a6"
Sep 11 17:34:30.448922 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:34:30.448844    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d87ef3cdc4d2be6fda9b8bae588afb10791b604a4916e963f9bfea01b74cf8a6"} err="failed to get container status \"d87ef3cdc4d2be6fda9b8bae588afb10791b604a4916e963f9bfea01b74cf8a6\": rpc error: code = NotFound desc = could not find container \"d87ef3cdc4d2be6fda9b8bae588afb10791b604a4916e963f9bfea01b74cf8a6\": container with ID starting with d87ef3cdc4d2be6fda9b8bae588afb10791b604a4916e963f9bfea01b74cf8a6 not found: ID does not exist"
Sep 11 17:34:30.461176 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:34:30.461103    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-49e398-86698d695f-f447p"]
Sep 11 17:34:30.466819 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:34:30.466798    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-49e398-86698d695f-f447p"]
Sep 11 17:34:30.499161 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:34:30.499135    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:34:30.599696 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:34:30.599663    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:34:31.451406 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:34:31.451376    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:34:31.950650 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:34:31.950613    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="107916e5-934a-406e-b26c-b668814b6285" path="/var/lib/kubelet/pods/107916e5-934a-406e-b26c-b668814b6285/volumes"
Sep 11 17:34:32.131037 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:34:32.131002    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:34:32.920023 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:34:32.919982    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:34:33.838209 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:34:33.838166    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:34:35.645171 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:34:35.645131    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:34:44.279144 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:34:44.279094    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:34:50.289515 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:34:50.289436    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:34:50.362443 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:34:50.362401    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:34:50.443367 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:34:50.443334    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:35:00.962802 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:00.962758    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:35:01.727674 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:01.727633    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:35:03.199672 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:03.199636    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-144bfa/premium-w4-144bfa-kserve-5b5658659d-7v9nm"]
Sep 11 17:35:03.200094 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:03.200066    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="107916e5-934a-406e-b26c-b668814b6285" containerName="maas-api"
Sep 11 17:35:03.200094 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:03.200086    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="107916e5-934a-406e-b26c-b668814b6285" containerName="maas-api"
Sep 11 17:35:03.200253 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:03.200234    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="1b39a4c0-6d65-48fb-8d38-9a9523b9c154" containerName="istio-proxy"
Sep 11 17:35:03.200253 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:03.200251    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="107916e5-934a-406e-b26c-b668814b6285" containerName="maas-api"
Sep 11 17:35:03.203337 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:03.203317    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-144bfa/premium-w4-144bfa-kserve-5b5658659d-7v9nm"
Sep 11 17:35:03.206356 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:03.206330    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-144bfa\"/\"openshift-service-ca.crt\""
Sep 11 17:35:03.206477 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:03.206330    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-144bfa\"/\"default-dockercfg-chr7w\""
Sep 11 17:35:03.206477 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:03.206334    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-144bfa\"/\"kube-root-ca.crt\""
Sep 11 17:35:03.206477 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:03.206405    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-144bfa\"/\"premium-w4-144bfa-kserve-self-signed-certs\""
Sep 11 17:35:03.213833 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:03.213803    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-144bfa/premium-w4-144bfa-kserve-5b5658659d-7v9nm"]
Sep 11 17:35:03.241538 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:03.241495    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:35:03.389935 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:03.389903    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/01c753eb-e3ca-4a2b-8b9b-c3cc7b672d7d-home\") pod \"premium-w4-144bfa-kserve-5b5658659d-7v9nm\" (UID: \"01c753eb-e3ca-4a2b-8b9b-c3cc7b672d7d\") " pod="e2e-models-e2e-worker-w4-144bfa/premium-w4-144bfa-kserve-5b5658659d-7v9nm"
Sep 11 17:35:03.390162 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:03.389942    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/01c753eb-e3ca-4a2b-8b9b-c3cc7b672d7d-dshm\") pod \"premium-w4-144bfa-kserve-5b5658659d-7v9nm\" (UID: \"01c753eb-e3ca-4a2b-8b9b-c3cc7b672d7d\") " pod="e2e-models-e2e-worker-w4-144bfa/premium-w4-144bfa-kserve-5b5658659d-7v9nm"
Sep 11 17:35:03.390162 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:03.389971    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/01c753eb-e3ca-4a2b-8b9b-c3cc7b672d7d-tmp-dir\") pod \"premium-w4-144bfa-kserve-5b5658659d-7v9nm\" (UID: \"01c753eb-e3ca-4a2b-8b9b-c3cc7b672d7d\") " pod="e2e-models-e2e-worker-w4-144bfa/premium-w4-144bfa-kserve-5b5658659d-7v9nm"
Sep 11 17:35:03.390162 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:03.390052    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wkrps\" (UniqueName: \"kubernetes.io/projected/01c753eb-e3ca-4a2b-8b9b-c3cc7b672d7d-kube-api-access-wkrps\") pod \"premium-w4-144bfa-kserve-5b5658659d-7v9nm\" (UID: \"01c753eb-e3ca-4a2b-8b9b-c3cc7b672d7d\") " pod="e2e-models-e2e-worker-w4-144bfa/premium-w4-144bfa-kserve-5b5658659d-7v9nm"
Sep 11 17:35:03.390162 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:03.390093    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/01c753eb-e3ca-4a2b-8b9b-c3cc7b672d7d-tls-certs\") pod \"premium-w4-144bfa-kserve-5b5658659d-7v9nm\" (UID: \"01c753eb-e3ca-4a2b-8b9b-c3cc7b672d7d\") " pod="e2e-models-e2e-worker-w4-144bfa/premium-w4-144bfa-kserve-5b5658659d-7v9nm"
Sep 11 17:35:03.390162 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:03.390133    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/01c753eb-e3ca-4a2b-8b9b-c3cc7b672d7d-model-cache\") pod \"premium-w4-144bfa-kserve-5b5658659d-7v9nm\" (UID: \"01c753eb-e3ca-4a2b-8b9b-c3cc7b672d7d\") " pod="e2e-models-e2e-worker-w4-144bfa/premium-w4-144bfa-kserve-5b5658659d-7v9nm"
Sep 11 17:35:03.491473 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:03.491364    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/01c753eb-e3ca-4a2b-8b9b-c3cc7b672d7d-home\") pod \"premium-w4-144bfa-kserve-5b5658659d-7v9nm\" (UID: \"01c753eb-e3ca-4a2b-8b9b-c3cc7b672d7d\") " pod="e2e-models-e2e-worker-w4-144bfa/premium-w4-144bfa-kserve-5b5658659d-7v9nm"
Sep 11 17:35:03.491473 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:03.491425    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/01c753eb-e3ca-4a2b-8b9b-c3cc7b672d7d-dshm\") pod \"premium-w4-144bfa-kserve-5b5658659d-7v9nm\" (UID: \"01c753eb-e3ca-4a2b-8b9b-c3cc7b672d7d\") " pod="e2e-models-e2e-worker-w4-144bfa/premium-w4-144bfa-kserve-5b5658659d-7v9nm"
Sep 11 17:35:03.491473 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:03.491460    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/01c753eb-e3ca-4a2b-8b9b-c3cc7b672d7d-tmp-dir\") pod \"premium-w4-144bfa-kserve-5b5658659d-7v9nm\" (UID: \"01c753eb-e3ca-4a2b-8b9b-c3cc7b672d7d\") " pod="e2e-models-e2e-worker-w4-144bfa/premium-w4-144bfa-kserve-5b5658659d-7v9nm"
Sep 11 17:35:03.491759 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:03.491499    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wkrps\" (UniqueName: \"kubernetes.io/projected/01c753eb-e3ca-4a2b-8b9b-c3cc7b672d7d-kube-api-access-wkrps\") pod \"premium-w4-144bfa-kserve-5b5658659d-7v9nm\" (UID: \"01c753eb-e3ca-4a2b-8b9b-c3cc7b672d7d\") " pod="e2e-models-e2e-worker-w4-144bfa/premium-w4-144bfa-kserve-5b5658659d-7v9nm"
Sep 11 17:35:03.491759 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:03.491536    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/01c753eb-e3ca-4a2b-8b9b-c3cc7b672d7d-tls-certs\") pod \"premium-w4-144bfa-kserve-5b5658659d-7v9nm\" (UID: \"01c753eb-e3ca-4a2b-8b9b-c3cc7b672d7d\") " pod="e2e-models-e2e-worker-w4-144bfa/premium-w4-144bfa-kserve-5b5658659d-7v9nm"
Sep 11 17:35:03.491759 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:03.491559    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/01c753eb-e3ca-4a2b-8b9b-c3cc7b672d7d-model-cache\") pod \"premium-w4-144bfa-kserve-5b5658659d-7v9nm\" (UID: \"01c753eb-e3ca-4a2b-8b9b-c3cc7b672d7d\") " pod="e2e-models-e2e-worker-w4-144bfa/premium-w4-144bfa-kserve-5b5658659d-7v9nm"
Sep 11 17:35:03.491874 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:03.491823    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/01c753eb-e3ca-4a2b-8b9b-c3cc7b672d7d-home\") pod \"premium-w4-144bfa-kserve-5b5658659d-7v9nm\" (UID: \"01c753eb-e3ca-4a2b-8b9b-c3cc7b672d7d\") " pod="e2e-models-e2e-worker-w4-144bfa/premium-w4-144bfa-kserve-5b5658659d-7v9nm"
Sep 11 17:35:03.491874 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:03.491866    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/01c753eb-e3ca-4a2b-8b9b-c3cc7b672d7d-tmp-dir\") pod \"premium-w4-144bfa-kserve-5b5658659d-7v9nm\" (UID: \"01c753eb-e3ca-4a2b-8b9b-c3cc7b672d7d\") " pod="e2e-models-e2e-worker-w4-144bfa/premium-w4-144bfa-kserve-5b5658659d-7v9nm"
Sep 11 17:35:03.492013 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:03.491985    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/01c753eb-e3ca-4a2b-8b9b-c3cc7b672d7d-model-cache\") pod \"premium-w4-144bfa-kserve-5b5658659d-7v9nm\" (UID: \"01c753eb-e3ca-4a2b-8b9b-c3cc7b672d7d\") " pod="e2e-models-e2e-worker-w4-144bfa/premium-w4-144bfa-kserve-5b5658659d-7v9nm"
Sep 11 17:35:03.493811 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:03.493791    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/01c753eb-e3ca-4a2b-8b9b-c3cc7b672d7d-dshm\") pod \"premium-w4-144bfa-kserve-5b5658659d-7v9nm\" (UID: \"01c753eb-e3ca-4a2b-8b9b-c3cc7b672d7d\") " pod="e2e-models-e2e-worker-w4-144bfa/premium-w4-144bfa-kserve-5b5658659d-7v9nm"
Sep 11 17:35:03.494272 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:03.494250    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/01c753eb-e3ca-4a2b-8b9b-c3cc7b672d7d-tls-certs\") pod \"premium-w4-144bfa-kserve-5b5658659d-7v9nm\" (UID: \"01c753eb-e3ca-4a2b-8b9b-c3cc7b672d7d\") " pod="e2e-models-e2e-worker-w4-144bfa/premium-w4-144bfa-kserve-5b5658659d-7v9nm"
Sep 11 17:35:03.499353 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:03.499328    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wkrps\" (UniqueName: \"kubernetes.io/projected/01c753eb-e3ca-4a2b-8b9b-c3cc7b672d7d-kube-api-access-wkrps\") pod \"premium-w4-144bfa-kserve-5b5658659d-7v9nm\" (UID: \"01c753eb-e3ca-4a2b-8b9b-c3cc7b672d7d\") " pod="e2e-models-e2e-worker-w4-144bfa/premium-w4-144bfa-kserve-5b5658659d-7v9nm"
Sep 11 17:35:03.514247 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:03.514210    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-144bfa/premium-w4-144bfa-kserve-5b5658659d-7v9nm"
Sep 11 17:35:03.787672 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:03.787644    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-144bfa/premium-w4-144bfa-kserve-5b5658659d-7v9nm"]
Sep 11 17:35:03.789003 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:35:03.788974    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod01c753eb_e3ca_4a2b_8b9b_c3cc7b672d7d.slice/crio-c70aeaa5ae45a5528062b573f8f05866690b36c0ab2eddbc143baeb569a0ba59 WatchSource:0}: Error finding container c70aeaa5ae45a5528062b573f8f05866690b36c0ab2eddbc143baeb569a0ba59: Status 404 returned error can't find the container with id c70aeaa5ae45a5528062b573f8f05866690b36c0ab2eddbc143baeb569a0ba59
Sep 11 17:35:04.639811 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:04.639773    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-144bfa/premium-w4-144bfa-kserve-5b5658659d-7v9nm" event={"ID":"01c753eb-e3ca-4a2b-8b9b-c3cc7b672d7d","Type":"ContainerStarted","Data":"a174e2857e9e118fd25945086e1a141898e7c83254e39debfb06d0c48f793f2b"}
Sep 11 17:35:04.640341 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:04.639827    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-144bfa/premium-w4-144bfa-kserve-5b5658659d-7v9nm" event={"ID":"01c753eb-e3ca-4a2b-8b9b-c3cc7b672d7d","Type":"ContainerStarted","Data":"c70aeaa5ae45a5528062b573f8f05866690b36c0ab2eddbc143baeb569a0ba59"}
Sep 11 17:35:04.660302 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:04.660249    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-144bfa/premium-w4-144bfa-kserve-5b5658659d-7v9nm" podStartSLOduration=1.660232345 podStartE2EDuration="1.660232345s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:35:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:35:04.657639324 +0000 UTC m=+999.178930845" watchObservedRunningTime="2026-09-11 17:35:04.660232345 +0000 UTC m=+999.181523866"
Sep 11 17:35:07.434755 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:07.434718    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:35:07.500302 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:07.500264    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:35:07.575130 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:07.575072    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:35:07.674978 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:07.674946    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:35:07.775774 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:07.775676    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:35:08.532466 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:08.532428    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:35:08.637150 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:08.637079    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:35:10.051375 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:10.051340    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:35:12.378244 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:12.378203    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:35:12.595197 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:12.595158    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-autores-49e398-openshift-default-66cbbd759d-nf4cx"]
Sep 11 17:35:12.595455 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:12.595429    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-autores-49e398-openshift-default-66cbbd759d-nf4cx" podUID="8d4763e0-c0ce-4b9a-8a05-ff4f718b96f0" containerName="istio-proxy" containerID="cri-o://1a3257d07557fccd8f9c1155dcbb8e3e87106db299bf99a2df5f37bebe89299d" gracePeriod=30
Sep 11 17:35:12.617719 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:12.617683    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:35:13.105961 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:13.105932    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:35:13.296313 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:13.296280    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-explicit-ec3fed-openshift-default-6fdf496464-f2z6f"]
Sep 11 17:35:13.300213 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:13.300188    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-ec3fed-openshift-default-6fdf496464-f2z6f"
Sep 11 17:35:13.302402 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:13.302381    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-explicit-ec3fed-openshift-default-dockercfg-45w7l\""
Sep 11 17:35:13.311980 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:13.311958    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-explicit-ec3fed-openshift-default-6fdf496464-f2z6f"]
Sep 11 17:35:13.325948 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:13.325920    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:35:13.383672 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:13.383581    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/411db317-1635-45d6-8a64-d6eaf0b4ae92-istio-data\") pod \"e2e-explicit-ec3fed-openshift-default-6fdf496464-f2z6f\" (UID: \"411db317-1635-45d6-8a64-d6eaf0b4ae92\") " pod="openshift-ingress/e2e-explicit-ec3fed-openshift-default-6fdf496464-f2z6f"
Sep 11 17:35:13.383672 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:13.383619    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/411db317-1635-45d6-8a64-d6eaf0b4ae92-istio-envoy\") pod \"e2e-explicit-ec3fed-openshift-default-6fdf496464-f2z6f\" (UID: \"411db317-1635-45d6-8a64-d6eaf0b4ae92\") " pod="openshift-ingress/e2e-explicit-ec3fed-openshift-default-6fdf496464-f2z6f"
Sep 11 17:35:13.383672 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:13.383638    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/411db317-1635-45d6-8a64-d6eaf0b4ae92-workload-certs\") pod \"e2e-explicit-ec3fed-openshift-default-6fdf496464-f2z6f\" (UID: \"411db317-1635-45d6-8a64-d6eaf0b4ae92\") " pod="openshift-ingress/e2e-explicit-ec3fed-openshift-default-6fdf496464-f2z6f"
Sep 11 17:35:13.384158 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:13.383722    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cjtxs\" (UniqueName: \"kubernetes.io/projected/411db317-1635-45d6-8a64-d6eaf0b4ae92-kube-api-access-cjtxs\") pod \"e2e-explicit-ec3fed-openshift-default-6fdf496464-f2z6f\" (UID: \"411db317-1635-45d6-8a64-d6eaf0b4ae92\") " pod="openshift-ingress/e2e-explicit-ec3fed-openshift-default-6fdf496464-f2z6f"
Sep 11 17:35:13.384158 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:13.383800    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/411db317-1635-45d6-8a64-d6eaf0b4ae92-istiod-ca-cert\") pod \"e2e-explicit-ec3fed-openshift-default-6fdf496464-f2z6f\" (UID: \"411db317-1635-45d6-8a64-d6eaf0b4ae92\") " pod="openshift-ingress/e2e-explicit-ec3fed-openshift-default-6fdf496464-f2z6f"
Sep 11 17:35:13.384158 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:13.383867    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/411db317-1635-45d6-8a64-d6eaf0b4ae92-credential-socket\") pod \"e2e-explicit-ec3fed-openshift-default-6fdf496464-f2z6f\" (UID: \"411db317-1635-45d6-8a64-d6eaf0b4ae92\") " pod="openshift-ingress/e2e-explicit-ec3fed-openshift-default-6fdf496464-f2z6f"
Sep 11 17:35:13.384158 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:13.383934    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/411db317-1635-45d6-8a64-d6eaf0b4ae92-istio-token\") pod \"e2e-explicit-ec3fed-openshift-default-6fdf496464-f2z6f\" (UID: \"411db317-1635-45d6-8a64-d6eaf0b4ae92\") " pod="openshift-ingress/e2e-explicit-ec3fed-openshift-default-6fdf496464-f2z6f"
Sep 11 17:35:13.384158 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:13.383964    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/411db317-1635-45d6-8a64-d6eaf0b4ae92-workload-socket\") pod \"e2e-explicit-ec3fed-openshift-default-6fdf496464-f2z6f\" (UID: \"411db317-1635-45d6-8a64-d6eaf0b4ae92\") " pod="openshift-ingress/e2e-explicit-ec3fed-openshift-default-6fdf496464-f2z6f"
Sep 11 17:35:13.384158 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:13.383990    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/411db317-1635-45d6-8a64-d6eaf0b4ae92-istio-podinfo\") pod \"e2e-explicit-ec3fed-openshift-default-6fdf496464-f2z6f\" (UID: \"411db317-1635-45d6-8a64-d6eaf0b4ae92\") " pod="openshift-ingress/e2e-explicit-ec3fed-openshift-default-6fdf496464-f2z6f"
Sep 11 17:35:13.484862 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:13.484826    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/411db317-1635-45d6-8a64-d6eaf0b4ae92-istio-data\") pod \"e2e-explicit-ec3fed-openshift-default-6fdf496464-f2z6f\" (UID: \"411db317-1635-45d6-8a64-d6eaf0b4ae92\") " pod="openshift-ingress/e2e-explicit-ec3fed-openshift-default-6fdf496464-f2z6f"
Sep 11 17:35:13.484862 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:13.484862    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/411db317-1635-45d6-8a64-d6eaf0b4ae92-istio-envoy\") pod \"e2e-explicit-ec3fed-openshift-default-6fdf496464-f2z6f\" (UID: \"411db317-1635-45d6-8a64-d6eaf0b4ae92\") " pod="openshift-ingress/e2e-explicit-ec3fed-openshift-default-6fdf496464-f2z6f"
Sep 11 17:35:13.485183 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:13.484891    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/411db317-1635-45d6-8a64-d6eaf0b4ae92-workload-certs\") pod \"e2e-explicit-ec3fed-openshift-default-6fdf496464-f2z6f\" (UID: \"411db317-1635-45d6-8a64-d6eaf0b4ae92\") " pod="openshift-ingress/e2e-explicit-ec3fed-openshift-default-6fdf496464-f2z6f"
Sep 11 17:35:13.485183 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:13.484923    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cjtxs\" (UniqueName: \"kubernetes.io/projected/411db317-1635-45d6-8a64-d6eaf0b4ae92-kube-api-access-cjtxs\") pod \"e2e-explicit-ec3fed-openshift-default-6fdf496464-f2z6f\" (UID: \"411db317-1635-45d6-8a64-d6eaf0b4ae92\") " pod="openshift-ingress/e2e-explicit-ec3fed-openshift-default-6fdf496464-f2z6f"
Sep 11 17:35:13.485183 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:13.484977    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/411db317-1635-45d6-8a64-d6eaf0b4ae92-istiod-ca-cert\") pod \"e2e-explicit-ec3fed-openshift-default-6fdf496464-f2z6f\" (UID: \"411db317-1635-45d6-8a64-d6eaf0b4ae92\") " pod="openshift-ingress/e2e-explicit-ec3fed-openshift-default-6fdf496464-f2z6f"
Sep 11 17:35:13.485440 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:13.485070    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/411db317-1635-45d6-8a64-d6eaf0b4ae92-credential-socket\") pod \"e2e-explicit-ec3fed-openshift-default-6fdf496464-f2z6f\" (UID: \"411db317-1635-45d6-8a64-d6eaf0b4ae92\") " pod="openshift-ingress/e2e-explicit-ec3fed-openshift-default-6fdf496464-f2z6f"
Sep 11 17:35:13.485577 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:13.485477    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/411db317-1635-45d6-8a64-d6eaf0b4ae92-credential-socket\") pod \"e2e-explicit-ec3fed-openshift-default-6fdf496464-f2z6f\" (UID: \"411db317-1635-45d6-8a64-d6eaf0b4ae92\") " pod="openshift-ingress/e2e-explicit-ec3fed-openshift-default-6fdf496464-f2z6f"
Sep 11 17:35:13.485577 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:13.485383    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/411db317-1635-45d6-8a64-d6eaf0b4ae92-workload-certs\") pod \"e2e-explicit-ec3fed-openshift-default-6fdf496464-f2z6f\" (UID: \"411db317-1635-45d6-8a64-d6eaf0b4ae92\") " pod="openshift-ingress/e2e-explicit-ec3fed-openshift-default-6fdf496464-f2z6f"
Sep 11 17:35:13.485577 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:13.485342    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/411db317-1635-45d6-8a64-d6eaf0b4ae92-istio-data\") pod \"e2e-explicit-ec3fed-openshift-default-6fdf496464-f2z6f\" (UID: \"411db317-1635-45d6-8a64-d6eaf0b4ae92\") " pod="openshift-ingress/e2e-explicit-ec3fed-openshift-default-6fdf496464-f2z6f"
Sep 11 17:35:13.485577 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:13.485510    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/411db317-1635-45d6-8a64-d6eaf0b4ae92-istio-token\") pod \"e2e-explicit-ec3fed-openshift-default-6fdf496464-f2z6f\" (UID: \"411db317-1635-45d6-8a64-d6eaf0b4ae92\") " pod="openshift-ingress/e2e-explicit-ec3fed-openshift-default-6fdf496464-f2z6f"
Sep 11 17:35:13.485577 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:13.485573    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/411db317-1635-45d6-8a64-d6eaf0b4ae92-workload-socket\") pod \"e2e-explicit-ec3fed-openshift-default-6fdf496464-f2z6f\" (UID: \"411db317-1635-45d6-8a64-d6eaf0b4ae92\") " pod="openshift-ingress/e2e-explicit-ec3fed-openshift-default-6fdf496464-f2z6f"
Sep 11 17:35:13.485874 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:13.485612    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/411db317-1635-45d6-8a64-d6eaf0b4ae92-istio-podinfo\") pod \"e2e-explicit-ec3fed-openshift-default-6fdf496464-f2z6f\" (UID: \"411db317-1635-45d6-8a64-d6eaf0b4ae92\") " pod="openshift-ingress/e2e-explicit-ec3fed-openshift-default-6fdf496464-f2z6f"
Sep 11 17:35:13.485874 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:13.485736    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/411db317-1635-45d6-8a64-d6eaf0b4ae92-istiod-ca-cert\") pod \"e2e-explicit-ec3fed-openshift-default-6fdf496464-f2z6f\" (UID: \"411db317-1635-45d6-8a64-d6eaf0b4ae92\") " pod="openshift-ingress/e2e-explicit-ec3fed-openshift-default-6fdf496464-f2z6f"
Sep 11 17:35:13.485874 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:13.485858    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/411db317-1635-45d6-8a64-d6eaf0b4ae92-workload-socket\") pod \"e2e-explicit-ec3fed-openshift-default-6fdf496464-f2z6f\" (UID: \"411db317-1635-45d6-8a64-d6eaf0b4ae92\") " pod="openshift-ingress/e2e-explicit-ec3fed-openshift-default-6fdf496464-f2z6f"
Sep 11 17:35:13.487400 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:13.487374    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/411db317-1635-45d6-8a64-d6eaf0b4ae92-istio-envoy\") pod \"e2e-explicit-ec3fed-openshift-default-6fdf496464-f2z6f\" (UID: \"411db317-1635-45d6-8a64-d6eaf0b4ae92\") " pod="openshift-ingress/e2e-explicit-ec3fed-openshift-default-6fdf496464-f2z6f"
Sep 11 17:35:13.488011 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:13.487996    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/411db317-1635-45d6-8a64-d6eaf0b4ae92-istio-podinfo\") pod \"e2e-explicit-ec3fed-openshift-default-6fdf496464-f2z6f\" (UID: \"411db317-1635-45d6-8a64-d6eaf0b4ae92\") " pod="openshift-ingress/e2e-explicit-ec3fed-openshift-default-6fdf496464-f2z6f"
Sep 11 17:35:13.492971 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:13.492948    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/411db317-1635-45d6-8a64-d6eaf0b4ae92-istio-token\") pod \"e2e-explicit-ec3fed-openshift-default-6fdf496464-f2z6f\" (UID: \"411db317-1635-45d6-8a64-d6eaf0b4ae92\") " pod="openshift-ingress/e2e-explicit-ec3fed-openshift-default-6fdf496464-f2z6f"
Sep 11 17:35:13.492971 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:13.492963    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cjtxs\" (UniqueName: \"kubernetes.io/projected/411db317-1635-45d6-8a64-d6eaf0b4ae92-kube-api-access-cjtxs\") pod \"e2e-explicit-ec3fed-openshift-default-6fdf496464-f2z6f\" (UID: \"411db317-1635-45d6-8a64-d6eaf0b4ae92\") " pod="openshift-ingress/e2e-explicit-ec3fed-openshift-default-6fdf496464-f2z6f"
Sep 11 17:35:13.514954 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:13.514923    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-144bfa/premium-w4-144bfa-kserve-5b5658659d-7v9nm"
Sep 11 17:35:13.515081 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:13.514970    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-144bfa/premium-w4-144bfa-kserve-5b5658659d-7v9nm"
Sep 11 17:35:13.527687 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:13.527661    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-144bfa/premium-w4-144bfa-kserve-5b5658659d-7v9nm"
Sep 11 17:35:13.534604 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:13.534577    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:35:13.611979 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:13.611938    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-ec3fed-openshift-default-6fdf496464-f2z6f"
Sep 11 17:35:13.716354 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:13.716292    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-144bfa/premium-w4-144bfa-kserve-5b5658659d-7v9nm"
Sep 11 17:35:13.894413 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:13.894382    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-explicit-ec3fed-openshift-default-6fdf496464-f2z6f"]
Sep 11 17:35:13.895911 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:35:13.895879    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod411db317_1635_45d6_8a64_d6eaf0b4ae92.slice/crio-d084bf47cb0b205e87dfa334acaf4339c8875b3e266edcc2dbd59ae7d0b78e71 WatchSource:0}: Error finding container d084bf47cb0b205e87dfa334acaf4339c8875b3e266edcc2dbd59ae7d0b78e71: Status 404 returned error can't find the container with id d084bf47cb0b205e87dfa334acaf4339c8875b3e266edcc2dbd59ae7d0b78e71
Sep 11 17:35:13.898491 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:13.898455    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 17:35:13.898579 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:13.898540    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 17:35:13.898619 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:13.898583    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 17:35:14.711669 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:14.711633    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-ec3fed-openshift-default-6fdf496464-f2z6f" event={"ID":"411db317-1635-45d6-8a64-d6eaf0b4ae92","Type":"ContainerStarted","Data":"52864b030ef5db52619d6f6b135b7e150e3a2ce74aff90f6b33bdad6ba403c33"}
Sep 11 17:35:14.711669 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:14.711687    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-ec3fed-openshift-default-6fdf496464-f2z6f" event={"ID":"411db317-1635-45d6-8a64-d6eaf0b4ae92","Type":"ContainerStarted","Data":"d084bf47cb0b205e87dfa334acaf4339c8875b3e266edcc2dbd59ae7d0b78e71"}
Sep 11 17:35:14.732926 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:14.732873    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-explicit-ec3fed-openshift-default-6fdf496464-f2z6f" podStartSLOduration=1.732857893 podStartE2EDuration="1.732857893s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:35:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:35:14.730226441 +0000 UTC m=+1009.251517974" watchObservedRunningTime="2026-09-11 17:35:14.732857893 +0000 UTC m=+1009.254149413"
Sep 11 17:35:15.613076 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:15.613031    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-explicit-ec3fed-openshift-default-6fdf496464-f2z6f"
Sep 11 17:35:15.618853 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:15.618826    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-explicit-ec3fed-openshift-default-6fdf496464-f2z6f"
Sep 11 17:35:15.718049 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:15.718015    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-explicit-ec3fed-openshift-default-6fdf496464-f2z6f"
Sep 11 17:35:15.719206 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:15.719186    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-explicit-ec3fed-openshift-default-6fdf496464-f2z6f"
Sep 11 17:35:15.845677 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:15.845637    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:35:15.947870 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:15.947770    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:35:16.513629 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:16.513595    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:35:16.617376 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:16.617333    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:35:16.713878 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:16.713848    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:35:16.909684 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:16.909650    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:35:17.732857 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:17.732826    2552 generic.go:356] "Generic (PLEG): container finished" podID="8d4763e0-c0ce-4b9a-8a05-ff4f718b96f0" containerID="1a3257d07557fccd8f9c1155dcbb8e3e87106db299bf99a2df5f37bebe89299d" exitCode=0
Sep 11 17:35:17.733040 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:17.732966    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:35:17.733040 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:17.733031    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-49e398-openshift-default-66cbbd759d-nf4cx" event={"ID":"8d4763e0-c0ce-4b9a-8a05-ff4f718b96f0","Type":"ContainerDied","Data":"1a3257d07557fccd8f9c1155dcbb8e3e87106db299bf99a2df5f37bebe89299d"}
Sep 11 17:35:17.733241 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:17.733084    2552 scope.go:117] "RemoveContainer" containerID="646c4823f006ad259a058824a1c60f5aab0dddea84724b709ab153aec5eeb7c7"
Sep 11 17:35:17.862996 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:17.862931    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-49e398-openshift-default-66cbbd759d-nf4cx"
Sep 11 17:35:17.933402 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:17.933369    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mp255\" (UniqueName: \"kubernetes.io/projected/8d4763e0-c0ce-4b9a-8a05-ff4f718b96f0-kube-api-access-mp255\") pod \"8d4763e0-c0ce-4b9a-8a05-ff4f718b96f0\" (UID: \"8d4763e0-c0ce-4b9a-8a05-ff4f718b96f0\") "
Sep 11 17:35:17.933813 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:17.933480    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8d4763e0-c0ce-4b9a-8a05-ff4f718b96f0-istio-podinfo\") pod \"8d4763e0-c0ce-4b9a-8a05-ff4f718b96f0\" (UID: \"8d4763e0-c0ce-4b9a-8a05-ff4f718b96f0\") "
Sep 11 17:35:17.933813 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:17.933506    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8d4763e0-c0ce-4b9a-8a05-ff4f718b96f0-istiod-ca-cert\") pod \"8d4763e0-c0ce-4b9a-8a05-ff4f718b96f0\" (UID: \"8d4763e0-c0ce-4b9a-8a05-ff4f718b96f0\") "
Sep 11 17:35:17.933813 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:17.933525    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8d4763e0-c0ce-4b9a-8a05-ff4f718b96f0-istio-envoy\") pod \"8d4763e0-c0ce-4b9a-8a05-ff4f718b96f0\" (UID: \"8d4763e0-c0ce-4b9a-8a05-ff4f718b96f0\") "
Sep 11 17:35:17.933813 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:17.933567    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8d4763e0-c0ce-4b9a-8a05-ff4f718b96f0-istio-token\") pod \"8d4763e0-c0ce-4b9a-8a05-ff4f718b96f0\" (UID: \"8d4763e0-c0ce-4b9a-8a05-ff4f718b96f0\") "
Sep 11 17:35:17.933813 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:17.933601    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8d4763e0-c0ce-4b9a-8a05-ff4f718b96f0-workload-certs\") pod \"8d4763e0-c0ce-4b9a-8a05-ff4f718b96f0\" (UID: \"8d4763e0-c0ce-4b9a-8a05-ff4f718b96f0\") "
Sep 11 17:35:17.933813 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:17.933675    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8d4763e0-c0ce-4b9a-8a05-ff4f718b96f0-workload-socket\") pod \"8d4763e0-c0ce-4b9a-8a05-ff4f718b96f0\" (UID: \"8d4763e0-c0ce-4b9a-8a05-ff4f718b96f0\") "
Sep 11 17:35:17.933813 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:17.933712    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8d4763e0-c0ce-4b9a-8a05-ff4f718b96f0-istio-data\") pod \"8d4763e0-c0ce-4b9a-8a05-ff4f718b96f0\" (UID: \"8d4763e0-c0ce-4b9a-8a05-ff4f718b96f0\") "
Sep 11 17:35:17.933813 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:17.933743    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8d4763e0-c0ce-4b9a-8a05-ff4f718b96f0-credential-socket\") pod \"8d4763e0-c0ce-4b9a-8a05-ff4f718b96f0\" (UID: \"8d4763e0-c0ce-4b9a-8a05-ff4f718b96f0\") "
Sep 11 17:35:17.934277 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:17.933973    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8d4763e0-c0ce-4b9a-8a05-ff4f718b96f0-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "8d4763e0-c0ce-4b9a-8a05-ff4f718b96f0" (UID: "8d4763e0-c0ce-4b9a-8a05-ff4f718b96f0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 17:35:17.934277 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:17.934127    2552 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8d4763e0-c0ce-4b9a-8a05-ff4f718b96f0-istiod-ca-cert\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:35:17.934277 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:17.934216    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8d4763e0-c0ce-4b9a-8a05-ff4f718b96f0-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "8d4763e0-c0ce-4b9a-8a05-ff4f718b96f0" (UID: "8d4763e0-c0ce-4b9a-8a05-ff4f718b96f0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:35:17.934446 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:17.934281    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8d4763e0-c0ce-4b9a-8a05-ff4f718b96f0-istio-data" (OuterVolumeSpecName: "istio-data") pod "8d4763e0-c0ce-4b9a-8a05-ff4f718b96f0" (UID: "8d4763e0-c0ce-4b9a-8a05-ff4f718b96f0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:35:17.934446 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:17.934388    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8d4763e0-c0ce-4b9a-8a05-ff4f718b96f0-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "8d4763e0-c0ce-4b9a-8a05-ff4f718b96f0" (UID: "8d4763e0-c0ce-4b9a-8a05-ff4f718b96f0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:35:17.934446 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:17.934401    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8d4763e0-c0ce-4b9a-8a05-ff4f718b96f0-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "8d4763e0-c0ce-4b9a-8a05-ff4f718b96f0" (UID: "8d4763e0-c0ce-4b9a-8a05-ff4f718b96f0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:35:17.936273 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:17.936225    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8d4763e0-c0ce-4b9a-8a05-ff4f718b96f0-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "8d4763e0-c0ce-4b9a-8a05-ff4f718b96f0" (UID: "8d4763e0-c0ce-4b9a-8a05-ff4f718b96f0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 17:35:17.943202 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:17.943168    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8d4763e0-c0ce-4b9a-8a05-ff4f718b96f0-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "8d4763e0-c0ce-4b9a-8a05-ff4f718b96f0" (UID: "8d4763e0-c0ce-4b9a-8a05-ff4f718b96f0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:35:17.943354 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:17.943205    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8d4763e0-c0ce-4b9a-8a05-ff4f718b96f0-kube-api-access-mp255" (OuterVolumeSpecName: "kube-api-access-mp255") pod "8d4763e0-c0ce-4b9a-8a05-ff4f718b96f0" (UID: "8d4763e0-c0ce-4b9a-8a05-ff4f718b96f0"). InnerVolumeSpecName "kube-api-access-mp255". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:35:17.943354 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:17.943230    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8d4763e0-c0ce-4b9a-8a05-ff4f718b96f0-istio-token" (OuterVolumeSpecName: "istio-token") pod "8d4763e0-c0ce-4b9a-8a05-ff4f718b96f0" (UID: "8d4763e0-c0ce-4b9a-8a05-ff4f718b96f0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:35:18.035747 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:18.035500    2552 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8d4763e0-c0ce-4b9a-8a05-ff4f718b96f0-istio-podinfo\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:35:18.035747 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:18.035535    2552 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8d4763e0-c0ce-4b9a-8a05-ff4f718b96f0-istio-envoy\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:35:18.035747 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:18.035548    2552 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8d4763e0-c0ce-4b9a-8a05-ff4f718b96f0-istio-token\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:35:18.035747 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:18.035561    2552 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8d4763e0-c0ce-4b9a-8a05-ff4f718b96f0-workload-certs\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:35:18.035747 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:18.035573    2552 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8d4763e0-c0ce-4b9a-8a05-ff4f718b96f0-workload-socket\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:35:18.035747 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:18.035587    2552 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8d4763e0-c0ce-4b9a-8a05-ff4f718b96f0-istio-data\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:35:18.035747 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:18.035600    2552 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8d4763e0-c0ce-4b9a-8a05-ff4f718b96f0-credential-socket\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:35:18.035747 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:18.035614    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mp255\" (UniqueName: \"kubernetes.io/projected/8d4763e0-c0ce-4b9a-8a05-ff4f718b96f0-kube-api-access-mp255\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:35:18.477232 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:18.477189    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:35:18.741853 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:18.741769    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-49e398-openshift-default-66cbbd759d-nf4cx"
Sep 11 17:35:18.741853 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:18.741801    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-49e398-openshift-default-66cbbd759d-nf4cx" event={"ID":"8d4763e0-c0ce-4b9a-8a05-ff4f718b96f0","Type":"ContainerDied","Data":"6601544c482ec1f6067c95d060d87ea4e31fc527cbce3b1031e95e691b0d4c4d"}
Sep 11 17:35:18.742152 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:18.741876    2552 scope.go:117] "RemoveContainer" containerID="1a3257d07557fccd8f9c1155dcbb8e3e87106db299bf99a2df5f37bebe89299d"
Sep 11 17:35:18.759223 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:18.759195    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-autores-49e398-openshift-default-66cbbd759d-nf4cx"]
Sep 11 17:35:18.764073 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:18.764046    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-autores-49e398-openshift-default-66cbbd759d-nf4cx"]
Sep 11 17:35:19.264126 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:19.264074    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:35:19.949650 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:19.949611    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8d4763e0-c0ce-4b9a-8a05-ff4f718b96f0" path="/var/lib/kubelet/pods/8d4763e0-c0ce-4b9a-8a05-ff4f718b96f0/volumes"
Sep 11 17:35:19.964356 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:19.964330    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-ec3fed-6ccdcff8f6-jggm8"]
Sep 11 17:35:19.964705 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:19.964694    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8d4763e0-c0ce-4b9a-8a05-ff4f718b96f0" containerName="istio-proxy"
Sep 11 17:35:19.964752 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:19.964707    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="8d4763e0-c0ce-4b9a-8a05-ff4f718b96f0" containerName="istio-proxy"
Sep 11 17:35:19.964791 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:19.964787    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="8d4763e0-c0ce-4b9a-8a05-ff4f718b96f0" containerName="istio-proxy"
Sep 11 17:35:19.969468 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:19.969448    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ec3fed-6ccdcff8f6-jggm8"
Sep 11 17:35:19.972251 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:19.972228    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-zp7dq\""
Sep 11 17:35:19.972668 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:19.972653    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-explicit-ec3fed-serving-cert\""
Sep 11 17:35:19.972759 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:19.972698    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 17:35:19.972759 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:19.972698    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 11 17:35:19.972859 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:19.972704    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 11 17:35:19.983932 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:19.983036    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-ec3fed-6ccdcff8f6-jggm8"]
Sep 11 17:35:20.010257 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:20.010214    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:35:20.053824 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:20.053790    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5jdzw\" (UniqueName: \"kubernetes.io/projected/48799580-f093-4a6e-aa8e-6f9d7d2ff78c-kube-api-access-5jdzw\") pod \"maas-api-e2e-explicit-ec3fed-6ccdcff8f6-jggm8\" (UID: \"48799580-f093-4a6e-aa8e-6f9d7d2ff78c\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ec3fed-6ccdcff8f6-jggm8"
Sep 11 17:35:20.053989 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:20.053842    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/48799580-f093-4a6e-aa8e-6f9d7d2ff78c-metrics-certs\") pod \"maas-api-e2e-explicit-ec3fed-6ccdcff8f6-jggm8\" (UID: \"48799580-f093-4a6e-aa8e-6f9d7d2ff78c\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ec3fed-6ccdcff8f6-jggm8"
Sep 11 17:35:20.053989 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:20.053919    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/48799580-f093-4a6e-aa8e-6f9d7d2ff78c-openshift-service-ca\") pod \"maas-api-e2e-explicit-ec3fed-6ccdcff8f6-jggm8\" (UID: \"48799580-f093-4a6e-aa8e-6f9d7d2ff78c\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ec3fed-6ccdcff8f6-jggm8"
Sep 11 17:35:20.053989 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:20.053942    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/48799580-f093-4a6e-aa8e-6f9d7d2ff78c-maas-api-tls\") pod \"maas-api-e2e-explicit-ec3fed-6ccdcff8f6-jggm8\" (UID: \"48799580-f093-4a6e-aa8e-6f9d7d2ff78c\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ec3fed-6ccdcff8f6-jggm8"
Sep 11 17:35:20.106076 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:20.106041    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:35:20.154675 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:20.154638    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/48799580-f093-4a6e-aa8e-6f9d7d2ff78c-maas-api-tls\") pod \"maas-api-e2e-explicit-ec3fed-6ccdcff8f6-jggm8\" (UID: \"48799580-f093-4a6e-aa8e-6f9d7d2ff78c\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ec3fed-6ccdcff8f6-jggm8"
Sep 11 17:35:20.154841 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:20.154763    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5jdzw\" (UniqueName: \"kubernetes.io/projected/48799580-f093-4a6e-aa8e-6f9d7d2ff78c-kube-api-access-5jdzw\") pod \"maas-api-e2e-explicit-ec3fed-6ccdcff8f6-jggm8\" (UID: \"48799580-f093-4a6e-aa8e-6f9d7d2ff78c\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ec3fed-6ccdcff8f6-jggm8"
Sep 11 17:35:20.154841 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:20.154803    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/48799580-f093-4a6e-aa8e-6f9d7d2ff78c-metrics-certs\") pod \"maas-api-e2e-explicit-ec3fed-6ccdcff8f6-jggm8\" (UID: \"48799580-f093-4a6e-aa8e-6f9d7d2ff78c\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ec3fed-6ccdcff8f6-jggm8"
Sep 11 17:35:20.154922 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:20.154869    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/48799580-f093-4a6e-aa8e-6f9d7d2ff78c-openshift-service-ca\") pod \"maas-api-e2e-explicit-ec3fed-6ccdcff8f6-jggm8\" (UID: \"48799580-f093-4a6e-aa8e-6f9d7d2ff78c\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ec3fed-6ccdcff8f6-jggm8"
Sep 11 17:35:20.155525 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:20.155502    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/48799580-f093-4a6e-aa8e-6f9d7d2ff78c-openshift-service-ca\") pod \"maas-api-e2e-explicit-ec3fed-6ccdcff8f6-jggm8\" (UID: \"48799580-f093-4a6e-aa8e-6f9d7d2ff78c\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ec3fed-6ccdcff8f6-jggm8"
Sep 11 17:35:20.157267 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:20.157245    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/48799580-f093-4a6e-aa8e-6f9d7d2ff78c-maas-api-tls\") pod \"maas-api-e2e-explicit-ec3fed-6ccdcff8f6-jggm8\" (UID: \"48799580-f093-4a6e-aa8e-6f9d7d2ff78c\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ec3fed-6ccdcff8f6-jggm8"
Sep 11 17:35:20.157356 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:20.157263    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/48799580-f093-4a6e-aa8e-6f9d7d2ff78c-metrics-certs\") pod \"maas-api-e2e-explicit-ec3fed-6ccdcff8f6-jggm8\" (UID: \"48799580-f093-4a6e-aa8e-6f9d7d2ff78c\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ec3fed-6ccdcff8f6-jggm8"
Sep 11 17:35:20.164736 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:20.164714    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5jdzw\" (UniqueName: \"kubernetes.io/projected/48799580-f093-4a6e-aa8e-6f9d7d2ff78c-kube-api-access-5jdzw\") pod \"maas-api-e2e-explicit-ec3fed-6ccdcff8f6-jggm8\" (UID: \"48799580-f093-4a6e-aa8e-6f9d7d2ff78c\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ec3fed-6ccdcff8f6-jggm8"
Sep 11 17:35:20.207140 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:20.207030    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:35:20.281972 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:20.281941    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ec3fed-6ccdcff8f6-jggm8"
Sep 11 17:35:20.545923 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:20.545848    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:35:20.574504 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:20.574469    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-ec3fed-6ccdcff8f6-jggm8"]
Sep 11 17:35:20.575681 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:35:20.575656    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48799580_f093_4a6e_aa8e_6f9d7d2ff78c.slice/crio-a0eb8516b322657b2a285a2bf8318075259b62705f971ba68f9557d80e77c766 WatchSource:0}: Error finding container a0eb8516b322657b2a285a2bf8318075259b62705f971ba68f9557d80e77c766: Status 404 returned error can't find the container with id a0eb8516b322657b2a285a2bf8318075259b62705f971ba68f9557d80e77c766
Sep 11 17:35:20.577292 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:20.577271    2552 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 17:35:20.641410 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:20.641374    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:35:20.755186 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:20.755150    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ec3fed-6ccdcff8f6-jggm8" event={"ID":"48799580-f093-4a6e-aa8e-6f9d7d2ff78c","Type":"ContainerStarted","Data":"a0eb8516b322657b2a285a2bf8318075259b62705f971ba68f9557d80e77c766"}
Sep 11 17:35:21.762971 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:21.762932    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ec3fed-6ccdcff8f6-jggm8" event={"ID":"48799580-f093-4a6e-aa8e-6f9d7d2ff78c","Type":"ContainerStarted","Data":"27203714baa7b184d47ab4d2f1283db8574fe016c2b66ae73d56ce14c29b491b"}
Sep 11 17:35:21.763401 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:21.762995    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ec3fed-6ccdcff8f6-jggm8"
Sep 11 17:35:21.780997 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:21.780944    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ec3fed-6ccdcff8f6-jggm8" podStartSLOduration=2.387123448 podStartE2EDuration="2.780927502s" totalImagesPullingTime="393.804054ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:35:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:35:21.778500126 +0000 UTC m=+1016.299791638" watchObservedRunningTime="2026-09-11 17:35:21.780927502 +0000 UTC m=+1016.302219072"
Sep 11 17:35:22.773010 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:22.772978    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:35:22.839044 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:22.839006    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:35:22.993013 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:22.992980    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:35:23.294034 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:23.293992    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:35:24.077458 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:24.077416    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:35:27.719377 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:27.719346    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:35:27.777492 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:27.777463    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ec3fed-6ccdcff8f6-jggm8"
Sep 11 17:35:29.149384 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:29.149351    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:35:29.854693 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:29.854648    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:35:29.950024 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:29.949988    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:35:31.040872 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:31.040832    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:35:31.730434 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:31.730392    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:35:32.549605 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:32.549573    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:35:33.327094 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:33.327062    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:35:34.054734 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:34.052964    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-65fdd4fd-cf4d7b694-pzmqq"]
Sep 11 17:35:34.054734 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:34.053636    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8d4763e0-c0ce-4b9a-8a05-ff4f718b96f0" containerName="istio-proxy"
Sep 11 17:35:34.054734 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:34.053656    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="8d4763e0-c0ce-4b9a-8a05-ff4f718b96f0" containerName="istio-proxy"
Sep 11 17:35:34.054734 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:34.053819    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="8d4763e0-c0ce-4b9a-8a05-ff4f718b96f0" containerName="istio-proxy"
Sep 11 17:35:34.059630 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:34.059599    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-65fdd4fd-cf4d7b694-pzmqq"
Sep 11 17:35:34.065884 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:34.063939    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-65fdd4fd-serving-cert\""
Sep 11 17:35:34.068505 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:34.068480    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-65fdd4fd-cf4d7b694-pzmqq"]
Sep 11 17:35:34.121691 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:34.121649    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:35:34.190814 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:34.190776    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e81cb3af-befe-4381-a08d-047fc9a13384-openshift-service-ca\") pod \"maas-api-e2e-mt-65fdd4fd-cf4d7b694-pzmqq\" (UID: \"e81cb3af-befe-4381-a08d-047fc9a13384\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-65fdd4fd-cf4d7b694-pzmqq"
Sep 11 17:35:34.191133 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:34.191097    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e81cb3af-befe-4381-a08d-047fc9a13384-metrics-certs\") pod \"maas-api-e2e-mt-65fdd4fd-cf4d7b694-pzmqq\" (UID: \"e81cb3af-befe-4381-a08d-047fc9a13384\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-65fdd4fd-cf4d7b694-pzmqq"
Sep 11 17:35:34.191390 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:34.191363    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e81cb3af-befe-4381-a08d-047fc9a13384-maas-api-tls\") pod \"maas-api-e2e-mt-65fdd4fd-cf4d7b694-pzmqq\" (UID: \"e81cb3af-befe-4381-a08d-047fc9a13384\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-65fdd4fd-cf4d7b694-pzmqq"
Sep 11 17:35:34.191570 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:34.191396    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9vzj4\" (UniqueName: \"kubernetes.io/projected/e81cb3af-befe-4381-a08d-047fc9a13384-kube-api-access-9vzj4\") pod \"maas-api-e2e-mt-65fdd4fd-cf4d7b694-pzmqq\" (UID: \"e81cb3af-befe-4381-a08d-047fc9a13384\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-65fdd4fd-cf4d7b694-pzmqq"
Sep 11 17:35:34.222507 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:34.222474    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:35:34.292795 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:34.292756    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e81cb3af-befe-4381-a08d-047fc9a13384-openshift-service-ca\") pod \"maas-api-e2e-mt-65fdd4fd-cf4d7b694-pzmqq\" (UID: \"e81cb3af-befe-4381-a08d-047fc9a13384\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-65fdd4fd-cf4d7b694-pzmqq"
Sep 11 17:35:34.292993 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:34.292809    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e81cb3af-befe-4381-a08d-047fc9a13384-metrics-certs\") pod \"maas-api-e2e-mt-65fdd4fd-cf4d7b694-pzmqq\" (UID: \"e81cb3af-befe-4381-a08d-047fc9a13384\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-65fdd4fd-cf4d7b694-pzmqq"
Sep 11 17:35:34.292993 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:34.292873    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e81cb3af-befe-4381-a08d-047fc9a13384-maas-api-tls\") pod \"maas-api-e2e-mt-65fdd4fd-cf4d7b694-pzmqq\" (UID: \"e81cb3af-befe-4381-a08d-047fc9a13384\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-65fdd4fd-cf4d7b694-pzmqq"
Sep 11 17:35:34.292993 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:34.292889    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9vzj4\" (UniqueName: \"kubernetes.io/projected/e81cb3af-befe-4381-a08d-047fc9a13384-kube-api-access-9vzj4\") pod \"maas-api-e2e-mt-65fdd4fd-cf4d7b694-pzmqq\" (UID: \"e81cb3af-befe-4381-a08d-047fc9a13384\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-65fdd4fd-cf4d7b694-pzmqq"
Sep 11 17:35:34.293484 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:34.293463    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e81cb3af-befe-4381-a08d-047fc9a13384-openshift-service-ca\") pod \"maas-api-e2e-mt-65fdd4fd-cf4d7b694-pzmqq\" (UID: \"e81cb3af-befe-4381-a08d-047fc9a13384\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-65fdd4fd-cf4d7b694-pzmqq"
Sep 11 17:35:34.295428 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:34.295409    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e81cb3af-befe-4381-a08d-047fc9a13384-maas-api-tls\") pod \"maas-api-e2e-mt-65fdd4fd-cf4d7b694-pzmqq\" (UID: \"e81cb3af-befe-4381-a08d-047fc9a13384\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-65fdd4fd-cf4d7b694-pzmqq"
Sep 11 17:35:34.295482 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:34.295436    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e81cb3af-befe-4381-a08d-047fc9a13384-metrics-certs\") pod \"maas-api-e2e-mt-65fdd4fd-cf4d7b694-pzmqq\" (UID: \"e81cb3af-befe-4381-a08d-047fc9a13384\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-65fdd4fd-cf4d7b694-pzmqq"
Sep 11 17:35:34.300516 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:34.300494    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9vzj4\" (UniqueName: \"kubernetes.io/projected/e81cb3af-befe-4381-a08d-047fc9a13384-kube-api-access-9vzj4\") pod \"maas-api-e2e-mt-65fdd4fd-cf4d7b694-pzmqq\" (UID: \"e81cb3af-befe-4381-a08d-047fc9a13384\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-65fdd4fd-cf4d7b694-pzmqq"
Sep 11 17:35:34.329436 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:34.329364    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:35:34.379291 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:34.379254    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-65fdd4fd-cf4d7b694-pzmqq"
Sep 11 17:35:34.673933 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:34.673889    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-65fdd4fd-cf4d7b694-pzmqq"]
Sep 11 17:35:34.677641 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:35:34.677608    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode81cb3af_befe_4381_a08d_047fc9a13384.slice/crio-efdf698c2a1e084fa372f988e573c98ece3c1f57531418a260bed4f1a81ae4c3 WatchSource:0}: Error finding container efdf698c2a1e084fa372f988e573c98ece3c1f57531418a260bed4f1a81ae4c3: Status 404 returned error can't find the container with id efdf698c2a1e084fa372f988e573c98ece3c1f57531418a260bed4f1a81ae4c3
Sep 11 17:35:34.713498 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:34.713467    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:35:34.804243 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:34.804206    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:35:34.862136 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:34.856078    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-65fdd4fd-cf4d7b694-pzmqq" event={"ID":"e81cb3af-befe-4381-a08d-047fc9a13384","Type":"ContainerStarted","Data":"efdf698c2a1e084fa372f988e573c98ece3c1f57531418a260bed4f1a81ae4c3"}
Sep 11 17:35:35.209663 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:35.209584    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:35:35.866303 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:35.866260    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-65fdd4fd-cf4d7b694-pzmqq" event={"ID":"e81cb3af-befe-4381-a08d-047fc9a13384","Type":"ContainerStarted","Data":"805a0053ca06f5710a6c3e91c39b9246961a6ec2e23f130b9bab7d442b49fdb9"}
Sep 11 17:35:35.866492 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:35.866395    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-65fdd4fd-cf4d7b694-pzmqq"
Sep 11 17:35:35.883285 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:35.883233    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-65fdd4fd-cf4d7b694-pzmqq" podStartSLOduration=1.471967808 podStartE2EDuration="1.883213884s" totalImagesPullingTime="411.246076ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:35:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:35:35.882140201 +0000 UTC m=+1030.403431721" watchObservedRunningTime="2026-09-11 17:35:35.883213884 +0000 UTC m=+1030.404505405"
Sep 11 17:35:35.963816 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:35.963755    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:35:36.497002 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:36.496929    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:35:36.767789 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:36.767688    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:35:37.554014 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:37.553870    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:35:38.279912 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:38.279869    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:35:39.263647 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:39.263613    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:35:40.015611 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:40.015575    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:35:40.344243 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:40.344207    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:35:40.444735 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:40.444698    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:35:40.907203 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:40.907098    2552 generic.go:356] "Generic (PLEG): container finished" podID="411db317-1635-45d6-8a64-d6eaf0b4ae92" containerID="52864b030ef5db52619d6f6b135b7e150e3a2ce74aff90f6b33bdad6ba403c33" exitCode=0
Sep 11 17:35:40.907203 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:40.907174    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-ec3fed-openshift-default-6fdf496464-f2z6f" event={"ID":"411db317-1635-45d6-8a64-d6eaf0b4ae92","Type":"ContainerDied","Data":"52864b030ef5db52619d6f6b135b7e150e3a2ce74aff90f6b33bdad6ba403c33"}
Sep 11 17:35:40.907704 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:40.907660    2552 scope.go:117] "RemoveContainer" containerID="52864b030ef5db52619d6f6b135b7e150e3a2ce74aff90f6b33bdad6ba403c33"
Sep 11 17:35:40.908897 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:40.908869    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 17:35:40.908987 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:40.908936    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 17:35:40.908987 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:40.908964    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 17:35:41.885875 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:41.885847    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-65fdd4fd-cf4d7b694-pzmqq"
Sep 11 17:35:41.916446 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:41.916415    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-ec3fed-openshift-default-6fdf496464-f2z6f" event={"ID":"411db317-1635-45d6-8a64-d6eaf0b4ae92","Type":"ContainerStarted","Data":"0247f032b085c9cd4a189a8ca6d74f84526d0bff3df2f5c940e38b7642b47cae"}
Sep 11 17:35:42.126506 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:42.126463    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:35:42.612849 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:42.612813    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-explicit-ec3fed-openshift-default-6fdf496464-f2z6f"
Sep 11 17:35:42.614143 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:42.614103    2552 patch_prober.go:28] interesting pod/e2e-explicit-ec3fed-openshift-default-6fdf496464-f2z6f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body=
Sep 11 17:35:42.614259 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:42.614176    2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-ec3fed-openshift-default-6fdf496464-f2z6f" podUID="411db317-1635-45d6-8a64-d6eaf0b4ae92" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused"
Sep 11 17:35:43.613026 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:43.612987    2552 patch_prober.go:28] interesting pod/e2e-explicit-ec3fed-openshift-default-6fdf496464-f2z6f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body=
Sep 11 17:35:43.613493 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:43.613060    2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-ec3fed-openshift-default-6fdf496464-f2z6f" podUID="411db317-1635-45d6-8a64-d6eaf0b4ae92" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused"
Sep 11 17:35:44.613045 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:44.613010    2552 patch_prober.go:28] interesting pod/e2e-explicit-ec3fed-openshift-default-6fdf496464-f2z6f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body=
Sep 11 17:35:44.613489 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:44.613067    2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-ec3fed-openshift-default-6fdf496464-f2z6f" podUID="411db317-1635-45d6-8a64-d6eaf0b4ae92" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused"
Sep 11 17:35:45.612712 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:45.612665    2552 patch_prober.go:28] interesting pod/e2e-explicit-ec3fed-openshift-default-6fdf496464-f2z6f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body=
Sep 11 17:35:45.612898 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:45.612745    2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-ec3fed-openshift-default-6fdf496464-f2z6f" podUID="411db317-1635-45d6-8a64-d6eaf0b4ae92" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused"
Sep 11 17:35:45.718937 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:45.718904    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-explicit-ec3fed-openshift-default-6fdf496464-f2z6f"
Sep 11 17:35:46.612704 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:46.612663    2552 patch_prober.go:28] interesting pod/e2e-explicit-ec3fed-openshift-default-6fdf496464-f2z6f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body=
Sep 11 17:35:46.612871 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:46.612729    2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-ec3fed-openshift-default-6fdf496464-f2z6f" podUID="411db317-1635-45d6-8a64-d6eaf0b4ae92" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused"
Sep 11 17:35:47.616539 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:47.616509    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-explicit-ec3fed-openshift-default-6fdf496464-f2z6f"
Sep 11 17:35:47.617497 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:47.617476    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-explicit-ec3fed-openshift-default-6fdf496464-f2z6f"
Sep 11 17:35:54.820673 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:54.820637    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:35:54.919553 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:54.919517    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:35:55.170237 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:55.170200    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:35:56.708475 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:56.708432    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:35:57.440046 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:57.440010    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:35:58.134099 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:58.134067    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:35:58.933578 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:58.933527    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:35:59.627013 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:59.626977    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:35:59.909422 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:35:59.909319    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:36:00.453870 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:00.453836    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:36:00.577418 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:00.577384    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:36:00.672835 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:00.672802    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:36:01.002559 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:01.002523    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:36:01.449605 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:01.449570    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:36:04.814616 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:04.814581    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:36:06.844724 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:06.844696    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:36:06.945386 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:06.945351    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:36:08.117544 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:08.117512    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:36:08.804603 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:08.804564    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:36:09.553912 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:09.553873    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:36:10.377727 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:10.377694    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:36:11.256331 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:11.256293    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:36:12.195613 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:12.195575    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:36:13.363203 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:13.363168    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:36:17.879484 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:17.879453    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-65fdd4fd-cf4d7b694-pzmqq"]
Sep 11 17:36:17.879954 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:17.879663    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-65fdd4fd-cf4d7b694-pzmqq" podUID="e81cb3af-befe-4381-a08d-047fc9a13384" containerName="maas-api" containerID="cri-o://805a0053ca06f5710a6c3e91c39b9246961a6ec2e23f130b9bab7d442b49fdb9" gracePeriod=30
Sep 11 17:36:17.923485 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:17.923451    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:36:18.018647 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:18.018614    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:36:18.119725 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:18.119691    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:36:18.141811 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:18.141742    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-65fdd4fd-cf4d7b694-pzmqq"
Sep 11 17:36:18.147167 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:18.147143    2552 generic.go:356] "Generic (PLEG): container finished" podID="e81cb3af-befe-4381-a08d-047fc9a13384" containerID="805a0053ca06f5710a6c3e91c39b9246961a6ec2e23f130b9bab7d442b49fdb9" exitCode=0
Sep 11 17:36:18.147282 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:18.147190    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-65fdd4fd-cf4d7b694-pzmqq"
Sep 11 17:36:18.147282 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:18.147233    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-65fdd4fd-cf4d7b694-pzmqq" event={"ID":"e81cb3af-befe-4381-a08d-047fc9a13384","Type":"ContainerDied","Data":"805a0053ca06f5710a6c3e91c39b9246961a6ec2e23f130b9bab7d442b49fdb9"}
Sep 11 17:36:18.147365 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:18.147287    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-65fdd4fd-cf4d7b694-pzmqq" event={"ID":"e81cb3af-befe-4381-a08d-047fc9a13384","Type":"ContainerDied","Data":"efdf698c2a1e084fa372f988e573c98ece3c1f57531418a260bed4f1a81ae4c3"}
Sep 11 17:36:18.147365 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:18.147319    2552 scope.go:117] "RemoveContainer" containerID="805a0053ca06f5710a6c3e91c39b9246961a6ec2e23f130b9bab7d442b49fdb9"
Sep 11 17:36:18.159945 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:18.159628    2552 scope.go:117] "RemoveContainer" containerID="805a0053ca06f5710a6c3e91c39b9246961a6ec2e23f130b9bab7d442b49fdb9"
Sep 11 17:36:18.160074 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:36:18.160036    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"805a0053ca06f5710a6c3e91c39b9246961a6ec2e23f130b9bab7d442b49fdb9\": container with ID starting with 805a0053ca06f5710a6c3e91c39b9246961a6ec2e23f130b9bab7d442b49fdb9 not found: ID does not exist" containerID="805a0053ca06f5710a6c3e91c39b9246961a6ec2e23f130b9bab7d442b49fdb9"
Sep 11 17:36:18.160159 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:18.160061    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"805a0053ca06f5710a6c3e91c39b9246961a6ec2e23f130b9bab7d442b49fdb9"} err="failed to get container status \"805a0053ca06f5710a6c3e91c39b9246961a6ec2e23f130b9bab7d442b49fdb9\": rpc error: code = NotFound desc = could not find container \"805a0053ca06f5710a6c3e91c39b9246961a6ec2e23f130b9bab7d442b49fdb9\": container with ID starting with 805a0053ca06f5710a6c3e91c39b9246961a6ec2e23f130b9bab7d442b49fdb9 not found: ID does not exist"
Sep 11 17:36:18.230188 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:18.230070    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:36:18.239123 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:18.239084    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9vzj4\" (UniqueName: \"kubernetes.io/projected/e81cb3af-befe-4381-a08d-047fc9a13384-kube-api-access-9vzj4\") pod \"e81cb3af-befe-4381-a08d-047fc9a13384\" (UID: \"e81cb3af-befe-4381-a08d-047fc9a13384\") "
Sep 11 17:36:18.239295 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:18.239166    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e81cb3af-befe-4381-a08d-047fc9a13384-openshift-service-ca\") pod \"e81cb3af-befe-4381-a08d-047fc9a13384\" (UID: \"e81cb3af-befe-4381-a08d-047fc9a13384\") "
Sep 11 17:36:18.239622 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:18.239279    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e81cb3af-befe-4381-a08d-047fc9a13384-maas-api-tls\") pod \"e81cb3af-befe-4381-a08d-047fc9a13384\" (UID: \"e81cb3af-befe-4381-a08d-047fc9a13384\") "
Sep 11 17:36:18.239730 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:18.239513    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e81cb3af-befe-4381-a08d-047fc9a13384-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e81cb3af-befe-4381-a08d-047fc9a13384" (UID: "e81cb3af-befe-4381-a08d-047fc9a13384"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 17:36:18.239797 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:18.239783    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e81cb3af-befe-4381-a08d-047fc9a13384-metrics-certs\") pod \"e81cb3af-befe-4381-a08d-047fc9a13384\" (UID: \"e81cb3af-befe-4381-a08d-047fc9a13384\") "
Sep 11 17:36:18.240559 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:18.240537    2552 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e81cb3af-befe-4381-a08d-047fc9a13384-openshift-service-ca\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:36:18.241604 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:18.241558    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e81cb3af-befe-4381-a08d-047fc9a13384-kube-api-access-9vzj4" (OuterVolumeSpecName: "kube-api-access-9vzj4") pod "e81cb3af-befe-4381-a08d-047fc9a13384" (UID: "e81cb3af-befe-4381-a08d-047fc9a13384"). InnerVolumeSpecName "kube-api-access-9vzj4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:36:18.241769 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:18.241747    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e81cb3af-befe-4381-a08d-047fc9a13384-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e81cb3af-befe-4381-a08d-047fc9a13384" (UID: "e81cb3af-befe-4381-a08d-047fc9a13384"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 17:36:18.242228 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:18.242207    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e81cb3af-befe-4381-a08d-047fc9a13384-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "e81cb3af-befe-4381-a08d-047fc9a13384" (UID: "e81cb3af-befe-4381-a08d-047fc9a13384"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 17:36:18.325414 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:18.325380    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:36:18.341578 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:18.341550    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9vzj4\" (UniqueName: \"kubernetes.io/projected/e81cb3af-befe-4381-a08d-047fc9a13384-kube-api-access-9vzj4\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:36:18.341578 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:18.341578    2552 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e81cb3af-befe-4381-a08d-047fc9a13384-maas-api-tls\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:36:18.341768 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:18.341591    2552 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e81cb3af-befe-4381-a08d-047fc9a13384-metrics-certs\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:36:18.480387 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:18.480350    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-65fdd4fd-cf4d7b694-pzmqq"]
Sep 11 17:36:18.480587 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:18.480404    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-65fdd4fd-cf4d7b694-pzmqq"]
Sep 11 17:36:18.586879 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:18.586819    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:36:18.761781 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:18.761650    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:36:19.575874 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:19.575829    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:36:19.950622 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:19.950587    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e81cb3af-befe-4381-a08d-047fc9a13384" path="/var/lib/kubelet/pods/e81cb3af-befe-4381-a08d-047fc9a13384/volumes"
Sep 11 17:36:20.686699 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:20.686656    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:36:21.524513 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:21.524480    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:36:22.335761 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:22.335728    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:36:23.209746 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:23.209710    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:36:24.879960 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:24.879922    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:36:24.980607 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:24.980576    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:36:25.140059 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:25.139965    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:36:25.587382 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:25.587286    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:36:25.756500 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:25.756457    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:36:25.871306 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:25.871278    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:36:31.988838 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:31.988804    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:36:32.100158 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:32.100124    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:36:32.218343 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:32.218293    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:36:32.366929 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:32.366888    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:36:36.838597 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:36.838554    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:36:38.361827 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:38.361790    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:36:38.461762 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:38.461729    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:36:38.562344 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:38.562303    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:36:39.234723 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:39.234692    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:36:39.555937 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:39.555860    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w2-290683-openshift-default-57c7b78dd6-q77mh"]
Sep 11 17:36:39.556313 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:39.556292    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e81cb3af-befe-4381-a08d-047fc9a13384" containerName="maas-api"
Sep 11 17:36:39.556313 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:39.556310    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="e81cb3af-befe-4381-a08d-047fc9a13384" containerName="maas-api"
Sep 11 17:36:39.556396 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:39.556383    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="e81cb3af-befe-4381-a08d-047fc9a13384" containerName="maas-api"
Sep 11 17:36:39.560874 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:39.560848    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-290683-openshift-default-57c7b78dd6-q77mh"
Sep 11 17:36:39.563537 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:39.563519    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w2-290683-openshift-default-dockercfg-zhmqz\""
Sep 11 17:36:39.575884 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:39.575825    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:36:39.578037 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:39.578013    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-290683-openshift-default-57c7b78dd6-q77mh"]
Sep 11 17:36:39.651919 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:39.651880    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/34c43407-b150-4315-b2d4-b1413944bdd3-istio-token\") pod \"e2e-shared-a-w2-290683-openshift-default-57c7b78dd6-q77mh\" (UID: \"34c43407-b150-4315-b2d4-b1413944bdd3\") " pod="openshift-ingress/e2e-shared-a-w2-290683-openshift-default-57c7b78dd6-q77mh"
Sep 11 17:36:39.652323 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:39.652302    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7gclm\" (UniqueName: \"kubernetes.io/projected/34c43407-b150-4315-b2d4-b1413944bdd3-kube-api-access-7gclm\") pod \"e2e-shared-a-w2-290683-openshift-default-57c7b78dd6-q77mh\" (UID: \"34c43407-b150-4315-b2d4-b1413944bdd3\") " pod="openshift-ingress/e2e-shared-a-w2-290683-openshift-default-57c7b78dd6-q77mh"
Sep 11 17:36:39.652519 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:39.652505    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/34c43407-b150-4315-b2d4-b1413944bdd3-istiod-ca-cert\") pod \"e2e-shared-a-w2-290683-openshift-default-57c7b78dd6-q77mh\" (UID: \"34c43407-b150-4315-b2d4-b1413944bdd3\") " pod="openshift-ingress/e2e-shared-a-w2-290683-openshift-default-57c7b78dd6-q77mh"
Sep 11 17:36:39.652741 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:39.652724    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/34c43407-b150-4315-b2d4-b1413944bdd3-istio-data\") pod \"e2e-shared-a-w2-290683-openshift-default-57c7b78dd6-q77mh\" (UID: \"34c43407-b150-4315-b2d4-b1413944bdd3\") " pod="openshift-ingress/e2e-shared-a-w2-290683-openshift-default-57c7b78dd6-q77mh"
Sep 11 17:36:39.652878 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:39.652864    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/34c43407-b150-4315-b2d4-b1413944bdd3-istio-podinfo\") pod \"e2e-shared-a-w2-290683-openshift-default-57c7b78dd6-q77mh\" (UID: \"34c43407-b150-4315-b2d4-b1413944bdd3\") " pod="openshift-ingress/e2e-shared-a-w2-290683-openshift-default-57c7b78dd6-q77mh"
Sep 11 17:36:39.653015 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:39.653000    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/34c43407-b150-4315-b2d4-b1413944bdd3-workload-socket\") pod \"e2e-shared-a-w2-290683-openshift-default-57c7b78dd6-q77mh\" (UID: \"34c43407-b150-4315-b2d4-b1413944bdd3\") " pod="openshift-ingress/e2e-shared-a-w2-290683-openshift-default-57c7b78dd6-q77mh"
Sep 11 17:36:39.653272 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:39.653247    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/34c43407-b150-4315-b2d4-b1413944bdd3-credential-socket\") pod \"e2e-shared-a-w2-290683-openshift-default-57c7b78dd6-q77mh\" (UID: \"34c43407-b150-4315-b2d4-b1413944bdd3\") " pod="openshift-ingress/e2e-shared-a-w2-290683-openshift-default-57c7b78dd6-q77mh"
Sep 11 17:36:39.653471 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:39.653448    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/34c43407-b150-4315-b2d4-b1413944bdd3-istio-envoy\") pod \"e2e-shared-a-w2-290683-openshift-default-57c7b78dd6-q77mh\" (UID: \"34c43407-b150-4315-b2d4-b1413944bdd3\") " pod="openshift-ingress/e2e-shared-a-w2-290683-openshift-default-57c7b78dd6-q77mh"
Sep 11 17:36:39.653588 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:39.653490    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/34c43407-b150-4315-b2d4-b1413944bdd3-workload-certs\") pod \"e2e-shared-a-w2-290683-openshift-default-57c7b78dd6-q77mh\" (UID: \"34c43407-b150-4315-b2d4-b1413944bdd3\") " pod="openshift-ingress/e2e-shared-a-w2-290683-openshift-default-57c7b78dd6-q77mh"
Sep 11 17:36:39.709043 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:39.709009    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:36:39.754379 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:39.754348    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/34c43407-b150-4315-b2d4-b1413944bdd3-istio-data\") pod \"e2e-shared-a-w2-290683-openshift-default-57c7b78dd6-q77mh\" (UID: \"34c43407-b150-4315-b2d4-b1413944bdd3\") " pod="openshift-ingress/e2e-shared-a-w2-290683-openshift-default-57c7b78dd6-q77mh"
Sep 11 17:36:39.754586 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:39.754386    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/34c43407-b150-4315-b2d4-b1413944bdd3-istio-podinfo\") pod \"e2e-shared-a-w2-290683-openshift-default-57c7b78dd6-q77mh\" (UID: \"34c43407-b150-4315-b2d4-b1413944bdd3\") " pod="openshift-ingress/e2e-shared-a-w2-290683-openshift-default-57c7b78dd6-q77mh"
Sep 11 17:36:39.754586 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:39.754406    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/34c43407-b150-4315-b2d4-b1413944bdd3-workload-socket\") pod \"e2e-shared-a-w2-290683-openshift-default-57c7b78dd6-q77mh\" (UID: \"34c43407-b150-4315-b2d4-b1413944bdd3\") " pod="openshift-ingress/e2e-shared-a-w2-290683-openshift-default-57c7b78dd6-q77mh"
Sep 11 17:36:39.754586 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:39.754437    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/34c43407-b150-4315-b2d4-b1413944bdd3-credential-socket\") pod \"e2e-shared-a-w2-290683-openshift-default-57c7b78dd6-q77mh\" (UID: \"34c43407-b150-4315-b2d4-b1413944bdd3\") " pod="openshift-ingress/e2e-shared-a-w2-290683-openshift-default-57c7b78dd6-q77mh"
Sep 11 17:36:39.754586 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:39.754480    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/34c43407-b150-4315-b2d4-b1413944bdd3-istio-envoy\") pod \"e2e-shared-a-w2-290683-openshift-default-57c7b78dd6-q77mh\" (UID: \"34c43407-b150-4315-b2d4-b1413944bdd3\") " pod="openshift-ingress/e2e-shared-a-w2-290683-openshift-default-57c7b78dd6-q77mh"
Sep 11 17:36:39.754586 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:39.754511    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/34c43407-b150-4315-b2d4-b1413944bdd3-workload-certs\") pod \"e2e-shared-a-w2-290683-openshift-default-57c7b78dd6-q77mh\" (UID: \"34c43407-b150-4315-b2d4-b1413944bdd3\") " pod="openshift-ingress/e2e-shared-a-w2-290683-openshift-default-57c7b78dd6-q77mh"
Sep 11 17:36:39.754586 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:39.754550    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/34c43407-b150-4315-b2d4-b1413944bdd3-istio-token\") pod \"e2e-shared-a-w2-290683-openshift-default-57c7b78dd6-q77mh\" (UID: \"34c43407-b150-4315-b2d4-b1413944bdd3\") " pod="openshift-ingress/e2e-shared-a-w2-290683-openshift-default-57c7b78dd6-q77mh"
Sep 11 17:36:39.754586 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:39.754587    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7gclm\" (UniqueName: \"kubernetes.io/projected/34c43407-b150-4315-b2d4-b1413944bdd3-kube-api-access-7gclm\") pod \"e2e-shared-a-w2-290683-openshift-default-57c7b78dd6-q77mh\" (UID: \"34c43407-b150-4315-b2d4-b1413944bdd3\") " pod="openshift-ingress/e2e-shared-a-w2-290683-openshift-default-57c7b78dd6-q77mh"
Sep 11 17:36:39.754998 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:39.754637    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/34c43407-b150-4315-b2d4-b1413944bdd3-istiod-ca-cert\") pod \"e2e-shared-a-w2-290683-openshift-default-57c7b78dd6-q77mh\" (UID: \"34c43407-b150-4315-b2d4-b1413944bdd3\") " pod="openshift-ingress/e2e-shared-a-w2-290683-openshift-default-57c7b78dd6-q77mh"
Sep 11 17:36:39.754998 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:39.754823    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/34c43407-b150-4315-b2d4-b1413944bdd3-istio-data\") pod \"e2e-shared-a-w2-290683-openshift-default-57c7b78dd6-q77mh\" (UID: \"34c43407-b150-4315-b2d4-b1413944bdd3\") " pod="openshift-ingress/e2e-shared-a-w2-290683-openshift-default-57c7b78dd6-q77mh"
Sep 11 17:36:39.754998 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:39.754840    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/34c43407-b150-4315-b2d4-b1413944bdd3-workload-socket\") pod \"e2e-shared-a-w2-290683-openshift-default-57c7b78dd6-q77mh\" (UID: \"34c43407-b150-4315-b2d4-b1413944bdd3\") " pod="openshift-ingress/e2e-shared-a-w2-290683-openshift-default-57c7b78dd6-q77mh"
Sep 11 17:36:39.754998 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:39.754922    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/34c43407-b150-4315-b2d4-b1413944bdd3-credential-socket\") pod \"e2e-shared-a-w2-290683-openshift-default-57c7b78dd6-q77mh\" (UID: \"34c43407-b150-4315-b2d4-b1413944bdd3\") " pod="openshift-ingress/e2e-shared-a-w2-290683-openshift-default-57c7b78dd6-q77mh"
Sep 11 17:36:39.755250 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:39.755137    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/34c43407-b150-4315-b2d4-b1413944bdd3-workload-certs\") pod \"e2e-shared-a-w2-290683-openshift-default-57c7b78dd6-q77mh\" (UID: \"34c43407-b150-4315-b2d4-b1413944bdd3\") " pod="openshift-ingress/e2e-shared-a-w2-290683-openshift-default-57c7b78dd6-q77mh"
Sep 11 17:36:39.755388 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:39.755371    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/34c43407-b150-4315-b2d4-b1413944bdd3-istiod-ca-cert\") pod \"e2e-shared-a-w2-290683-openshift-default-57c7b78dd6-q77mh\" (UID: \"34c43407-b150-4315-b2d4-b1413944bdd3\") " pod="openshift-ingress/e2e-shared-a-w2-290683-openshift-default-57c7b78dd6-q77mh"
Sep 11 17:36:39.756954 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:39.756932    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/34c43407-b150-4315-b2d4-b1413944bdd3-istio-envoy\") pod \"e2e-shared-a-w2-290683-openshift-default-57c7b78dd6-q77mh\" (UID: \"34c43407-b150-4315-b2d4-b1413944bdd3\") " pod="openshift-ingress/e2e-shared-a-w2-290683-openshift-default-57c7b78dd6-q77mh"
Sep 11 17:36:39.757144 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:39.757105    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/34c43407-b150-4315-b2d4-b1413944bdd3-istio-podinfo\") pod \"e2e-shared-a-w2-290683-openshift-default-57c7b78dd6-q77mh\" (UID: \"34c43407-b150-4315-b2d4-b1413944bdd3\") " pod="openshift-ingress/e2e-shared-a-w2-290683-openshift-default-57c7b78dd6-q77mh"
Sep 11 17:36:39.765194 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:39.765169    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7gclm\" (UniqueName: \"kubernetes.io/projected/34c43407-b150-4315-b2d4-b1413944bdd3-kube-api-access-7gclm\") pod \"e2e-shared-a-w2-290683-openshift-default-57c7b78dd6-q77mh\" (UID: \"34c43407-b150-4315-b2d4-b1413944bdd3\") " pod="openshift-ingress/e2e-shared-a-w2-290683-openshift-default-57c7b78dd6-q77mh"
Sep 11 17:36:39.765648 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:39.765627    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/34c43407-b150-4315-b2d4-b1413944bdd3-istio-token\") pod \"e2e-shared-a-w2-290683-openshift-default-57c7b78dd6-q77mh\" (UID: \"34c43407-b150-4315-b2d4-b1413944bdd3\") " pod="openshift-ingress/e2e-shared-a-w2-290683-openshift-default-57c7b78dd6-q77mh"
Sep 11 17:36:39.872310 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:39.872269    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-290683-openshift-default-57c7b78dd6-q77mh"
Sep 11 17:36:40.151292 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:40.151256    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-290683-openshift-default-57c7b78dd6-q77mh"]
Sep 11 17:36:40.152166 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:36:40.152133    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34c43407_b150_4315_b2d4_b1413944bdd3.slice/crio-7f76ee46a11348295eaaa9ea561905984d6adb6efb1349274cecdc4aa8edb60c WatchSource:0}: Error finding container 7f76ee46a11348295eaaa9ea561905984d6adb6efb1349274cecdc4aa8edb60c: Status 404 returned error can't find the container with id 7f76ee46a11348295eaaa9ea561905984d6adb6efb1349274cecdc4aa8edb60c
Sep 11 17:36:40.154399 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:40.154371    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 17:36:40.154464 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:40.154432    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 17:36:40.154464 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:40.154458    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 17:36:40.289490 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:40.289454    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-290683-openshift-default-57c7b78dd6-q77mh" event={"ID":"34c43407-b150-4315-b2d4-b1413944bdd3","Type":"ContainerStarted","Data":"9f5858a971d9e8f8f91f9b412e2797ba213b59c52ec2d31e9d9d5e236461056c"}
Sep 11 17:36:40.289651 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:40.289520    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-290683-openshift-default-57c7b78dd6-q77mh" event={"ID":"34c43407-b150-4315-b2d4-b1413944bdd3","Type":"ContainerStarted","Data":"7f76ee46a11348295eaaa9ea561905984d6adb6efb1349274cecdc4aa8edb60c"}
Sep 11 17:36:40.326236 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:40.325683    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w2-290683-openshift-default-57c7b78dd6-q77mh" podStartSLOduration=1.325499474 podStartE2EDuration="1.325499474s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:36:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:36:40.317435981 +0000 UTC m=+1094.838727503" watchObservedRunningTime="2026-09-11 17:36:40.325499474 +0000 UTC m=+1094.846790997"
Sep 11 17:36:40.450868 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:40.450764    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:36:40.872784 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:40.872750    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-290683-openshift-default-57c7b78dd6-q77mh"
Sep 11 17:36:41.573455 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:41.573422    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:36:41.878190 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:41.878163    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-290683-openshift-default-57c7b78dd6-q77mh"
Sep 11 17:36:42.245527 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:42.245426    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:36:42.302243 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:42.302209    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-290683-openshift-default-57c7b78dd6-q77mh"
Sep 11 17:36:42.303073 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:42.303051    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-290683-openshift-default-57c7b78dd6-q77mh"
Sep 11 17:36:42.351948 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:42.351915    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:36:42.436749 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:42.436712    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:36:43.482960 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:43.482921    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:36:44.313747 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:44.313714    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:36:45.095213 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:45.095172    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:36:45.889923 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:45.889872    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:36:46.720559 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:46.720523    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-290683-74d74bd8b7-qpj8k"]
Sep 11 17:36:46.725190 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:46.725159    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-290683-74d74bd8b7-qpj8k"
Sep 11 17:36:46.727967 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:46.727942    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w2-290683-serving-cert\""
Sep 11 17:36:46.735312 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:46.735284    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-290683-74d74bd8b7-qpj8k"]
Sep 11 17:36:46.755481 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:46.755444    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:36:46.827301 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:46.827257    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e9cb1a4-4a5f-417d-9ad4-9769dd1ae1fc-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-290683-74d74bd8b7-qpj8k\" (UID: \"9e9cb1a4-4a5f-417d-9ad4-9769dd1ae1fc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-290683-74d74bd8b7-qpj8k"
Sep 11 17:36:46.827648 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:46.827622    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9e9cb1a4-4a5f-417d-9ad4-9769dd1ae1fc-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-290683-74d74bd8b7-qpj8k\" (UID: \"9e9cb1a4-4a5f-417d-9ad4-9769dd1ae1fc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-290683-74d74bd8b7-qpj8k"
Sep 11 17:36:46.827945 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:46.827925    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9e9cb1a4-4a5f-417d-9ad4-9769dd1ae1fc-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-290683-74d74bd8b7-qpj8k\" (UID: \"9e9cb1a4-4a5f-417d-9ad4-9769dd1ae1fc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-290683-74d74bd8b7-qpj8k"
Sep 11 17:36:46.828178 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:46.828161    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nh7md\" (UniqueName: \"kubernetes.io/projected/9e9cb1a4-4a5f-417d-9ad4-9769dd1ae1fc-kube-api-access-nh7md\") pod \"maas-api-e2e-shared-a-w2-290683-74d74bd8b7-qpj8k\" (UID: \"9e9cb1a4-4a5f-417d-9ad4-9769dd1ae1fc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-290683-74d74bd8b7-qpj8k"
Sep 11 17:36:46.868006 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:46.867972    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:36:46.929554 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:46.929523    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9e9cb1a4-4a5f-417d-9ad4-9769dd1ae1fc-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-290683-74d74bd8b7-qpj8k\" (UID: \"9e9cb1a4-4a5f-417d-9ad4-9769dd1ae1fc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-290683-74d74bd8b7-qpj8k"
Sep 11 17:36:46.929753 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:46.929569    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nh7md\" (UniqueName: \"kubernetes.io/projected/9e9cb1a4-4a5f-417d-9ad4-9769dd1ae1fc-kube-api-access-nh7md\") pod \"maas-api-e2e-shared-a-w2-290683-74d74bd8b7-qpj8k\" (UID: \"9e9cb1a4-4a5f-417d-9ad4-9769dd1ae1fc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-290683-74d74bd8b7-qpj8k"
Sep 11 17:36:46.929753 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:36:46.929676    2552 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-290683-serving-cert: secret "maas-api-e2e-shared-a-w2-290683-serving-cert" not found
Sep 11 17:36:46.929753 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:46.929695    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e9cb1a4-4a5f-417d-9ad4-9769dd1ae1fc-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-290683-74d74bd8b7-qpj8k\" (UID: \"9e9cb1a4-4a5f-417d-9ad4-9769dd1ae1fc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-290683-74d74bd8b7-qpj8k"
Sep 11 17:36:46.929753 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:36:46.929738    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e9cb1a4-4a5f-417d-9ad4-9769dd1ae1fc-maas-api-tls podName:9e9cb1a4-4a5f-417d-9ad4-9769dd1ae1fc nodeName:}" failed. No retries permitted until 2026-09-11 17:36:47.429719741 +0000 UTC m=+1101.951011241 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/9e9cb1a4-4a5f-417d-9ad4-9769dd1ae1fc-maas-api-tls") pod "maas-api-e2e-shared-a-w2-290683-74d74bd8b7-qpj8k" (UID: "9e9cb1a4-4a5f-417d-9ad4-9769dd1ae1fc") : secret "maas-api-e2e-shared-a-w2-290683-serving-cert" not found
Sep 11 17:36:46.929983 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:46.929809    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9e9cb1a4-4a5f-417d-9ad4-9769dd1ae1fc-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-290683-74d74bd8b7-qpj8k\" (UID: \"9e9cb1a4-4a5f-417d-9ad4-9769dd1ae1fc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-290683-74d74bd8b7-qpj8k"
Sep 11 17:36:46.930418 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:46.930400    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e9cb1a4-4a5f-417d-9ad4-9769dd1ae1fc-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-290683-74d74bd8b7-qpj8k\" (UID: \"9e9cb1a4-4a5f-417d-9ad4-9769dd1ae1fc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-290683-74d74bd8b7-qpj8k"
Sep 11 17:36:46.932282 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:46.932256    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9e9cb1a4-4a5f-417d-9ad4-9769dd1ae1fc-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-290683-74d74bd8b7-qpj8k\" (UID: \"9e9cb1a4-4a5f-417d-9ad4-9769dd1ae1fc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-290683-74d74bd8b7-qpj8k"
Sep 11 17:36:46.940023 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:46.939999    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nh7md\" (UniqueName: \"kubernetes.io/projected/9e9cb1a4-4a5f-417d-9ad4-9769dd1ae1fc-kube-api-access-nh7md\") pod \"maas-api-e2e-shared-a-w2-290683-74d74bd8b7-qpj8k\" (UID: \"9e9cb1a4-4a5f-417d-9ad4-9769dd1ae1fc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-290683-74d74bd8b7-qpj8k"
Sep 11 17:36:47.221779 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:47.221741    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:36:47.435577 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:47.435538    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9e9cb1a4-4a5f-417d-9ad4-9769dd1ae1fc-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-290683-74d74bd8b7-qpj8k\" (UID: \"9e9cb1a4-4a5f-417d-9ad4-9769dd1ae1fc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-290683-74d74bd8b7-qpj8k"
Sep 11 17:36:47.439027 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:47.438965    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9e9cb1a4-4a5f-417d-9ad4-9769dd1ae1fc-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-290683-74d74bd8b7-qpj8k\" (UID: \"9e9cb1a4-4a5f-417d-9ad4-9769dd1ae1fc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-290683-74d74bd8b7-qpj8k"
Sep 11 17:36:47.491765 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:47.491658    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:36:47.607905 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:47.607871    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:36:47.639372 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:47.639336    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-290683-74d74bd8b7-qpj8k"
Sep 11 17:36:47.926084 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:47.926051    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-290683-74d74bd8b7-qpj8k"]
Sep 11 17:36:47.926806 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:36:47.926782    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e9cb1a4_4a5f_417d_9ad4_9769dd1ae1fc.slice/crio-cec81e42b19d1a5ec52e4a4b193cd1bd2a8eb7c32c208f81400e0a04c09600de WatchSource:0}: Error finding container cec81e42b19d1a5ec52e4a4b193cd1bd2a8eb7c32c208f81400e0a04c09600de: Status 404 returned error can't find the container with id cec81e42b19d1a5ec52e4a4b193cd1bd2a8eb7c32c208f81400e0a04c09600de
Sep 11 17:36:48.338841 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:48.338777    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-290683-74d74bd8b7-qpj8k" event={"ID":"9e9cb1a4-4a5f-417d-9ad4-9769dd1ae1fc","Type":"ContainerStarted","Data":"cec81e42b19d1a5ec52e4a4b193cd1bd2a8eb7c32c208f81400e0a04c09600de"}
Sep 11 17:36:49.349617 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:49.349572    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-290683-74d74bd8b7-qpj8k" event={"ID":"9e9cb1a4-4a5f-417d-9ad4-9769dd1ae1fc","Type":"ContainerStarted","Data":"7610238f9f5881c24a588050cf2460416f4020e498fa06792cddfe633600c9d4"}
Sep 11 17:36:49.350174 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:49.349671    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-290683-74d74bd8b7-qpj8k"
Sep 11 17:36:49.375988 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:49.375932    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-290683-74d74bd8b7-qpj8k" podStartSLOduration=2.902536917 podStartE2EDuration="3.375915053s" totalImagesPullingTime="473.378136ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:36:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:36:49.373192465 +0000 UTC m=+1103.894483987" watchObservedRunningTime="2026-09-11 17:36:49.375915053 +0000 UTC m=+1103.897206575"
Sep 11 17:36:50.773683 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:50.773651    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:36:55.360892 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:36:55.360863    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-290683-74d74bd8b7-qpj8k"
Sep 11 17:37:04.467258 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:04.467221    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:37:07.164567 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:07.164528    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:37:08.093911 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:08.093883    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:37:09.756212 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:09.756171    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:37:10.687437 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:10.687386    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:37:11.674421 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:11.674372    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:37:12.326520 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:12.326482    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:37:12.431688 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:12.431657    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:37:12.641913 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:12.641869    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:37:12.743025 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:12.742991    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:37:12.934286 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:12.934204    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w2-2f0a78-openshift-default-5f8fffc99b-4rrv5"]
Sep 11 17:37:12.937933 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:12.937906    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-2f0a78-openshift-default-5f8fffc99b-4rrv5"
Sep 11 17:37:12.941237 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:12.941164    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w2-2f0a78-openshift-default-dockercfg-vjmjq\""
Sep 11 17:37:12.965696 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:12.965667    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:37:12.966756 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:12.966722    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-2f0a78-openshift-default-5f8fffc99b-4rrv5"]
Sep 11 17:37:12.996162 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:12.996122    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-85kgq\" (UniqueName: \"kubernetes.io/projected/bb3e4bcb-fefa-4488-be59-1ac3b5b56bd4-kube-api-access-85kgq\") pod \"e2e-shared-b-w2-2f0a78-openshift-default-5f8fffc99b-4rrv5\" (UID: \"bb3e4bcb-fefa-4488-be59-1ac3b5b56bd4\") " pod="openshift-ingress/e2e-shared-b-w2-2f0a78-openshift-default-5f8fffc99b-4rrv5"
Sep 11 17:37:12.996332 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:12.996181    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bb3e4bcb-fefa-4488-be59-1ac3b5b56bd4-istio-data\") pod \"e2e-shared-b-w2-2f0a78-openshift-default-5f8fffc99b-4rrv5\" (UID: \"bb3e4bcb-fefa-4488-be59-1ac3b5b56bd4\") " pod="openshift-ingress/e2e-shared-b-w2-2f0a78-openshift-default-5f8fffc99b-4rrv5"
Sep 11 17:37:12.996332 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:12.996237    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bb3e4bcb-fefa-4488-be59-1ac3b5b56bd4-istio-token\") pod \"e2e-shared-b-w2-2f0a78-openshift-default-5f8fffc99b-4rrv5\" (UID: \"bb3e4bcb-fefa-4488-be59-1ac3b5b56bd4\") " pod="openshift-ingress/e2e-shared-b-w2-2f0a78-openshift-default-5f8fffc99b-4rrv5"
Sep 11 17:37:12.996427 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:12.996330    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb3e4bcb-fefa-4488-be59-1ac3b5b56bd4-credential-socket\") pod \"e2e-shared-b-w2-2f0a78-openshift-default-5f8fffc99b-4rrv5\" (UID: \"bb3e4bcb-fefa-4488-be59-1ac3b5b56bd4\") " pod="openshift-ingress/e2e-shared-b-w2-2f0a78-openshift-default-5f8fffc99b-4rrv5"
Sep 11 17:37:12.996427 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:12.996364    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bb3e4bcb-fefa-4488-be59-1ac3b5b56bd4-istio-envoy\") pod \"e2e-shared-b-w2-2f0a78-openshift-default-5f8fffc99b-4rrv5\" (UID: \"bb3e4bcb-fefa-4488-be59-1ac3b5b56bd4\") " pod="openshift-ingress/e2e-shared-b-w2-2f0a78-openshift-default-5f8fffc99b-4rrv5"
Sep 11 17:37:12.996427 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:12.996393    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bb3e4bcb-fefa-4488-be59-1ac3b5b56bd4-workload-certs\") pod \"e2e-shared-b-w2-2f0a78-openshift-default-5f8fffc99b-4rrv5\" (UID: \"bb3e4bcb-fefa-4488-be59-1ac3b5b56bd4\") " pod="openshift-ingress/e2e-shared-b-w2-2f0a78-openshift-default-5f8fffc99b-4rrv5"
Sep 11 17:37:12.996538 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:12.996427    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bb3e4bcb-fefa-4488-be59-1ac3b5b56bd4-istiod-ca-cert\") pod \"e2e-shared-b-w2-2f0a78-openshift-default-5f8fffc99b-4rrv5\" (UID: \"bb3e4bcb-fefa-4488-be59-1ac3b5b56bd4\") " pod="openshift-ingress/e2e-shared-b-w2-2f0a78-openshift-default-5f8fffc99b-4rrv5"
Sep 11 17:37:12.996581 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:12.996540    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bb3e4bcb-fefa-4488-be59-1ac3b5b56bd4-istio-podinfo\") pod \"e2e-shared-b-w2-2f0a78-openshift-default-5f8fffc99b-4rrv5\" (UID: \"bb3e4bcb-fefa-4488-be59-1ac3b5b56bd4\") " pod="openshift-ingress/e2e-shared-b-w2-2f0a78-openshift-default-5f8fffc99b-4rrv5"
Sep 11 17:37:12.996626 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:12.996608    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb3e4bcb-fefa-4488-be59-1ac3b5b56bd4-workload-socket\") pod \"e2e-shared-b-w2-2f0a78-openshift-default-5f8fffc99b-4rrv5\" (UID: \"bb3e4bcb-fefa-4488-be59-1ac3b5b56bd4\") " pod="openshift-ingress/e2e-shared-b-w2-2f0a78-openshift-default-5f8fffc99b-4rrv5"
Sep 11 17:37:13.097788 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:13.097756    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb3e4bcb-fefa-4488-be59-1ac3b5b56bd4-credential-socket\") pod \"e2e-shared-b-w2-2f0a78-openshift-default-5f8fffc99b-4rrv5\" (UID: \"bb3e4bcb-fefa-4488-be59-1ac3b5b56bd4\") " pod="openshift-ingress/e2e-shared-b-w2-2f0a78-openshift-default-5f8fffc99b-4rrv5"
Sep 11 17:37:13.097963 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:13.097804    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bb3e4bcb-fefa-4488-be59-1ac3b5b56bd4-istio-envoy\") pod \"e2e-shared-b-w2-2f0a78-openshift-default-5f8fffc99b-4rrv5\" (UID: \"bb3e4bcb-fefa-4488-be59-1ac3b5b56bd4\") " pod="openshift-ingress/e2e-shared-b-w2-2f0a78-openshift-default-5f8fffc99b-4rrv5"
Sep 11 17:37:13.097963 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:13.097839    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bb3e4bcb-fefa-4488-be59-1ac3b5b56bd4-workload-certs\") pod \"e2e-shared-b-w2-2f0a78-openshift-default-5f8fffc99b-4rrv5\" (UID: \"bb3e4bcb-fefa-4488-be59-1ac3b5b56bd4\") " pod="openshift-ingress/e2e-shared-b-w2-2f0a78-openshift-default-5f8fffc99b-4rrv5"
Sep 11 17:37:13.097963 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:13.097866    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bb3e4bcb-fefa-4488-be59-1ac3b5b56bd4-istiod-ca-cert\") pod \"e2e-shared-b-w2-2f0a78-openshift-default-5f8fffc99b-4rrv5\" (UID: \"bb3e4bcb-fefa-4488-be59-1ac3b5b56bd4\") " pod="openshift-ingress/e2e-shared-b-w2-2f0a78-openshift-default-5f8fffc99b-4rrv5"
Sep 11 17:37:13.097963 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:13.097938    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bb3e4bcb-fefa-4488-be59-1ac3b5b56bd4-istio-podinfo\") pod \"e2e-shared-b-w2-2f0a78-openshift-default-5f8fffc99b-4rrv5\" (UID: \"bb3e4bcb-fefa-4488-be59-1ac3b5b56bd4\") " pod="openshift-ingress/e2e-shared-b-w2-2f0a78-openshift-default-5f8fffc99b-4rrv5"
Sep 11 17:37:13.098234 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:13.097980    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb3e4bcb-fefa-4488-be59-1ac3b5b56bd4-workload-socket\") pod \"e2e-shared-b-w2-2f0a78-openshift-default-5f8fffc99b-4rrv5\" (UID: \"bb3e4bcb-fefa-4488-be59-1ac3b5b56bd4\") " pod="openshift-ingress/e2e-shared-b-w2-2f0a78-openshift-default-5f8fffc99b-4rrv5"
Sep 11 17:37:13.098326 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:13.098299    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb3e4bcb-fefa-4488-be59-1ac3b5b56bd4-credential-socket\") pod \"e2e-shared-b-w2-2f0a78-openshift-default-5f8fffc99b-4rrv5\" (UID: \"bb3e4bcb-fefa-4488-be59-1ac3b5b56bd4\") " pod="openshift-ingress/e2e-shared-b-w2-2f0a78-openshift-default-5f8fffc99b-4rrv5"
Sep 11 17:37:13.098383 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:13.098302    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-85kgq\" (UniqueName: \"kubernetes.io/projected/bb3e4bcb-fefa-4488-be59-1ac3b5b56bd4-kube-api-access-85kgq\") pod \"e2e-shared-b-w2-2f0a78-openshift-default-5f8fffc99b-4rrv5\" (UID: \"bb3e4bcb-fefa-4488-be59-1ac3b5b56bd4\") " pod="openshift-ingress/e2e-shared-b-w2-2f0a78-openshift-default-5f8fffc99b-4rrv5"
Sep 11 17:37:13.098443 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:13.098380    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bb3e4bcb-fefa-4488-be59-1ac3b5b56bd4-istio-data\") pod \"e2e-shared-b-w2-2f0a78-openshift-default-5f8fffc99b-4rrv5\" (UID: \"bb3e4bcb-fefa-4488-be59-1ac3b5b56bd4\") " pod="openshift-ingress/e2e-shared-b-w2-2f0a78-openshift-default-5f8fffc99b-4rrv5"
Sep 11 17:37:13.098494 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:13.098447    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bb3e4bcb-fefa-4488-be59-1ac3b5b56bd4-istio-token\") pod \"e2e-shared-b-w2-2f0a78-openshift-default-5f8fffc99b-4rrv5\" (UID: \"bb3e4bcb-fefa-4488-be59-1ac3b5b56bd4\") " pod="openshift-ingress/e2e-shared-b-w2-2f0a78-openshift-default-5f8fffc99b-4rrv5"
Sep 11 17:37:13.098770 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:13.098744    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bb3e4bcb-fefa-4488-be59-1ac3b5b56bd4-istio-data\") pod \"e2e-shared-b-w2-2f0a78-openshift-default-5f8fffc99b-4rrv5\" (UID: \"bb3e4bcb-fefa-4488-be59-1ac3b5b56bd4\") " pod="openshift-ingress/e2e-shared-b-w2-2f0a78-openshift-default-5f8fffc99b-4rrv5"
Sep 11 17:37:13.098878 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:13.098853    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bb3e4bcb-fefa-4488-be59-1ac3b5b56bd4-istiod-ca-cert\") pod \"e2e-shared-b-w2-2f0a78-openshift-default-5f8fffc99b-4rrv5\" (UID: \"bb3e4bcb-fefa-4488-be59-1ac3b5b56bd4\") " pod="openshift-ingress/e2e-shared-b-w2-2f0a78-openshift-default-5f8fffc99b-4rrv5"
Sep 11 17:37:13.098949 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:13.098935    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bb3e4bcb-fefa-4488-be59-1ac3b5b56bd4-workload-certs\") pod \"e2e-shared-b-w2-2f0a78-openshift-default-5f8fffc99b-4rrv5\" (UID: \"bb3e4bcb-fefa-4488-be59-1ac3b5b56bd4\") " pod="openshift-ingress/e2e-shared-b-w2-2f0a78-openshift-default-5f8fffc99b-4rrv5"
Sep 11 17:37:13.099006 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:13.098983    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb3e4bcb-fefa-4488-be59-1ac3b5b56bd4-workload-socket\") pod \"e2e-shared-b-w2-2f0a78-openshift-default-5f8fffc99b-4rrv5\" (UID: \"bb3e4bcb-fefa-4488-be59-1ac3b5b56bd4\") " pod="openshift-ingress/e2e-shared-b-w2-2f0a78-openshift-default-5f8fffc99b-4rrv5"
Sep 11 17:37:13.100235 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:13.100211    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bb3e4bcb-fefa-4488-be59-1ac3b5b56bd4-istio-envoy\") pod \"e2e-shared-b-w2-2f0a78-openshift-default-5f8fffc99b-4rrv5\" (UID: \"bb3e4bcb-fefa-4488-be59-1ac3b5b56bd4\") " pod="openshift-ingress/e2e-shared-b-w2-2f0a78-openshift-default-5f8fffc99b-4rrv5"
Sep 11 17:37:13.100809 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:13.100768    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bb3e4bcb-fefa-4488-be59-1ac3b5b56bd4-istio-podinfo\") pod \"e2e-shared-b-w2-2f0a78-openshift-default-5f8fffc99b-4rrv5\" (UID: \"bb3e4bcb-fefa-4488-be59-1ac3b5b56bd4\") " pod="openshift-ingress/e2e-shared-b-w2-2f0a78-openshift-default-5f8fffc99b-4rrv5"
Sep 11 17:37:13.107070 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:13.107049    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-85kgq\" (UniqueName: \"kubernetes.io/projected/bb3e4bcb-fefa-4488-be59-1ac3b5b56bd4-kube-api-access-85kgq\") pod \"e2e-shared-b-w2-2f0a78-openshift-default-5f8fffc99b-4rrv5\" (UID: \"bb3e4bcb-fefa-4488-be59-1ac3b5b56bd4\") " pod="openshift-ingress/e2e-shared-b-w2-2f0a78-openshift-default-5f8fffc99b-4rrv5"
Sep 11 17:37:13.107742 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:13.107715    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bb3e4bcb-fefa-4488-be59-1ac3b5b56bd4-istio-token\") pod \"e2e-shared-b-w2-2f0a78-openshift-default-5f8fffc99b-4rrv5\" (UID: \"bb3e4bcb-fefa-4488-be59-1ac3b5b56bd4\") " pod="openshift-ingress/e2e-shared-b-w2-2f0a78-openshift-default-5f8fffc99b-4rrv5"
Sep 11 17:37:13.250819 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:13.250728    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-2f0a78-openshift-default-5f8fffc99b-4rrv5"
Sep 11 17:37:13.541757 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:13.541729    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-2f0a78-openshift-default-5f8fffc99b-4rrv5"]
Sep 11 17:37:13.544414 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:37:13.544374    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb3e4bcb_fefa_4488_be59_1ac3b5b56bd4.slice/crio-2b2214b3532b31b0d169d196187b45d941b03f79af654e317cd012cafcdd36b9 WatchSource:0}: Error finding container 2b2214b3532b31b0d169d196187b45d941b03f79af654e317cd012cafcdd36b9: Status 404 returned error can't find the container with id 2b2214b3532b31b0d169d196187b45d941b03f79af654e317cd012cafcdd36b9
Sep 11 17:37:13.547218 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:13.547179    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 17:37:13.547396 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:13.547272    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 17:37:13.547396 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:13.547318    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 17:37:13.646788 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:13.646691    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:37:14.232453 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:14.232414    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:37:14.521292 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:14.521200    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-2f0a78-openshift-default-5f8fffc99b-4rrv5" event={"ID":"bb3e4bcb-fefa-4488-be59-1ac3b5b56bd4","Type":"ContainerStarted","Data":"a7e64f6828bdcccf84f762dcb31cff7fd1c20bbc0f4298312fd6c95c3b7099b4"}
Sep 11 17:37:14.521292 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:14.521252    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-2f0a78-openshift-default-5f8fffc99b-4rrv5" event={"ID":"bb3e4bcb-fefa-4488-be59-1ac3b5b56bd4","Type":"ContainerStarted","Data":"2b2214b3532b31b0d169d196187b45d941b03f79af654e317cd012cafcdd36b9"}
Sep 11 17:37:14.554341 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:14.554290    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w2-2f0a78-openshift-default-5f8fffc99b-4rrv5" podStartSLOduration=2.554273263 podStartE2EDuration="2.554273263s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:37:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:37:14.553005233 +0000 UTC m=+1129.074296767" watchObservedRunningTime="2026-09-11 17:37:14.554273263 +0000 UTC m=+1129.075564783"
Sep 11 17:37:14.586467 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:14.586435    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:37:15.251469 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:15.251431    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-2f0a78-openshift-default-5f8fffc99b-4rrv5"
Sep 11 17:37:15.256974 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:15.256949    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-2f0a78-openshift-default-5f8fffc99b-4rrv5"
Sep 11 17:37:15.528875 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:15.528785    2552 generic.go:356] "Generic (PLEG): container finished" podID="34c43407-b150-4315-b2d4-b1413944bdd3" containerID="9f5858a971d9e8f8f91f9b412e2797ba213b59c52ec2d31e9d9d5e236461056c" exitCode=0
Sep 11 17:37:15.529080 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:15.528869    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-290683-openshift-default-57c7b78dd6-q77mh" event={"ID":"34c43407-b150-4315-b2d4-b1413944bdd3","Type":"ContainerDied","Data":"9f5858a971d9e8f8f91f9b412e2797ba213b59c52ec2d31e9d9d5e236461056c"}
Sep 11 17:37:15.529353 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:15.529327    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-2f0a78-openshift-default-5f8fffc99b-4rrv5"
Sep 11 17:37:15.529561 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:15.529540    2552 scope.go:117] "RemoveContainer" containerID="9f5858a971d9e8f8f91f9b412e2797ba213b59c52ec2d31e9d9d5e236461056c"
Sep 11 17:37:15.530373 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:15.530224    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-2f0a78-openshift-default-5f8fffc99b-4rrv5"
Sep 11 17:37:15.530717 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:15.530693    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 17:37:15.530764 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:15.530755    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 17:37:15.530805 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:15.530792    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 17:37:15.826457 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:15.826369    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:37:16.540493 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:16.540457    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-290683-openshift-default-57c7b78dd6-q77mh" event={"ID":"34c43407-b150-4315-b2d4-b1413944bdd3","Type":"ContainerStarted","Data":"a05ccad1ecbfdbc40d7efea851903347f1edad657b01f11184ec50f50488f722"}
Sep 11 17:37:16.630898 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:16.630869    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:37:16.723268 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:16.723236    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:37:16.872780 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:16.872748    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-290683-openshift-default-57c7b78dd6-q77mh"
Sep 11 17:37:16.872991 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:16.872973    2552 patch_prober.go:28] interesting pod/e2e-shared-a-w2-290683-openshift-default-57c7b78dd6-q77mh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" start-of-body=
Sep 11 17:37:16.873048 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:16.873013    2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-290683-openshift-default-57c7b78dd6-q77mh" podUID="34c43407-b150-4315-b2d4-b1413944bdd3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused"
Sep 11 17:37:17.688053 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:17.688020    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:37:17.873224 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:17.873190    2552 patch_prober.go:28] interesting pod/e2e-shared-a-w2-290683-openshift-default-57c7b78dd6-q77mh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" start-of-body=
Sep 11 17:37:17.873422 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:17.873262    2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-290683-openshift-default-57c7b78dd6-q77mh" podUID="34c43407-b150-4315-b2d4-b1413944bdd3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused"
Sep 11 17:37:18.741267 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:18.741097    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:37:18.873370 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:18.873330    2552 patch_prober.go:28] interesting pod/e2e-shared-a-w2-290683-openshift-default-57c7b78dd6-q77mh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" start-of-body=
Sep 11 17:37:18.873543 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:18.873398    2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-290683-openshift-default-57c7b78dd6-q77mh" podUID="34c43407-b150-4315-b2d4-b1413944bdd3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused"
Sep 11 17:37:19.522202 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:19.522154    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:37:19.872972 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:19.872933    2552 patch_prober.go:28] interesting pod/e2e-shared-a-w2-290683-openshift-default-57c7b78dd6-q77mh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" start-of-body=
Sep 11 17:37:19.873417 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:19.872990    2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-290683-openshift-default-57c7b78dd6-q77mh" podUID="34c43407-b150-4315-b2d4-b1413944bdd3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused"
Sep 11 17:37:20.423467 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:20.423401    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:37:20.873395 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:20.873350    2552 patch_prober.go:28] interesting pod/e2e-shared-a-w2-290683-openshift-default-57c7b78dd6-q77mh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" start-of-body=
Sep 11 17:37:20.873891 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:20.873418    2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-290683-openshift-default-57c7b78dd6-q77mh" podUID="34c43407-b150-4315-b2d4-b1413944bdd3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused"
Sep 11 17:37:21.178173 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:21.178033    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:37:21.877232 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:21.877196    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-290683-openshift-default-57c7b78dd6-q77mh"
Sep 11 17:37:21.877706 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:21.877574    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-290683-openshift-default-57c7b78dd6-q77mh"
Sep 11 17:37:21.878222 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:21.878200    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-290683-openshift-default-57c7b78dd6-q77mh"
Sep 11 17:37:21.998548 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:21.998508    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:37:22.850804 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:22.850772    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-2f0a78-ccb9cd454-8h8gh"]
Sep 11 17:37:22.853985 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:22.853968    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-2f0a78-ccb9cd454-8h8gh"
Sep 11 17:37:22.862531 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:22.862509    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w2-2f0a78-serving-cert\""
Sep 11 17:37:22.874785 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:22.874760    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-2f0a78-ccb9cd454-8h8gh"]
Sep 11 17:37:22.903039 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:22.903006    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t4c8t\" (UniqueName: \"kubernetes.io/projected/474d5c3f-d5e3-40bf-bc71-c9584b4efeec-kube-api-access-t4c8t\") pod \"maas-api-e2e-shared-b-w2-2f0a78-ccb9cd454-8h8gh\" (UID: \"474d5c3f-d5e3-40bf-bc71-c9584b4efeec\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-2f0a78-ccb9cd454-8h8gh"
Sep 11 17:37:22.903494 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:22.903066    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/474d5c3f-d5e3-40bf-bc71-c9584b4efeec-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-2f0a78-ccb9cd454-8h8gh\" (UID: \"474d5c3f-d5e3-40bf-bc71-c9584b4efeec\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-2f0a78-ccb9cd454-8h8gh"
Sep 11 17:37:22.903494 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:22.903093    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/474d5c3f-d5e3-40bf-bc71-c9584b4efeec-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-2f0a78-ccb9cd454-8h8gh\" (UID: \"474d5c3f-d5e3-40bf-bc71-c9584b4efeec\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-2f0a78-ccb9cd454-8h8gh"
Sep 11 17:37:22.903494 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:22.903214    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/474d5c3f-d5e3-40bf-bc71-c9584b4efeec-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-2f0a78-ccb9cd454-8h8gh\" (UID: \"474d5c3f-d5e3-40bf-bc71-c9584b4efeec\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-2f0a78-ccb9cd454-8h8gh"
Sep 11 17:37:22.904278 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:22.904251    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:37:22.998070 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:22.998033    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:37:23.004336 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:23.004311    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t4c8t\" (UniqueName: \"kubernetes.io/projected/474d5c3f-d5e3-40bf-bc71-c9584b4efeec-kube-api-access-t4c8t\") pod \"maas-api-e2e-shared-b-w2-2f0a78-ccb9cd454-8h8gh\" (UID: \"474d5c3f-d5e3-40bf-bc71-c9584b4efeec\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-2f0a78-ccb9cd454-8h8gh"
Sep 11 17:37:23.004473 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:23.004356    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/474d5c3f-d5e3-40bf-bc71-c9584b4efeec-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-2f0a78-ccb9cd454-8h8gh\" (UID: \"474d5c3f-d5e3-40bf-bc71-c9584b4efeec\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-2f0a78-ccb9cd454-8h8gh"
Sep 11 17:37:23.004473 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:23.004375    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/474d5c3f-d5e3-40bf-bc71-c9584b4efeec-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-2f0a78-ccb9cd454-8h8gh\" (UID: \"474d5c3f-d5e3-40bf-bc71-c9584b4efeec\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-2f0a78-ccb9cd454-8h8gh"
Sep 11 17:37:23.004587 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:23.004497    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/474d5c3f-d5e3-40bf-bc71-c9584b4efeec-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-2f0a78-ccb9cd454-8h8gh\" (UID: \"474d5c3f-d5e3-40bf-bc71-c9584b4efeec\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-2f0a78-ccb9cd454-8h8gh"
Sep 11 17:37:23.004647 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:37:23.004612    2552 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-2f0a78-serving-cert: secret "maas-api-e2e-shared-b-w2-2f0a78-serving-cert" not found
Sep 11 17:37:23.004698 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:37:23.004682    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/474d5c3f-d5e3-40bf-bc71-c9584b4efeec-maas-api-tls podName:474d5c3f-d5e3-40bf-bc71-c9584b4efeec nodeName:}" failed. No retries permitted until 2026-09-11 17:37:23.504661499 +0000 UTC m=+1138.025953003 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/474d5c3f-d5e3-40bf-bc71-c9584b4efeec-maas-api-tls") pod "maas-api-e2e-shared-b-w2-2f0a78-ccb9cd454-8h8gh" (UID: "474d5c3f-d5e3-40bf-bc71-c9584b4efeec") : secret "maas-api-e2e-shared-b-w2-2f0a78-serving-cert" not found
Sep 11 17:37:23.005085 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:23.005054    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/474d5c3f-d5e3-40bf-bc71-c9584b4efeec-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-2f0a78-ccb9cd454-8h8gh\" (UID: \"474d5c3f-d5e3-40bf-bc71-c9584b4efeec\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-2f0a78-ccb9cd454-8h8gh"
Sep 11 17:37:23.006993 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:23.006970    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/474d5c3f-d5e3-40bf-bc71-c9584b4efeec-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-2f0a78-ccb9cd454-8h8gh\" (UID: \"474d5c3f-d5e3-40bf-bc71-c9584b4efeec\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-2f0a78-ccb9cd454-8h8gh"
Sep 11 17:37:23.018729 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:23.018705    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t4c8t\" (UniqueName: \"kubernetes.io/projected/474d5c3f-d5e3-40bf-bc71-c9584b4efeec-kube-api-access-t4c8t\") pod \"maas-api-e2e-shared-b-w2-2f0a78-ccb9cd454-8h8gh\" (UID: \"474d5c3f-d5e3-40bf-bc71-c9584b4efeec\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-2f0a78-ccb9cd454-8h8gh"
Sep 11 17:37:23.509865 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:23.509826    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/474d5c3f-d5e3-40bf-bc71-c9584b4efeec-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-2f0a78-ccb9cd454-8h8gh\" (UID: \"474d5c3f-d5e3-40bf-bc71-c9584b4efeec\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-2f0a78-ccb9cd454-8h8gh"
Sep 11 17:37:23.512354 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:23.512328    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/474d5c3f-d5e3-40bf-bc71-c9584b4efeec-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-2f0a78-ccb9cd454-8h8gh\" (UID: \"474d5c3f-d5e3-40bf-bc71-c9584b4efeec\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-2f0a78-ccb9cd454-8h8gh"
Sep 11 17:37:23.550973 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:23.550877    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:37:23.744818 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:23.744784    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:37:23.764750 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:23.764681    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-2f0a78-ccb9cd454-8h8gh"
Sep 11 17:37:23.884287 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:23.884259    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:37:24.070084 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:24.068690    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-2f0a78-ccb9cd454-8h8gh"]
Sep 11 17:37:24.077065 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:37:24.077029    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod474d5c3f_d5e3_40bf_bc71_c9584b4efeec.slice/crio-ad64bb06798477a3162ba1a6db40fb90982f0a08568dc4ad0dc3c260c03ca9d9 WatchSource:0}: Error finding container ad64bb06798477a3162ba1a6db40fb90982f0a08568dc4ad0dc3c260c03ca9d9: Status 404 returned error can't find the container with id ad64bb06798477a3162ba1a6db40fb90982f0a08568dc4ad0dc3c260c03ca9d9
Sep 11 17:37:24.604035 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:24.603998    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-2f0a78-ccb9cd454-8h8gh" event={"ID":"474d5c3f-d5e3-40bf-bc71-c9584b4efeec","Type":"ContainerStarted","Data":"7ced25eb7e951ae19a69c03b411e4feee1f99b0ffedc049d4436ea583a51f0b6"}
Sep 11 17:37:24.604242 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:24.604094    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-2f0a78-ccb9cd454-8h8gh" event={"ID":"474d5c3f-d5e3-40bf-bc71-c9584b4efeec","Type":"ContainerStarted","Data":"ad64bb06798477a3162ba1a6db40fb90982f0a08568dc4ad0dc3c260c03ca9d9"}
Sep 11 17:37:24.604242 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:24.604141    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-2f0a78-ccb9cd454-8h8gh"
Sep 11 17:37:24.637547 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:24.637484    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-2f0a78-ccb9cd454-8h8gh" podStartSLOduration=2.211375406 podStartE2EDuration="2.637462733s" totalImagesPullingTime="426.087327ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:37:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:37:24.635888615 +0000 UTC m=+1139.157180137" watchObservedRunningTime="2026-09-11 17:37:24.637462733 +0000 UTC m=+1139.158754254"
Sep 11 17:37:30.616551 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:30.616522    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-2f0a78-ccb9cd454-8h8gh"
Sep 11 17:37:40.632928 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:40.632896    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:37:41.538226 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:41.538181    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:37:42.287091 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:42.287049    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:37:42.741221 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:42.741190    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:37:43.128687 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:43.128650    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:37:43.965238 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:43.965200    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:37:44.973299 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:44.973191    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:37:45.624447 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:45.624420    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:37:45.930682 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:45.930582    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:37:46.619484 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:46.619450    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:37:46.722503 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:46.722466    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:37:47.363679 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:47.363636    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7686-43624b"]
Sep 11 17:37:47.367100 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:47.367073    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7686-43624b"
Sep 11 17:37:47.369139 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:47.369100    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-6ptn5\""
Sep 11 17:37:47.379022 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:47.378999    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7686-43624b"]
Sep 11 17:37:47.473821 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:47.473787    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f5wml\" (UniqueName: \"kubernetes.io/projected/1ed67833-1c90-44e1-b81a-d6cdc0493a8d-kube-api-access-f5wml\") pod \"test-curl-7686-43624b\" (UID: \"1ed67833-1c90-44e1-b81a-d6cdc0493a8d\") " pod="opendatahub/test-curl-7686-43624b"
Sep 11 17:37:47.574623 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:47.574588    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f5wml\" (UniqueName: \"kubernetes.io/projected/1ed67833-1c90-44e1-b81a-d6cdc0493a8d-kube-api-access-f5wml\") pod \"test-curl-7686-43624b\" (UID: \"1ed67833-1c90-44e1-b81a-d6cdc0493a8d\") " pod="opendatahub/test-curl-7686-43624b"
Sep 11 17:37:47.602291 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:47.602260    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f5wml\" (UniqueName: \"kubernetes.io/projected/1ed67833-1c90-44e1-b81a-d6cdc0493a8d-kube-api-access-f5wml\") pod \"test-curl-7686-43624b\" (UID: \"1ed67833-1c90-44e1-b81a-d6cdc0493a8d\") " pod="opendatahub/test-curl-7686-43624b"
Sep 11 17:37:47.675871 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:47.675779    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7686-43624b"
Sep 11 17:37:47.786869 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:47.786839    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:37:47.981921 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:47.981894    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7686-43624b"]
Sep 11 17:37:48.669554 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:48.669516    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:37:48.779227 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:48.779189    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7686-43624b" event={"ID":"1ed67833-1c90-44e1-b81a-d6cdc0493a8d","Type":"ContainerStarted","Data":"c3deb1cc13e54a01d0536e0b726118f584acb8d390184c64027fd1fdebd69d46"}
Sep 11 17:37:48.779644 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:48.779403    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7686-43624b" event={"ID":"1ed67833-1c90-44e1-b81a-d6cdc0493a8d","Type":"ContainerStarted","Data":"acae578715f406395044f1c5de2da0af9ff02a067cf709ebdcab50412d231015"}
Sep 11 17:37:48.797545 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:48.797499    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7686-43624b" podStartSLOduration=1.7974839089999999 podStartE2EDuration="1.797483909s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:37:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:37:48.794450971 +0000 UTC m=+1163.315742491" watchObservedRunningTime="2026-09-11 17:37:48.797483909 +0000 UTC m=+1163.318775430"
Sep 11 17:37:49.454206 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:49.454168    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7686-43624b"]
Sep 11 17:37:49.464134 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:49.462626    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7686-43624b"]
Sep 11 17:37:49.519104 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:49.519070    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:37:49.573961 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:49.573916    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7686-99b46e"]
Sep 11 17:37:49.574739 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:49.574709    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1ed67833-1c90-44e1-b81a-d6cdc0493a8d" containerName="test-curl-7686-43624b"
Sep 11 17:37:49.574868 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:49.574742    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="1ed67833-1c90-44e1-b81a-d6cdc0493a8d" containerName="test-curl-7686-43624b"
Sep 11 17:37:49.574924 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:49.574895    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="1ed67833-1c90-44e1-b81a-d6cdc0493a8d" containerName="test-curl-7686-43624b"
Sep 11 17:37:49.577748 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:49.577725    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7686-99b46e"
Sep 11 17:37:49.586156 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:49.586132    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7686-99b46e"]
Sep 11 17:37:49.594035 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:49.594013    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-486mr\" (UniqueName: \"kubernetes.io/projected/2b49f217-0f6b-4df0-8d0e-c76e9443695c-kube-api-access-486mr\") pod \"test-curl-7686-99b46e\" (UID: \"2b49f217-0f6b-4df0-8d0e-c76e9443695c\") " pod="opendatahub/test-curl-7686-99b46e"
Sep 11 17:37:49.694746 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:49.694711    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-486mr\" (UniqueName: \"kubernetes.io/projected/2b49f217-0f6b-4df0-8d0e-c76e9443695c-kube-api-access-486mr\") pod \"test-curl-7686-99b46e\" (UID: \"2b49f217-0f6b-4df0-8d0e-c76e9443695c\") " pod="opendatahub/test-curl-7686-99b46e"
Sep 11 17:37:49.703974 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:49.703938    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-486mr\" (UniqueName: \"kubernetes.io/projected/2b49f217-0f6b-4df0-8d0e-c76e9443695c-kube-api-access-486mr\") pod \"test-curl-7686-99b46e\" (UID: \"2b49f217-0f6b-4df0-8d0e-c76e9443695c\") " pod="opendatahub/test-curl-7686-99b46e"
Sep 11 17:37:49.785552 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:37:49.785469    2552 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7686-43624b" container="test-curl-7686-43624b"
Sep 11 17:37:49.892005 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:49.891969    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7686-99b46e"
Sep 11 17:37:50.203940 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:50.203915    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7686-99b46e"]
Sep 11 17:37:50.206272 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:37:50.206243    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2b49f217_0f6b_4df0_8d0e_c76e9443695c.slice/crio-81030a8e43df37fb38cef2906c7a5259291cf87bc01b4dfd7c33c9bc711c8c1a WatchSource:0}: Error finding container 81030a8e43df37fb38cef2906c7a5259291cf87bc01b4dfd7c33c9bc711c8c1a: Status 404 returned error can't find the container with id 81030a8e43df37fb38cef2906c7a5259291cf87bc01b4dfd7c33c9bc711c8c1a
Sep 11 17:37:50.471434 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:50.471323    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:37:50.793167 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:50.793060    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7686-99b46e" event={"ID":"2b49f217-0f6b-4df0-8d0e-c76e9443695c","Type":"ContainerStarted","Data":"71a5e6a860f332159d5505a28417d7187786f023b4890f992793fef3a8ce38ae"}
Sep 11 17:37:50.793167 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:50.793101    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7686-99b46e" event={"ID":"2b49f217-0f6b-4df0-8d0e-c76e9443695c","Type":"ContainerStarted","Data":"81030a8e43df37fb38cef2906c7a5259291cf87bc01b4dfd7c33c9bc711c8c1a"}
Sep 11 17:37:50.793167 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:50.793143    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7686-43624b" podUID="1ed67833-1c90-44e1-b81a-d6cdc0493a8d" containerName="test-curl-7686-43624b" containerID="cri-o://c3deb1cc13e54a01d0536e0b726118f584acb8d390184c64027fd1fdebd69d46" gracePeriod=2
Sep 11 17:37:50.830317 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:50.830259    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7686-99b46e" podStartSLOduration=1.830239406 podStartE2EDuration="1.830239406s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:37:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:37:50.826594621 +0000 UTC m=+1165.347886142" watchObservedRunningTime="2026-09-11 17:37:50.830239406 +0000 UTC m=+1165.351530929"
Sep 11 17:37:51.579234 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:51.579201    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:37:51.773446 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:51.773353    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7686-99b46e"]
Sep 11 17:37:51.791577 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:51.791546    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7686-99b46e"]
Sep 11 17:37:51.801322 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:37:51.801295    2552 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7686-99b46e" container="test-curl-7686-99b46e"
Sep 11 17:37:51.931618 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:51.931585    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7686-a7856a"]
Sep 11 17:37:51.932060 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:51.932046    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2b49f217-0f6b-4df0-8d0e-c76e9443695c" containerName="test-curl-7686-99b46e"
Sep 11 17:37:51.932126 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:51.932061    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="2b49f217-0f6b-4df0-8d0e-c76e9443695c" containerName="test-curl-7686-99b46e"
Sep 11 17:37:51.932188 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:51.932176    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="2b49f217-0f6b-4df0-8d0e-c76e9443695c" containerName="test-curl-7686-99b46e"
Sep 11 17:37:51.935166 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:51.935142    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7686-a7856a"
Sep 11 17:37:51.997626 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:51.997596    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7686-a7856a"]
Sep 11 17:37:52.021152 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:52.021096    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mbw2p\" (UniqueName: \"kubernetes.io/projected/9f6a64bb-4e91-41b7-abca-53a3cc9acdfa-kube-api-access-mbw2p\") pod \"test-curl-7686-a7856a\" (UID: \"9f6a64bb-4e91-41b7-abca-53a3cc9acdfa\") " pod="opendatahub/test-curl-7686-a7856a"
Sep 11 17:37:52.121745 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:52.121714    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mbw2p\" (UniqueName: \"kubernetes.io/projected/9f6a64bb-4e91-41b7-abca-53a3cc9acdfa-kube-api-access-mbw2p\") pod \"test-curl-7686-a7856a\" (UID: \"9f6a64bb-4e91-41b7-abca-53a3cc9acdfa\") " pod="opendatahub/test-curl-7686-a7856a"
Sep 11 17:37:52.131854 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:52.131824    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mbw2p\" (UniqueName: \"kubernetes.io/projected/9f6a64bb-4e91-41b7-abca-53a3cc9acdfa-kube-api-access-mbw2p\") pod \"test-curl-7686-a7856a\" (UID: \"9f6a64bb-4e91-41b7-abca-53a3cc9acdfa\") " pod="opendatahub/test-curl-7686-a7856a"
Sep 11 17:37:52.269948 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:52.269919    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7686-a7856a"
Sep 11 17:37:52.553267 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:52.553190    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7686-a7856a"]
Sep 11 17:37:52.557182 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:37:52.557142    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9f6a64bb_4e91_41b7_abca_53a3cc9acdfa.slice/crio-fd61184b3a8c909caa8f20da7e4fa56546fc3d56cf66c451309c0ab31a5c30f1 WatchSource:0}: Error finding container fd61184b3a8c909caa8f20da7e4fa56546fc3d56cf66c451309c0ab31a5c30f1: Status 404 returned error can't find the container with id fd61184b3a8c909caa8f20da7e4fa56546fc3d56cf66c451309c0ab31a5c30f1
Sep 11 17:37:52.588663 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:52.588631    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:37:52.810661 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:52.810631    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7686-a7856a" event={"ID":"9f6a64bb-4e91-41b7-abca-53a3cc9acdfa","Type":"ContainerStarted","Data":"fba952a6128ced15d238433934e2df8391d0200eb52a57cb50c8a5c9bab27470"}
Sep 11 17:37:52.811032 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:52.810681    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7686-a7856a" event={"ID":"9f6a64bb-4e91-41b7-abca-53a3cc9acdfa","Type":"ContainerStarted","Data":"fd61184b3a8c909caa8f20da7e4fa56546fc3d56cf66c451309c0ab31a5c30f1"}
Sep 11 17:37:52.811032 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:52.810816    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7686-99b46e" podUID="2b49f217-0f6b-4df0-8d0e-c76e9443695c" containerName="test-curl-7686-99b46e" containerID="cri-o://71a5e6a860f332159d5505a28417d7187786f023b4890f992793fef3a8ce38ae" gracePeriod=2
Sep 11 17:37:52.841817 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:52.841756    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7686-a7856a" podStartSLOduration=1.841724223 podStartE2EDuration="1.841724223s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:37:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:37:52.840617596 +0000 UTC m=+1167.361909121" watchObservedRunningTime="2026-09-11 17:37:52.841724223 +0000 UTC m=+1167.363015744"
Sep 11 17:37:52.958724 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:52.958343    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7686-43624b"
Sep 11 17:37:53.033229 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:53.033207    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f5wml\" (UniqueName: \"kubernetes.io/projected/1ed67833-1c90-44e1-b81a-d6cdc0493a8d-kube-api-access-f5wml\") pod \"1ed67833-1c90-44e1-b81a-d6cdc0493a8d\" (UID: \"1ed67833-1c90-44e1-b81a-d6cdc0493a8d\") "
Sep 11 17:37:53.035490 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:53.035458    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1ed67833-1c90-44e1-b81a-d6cdc0493a8d-kube-api-access-f5wml" (OuterVolumeSpecName: "kube-api-access-f5wml") pod "1ed67833-1c90-44e1-b81a-d6cdc0493a8d" (UID: "1ed67833-1c90-44e1-b81a-d6cdc0493a8d"). InnerVolumeSpecName "kube-api-access-f5wml". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:37:53.134925 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:53.134887    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f5wml\" (UniqueName: \"kubernetes.io/projected/1ed67833-1c90-44e1-b81a-d6cdc0493a8d-kube-api-access-f5wml\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:37:53.635221 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:53.635180    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7686-a7856a"]
Sep 11 17:37:53.648501 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:53.648468    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7686-a7856a"]
Sep 11 17:37:53.698946 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:53.698852    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:37:53.795720 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:53.795687    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7686-9598c7"]
Sep 11 17:37:53.796196 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:53.796181    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9f6a64bb-4e91-41b7-abca-53a3cc9acdfa" containerName="test-curl-7686-a7856a"
Sep 11 17:37:53.796253 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:53.796197    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f6a64bb-4e91-41b7-abca-53a3cc9acdfa" containerName="test-curl-7686-a7856a"
Sep 11 17:37:53.796296 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:53.796264    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="9f6a64bb-4e91-41b7-abca-53a3cc9acdfa" containerName="test-curl-7686-a7856a"
Sep 11 17:37:53.799237 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:53.799222    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7686-9598c7"
Sep 11 17:37:53.819214 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:53.819176    2552 generic.go:356] "Generic (PLEG): container finished" podID="1ed67833-1c90-44e1-b81a-d6cdc0493a8d" containerID="c3deb1cc13e54a01d0536e0b726118f584acb8d390184c64027fd1fdebd69d46" exitCode=137
Sep 11 17:37:53.819665 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:53.819240    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7686-43624b"
Sep 11 17:37:53.819665 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:53.819279    2552 scope.go:117] "RemoveContainer" containerID="c3deb1cc13e54a01d0536e0b726118f584acb8d390184c64027fd1fdebd69d46"
Sep 11 17:37:53.819856 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:37:53.819755    2552 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7686-a7856a" container="test-curl-7686-a7856a"
Sep 11 17:37:53.822855 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:53.822818    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7686-9598c7"]
Sep 11 17:37:53.841055 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:53.841030    2552 scope.go:117] "RemoveContainer" containerID="c3deb1cc13e54a01d0536e0b726118f584acb8d390184c64027fd1fdebd69d46"
Sep 11 17:37:53.841500 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:37:53.841464    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c3deb1cc13e54a01d0536e0b726118f584acb8d390184c64027fd1fdebd69d46\": container with ID starting with c3deb1cc13e54a01d0536e0b726118f584acb8d390184c64027fd1fdebd69d46 not found: ID does not exist" containerID="c3deb1cc13e54a01d0536e0b726118f584acb8d390184c64027fd1fdebd69d46"
Sep 11 17:37:53.841639 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:53.841511    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c3deb1cc13e54a01d0536e0b726118f584acb8d390184c64027fd1fdebd69d46"} err="failed to get container status \"c3deb1cc13e54a01d0536e0b726118f584acb8d390184c64027fd1fdebd69d46\": rpc error: code = NotFound desc = could not find container \"c3deb1cc13e54a01d0536e0b726118f584acb8d390184c64027fd1fdebd69d46\": container with ID starting with c3deb1cc13e54a01d0536e0b726118f584acb8d390184c64027fd1fdebd69d46 not found: ID does not exist"
Sep 11 17:37:53.842019 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:53.841988    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mnznr\" (UniqueName: \"kubernetes.io/projected/997c7aea-30b2-4df6-a41c-5a4a6d94f9f0-kube-api-access-mnznr\") pod \"test-curl-7686-9598c7\" (UID: \"997c7aea-30b2-4df6-a41c-5a4a6d94f9f0\") " pod="opendatahub/test-curl-7686-9598c7"
Sep 11 17:37:53.857646 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:37:53.857603    2552 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7686-43624b\" is forbidden: User \"system:node:ip-10-0-140-51.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-51.ec2.internal' and this object" podUID="1ed67833-1c90-44e1-b81a-d6cdc0493a8d" pod="opendatahub/test-curl-7686-43624b"
Sep 11 17:37:53.943334 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:53.943243    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mnznr\" (UniqueName: \"kubernetes.io/projected/997c7aea-30b2-4df6-a41c-5a4a6d94f9f0-kube-api-access-mnznr\") pod \"test-curl-7686-9598c7\" (UID: \"997c7aea-30b2-4df6-a41c-5a4a6d94f9f0\") " pod="opendatahub/test-curl-7686-9598c7"
Sep 11 17:37:53.950029 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:53.949997    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1ed67833-1c90-44e1-b81a-d6cdc0493a8d" path="/var/lib/kubelet/pods/1ed67833-1c90-44e1-b81a-d6cdc0493a8d/volumes"
Sep 11 17:37:53.966792 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:53.966761    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mnznr\" (UniqueName: \"kubernetes.io/projected/997c7aea-30b2-4df6-a41c-5a4a6d94f9f0-kube-api-access-mnznr\") pod \"test-curl-7686-9598c7\" (UID: \"997c7aea-30b2-4df6-a41c-5a4a6d94f9f0\") " pod="opendatahub/test-curl-7686-9598c7"
Sep 11 17:37:54.108536 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:54.108497    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7686-9598c7"
Sep 11 17:37:54.400363 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:37:54.400330    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod997c7aea_30b2_4df6_a41c_5a4a6d94f9f0.slice/crio-958b3066c8bbbdf545ad957913a6687e7d51784e11075eedb13e6d0c29791306 WatchSource:0}: Error finding container 958b3066c8bbbdf545ad957913a6687e7d51784e11075eedb13e6d0c29791306: Status 404 returned error can't find the container with id 958b3066c8bbbdf545ad957913a6687e7d51784e11075eedb13e6d0c29791306
Sep 11 17:37:54.401732 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:54.401700    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7686-9598c7"]
Sep 11 17:37:54.675599 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:54.675518    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-ec3fed-6ccdcff8f6-jggm8"]
Sep 11 17:37:54.675745 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:54.675707    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ec3fed-6ccdcff8f6-jggm8" podUID="48799580-f093-4a6e-aa8e-6f9d7d2ff78c" containerName="maas-api" containerID="cri-o://27203714baa7b184d47ab4d2f1283db8574fe016c2b66ae73d56ce14c29b491b" gracePeriod=30
Sep 11 17:37:54.715906 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:54.715876    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:37:54.831847 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:54.831811    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7686-9598c7" event={"ID":"997c7aea-30b2-4df6-a41c-5a4a6d94f9f0","Type":"ContainerStarted","Data":"c30cc70352d50ba65fa459047a108f97be9b39b341f54a89bdce37f9470329e7"}
Sep 11 17:37:54.832222 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:54.831867    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7686-9598c7" event={"ID":"997c7aea-30b2-4df6-a41c-5a4a6d94f9f0","Type":"ContainerStarted","Data":"958b3066c8bbbdf545ad957913a6687e7d51784e11075eedb13e6d0c29791306"}
Sep 11 17:37:54.834402 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:54.834374    2552 generic.go:356] "Generic (PLEG): container finished" podID="48799580-f093-4a6e-aa8e-6f9d7d2ff78c" containerID="27203714baa7b184d47ab4d2f1283db8574fe016c2b66ae73d56ce14c29b491b" exitCode=0
Sep 11 17:37:54.834665 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:54.834642    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7686-a7856a" podUID="9f6a64bb-4e91-41b7-abca-53a3cc9acdfa" containerName="test-curl-7686-a7856a" containerID="cri-o://fba952a6128ced15d238433934e2df8391d0200eb52a57cb50c8a5c9bab27470" gracePeriod=2
Sep 11 17:37:54.834749 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:54.834721    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ec3fed-6ccdcff8f6-jggm8" event={"ID":"48799580-f093-4a6e-aa8e-6f9d7d2ff78c","Type":"ContainerDied","Data":"27203714baa7b184d47ab4d2f1283db8574fe016c2b66ae73d56ce14c29b491b"}
Sep 11 17:37:54.879045 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:54.878670    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:37:54.889846 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:54.889702    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7686-9598c7" podStartSLOduration=1.8896817289999999 podStartE2EDuration="1.889681729s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:37:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:37:54.876962844 +0000 UTC m=+1169.398254381" watchObservedRunningTime="2026-09-11 17:37:54.889681729 +0000 UTC m=+1169.410973250"
Sep 11 17:37:54.948172 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:54.942330    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ec3fed-6ccdcff8f6-jggm8"
Sep 11 17:37:54.983408 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:54.983383    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7686-99b46e"
Sep 11 17:37:55.040228 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:55.040077    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:37:55.062577 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:55.062553    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-486mr\" (UniqueName: \"kubernetes.io/projected/2b49f217-0f6b-4df0-8d0e-c76e9443695c-kube-api-access-486mr\") pod \"2b49f217-0f6b-4df0-8d0e-c76e9443695c\" (UID: \"2b49f217-0f6b-4df0-8d0e-c76e9443695c\") "
Sep 11 17:37:55.062737 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:55.062650    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/48799580-f093-4a6e-aa8e-6f9d7d2ff78c-maas-api-tls\") pod \"48799580-f093-4a6e-aa8e-6f9d7d2ff78c\" (UID: \"48799580-f093-4a6e-aa8e-6f9d7d2ff78c\") "
Sep 11 17:37:55.062737 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:55.062702    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5jdzw\" (UniqueName: \"kubernetes.io/projected/48799580-f093-4a6e-aa8e-6f9d7d2ff78c-kube-api-access-5jdzw\") pod \"48799580-f093-4a6e-aa8e-6f9d7d2ff78c\" (UID: \"48799580-f093-4a6e-aa8e-6f9d7d2ff78c\") "
Sep 11 17:37:55.062737 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:55.062717    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/48799580-f093-4a6e-aa8e-6f9d7d2ff78c-metrics-certs\") pod \"48799580-f093-4a6e-aa8e-6f9d7d2ff78c\" (UID: \"48799580-f093-4a6e-aa8e-6f9d7d2ff78c\") "
Sep 11 17:37:55.062940 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:55.062746    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/48799580-f093-4a6e-aa8e-6f9d7d2ff78c-openshift-service-ca\") pod \"48799580-f093-4a6e-aa8e-6f9d7d2ff78c\" (UID: \"48799580-f093-4a6e-aa8e-6f9d7d2ff78c\") "
Sep 11 17:37:55.063250 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:55.063202    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/48799580-f093-4a6e-aa8e-6f9d7d2ff78c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "48799580-f093-4a6e-aa8e-6f9d7d2ff78c" (UID: "48799580-f093-4a6e-aa8e-6f9d7d2ff78c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 17:37:55.065406 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:55.065379    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/48799580-f093-4a6e-aa8e-6f9d7d2ff78c-kube-api-access-5jdzw" (OuterVolumeSpecName: "kube-api-access-5jdzw") pod "48799580-f093-4a6e-aa8e-6f9d7d2ff78c" (UID: "48799580-f093-4a6e-aa8e-6f9d7d2ff78c"). InnerVolumeSpecName "kube-api-access-5jdzw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:37:55.065506 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:55.065404    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2b49f217-0f6b-4df0-8d0e-c76e9443695c-kube-api-access-486mr" (OuterVolumeSpecName: "kube-api-access-486mr") pod "2b49f217-0f6b-4df0-8d0e-c76e9443695c" (UID: "2b49f217-0f6b-4df0-8d0e-c76e9443695c"). InnerVolumeSpecName "kube-api-access-486mr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:37:55.065506 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:55.065417    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/48799580-f093-4a6e-aa8e-6f9d7d2ff78c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "48799580-f093-4a6e-aa8e-6f9d7d2ff78c" (UID: "48799580-f093-4a6e-aa8e-6f9d7d2ff78c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 17:37:55.065584 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:55.065513    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/48799580-f093-4a6e-aa8e-6f9d7d2ff78c-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "48799580-f093-4a6e-aa8e-6f9d7d2ff78c" (UID: "48799580-f093-4a6e-aa8e-6f9d7d2ff78c"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 17:37:55.171825 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:55.171220    2552 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/48799580-f093-4a6e-aa8e-6f9d7d2ff78c-maas-api-tls\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:37:55.171825 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:55.171252    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5jdzw\" (UniqueName: \"kubernetes.io/projected/48799580-f093-4a6e-aa8e-6f9d7d2ff78c-kube-api-access-5jdzw\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:37:55.171825 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:55.171267    2552 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/48799580-f093-4a6e-aa8e-6f9d7d2ff78c-metrics-certs\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:37:55.171825 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:55.171280    2552 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/48799580-f093-4a6e-aa8e-6f9d7d2ff78c-openshift-service-ca\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:37:55.171825 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:55.171296    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-486mr\" (UniqueName: \"kubernetes.io/projected/2b49f217-0f6b-4df0-8d0e-c76e9443695c-kube-api-access-486mr\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:37:55.200744 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:55.200647    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:37:55.460275 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:55.460185    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:37:55.658238 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:55.658208    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:37:55.783484 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:55.783391    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7686-9598c7"]
Sep 11 17:37:55.788573 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:55.788544    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7686-9598c7"]
Sep 11 17:37:55.844328 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:55.844297    2552 generic.go:356] "Generic (PLEG): container finished" podID="bb3e4bcb-fefa-4488-be59-1ac3b5b56bd4" containerID="a7e64f6828bdcccf84f762dcb31cff7fd1c20bbc0f4298312fd6c95c3b7099b4" exitCode=0
Sep 11 17:37:55.844747 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:55.844375    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-2f0a78-openshift-default-5f8fffc99b-4rrv5" event={"ID":"bb3e4bcb-fefa-4488-be59-1ac3b5b56bd4","Type":"ContainerDied","Data":"a7e64f6828bdcccf84f762dcb31cff7fd1c20bbc0f4298312fd6c95c3b7099b4"}
Sep 11 17:37:55.844922 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:55.844904    2552 scope.go:117] "RemoveContainer" containerID="a7e64f6828bdcccf84f762dcb31cff7fd1c20bbc0f4298312fd6c95c3b7099b4"
Sep 11 17:37:55.846014 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:55.846000    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ec3fed-6ccdcff8f6-jggm8"
Sep 11 17:37:55.846099 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:55.846035    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ec3fed-6ccdcff8f6-jggm8" event={"ID":"48799580-f093-4a6e-aa8e-6f9d7d2ff78c","Type":"ContainerDied","Data":"a0eb8516b322657b2a285a2bf8318075259b62705f971ba68f9557d80e77c766"}
Sep 11 17:37:55.846099 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:55.846079    2552 scope.go:117] "RemoveContainer" containerID="27203714baa7b184d47ab4d2f1283db8574fe016c2b66ae73d56ce14c29b491b"
Sep 11 17:37:55.846276 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:55.846247    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 17:37:55.846366 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:55.846348    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 17:37:55.846418 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:55.846400    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 17:37:55.848023 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:55.848002    2552 generic.go:356] "Generic (PLEG): container finished" podID="2b49f217-0f6b-4df0-8d0e-c76e9443695c" containerID="71a5e6a860f332159d5505a28417d7187786f023b4890f992793fef3a8ce38ae" exitCode=137
Sep 11 17:37:55.848143 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:55.848052    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7686-99b46e"
Sep 11 17:37:55.850935 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:37:55.850917    2552 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7686-9598c7" container="test-curl-7686-9598c7"
Sep 11 17:37:55.861434 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:55.861417    2552 scope.go:117] "RemoveContainer" containerID="71a5e6a860f332159d5505a28417d7187786f023b4890f992793fef3a8ce38ae"
Sep 11 17:37:55.870949 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:55.870931    2552 scope.go:117] "RemoveContainer" containerID="71a5e6a860f332159d5505a28417d7187786f023b4890f992793fef3a8ce38ae"
Sep 11 17:37:55.871259 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:37:55.871235    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"71a5e6a860f332159d5505a28417d7187786f023b4890f992793fef3a8ce38ae\": container with ID starting with 71a5e6a860f332159d5505a28417d7187786f023b4890f992793fef3a8ce38ae not found: ID does not exist" containerID="71a5e6a860f332159d5505a28417d7187786f023b4890f992793fef3a8ce38ae"
Sep 11 17:37:55.871372 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:55.871263    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"71a5e6a860f332159d5505a28417d7187786f023b4890f992793fef3a8ce38ae"} err="failed to get container status \"71a5e6a860f332159d5505a28417d7187786f023b4890f992793fef3a8ce38ae\": rpc error: code = NotFound desc = could not find container \"71a5e6a860f332159d5505a28417d7187786f023b4890f992793fef3a8ce38ae\": container with ID starting with 71a5e6a860f332159d5505a28417d7187786f023b4890f992793fef3a8ce38ae not found: ID does not exist"
Sep 11 17:37:55.903727 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:37:55.903699    2552 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7686-99b46e\" is forbidden: User \"system:node:ip-10-0-140-51.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-51.ec2.internal' and this object" podUID="2b49f217-0f6b-4df0-8d0e-c76e9443695c" pod="opendatahub/test-curl-7686-99b46e"
Sep 11 17:37:55.954782 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:55.954685    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2b49f217-0f6b-4df0-8d0e-c76e9443695c" path="/var/lib/kubelet/pods/2b49f217-0f6b-4df0-8d0e-c76e9443695c/volumes"
Sep 11 17:37:55.955063 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:55.955043    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-ec3fed-6ccdcff8f6-jggm8"]
Sep 11 17:37:55.955182 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:55.955082    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-ec3fed-6ccdcff8f6-jggm8"]
Sep 11 17:37:55.955434 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:37:55.955412    2552 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7686-99b46e\" is forbidden: User \"system:node:ip-10-0-140-51.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-51.ec2.internal' and this object" podUID="2b49f217-0f6b-4df0-8d0e-c76e9443695c" pod="opendatahub/test-curl-7686-99b46e"
Sep 11 17:37:56.522587 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:56.522546    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:37:56.857394 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:56.857369    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-2f0a78-openshift-default-5f8fffc99b-4rrv5" event={"ID":"bb3e4bcb-fefa-4488-be59-1ac3b5b56bd4","Type":"ContainerStarted","Data":"da49bff43604ec2aa054bd0dda6ed030e07a2a927f1baa12ac4f7a2ce014633b"}
Sep 11 17:37:56.859699 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:56.859672    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7686-9598c7" podUID="997c7aea-30b2-4df6-a41c-5a4a6d94f9f0" containerName="test-curl-7686-9598c7" containerID="cri-o://c30cc70352d50ba65fa459047a108f97be9b39b341f54a89bdce37f9470329e7" gracePeriod=2
Sep 11 17:37:56.997848 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:56.997813    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:37:57.071580 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:57.071555    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7686-a7856a"
Sep 11 17:37:57.192789 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:57.192684    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mbw2p\" (UniqueName: \"kubernetes.io/projected/9f6a64bb-4e91-41b7-abca-53a3cc9acdfa-kube-api-access-mbw2p\") pod \"9f6a64bb-4e91-41b7-abca-53a3cc9acdfa\" (UID: \"9f6a64bb-4e91-41b7-abca-53a3cc9acdfa\") "
Sep 11 17:37:57.194828 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:57.194793    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9f6a64bb-4e91-41b7-abca-53a3cc9acdfa-kube-api-access-mbw2p" (OuterVolumeSpecName: "kube-api-access-mbw2p") pod "9f6a64bb-4e91-41b7-abca-53a3cc9acdfa" (UID: "9f6a64bb-4e91-41b7-abca-53a3cc9acdfa"). InnerVolumeSpecName "kube-api-access-mbw2p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:37:57.235165 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:57.235060    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:37:57.253415 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:57.252484    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-2f0a78-openshift-default-5f8fffc99b-4rrv5"
Sep 11 17:37:57.258139 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:57.255169    2552 patch_prober.go:28] interesting pod/e2e-shared-b-w2-2f0a78-openshift-default-5f8fffc99b-4rrv5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body=
Sep 11 17:37:57.258139 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:57.255229    2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-2f0a78-openshift-default-5f8fffc99b-4rrv5" podUID="bb3e4bcb-fefa-4488-be59-1ac3b5b56bd4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused"
Sep 11 17:37:57.294688 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:57.294655    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mbw2p\" (UniqueName: \"kubernetes.io/projected/9f6a64bb-4e91-41b7-abca-53a3cc9acdfa-kube-api-access-mbw2p\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:37:57.566687 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:57.566589    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:37:57.873392 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:57.873361    2552 generic.go:356] "Generic (PLEG): container finished" podID="9f6a64bb-4e91-41b7-abca-53a3cc9acdfa" containerID="fba952a6128ced15d238433934e2df8391d0200eb52a57cb50c8a5c9bab27470" exitCode=137
Sep 11 17:37:57.873835 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:57.873726    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7686-a7856a"
Sep 11 17:37:57.873971 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:57.873952    2552 scope.go:117] "RemoveContainer" containerID="fba952a6128ced15d238433934e2df8391d0200eb52a57cb50c8a5c9bab27470"
Sep 11 17:37:57.884621 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:37:57.884592    2552 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7686-a7856a\" is forbidden: User \"system:node:ip-10-0-140-51.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-51.ec2.internal' and this object" podUID="9f6a64bb-4e91-41b7-abca-53a3cc9acdfa" pod="opendatahub/test-curl-7686-a7856a"
Sep 11 17:37:57.884937 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:57.884917    2552 scope.go:117] "RemoveContainer" containerID="fba952a6128ced15d238433934e2df8391d0200eb52a57cb50c8a5c9bab27470"
Sep 11 17:37:57.885264 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:37:57.885238    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fba952a6128ced15d238433934e2df8391d0200eb52a57cb50c8a5c9bab27470\": container with ID starting with fba952a6128ced15d238433934e2df8391d0200eb52a57cb50c8a5c9bab27470 not found: ID does not exist" containerID="fba952a6128ced15d238433934e2df8391d0200eb52a57cb50c8a5c9bab27470"
Sep 11 17:37:57.885329 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:57.885272    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fba952a6128ced15d238433934e2df8391d0200eb52a57cb50c8a5c9bab27470"} err="failed to get container status \"fba952a6128ced15d238433934e2df8391d0200eb52a57cb50c8a5c9bab27470\": rpc error: code = NotFound desc = could not find container \"fba952a6128ced15d238433934e2df8391d0200eb52a57cb50c8a5c9bab27470\": container with ID starting with fba952a6128ced15d238433934e2df8391d0200eb52a57cb50c8a5c9bab27470 not found: ID does not exist"
Sep 11 17:37:57.965137 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:57.956297    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="48799580-f093-4a6e-aa8e-6f9d7d2ff78c" path="/var/lib/kubelet/pods/48799580-f093-4a6e-aa8e-6f9d7d2ff78c/volumes"
Sep 11 17:37:57.965137 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:57.956800    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9f6a64bb-4e91-41b7-abca-53a3cc9acdfa" path="/var/lib/kubelet/pods/9f6a64bb-4e91-41b7-abca-53a3cc9acdfa/volumes"
Sep 11 17:37:58.251866 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:58.251754    2552 patch_prober.go:28] interesting pod/e2e-shared-b-w2-2f0a78-openshift-default-5f8fffc99b-4rrv5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body=
Sep 11 17:37:58.251866 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:58.251837    2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-2f0a78-openshift-default-5f8fffc99b-4rrv5" podUID="bb3e4bcb-fefa-4488-be59-1ac3b5b56bd4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused"
Sep 11 17:37:58.369608 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:58.369516    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:37:59.087056 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:59.087030    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7686-9598c7"
Sep 11 17:37:59.187386 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:59.187302    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:37:59.212864 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:59.212831    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mnznr\" (UniqueName: \"kubernetes.io/projected/997c7aea-30b2-4df6-a41c-5a4a6d94f9f0-kube-api-access-mnznr\") pod \"997c7aea-30b2-4df6-a41c-5a4a6d94f9f0\" (UID: \"997c7aea-30b2-4df6-a41c-5a4a6d94f9f0\") "
Sep 11 17:37:59.217178 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:59.217140    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/997c7aea-30b2-4df6-a41c-5a4a6d94f9f0-kube-api-access-mnznr" (OuterVolumeSpecName: "kube-api-access-mnznr") pod "997c7aea-30b2-4df6-a41c-5a4a6d94f9f0" (UID: "997c7aea-30b2-4df6-a41c-5a4a6d94f9f0"). InnerVolumeSpecName "kube-api-access-mnznr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:37:59.252183 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:59.251707    2552 patch_prober.go:28] interesting pod/e2e-shared-b-w2-2f0a78-openshift-default-5f8fffc99b-4rrv5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body=
Sep 11 17:37:59.252183 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:59.251790    2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-2f0a78-openshift-default-5f8fffc99b-4rrv5" podUID="bb3e4bcb-fefa-4488-be59-1ac3b5b56bd4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused"
Sep 11 17:37:59.314624 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:59.314584    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mnznr\" (UniqueName: \"kubernetes.io/projected/997c7aea-30b2-4df6-a41c-5a4a6d94f9f0-kube-api-access-mnznr\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:37:59.889261 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:59.889225    2552 generic.go:356] "Generic (PLEG): container finished" podID="997c7aea-30b2-4df6-a41c-5a4a6d94f9f0" containerID="c30cc70352d50ba65fa459047a108f97be9b39b341f54a89bdce37f9470329e7" exitCode=137
Sep 11 17:37:59.889456 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:59.889279    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7686-9598c7"
Sep 11 17:37:59.889456 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:59.889331    2552 scope.go:117] "RemoveContainer" containerID="c30cc70352d50ba65fa459047a108f97be9b39b341f54a89bdce37f9470329e7"
Sep 11 17:37:59.900309 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:37:59.900273    2552 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7686-9598c7\" is forbidden: User \"system:node:ip-10-0-140-51.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-51.ec2.internal' and this object" podUID="997c7aea-30b2-4df6-a41c-5a4a6d94f9f0" pod="opendatahub/test-curl-7686-9598c7"
Sep 11 17:37:59.900399 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:59.900323    2552 scope.go:117] "RemoveContainer" containerID="c30cc70352d50ba65fa459047a108f97be9b39b341f54a89bdce37f9470329e7"
Sep 11 17:37:59.900654 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:37:59.900632    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c30cc70352d50ba65fa459047a108f97be9b39b341f54a89bdce37f9470329e7\": container with ID starting with c30cc70352d50ba65fa459047a108f97be9b39b341f54a89bdce37f9470329e7 not found: ID does not exist" containerID="c30cc70352d50ba65fa459047a108f97be9b39b341f54a89bdce37f9470329e7"
Sep 11 17:37:59.900752 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:59.900667    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c30cc70352d50ba65fa459047a108f97be9b39b341f54a89bdce37f9470329e7"} err="failed to get container status \"c30cc70352d50ba65fa459047a108f97be9b39b341f54a89bdce37f9470329e7\": rpc error: code = NotFound desc = could not find container \"c30cc70352d50ba65fa459047a108f97be9b39b341f54a89bdce37f9470329e7\": container with ID starting with c30cc70352d50ba65fa459047a108f97be9b39b341f54a89bdce37f9470329e7 not found: ID does not exist"
Sep 11 17:37:59.957783 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:37:59.957741    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="997c7aea-30b2-4df6-a41c-5a4a6d94f9f0" path="/var/lib/kubelet/pods/997c7aea-30b2-4df6-a41c-5a4a6d94f9f0/volumes"
Sep 11 17:38:00.054173 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:38:00.054143    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:38:00.251585 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:38:00.251487    2552 patch_prober.go:28] interesting pod/e2e-shared-b-w2-2f0a78-openshift-default-5f8fffc99b-4rrv5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body=
Sep 11 17:38:00.251585 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:38:00.251555    2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-2f0a78-openshift-default-5f8fffc99b-4rrv5" podUID="bb3e4bcb-fefa-4488-be59-1ac3b5b56bd4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused"
Sep 11 17:38:00.431174 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:38:00.431137    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:38:00.529659 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:38:00.529580    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-2f0a78-openshift-default-5f8fffc99b-4rrv5"
Sep 11 17:38:00.547477 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:38:00.547446    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:38:00.959323 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:38:00.959291    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:38:01.180092 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:38:01.180049    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:38:01.251607 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:38:01.251513    2552 patch_prober.go:28] interesting pod/e2e-shared-b-w2-2f0a78-openshift-default-5f8fffc99b-4rrv5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body=
Sep 11 17:38:01.252159 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:38:01.252129    2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-2f0a78-openshift-default-5f8fffc99b-4rrv5" podUID="bb3e4bcb-fefa-4488-be59-1ac3b5b56bd4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused"
Sep 11 17:38:02.043547 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:38:02.043466    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:38:02.251814 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:38:02.251757    2552 patch_prober.go:28] interesting pod/e2e-shared-b-w2-2f0a78-openshift-default-5f8fffc99b-4rrv5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body=
Sep 11 17:38:02.252350 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:38:02.251836    2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-2f0a78-openshift-default-5f8fffc99b-4rrv5" podUID="bb3e4bcb-fefa-4488-be59-1ac3b5b56bd4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused"
Sep 11 17:38:02.928449 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:38:02.928369    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-6709f1a7-555b6ffbfd-ccd9z"]
Sep 11 17:38:02.928922 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:38:02.928903    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="997c7aea-30b2-4df6-a41c-5a4a6d94f9f0" containerName="test-curl-7686-9598c7"
Sep 11 17:38:02.928922 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:38:02.928923    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="997c7aea-30b2-4df6-a41c-5a4a6d94f9f0" containerName="test-curl-7686-9598c7"
Sep 11 17:38:02.929089 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:38:02.928957    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="48799580-f093-4a6e-aa8e-6f9d7d2ff78c" containerName="maas-api"
Sep 11 17:38:02.929089 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:38:02.928966    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="48799580-f093-4a6e-aa8e-6f9d7d2ff78c" containerName="maas-api"
Sep 11 17:38:02.929089 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:38:02.929082    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="997c7aea-30b2-4df6-a41c-5a4a6d94f9f0" containerName="test-curl-7686-9598c7"
Sep 11 17:38:02.929356 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:38:02.929101    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="48799580-f093-4a6e-aa8e-6f9d7d2ff78c" containerName="maas-api"
Sep 11 17:38:02.933471 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:38:02.933453    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6709f1a7-555b6ffbfd-ccd9z"
Sep 11 17:38:02.936037 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:38:02.936018    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-6709f1a7-serving-cert\""
Sep 11 17:38:02.946147 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:38:02.946089    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-6709f1a7-555b6ffbfd-ccd9z"]
Sep 11 17:38:02.974063 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:38:02.974028    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:38:03.056770 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:38:03.056734    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3fe17cc3-fdf3-4268-bf76-3ecc00c45bfb-metrics-certs\") pod \"maas-api-e2e-mt-6709f1a7-555b6ffbfd-ccd9z\" (UID: \"3fe17cc3-fdf3-4268-bf76-3ecc00c45bfb\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6709f1a7-555b6ffbfd-ccd9z"
Sep 11 17:38:03.056955 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:38:03.056840    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4nqxq\" (UniqueName: \"kubernetes.io/projected/3fe17cc3-fdf3-4268-bf76-3ecc00c45bfb-kube-api-access-4nqxq\") pod \"maas-api-e2e-mt-6709f1a7-555b6ffbfd-ccd9z\" (UID: \"3fe17cc3-fdf3-4268-bf76-3ecc00c45bfb\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6709f1a7-555b6ffbfd-ccd9z"
Sep 11 17:38:03.057045 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:38:03.057014    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3fe17cc3-fdf3-4268-bf76-3ecc00c45bfb-maas-api-tls\") pod \"maas-api-e2e-mt-6709f1a7-555b6ffbfd-ccd9z\" (UID: \"3fe17cc3-fdf3-4268-bf76-3ecc00c45bfb\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6709f1a7-555b6ffbfd-ccd9z"
Sep 11 17:38:03.057130 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:38:03.057049    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3fe17cc3-fdf3-4268-bf76-3ecc00c45bfb-openshift-service-ca\") pod \"maas-api-e2e-mt-6709f1a7-555b6ffbfd-ccd9z\" (UID: \"3fe17cc3-fdf3-4268-bf76-3ecc00c45bfb\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6709f1a7-555b6ffbfd-ccd9z"
Sep 11 17:38:03.074304 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:38:03.074270    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:38:03.158107 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:38:03.158070    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3fe17cc3-fdf3-4268-bf76-3ecc00c45bfb-maas-api-tls\") pod \"maas-api-e2e-mt-6709f1a7-555b6ffbfd-ccd9z\" (UID: \"3fe17cc3-fdf3-4268-bf76-3ecc00c45bfb\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6709f1a7-555b6ffbfd-ccd9z"
Sep 11 17:38:03.158327 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:38:03.158136    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3fe17cc3-fdf3-4268-bf76-3ecc00c45bfb-openshift-service-ca\") pod \"maas-api-e2e-mt-6709f1a7-555b6ffbfd-ccd9z\" (UID: \"3fe17cc3-fdf3-4268-bf76-3ecc00c45bfb\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6709f1a7-555b6ffbfd-ccd9z"
Sep 11 17:38:03.158327 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:38:03.158208    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3fe17cc3-fdf3-4268-bf76-3ecc00c45bfb-metrics-certs\") pod \"maas-api-e2e-mt-6709f1a7-555b6ffbfd-ccd9z\" (UID: \"3fe17cc3-fdf3-4268-bf76-3ecc00c45bfb\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6709f1a7-555b6ffbfd-ccd9z"
Sep 11 17:38:03.158327 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:38:03.158248    2552 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-6709f1a7-serving-cert: secret "maas-api-e2e-mt-6709f1a7-serving-cert" not found
Sep 11 17:38:03.158327 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:38:03.158317    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3fe17cc3-fdf3-4268-bf76-3ecc00c45bfb-maas-api-tls podName:3fe17cc3-fdf3-4268-bf76-3ecc00c45bfb nodeName:}" failed. No retries permitted until 2026-09-11 17:38:03.658295302 +0000 UTC m=+1178.179586807 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/3fe17cc3-fdf3-4268-bf76-3ecc00c45bfb-maas-api-tls") pod "maas-api-e2e-mt-6709f1a7-555b6ffbfd-ccd9z" (UID: "3fe17cc3-fdf3-4268-bf76-3ecc00c45bfb") : secret "maas-api-e2e-mt-6709f1a7-serving-cert" not found
Sep 11 17:38:03.158327 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:38:03.158254    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4nqxq\" (UniqueName: \"kubernetes.io/projected/3fe17cc3-fdf3-4268-bf76-3ecc00c45bfb-kube-api-access-4nqxq\") pod \"maas-api-e2e-mt-6709f1a7-555b6ffbfd-ccd9z\" (UID: \"3fe17cc3-fdf3-4268-bf76-3ecc00c45bfb\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6709f1a7-555b6ffbfd-ccd9z"
Sep 11 17:38:03.158869 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:38:03.158845    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3fe17cc3-fdf3-4268-bf76-3ecc00c45bfb-openshift-service-ca\") pod \"maas-api-e2e-mt-6709f1a7-555b6ffbfd-ccd9z\" (UID: \"3fe17cc3-fdf3-4268-bf76-3ecc00c45bfb\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6709f1a7-555b6ffbfd-ccd9z"
Sep 11 17:38:03.160787 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:38:03.160765    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3fe17cc3-fdf3-4268-bf76-3ecc00c45bfb-metrics-certs\") pod \"maas-api-e2e-mt-6709f1a7-555b6ffbfd-ccd9z\" (UID: \"3fe17cc3-fdf3-4268-bf76-3ecc00c45bfb\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6709f1a7-555b6ffbfd-ccd9z"
Sep 11 17:38:03.170170 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:38:03.170144    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4nqxq\" (UniqueName: \"kubernetes.io/projected/3fe17cc3-fdf3-4268-bf76-3ecc00c45bfb-kube-api-access-4nqxq\") pod \"maas-api-e2e-mt-6709f1a7-555b6ffbfd-ccd9z\" (UID: \"3fe17cc3-fdf3-4268-bf76-3ecc00c45bfb\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6709f1a7-555b6ffbfd-ccd9z"
Sep 11 17:38:03.256384 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:38:03.256304    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-2f0a78-openshift-default-5f8fffc99b-4rrv5"
Sep 11 17:38:03.257504 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:38:03.257483    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-2f0a78-openshift-default-5f8fffc99b-4rrv5"
Sep 11 17:38:03.294025 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:38:03.293984    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:38:03.538996 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:38:03.538908    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:38:03.644366 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:38:03.644327    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:38:03.663710 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:38:03.663675    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3fe17cc3-fdf3-4268-bf76-3ecc00c45bfb-maas-api-tls\") pod \"maas-api-e2e-mt-6709f1a7-555b6ffbfd-ccd9z\" (UID: \"3fe17cc3-fdf3-4268-bf76-3ecc00c45bfb\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6709f1a7-555b6ffbfd-ccd9z"
Sep 11 17:38:03.666221 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:38:03.666189    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3fe17cc3-fdf3-4268-bf76-3ecc00c45bfb-maas-api-tls\") pod \"maas-api-e2e-mt-6709f1a7-555b6ffbfd-ccd9z\" (UID: \"3fe17cc3-fdf3-4268-bf76-3ecc00c45bfb\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6709f1a7-555b6ffbfd-ccd9z"
Sep 11 17:38:03.847642 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:38:03.847606    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6709f1a7-555b6ffbfd-ccd9z"
Sep 11 17:38:04.124855 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:38:04.124830    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-6709f1a7-555b6ffbfd-ccd9z"]
Sep 11 17:38:04.125483 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:38:04.125450    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3fe17cc3_fdf3_4268_bf76_3ecc00c45bfb.slice/crio-2177318dfc0d425619e3a4e5b1bce281cbc7af47e799a151c25cca26ae80f2bb WatchSource:0}: Error finding container 2177318dfc0d425619e3a4e5b1bce281cbc7af47e799a151c25cca26ae80f2bb: Status 404 returned error can't find the container with id 2177318dfc0d425619e3a4e5b1bce281cbc7af47e799a151c25cca26ae80f2bb
Sep 11 17:38:04.194878 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:38:04.194848    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:38:04.930574 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:38:04.930476    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6709f1a7-555b6ffbfd-ccd9z" event={"ID":"3fe17cc3-fdf3-4268-bf76-3ecc00c45bfb","Type":"ContainerStarted","Data":"27e7cc98a3bc625b48e442ecb7b54cf154daf7d0ceebd04aabc6f50fbcb36a34"}
Sep 11 17:38:04.930574 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:38:04.930536    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6709f1a7-555b6ffbfd-ccd9z" event={"ID":"3fe17cc3-fdf3-4268-bf76-3ecc00c45bfb","Type":"ContainerStarted","Data":"2177318dfc0d425619e3a4e5b1bce281cbc7af47e799a151c25cca26ae80f2bb"}
Sep 11 17:38:04.931065 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:38:04.930597    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6709f1a7-555b6ffbfd-ccd9z"
Sep 11 17:38:04.958129 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:38:04.958056    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6709f1a7-555b6ffbfd-ccd9z" podStartSLOduration=2.434134434 podStartE2EDuration="2.958040942s" totalImagesPullingTime="523.906508ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:38:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:38:04.955747279 +0000 UTC m=+1179.477038799" watchObservedRunningTime="2026-09-11 17:38:04.958040942 +0000 UTC m=+1179.479332462"
Sep 11 17:38:05.026284 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:38:05.026249    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:38:05.975473 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:38:05.975444    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:38:07.069499 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:38:07.069461    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:38:07.994199 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:38:07.994163    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:38:09.063159 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:38:09.063125    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:38:09.373022 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:38:09.372988    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-explicit-ec3fed-openshift-default-6fdf496464-f2z6f"]
Sep 11 17:38:09.373258 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:38:09.373234    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-explicit-ec3fed-openshift-default-6fdf496464-f2z6f" podUID="411db317-1635-45d6-8a64-d6eaf0b4ae92" containerName="istio-proxy" containerID="cri-o://0247f032b085c9cd4a189a8ca6d74f84526d0bff3df2f5c940e38b7642b47cae" gracePeriod=30
Sep 11 17:38:09.383660 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:38:09.383633    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:38:09.889655 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:38:09.889619    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:38:10.297026 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:38:10.296943    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:38:10.452359 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:38:10.452331    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:38:10.943966 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:38:10.943938    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6709f1a7-555b6ffbfd-ccd9z"
Sep 11 17:38:11.792783 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:38:11.792750    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:38:12.021637 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:38:12.021606    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:38:12.144733 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:38:12.144697    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:38:14.625601 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:38:14.625577    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-ec3fed-openshift-default-6fdf496464-f2z6f"
Sep 11 17:38:14.782729 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:38:14.782630    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/411db317-1635-45d6-8a64-d6eaf0b4ae92-credential-socket\") pod \"411db317-1635-45d6-8a64-d6eaf0b4ae92\" (UID: \"411db317-1635-45d6-8a64-d6eaf0b4ae92\") "
Sep 11 17:38:14.782729 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:38:14.782677    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/411db317-1635-45d6-8a64-d6eaf0b4ae92-istio-envoy\") pod \"411db317-1635-45d6-8a64-d6eaf0b4ae92\" (UID: \"411db317-1635-45d6-8a64-d6eaf0b4ae92\") "
Sep 11 17:38:14.782729 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:38:14.782705    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cjtxs\" (UniqueName: \"kubernetes.io/projected/411db317-1635-45d6-8a64-d6eaf0b4ae92-kube-api-access-cjtxs\") pod \"411db317-1635-45d6-8a64-d6eaf0b4ae92\" (UID: \"411db317-1635-45d6-8a64-d6eaf0b4ae92\") "
Sep 11 17:38:14.783012 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:38:14.782746    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/411db317-1635-45d6-8a64-d6eaf0b4ae92-istio-podinfo\") pod \"411db317-1635-45d6-8a64-d6eaf0b4ae92\" (UID: \"411db317-1635-45d6-8a64-d6eaf0b4ae92\") "
Sep 11 17:38:14.783012 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:38:14.782816    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/411db317-1635-45d6-8a64-d6eaf0b4ae92-workload-socket\") pod \"411db317-1635-45d6-8a64-d6eaf0b4ae92\" (UID: \"411db317-1635-45d6-8a64-d6eaf0b4ae92\") "
Sep 11 17:38:14.783012 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:38:14.782859    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/411db317-1635-45d6-8a64-d6eaf0b4ae92-istiod-ca-cert\") pod \"411db317-1635-45d6-8a64-d6eaf0b4ae92\" (UID: \"411db317-1635-45d6-8a64-d6eaf0b4ae92\") "
Sep 11 17:38:14.783012 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:38:14.782912    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/411db317-1635-45d6-8a64-d6eaf0b4ae92-istio-token\") pod \"411db317-1635-45d6-8a64-d6eaf0b4ae92\" (UID: \"411db317-1635-45d6-8a64-d6eaf0b4ae92\") "
Sep 11 17:38:14.783012 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:38:14.782954    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/411db317-1635-45d6-8a64-d6eaf0b4ae92-istio-data\") pod \"411db317-1635-45d6-8a64-d6eaf0b4ae92\" (UID: \"411db317-1635-45d6-8a64-d6eaf0b4ae92\") "
Sep 11 17:38:14.783012 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:38:14.782962    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/411db317-1635-45d6-8a64-d6eaf0b4ae92-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "411db317-1635-45d6-8a64-d6eaf0b4ae92" (UID: "411db317-1635-45d6-8a64-d6eaf0b4ae92"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:38:14.783012 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:38:14.782980    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/411db317-1635-45d6-8a64-d6eaf0b4ae92-workload-certs\") pod \"411db317-1635-45d6-8a64-d6eaf0b4ae92\" (UID: \"411db317-1635-45d6-8a64-d6eaf0b4ae92\") "
Sep 11 17:38:14.783418 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:38:14.783212    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/411db317-1635-45d6-8a64-d6eaf0b4ae92-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "411db317-1635-45d6-8a64-d6eaf0b4ae92" (UID: "411db317-1635-45d6-8a64-d6eaf0b4ae92"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:38:14.783418 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:38:14.783238    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/411db317-1635-45d6-8a64-d6eaf0b4ae92-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "411db317-1635-45d6-8a64-d6eaf0b4ae92" (UID: "411db317-1635-45d6-8a64-d6eaf0b4ae92"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:38:14.783418 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:38:14.783267    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/411db317-1635-45d6-8a64-d6eaf0b4ae92-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "411db317-1635-45d6-8a64-d6eaf0b4ae92" (UID: "411db317-1635-45d6-8a64-d6eaf0b4ae92"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 17:38:14.783595 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:38:14.783437    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/411db317-1635-45d6-8a64-d6eaf0b4ae92-istio-data" (OuterVolumeSpecName: "istio-data") pod "411db317-1635-45d6-8a64-d6eaf0b4ae92" (UID: "411db317-1635-45d6-8a64-d6eaf0b4ae92"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:38:14.783595 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:38:14.783569    2552 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/411db317-1635-45d6-8a64-d6eaf0b4ae92-credential-socket\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:38:14.783595 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:38:14.783591    2552 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/411db317-1635-45d6-8a64-d6eaf0b4ae92-workload-socket\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:38:14.783736 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:38:14.783604    2552 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/411db317-1635-45d6-8a64-d6eaf0b4ae92-istiod-ca-cert\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:38:14.783736 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:38:14.783617    2552 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/411db317-1635-45d6-8a64-d6eaf0b4ae92-istio-data\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:38:14.783736 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:38:14.783629    2552 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/411db317-1635-45d6-8a64-d6eaf0b4ae92-workload-certs\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:38:14.785707 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:38:14.785680    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/411db317-1635-45d6-8a64-d6eaf0b4ae92-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "411db317-1635-45d6-8a64-d6eaf0b4ae92" (UID: "411db317-1635-45d6-8a64-d6eaf0b4ae92"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:38:14.785853 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:38:14.785714    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/411db317-1635-45d6-8a64-d6eaf0b4ae92-istio-token" (OuterVolumeSpecName: "istio-token") pod "411db317-1635-45d6-8a64-d6eaf0b4ae92" (UID: "411db317-1635-45d6-8a64-d6eaf0b4ae92"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:38:14.785853 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:38:14.785829    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/411db317-1635-45d6-8a64-d6eaf0b4ae92-kube-api-access-cjtxs" (OuterVolumeSpecName: "kube-api-access-cjtxs") pod "411db317-1635-45d6-8a64-d6eaf0b4ae92" (UID: "411db317-1635-45d6-8a64-d6eaf0b4ae92"). InnerVolumeSpecName "kube-api-access-cjtxs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:38:14.785853 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:38:14.785829    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/411db317-1635-45d6-8a64-d6eaf0b4ae92-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "411db317-1635-45d6-8a64-d6eaf0b4ae92" (UID: "411db317-1635-45d6-8a64-d6eaf0b4ae92"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 17:38:14.884622 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:38:14.884586    2552 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/411db317-1635-45d6-8a64-d6eaf0b4ae92-istio-envoy\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:38:14.884622 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:38:14.884615    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cjtxs\" (UniqueName: \"kubernetes.io/projected/411db317-1635-45d6-8a64-d6eaf0b4ae92-kube-api-access-cjtxs\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:38:14.884622 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:38:14.884627    2552 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/411db317-1635-45d6-8a64-d6eaf0b4ae92-istio-podinfo\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:38:14.884845 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:38:14.884635    2552 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/411db317-1635-45d6-8a64-d6eaf0b4ae92-istio-token\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:38:15.006547 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:38:15.006507    2552 generic.go:356] "Generic (PLEG): container finished" podID="411db317-1635-45d6-8a64-d6eaf0b4ae92" containerID="0247f032b085c9cd4a189a8ca6d74f84526d0bff3df2f5c940e38b7642b47cae" exitCode=0
Sep 11 17:38:15.006753 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:38:15.006560    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-ec3fed-openshift-default-6fdf496464-f2z6f"
Sep 11 17:38:15.006753 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:38:15.006583    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-ec3fed-openshift-default-6fdf496464-f2z6f" event={"ID":"411db317-1635-45d6-8a64-d6eaf0b4ae92","Type":"ContainerDied","Data":"0247f032b085c9cd4a189a8ca6d74f84526d0bff3df2f5c940e38b7642b47cae"}
Sep 11 17:38:15.006753 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:38:15.006638    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-ec3fed-openshift-default-6fdf496464-f2z6f" event={"ID":"411db317-1635-45d6-8a64-d6eaf0b4ae92","Type":"ContainerDied","Data":"d084bf47cb0b205e87dfa334acaf4339c8875b3e266edcc2dbd59ae7d0b78e71"}
Sep 11 17:38:15.006753 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:38:15.006660    2552 scope.go:117] "RemoveContainer" containerID="0247f032b085c9cd4a189a8ca6d74f84526d0bff3df2f5c940e38b7642b47cae"
Sep 11 17:38:15.018859 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:38:15.018837    2552 scope.go:117] "RemoveContainer" containerID="52864b030ef5db52619d6f6b135b7e150e3a2ce74aff90f6b33bdad6ba403c33"
Sep 11 17:38:15.027969 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:38:15.027950    2552 scope.go:117] "RemoveContainer" containerID="0247f032b085c9cd4a189a8ca6d74f84526d0bff3df2f5c940e38b7642b47cae"
Sep 11 17:38:15.028247 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:38:15.028228    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0247f032b085c9cd4a189a8ca6d74f84526d0bff3df2f5c940e38b7642b47cae\": container with ID starting with 0247f032b085c9cd4a189a8ca6d74f84526d0bff3df2f5c940e38b7642b47cae not found: ID does not exist" containerID="0247f032b085c9cd4a189a8ca6d74f84526d0bff3df2f5c940e38b7642b47cae"
Sep 11 17:38:15.028309 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:38:15.028255    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0247f032b085c9cd4a189a8ca6d74f84526d0bff3df2f5c940e38b7642b47cae"} err="failed to get container status \"0247f032b085c9cd4a189a8ca6d74f84526d0bff3df2f5c940e38b7642b47cae\": rpc error: code = NotFound desc = could not find container \"0247f032b085c9cd4a189a8ca6d74f84526d0bff3df2f5c940e38b7642b47cae\": container with ID starting with 0247f032b085c9cd4a189a8ca6d74f84526d0bff3df2f5c940e38b7642b47cae not found: ID does not exist"
Sep 11 17:38:15.028309 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:38:15.028272    2552 scope.go:117] "RemoveContainer" containerID="52864b030ef5db52619d6f6b135b7e150e3a2ce74aff90f6b33bdad6ba403c33"
Sep 11 17:38:15.028560 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:38:15.028538    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"52864b030ef5db52619d6f6b135b7e150e3a2ce74aff90f6b33bdad6ba403c33\": container with ID starting with 52864b030ef5db52619d6f6b135b7e150e3a2ce74aff90f6b33bdad6ba403c33 not found: ID does not exist" containerID="52864b030ef5db52619d6f6b135b7e150e3a2ce74aff90f6b33bdad6ba403c33"
Sep 11 17:38:15.028625 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:38:15.028571    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"52864b030ef5db52619d6f6b135b7e150e3a2ce74aff90f6b33bdad6ba403c33"} err="failed to get container status \"52864b030ef5db52619d6f6b135b7e150e3a2ce74aff90f6b33bdad6ba403c33\": rpc error: code = NotFound desc = could not find container \"52864b030ef5db52619d6f6b135b7e150e3a2ce74aff90f6b33bdad6ba403c33\": container with ID starting with 52864b030ef5db52619d6f6b135b7e150e3a2ce74aff90f6b33bdad6ba403c33 not found: ID does not exist"
Sep 11 17:38:15.049527 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:38:15.049443    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-explicit-ec3fed-openshift-default-6fdf496464-f2z6f"]
Sep 11 17:38:15.062743 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:38:15.062710    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-explicit-ec3fed-openshift-default-6fdf496464-f2z6f"]
Sep 11 17:38:15.951120 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:38:15.951078    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="411db317-1635-45d6-8a64-d6eaf0b4ae92" path="/var/lib/kubelet/pods/411db317-1635-45d6-8a64-d6eaf0b4ae92/volumes"
Sep 11 17:38:20.394415 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:38:20.394382    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:38:25.333934 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:38:25.333900    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:38:25.742235 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:38:25.742191    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:38:25.997030 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:38:25.996949    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-hg7p8_eb5f3be1-2e36-40ba-b3a0-3bec413056e8/console-operator/1.log"
Sep 11 17:38:26.000710 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:38:26.000685    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6k9tr_0620c0a0-8070-4307-86d9-c69a76ea14a9/ovn-acl-logging/0.log"
Sep 11 17:38:26.003831 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:38:26.003809    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-hg7p8_eb5f3be1-2e36-40ba-b3a0-3bec413056e8/console-operator/1.log"
Sep 11 17:38:26.007409 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:38:26.007389    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6k9tr_0620c0a0-8070-4307-86d9-c69a76ea14a9/ovn-acl-logging/0.log"
Sep 11 17:38:26.805525 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:38:26.805494    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:38:27.871958 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:38:27.871921    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:38:28.756539 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:38:28.756505    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:38:29.765748 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:38:29.765712    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:38:30.774284 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:38:30.774248    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:38:31.645184 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:38:31.645149    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:38:32.432265 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:38:32.432226    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:38:41.225843 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:38:41.225812    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:38:44.415204 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:38:44.415169    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-08c6b2/test-model-08c6b2-kserve-78ff75c84c-fkhts"]
Sep 11 17:38:44.450415 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:38:44.450381    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:38:51.723379 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:38:51.723345    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:38:59.434494 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:38:59.434449    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-08c6b2/test-model-08c6b2-kserve-78ff75c84c-fkhts" podUID="3f65c572-1df7-45a1-bbf1-4e42e9373fa5" containerName="main" containerID="cri-o://bb2087a15121b0bf4a77c3281aac86e380a72cf8544609d3209830c472db4da6" gracePeriod=45
Sep 11 17:38:59.692749 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:38:59.692694    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-08c6b2/test-model-08c6b2-kserve-78ff75c84c-fkhts"
Sep 11 17:38:59.745734 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:38:59.745703    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g6xz8\" (UniqueName: \"kubernetes.io/projected/3f65c572-1df7-45a1-bbf1-4e42e9373fa5-kube-api-access-g6xz8\") pod \"3f65c572-1df7-45a1-bbf1-4e42e9373fa5\" (UID: \"3f65c572-1df7-45a1-bbf1-4e42e9373fa5\") "
Sep 11 17:38:59.745891 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:38:59.745760    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3f65c572-1df7-45a1-bbf1-4e42e9373fa5-tmp-dir\") pod \"3f65c572-1df7-45a1-bbf1-4e42e9373fa5\" (UID: \"3f65c572-1df7-45a1-bbf1-4e42e9373fa5\") "
Sep 11 17:38:59.745891 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:38:59.745792    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3f65c572-1df7-45a1-bbf1-4e42e9373fa5-home\") pod \"3f65c572-1df7-45a1-bbf1-4e42e9373fa5\" (UID: \"3f65c572-1df7-45a1-bbf1-4e42e9373fa5\") "
Sep 11 17:38:59.745891 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:38:59.745823    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3f65c572-1df7-45a1-bbf1-4e42e9373fa5-dshm\") pod \"3f65c572-1df7-45a1-bbf1-4e42e9373fa5\" (UID: \"3f65c572-1df7-45a1-bbf1-4e42e9373fa5\") "
Sep 11 17:38:59.745891 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:38:59.745846    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3f65c572-1df7-45a1-bbf1-4e42e9373fa5-tls-certs\") pod \"3f65c572-1df7-45a1-bbf1-4e42e9373fa5\" (UID: \"3f65c572-1df7-45a1-bbf1-4e42e9373fa5\") "
Sep 11 17:38:59.746137 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:38:59.745905    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3f65c572-1df7-45a1-bbf1-4e42e9373fa5-model-cache\") pod \"3f65c572-1df7-45a1-bbf1-4e42e9373fa5\" (UID: \"3f65c572-1df7-45a1-bbf1-4e42e9373fa5\") "
Sep 11 17:38:59.746227 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:38:59.746193    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f65c572-1df7-45a1-bbf1-4e42e9373fa5-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "3f65c572-1df7-45a1-bbf1-4e42e9373fa5" (UID: "3f65c572-1df7-45a1-bbf1-4e42e9373fa5"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:38:59.746383 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:38:59.746335    2552 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3f65c572-1df7-45a1-bbf1-4e42e9373fa5-tmp-dir\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:38:59.746514 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:38:59.746422    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f65c572-1df7-45a1-bbf1-4e42e9373fa5-model-cache" (OuterVolumeSpecName: "model-cache") pod "3f65c572-1df7-45a1-bbf1-4e42e9373fa5" (UID: "3f65c572-1df7-45a1-bbf1-4e42e9373fa5"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:38:59.746692 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:38:59.746672    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f65c572-1df7-45a1-bbf1-4e42e9373fa5-home" (OuterVolumeSpecName: "home") pod "3f65c572-1df7-45a1-bbf1-4e42e9373fa5" (UID: "3f65c572-1df7-45a1-bbf1-4e42e9373fa5"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:38:59.748242 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:38:59.748215    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3f65c572-1df7-45a1-bbf1-4e42e9373fa5-kube-api-access-g6xz8" (OuterVolumeSpecName: "kube-api-access-g6xz8") pod "3f65c572-1df7-45a1-bbf1-4e42e9373fa5" (UID: "3f65c572-1df7-45a1-bbf1-4e42e9373fa5"). InnerVolumeSpecName "kube-api-access-g6xz8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:38:59.748332 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:38:59.748243    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3f65c572-1df7-45a1-bbf1-4e42e9373fa5-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "3f65c572-1df7-45a1-bbf1-4e42e9373fa5" (UID: "3f65c572-1df7-45a1-bbf1-4e42e9373fa5"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 17:38:59.748332 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:38:59.748291    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f65c572-1df7-45a1-bbf1-4e42e9373fa5-dshm" (OuterVolumeSpecName: "dshm") pod "3f65c572-1df7-45a1-bbf1-4e42e9373fa5" (UID: "3f65c572-1df7-45a1-bbf1-4e42e9373fa5"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:38:59.847477 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:38:59.847446    2552 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3f65c572-1df7-45a1-bbf1-4e42e9373fa5-model-cache\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:38:59.847477 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:38:59.847474    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g6xz8\" (UniqueName: \"kubernetes.io/projected/3f65c572-1df7-45a1-bbf1-4e42e9373fa5-kube-api-access-g6xz8\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:38:59.847477 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:38:59.847483    2552 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3f65c572-1df7-45a1-bbf1-4e42e9373fa5-home\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:38:59.847759 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:38:59.847492    2552 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3f65c572-1df7-45a1-bbf1-4e42e9373fa5-dshm\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:38:59.847759 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:38:59.847500    2552 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3f65c572-1df7-45a1-bbf1-4e42e9373fa5-tls-certs\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:39:00.291040 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:00.290956    2552 generic.go:356] "Generic (PLEG): container finished" podID="3f65c572-1df7-45a1-bbf1-4e42e9373fa5" containerID="bb2087a15121b0bf4a77c3281aac86e380a72cf8544609d3209830c472db4da6" exitCode=0
Sep 11 17:39:00.291040 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:00.291002    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-08c6b2/test-model-08c6b2-kserve-78ff75c84c-fkhts"
Sep 11 17:39:00.291270 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:00.291043    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-08c6b2/test-model-08c6b2-kserve-78ff75c84c-fkhts" event={"ID":"3f65c572-1df7-45a1-bbf1-4e42e9373fa5","Type":"ContainerDied","Data":"bb2087a15121b0bf4a77c3281aac86e380a72cf8544609d3209830c472db4da6"}
Sep 11 17:39:00.291270 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:00.291091    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-08c6b2/test-model-08c6b2-kserve-78ff75c84c-fkhts" event={"ID":"3f65c572-1df7-45a1-bbf1-4e42e9373fa5","Type":"ContainerDied","Data":"1717ca8079bd419e66e76b39c0c8ab31e26592c9e6ae794289711c070f6f3852"}
Sep 11 17:39:00.291270 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:00.291124    2552 scope.go:117] "RemoveContainer" containerID="bb2087a15121b0bf4a77c3281aac86e380a72cf8544609d3209830c472db4da6"
Sep 11 17:39:00.300892 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:00.300875    2552 scope.go:117] "RemoveContainer" containerID="bb2087a15121b0bf4a77c3281aac86e380a72cf8544609d3209830c472db4da6"
Sep 11 17:39:00.301194 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:39:00.301175    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bb2087a15121b0bf4a77c3281aac86e380a72cf8544609d3209830c472db4da6\": container with ID starting with bb2087a15121b0bf4a77c3281aac86e380a72cf8544609d3209830c472db4da6 not found: ID does not exist" containerID="bb2087a15121b0bf4a77c3281aac86e380a72cf8544609d3209830c472db4da6"
Sep 11 17:39:00.301255 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:00.301203    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bb2087a15121b0bf4a77c3281aac86e380a72cf8544609d3209830c472db4da6"} err="failed to get container status \"bb2087a15121b0bf4a77c3281aac86e380a72cf8544609d3209830c472db4da6\": rpc error: code = NotFound desc = could not find container \"bb2087a15121b0bf4a77c3281aac86e380a72cf8544609d3209830c472db4da6\": container with ID starting with bb2087a15121b0bf4a77c3281aac86e380a72cf8544609d3209830c472db4da6 not found: ID does not exist"
Sep 11 17:39:00.307450 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:00.307427    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-08c6b2/test-model-08c6b2-kserve-78ff75c84c-fkhts"]
Sep 11 17:39:00.313147 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:00.313106    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-08c6b2/test-model-08c6b2-kserve-78ff75c84c-fkhts"]
Sep 11 17:39:01.950897 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:01.950859    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3f65c572-1df7-45a1-bbf1-4e42e9373fa5" path="/var/lib/kubelet/pods/3f65c572-1df7-45a1-bbf1-4e42e9373fa5/volumes"
Sep 11 17:39:06.102501 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:06.102443    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-6709f1a7/e2e-lifecycle-model-6709f1a7-kserve-7d6d8b6b5f-zmmwx"]
Sep 11 17:39:06.103017 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:06.102998    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3f65c572-1df7-45a1-bbf1-4e42e9373fa5" containerName="main"
Sep 11 17:39:06.103064 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:06.103021    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f65c572-1df7-45a1-bbf1-4e42e9373fa5" containerName="main"
Sep 11 17:39:06.103064 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:06.103030    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="411db317-1635-45d6-8a64-d6eaf0b4ae92" containerName="istio-proxy"
Sep 11 17:39:06.103064 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:06.103035    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="411db317-1635-45d6-8a64-d6eaf0b4ae92" containerName="istio-proxy"
Sep 11 17:39:06.103200 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:06.103122    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="411db317-1635-45d6-8a64-d6eaf0b4ae92" containerName="istio-proxy"
Sep 11 17:39:06.103200 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:06.103132    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="3f65c572-1df7-45a1-bbf1-4e42e9373fa5" containerName="main"
Sep 11 17:39:06.103200 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:06.103141    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="411db317-1635-45d6-8a64-d6eaf0b4ae92" containerName="istio-proxy"
Sep 11 17:39:06.107661 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:06.107638    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-6709f1a7/e2e-lifecycle-model-6709f1a7-kserve-7d6d8b6b5f-zmmwx"
Sep 11 17:39:06.109989 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:06.109968    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-6709f1a7\"/\"default-dockercfg-lttvb\""
Sep 11 17:39:06.110706 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:06.110683    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-6709f1a7\"/\"openshift-service-ca.crt\""
Sep 11 17:39:06.110896 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:06.110735    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-6709f1a7\"/\"kube-root-ca.crt\""
Sep 11 17:39:06.110896 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:06.110749    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-6709f1a7\"/\"e2e-lifecycle-model-6709f1a7-kserve-self-signed-certs\""
Sep 11 17:39:06.115522 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:06.115499    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-6709f1a7/e2e-lifecycle-model-6709f1a7-kserve-7d6d8b6b5f-zmmwx"]
Sep 11 17:39:06.191171 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:06.191136    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:06.209248 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:06.209216    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1fa2f90c-ef64-4ef9-990c-5d7880eeec02-tmp-dir\") pod \"e2e-lifecycle-model-6709f1a7-kserve-7d6d8b6b5f-zmmwx\" (UID: \"1fa2f90c-ef64-4ef9-990c-5d7880eeec02\") " pod="ai-tenant-e2e-mt-6709f1a7/e2e-lifecycle-model-6709f1a7-kserve-7d6d8b6b5f-zmmwx"
Sep 11 17:39:06.209390 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:06.209287    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1fa2f90c-ef64-4ef9-990c-5d7880eeec02-model-cache\") pod \"e2e-lifecycle-model-6709f1a7-kserve-7d6d8b6b5f-zmmwx\" (UID: \"1fa2f90c-ef64-4ef9-990c-5d7880eeec02\") " pod="ai-tenant-e2e-mt-6709f1a7/e2e-lifecycle-model-6709f1a7-kserve-7d6d8b6b5f-zmmwx"
Sep 11 17:39:06.209390 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:06.209323    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1fa2f90c-ef64-4ef9-990c-5d7880eeec02-dshm\") pod \"e2e-lifecycle-model-6709f1a7-kserve-7d6d8b6b5f-zmmwx\" (UID: \"1fa2f90c-ef64-4ef9-990c-5d7880eeec02\") " pod="ai-tenant-e2e-mt-6709f1a7/e2e-lifecycle-model-6709f1a7-kserve-7d6d8b6b5f-zmmwx"
Sep 11 17:39:06.209479 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:06.209416    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1fa2f90c-ef64-4ef9-990c-5d7880eeec02-home\") pod \"e2e-lifecycle-model-6709f1a7-kserve-7d6d8b6b5f-zmmwx\" (UID: \"1fa2f90c-ef64-4ef9-990c-5d7880eeec02\") " pod="ai-tenant-e2e-mt-6709f1a7/e2e-lifecycle-model-6709f1a7-kserve-7d6d8b6b5f-zmmwx"
Sep 11 17:39:06.209479 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:06.209454    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1fa2f90c-ef64-4ef9-990c-5d7880eeec02-tls-certs\") pod \"e2e-lifecycle-model-6709f1a7-kserve-7d6d8b6b5f-zmmwx\" (UID: \"1fa2f90c-ef64-4ef9-990c-5d7880eeec02\") " pod="ai-tenant-e2e-mt-6709f1a7/e2e-lifecycle-model-6709f1a7-kserve-7d6d8b6b5f-zmmwx"
Sep 11 17:39:06.209556 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:06.209493    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z7qdv\" (UniqueName: \"kubernetes.io/projected/1fa2f90c-ef64-4ef9-990c-5d7880eeec02-kube-api-access-z7qdv\") pod \"e2e-lifecycle-model-6709f1a7-kserve-7d6d8b6b5f-zmmwx\" (UID: \"1fa2f90c-ef64-4ef9-990c-5d7880eeec02\") " pod="ai-tenant-e2e-mt-6709f1a7/e2e-lifecycle-model-6709f1a7-kserve-7d6d8b6b5f-zmmwx"
Sep 11 17:39:06.310420 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:06.310388    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1fa2f90c-ef64-4ef9-990c-5d7880eeec02-tmp-dir\") pod \"e2e-lifecycle-model-6709f1a7-kserve-7d6d8b6b5f-zmmwx\" (UID: \"1fa2f90c-ef64-4ef9-990c-5d7880eeec02\") " pod="ai-tenant-e2e-mt-6709f1a7/e2e-lifecycle-model-6709f1a7-kserve-7d6d8b6b5f-zmmwx"
Sep 11 17:39:06.310606 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:06.310442    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1fa2f90c-ef64-4ef9-990c-5d7880eeec02-model-cache\") pod \"e2e-lifecycle-model-6709f1a7-kserve-7d6d8b6b5f-zmmwx\" (UID: \"1fa2f90c-ef64-4ef9-990c-5d7880eeec02\") " pod="ai-tenant-e2e-mt-6709f1a7/e2e-lifecycle-model-6709f1a7-kserve-7d6d8b6b5f-zmmwx"
Sep 11 17:39:06.310606 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:06.310476    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1fa2f90c-ef64-4ef9-990c-5d7880eeec02-dshm\") pod \"e2e-lifecycle-model-6709f1a7-kserve-7d6d8b6b5f-zmmwx\" (UID: \"1fa2f90c-ef64-4ef9-990c-5d7880eeec02\") " pod="ai-tenant-e2e-mt-6709f1a7/e2e-lifecycle-model-6709f1a7-kserve-7d6d8b6b5f-zmmwx"
Sep 11 17:39:06.310606 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:06.310516    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1fa2f90c-ef64-4ef9-990c-5d7880eeec02-home\") pod \"e2e-lifecycle-model-6709f1a7-kserve-7d6d8b6b5f-zmmwx\" (UID: \"1fa2f90c-ef64-4ef9-990c-5d7880eeec02\") " pod="ai-tenant-e2e-mt-6709f1a7/e2e-lifecycle-model-6709f1a7-kserve-7d6d8b6b5f-zmmwx"
Sep 11 17:39:06.310606 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:06.310540    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1fa2f90c-ef64-4ef9-990c-5d7880eeec02-tls-certs\") pod \"e2e-lifecycle-model-6709f1a7-kserve-7d6d8b6b5f-zmmwx\" (UID: \"1fa2f90c-ef64-4ef9-990c-5d7880eeec02\") " pod="ai-tenant-e2e-mt-6709f1a7/e2e-lifecycle-model-6709f1a7-kserve-7d6d8b6b5f-zmmwx"
Sep 11 17:39:06.310606 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:06.310579    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z7qdv\" (UniqueName: \"kubernetes.io/projected/1fa2f90c-ef64-4ef9-990c-5d7880eeec02-kube-api-access-z7qdv\") pod \"e2e-lifecycle-model-6709f1a7-kserve-7d6d8b6b5f-zmmwx\" (UID: \"1fa2f90c-ef64-4ef9-990c-5d7880eeec02\") " pod="ai-tenant-e2e-mt-6709f1a7/e2e-lifecycle-model-6709f1a7-kserve-7d6d8b6b5f-zmmwx"
Sep 11 17:39:06.310885 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:06.310838    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1fa2f90c-ef64-4ef9-990c-5d7880eeec02-tmp-dir\") pod \"e2e-lifecycle-model-6709f1a7-kserve-7d6d8b6b5f-zmmwx\" (UID: \"1fa2f90c-ef64-4ef9-990c-5d7880eeec02\") " pod="ai-tenant-e2e-mt-6709f1a7/e2e-lifecycle-model-6709f1a7-kserve-7d6d8b6b5f-zmmwx"
Sep 11 17:39:06.310928 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:06.310880    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1fa2f90c-ef64-4ef9-990c-5d7880eeec02-model-cache\") pod \"e2e-lifecycle-model-6709f1a7-kserve-7d6d8b6b5f-zmmwx\" (UID: \"1fa2f90c-ef64-4ef9-990c-5d7880eeec02\") " pod="ai-tenant-e2e-mt-6709f1a7/e2e-lifecycle-model-6709f1a7-kserve-7d6d8b6b5f-zmmwx"
Sep 11 17:39:06.310928 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:06.310899    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1fa2f90c-ef64-4ef9-990c-5d7880eeec02-home\") pod \"e2e-lifecycle-model-6709f1a7-kserve-7d6d8b6b5f-zmmwx\" (UID: \"1fa2f90c-ef64-4ef9-990c-5d7880eeec02\") " pod="ai-tenant-e2e-mt-6709f1a7/e2e-lifecycle-model-6709f1a7-kserve-7d6d8b6b5f-zmmwx"
Sep 11 17:39:06.312835 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:06.312809    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1fa2f90c-ef64-4ef9-990c-5d7880eeec02-dshm\") pod \"e2e-lifecycle-model-6709f1a7-kserve-7d6d8b6b5f-zmmwx\" (UID: \"1fa2f90c-ef64-4ef9-990c-5d7880eeec02\") " pod="ai-tenant-e2e-mt-6709f1a7/e2e-lifecycle-model-6709f1a7-kserve-7d6d8b6b5f-zmmwx"
Sep 11 17:39:06.313073 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:06.313059    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1fa2f90c-ef64-4ef9-990c-5d7880eeec02-tls-certs\") pod \"e2e-lifecycle-model-6709f1a7-kserve-7d6d8b6b5f-zmmwx\" (UID: \"1fa2f90c-ef64-4ef9-990c-5d7880eeec02\") " pod="ai-tenant-e2e-mt-6709f1a7/e2e-lifecycle-model-6709f1a7-kserve-7d6d8b6b5f-zmmwx"
Sep 11 17:39:06.318303 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:06.318283    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z7qdv\" (UniqueName: \"kubernetes.io/projected/1fa2f90c-ef64-4ef9-990c-5d7880eeec02-kube-api-access-z7qdv\") pod \"e2e-lifecycle-model-6709f1a7-kserve-7d6d8b6b5f-zmmwx\" (UID: \"1fa2f90c-ef64-4ef9-990c-5d7880eeec02\") " pod="ai-tenant-e2e-mt-6709f1a7/e2e-lifecycle-model-6709f1a7-kserve-7d6d8b6b5f-zmmwx"
Sep 11 17:39:06.419613 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:06.419517    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-6709f1a7/e2e-lifecycle-model-6709f1a7-kserve-7d6d8b6b5f-zmmwx"
Sep 11 17:39:06.700088 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:06.700061    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-6709f1a7/e2e-lifecycle-model-6709f1a7-kserve-7d6d8b6b5f-zmmwx"]
Sep 11 17:39:07.344882 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:07.344850    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-6709f1a7/e2e-lifecycle-model-6709f1a7-kserve-7d6d8b6b5f-zmmwx" event={"ID":"1fa2f90c-ef64-4ef9-990c-5d7880eeec02","Type":"ContainerStarted","Data":"1108ad80d922e3214719ed58ad3488a133e7a7eb019df592bf3ade0391589585"}
Sep 11 17:39:07.344882 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:07.344893    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-6709f1a7/e2e-lifecycle-model-6709f1a7-kserve-7d6d8b6b5f-zmmwx" event={"ID":"1fa2f90c-ef64-4ef9-990c-5d7880eeec02","Type":"ContainerStarted","Data":"4a1dc2a25c37c915b2fbd150990a313de0f648b5c8081a5ee35e5476b863b65a"}
Sep 11 17:39:07.361572 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:07.361517    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-6709f1a7/e2e-lifecycle-model-6709f1a7-kserve-7d6d8b6b5f-zmmwx" podStartSLOduration=1.361501117 podStartE2EDuration="1.361501117s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:39:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:39:07.360323266 +0000 UTC m=+1241.881614797" watchObservedRunningTime="2026-09-11 17:39:07.361501117 +0000 UTC m=+1241.882792637"
Sep 11 17:39:07.854780 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:07.854746    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:08.679626 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:08.679593    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:09.108552 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:09.108520    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:09.438499 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:09.438399    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:10.233340 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:10.233307    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:11.151882 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:11.151846    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:12.065503 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:12.065467    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:12.820898 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:12.820870    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:13.703804 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:13.703765    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:14.397914 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:14.397868    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:15.699289 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:15.699247    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-144bfa/e2e-conflict-b-a3b46de5-kserve-7654cbf8fc-hfwvc"]
Sep 11 17:39:15.699956 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:15.699933    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="411db317-1635-45d6-8a64-d6eaf0b4ae92" containerName="istio-proxy"
Sep 11 17:39:15.700050 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:15.699959    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="411db317-1635-45d6-8a64-d6eaf0b4ae92" containerName="istio-proxy"
Sep 11 17:39:15.704209 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:15.704187    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-144bfa/e2e-conflict-b-a3b46de5-kserve-7654cbf8fc-hfwvc"
Sep 11 17:39:15.706628 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:15.706603    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-144bfa\"/\"e2e-conflict-b-a3b46de5-kserve-self-signed-certs\""
Sep 11 17:39:15.709889 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:15.709867    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-144bfa/e2e-conflict-b-a3b46de5-kserve-7654cbf8fc-hfwvc"]
Sep 11 17:39:15.728314 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:15.728280    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:15.811266 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:15.811231    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e36986d4-829f-4706-b7ce-a7df8bfee750-home\") pod \"e2e-conflict-b-a3b46de5-kserve-7654cbf8fc-hfwvc\" (UID: \"e36986d4-829f-4706-b7ce-a7df8bfee750\") " pod="e2e-models-e2e-worker-w4-144bfa/e2e-conflict-b-a3b46de5-kserve-7654cbf8fc-hfwvc"
Sep 11 17:39:15.811495 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:15.811274    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e36986d4-829f-4706-b7ce-a7df8bfee750-tmp-dir\") pod \"e2e-conflict-b-a3b46de5-kserve-7654cbf8fc-hfwvc\" (UID: \"e36986d4-829f-4706-b7ce-a7df8bfee750\") " pod="e2e-models-e2e-worker-w4-144bfa/e2e-conflict-b-a3b46de5-kserve-7654cbf8fc-hfwvc"
Sep 11 17:39:15.811495 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:15.811302    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e36986d4-829f-4706-b7ce-a7df8bfee750-model-cache\") pod \"e2e-conflict-b-a3b46de5-kserve-7654cbf8fc-hfwvc\" (UID: \"e36986d4-829f-4706-b7ce-a7df8bfee750\") " pod="e2e-models-e2e-worker-w4-144bfa/e2e-conflict-b-a3b46de5-kserve-7654cbf8fc-hfwvc"
Sep 11 17:39:15.811495 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:15.811384    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e36986d4-829f-4706-b7ce-a7df8bfee750-tls-certs\") pod \"e2e-conflict-b-a3b46de5-kserve-7654cbf8fc-hfwvc\" (UID: \"e36986d4-829f-4706-b7ce-a7df8bfee750\") " pod="e2e-models-e2e-worker-w4-144bfa/e2e-conflict-b-a3b46de5-kserve-7654cbf8fc-hfwvc"
Sep 11 17:39:15.811495 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:15.811431    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e36986d4-829f-4706-b7ce-a7df8bfee750-dshm\") pod \"e2e-conflict-b-a3b46de5-kserve-7654cbf8fc-hfwvc\" (UID: \"e36986d4-829f-4706-b7ce-a7df8bfee750\") " pod="e2e-models-e2e-worker-w4-144bfa/e2e-conflict-b-a3b46de5-kserve-7654cbf8fc-hfwvc"
Sep 11 17:39:15.811683 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:15.811575    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lgf5s\" (UniqueName: \"kubernetes.io/projected/e36986d4-829f-4706-b7ce-a7df8bfee750-kube-api-access-lgf5s\") pod \"e2e-conflict-b-a3b46de5-kserve-7654cbf8fc-hfwvc\" (UID: \"e36986d4-829f-4706-b7ce-a7df8bfee750\") " pod="e2e-models-e2e-worker-w4-144bfa/e2e-conflict-b-a3b46de5-kserve-7654cbf8fc-hfwvc"
Sep 11 17:39:15.912319 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:15.912284    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lgf5s\" (UniqueName: \"kubernetes.io/projected/e36986d4-829f-4706-b7ce-a7df8bfee750-kube-api-access-lgf5s\") pod \"e2e-conflict-b-a3b46de5-kserve-7654cbf8fc-hfwvc\" (UID: \"e36986d4-829f-4706-b7ce-a7df8bfee750\") " pod="e2e-models-e2e-worker-w4-144bfa/e2e-conflict-b-a3b46de5-kserve-7654cbf8fc-hfwvc"
Sep 11 17:39:15.912512 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:15.912339    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e36986d4-829f-4706-b7ce-a7df8bfee750-home\") pod \"e2e-conflict-b-a3b46de5-kserve-7654cbf8fc-hfwvc\" (UID: \"e36986d4-829f-4706-b7ce-a7df8bfee750\") " pod="e2e-models-e2e-worker-w4-144bfa/e2e-conflict-b-a3b46de5-kserve-7654cbf8fc-hfwvc"
Sep 11 17:39:15.912512 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:15.912367    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e36986d4-829f-4706-b7ce-a7df8bfee750-tmp-dir\") pod \"e2e-conflict-b-a3b46de5-kserve-7654cbf8fc-hfwvc\" (UID: \"e36986d4-829f-4706-b7ce-a7df8bfee750\") " pod="e2e-models-e2e-worker-w4-144bfa/e2e-conflict-b-a3b46de5-kserve-7654cbf8fc-hfwvc"
Sep 11 17:39:15.912512 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:15.912389    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e36986d4-829f-4706-b7ce-a7df8bfee750-model-cache\") pod \"e2e-conflict-b-a3b46de5-kserve-7654cbf8fc-hfwvc\" (UID: \"e36986d4-829f-4706-b7ce-a7df8bfee750\") " pod="e2e-models-e2e-worker-w4-144bfa/e2e-conflict-b-a3b46de5-kserve-7654cbf8fc-hfwvc"
Sep 11 17:39:15.912512 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:15.912413    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e36986d4-829f-4706-b7ce-a7df8bfee750-tls-certs\") pod \"e2e-conflict-b-a3b46de5-kserve-7654cbf8fc-hfwvc\" (UID: \"e36986d4-829f-4706-b7ce-a7df8bfee750\") " pod="e2e-models-e2e-worker-w4-144bfa/e2e-conflict-b-a3b46de5-kserve-7654cbf8fc-hfwvc"
Sep 11 17:39:15.912512 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:15.912436    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e36986d4-829f-4706-b7ce-a7df8bfee750-dshm\") pod \"e2e-conflict-b-a3b46de5-kserve-7654cbf8fc-hfwvc\" (UID: \"e36986d4-829f-4706-b7ce-a7df8bfee750\") " pod="e2e-models-e2e-worker-w4-144bfa/e2e-conflict-b-a3b46de5-kserve-7654cbf8fc-hfwvc"
Sep 11 17:39:15.912857 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:15.912837    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e36986d4-829f-4706-b7ce-a7df8bfee750-home\") pod \"e2e-conflict-b-a3b46de5-kserve-7654cbf8fc-hfwvc\" (UID: \"e36986d4-829f-4706-b7ce-a7df8bfee750\") " pod="e2e-models-e2e-worker-w4-144bfa/e2e-conflict-b-a3b46de5-kserve-7654cbf8fc-hfwvc"
Sep 11 17:39:15.912898 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:15.912838    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e36986d4-829f-4706-b7ce-a7df8bfee750-tmp-dir\") pod \"e2e-conflict-b-a3b46de5-kserve-7654cbf8fc-hfwvc\" (UID: \"e36986d4-829f-4706-b7ce-a7df8bfee750\") " pod="e2e-models-e2e-worker-w4-144bfa/e2e-conflict-b-a3b46de5-kserve-7654cbf8fc-hfwvc"
Sep 11 17:39:15.912938 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:15.912902    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e36986d4-829f-4706-b7ce-a7df8bfee750-model-cache\") pod \"e2e-conflict-b-a3b46de5-kserve-7654cbf8fc-hfwvc\" (UID: \"e36986d4-829f-4706-b7ce-a7df8bfee750\") " pod="e2e-models-e2e-worker-w4-144bfa/e2e-conflict-b-a3b46de5-kserve-7654cbf8fc-hfwvc"
Sep 11 17:39:15.914782 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:15.914761    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e36986d4-829f-4706-b7ce-a7df8bfee750-dshm\") pod \"e2e-conflict-b-a3b46de5-kserve-7654cbf8fc-hfwvc\" (UID: \"e36986d4-829f-4706-b7ce-a7df8bfee750\") " pod="e2e-models-e2e-worker-w4-144bfa/e2e-conflict-b-a3b46de5-kserve-7654cbf8fc-hfwvc"
Sep 11 17:39:15.914969 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:15.914951    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e36986d4-829f-4706-b7ce-a7df8bfee750-tls-certs\") pod \"e2e-conflict-b-a3b46de5-kserve-7654cbf8fc-hfwvc\" (UID: \"e36986d4-829f-4706-b7ce-a7df8bfee750\") " pod="e2e-models-e2e-worker-w4-144bfa/e2e-conflict-b-a3b46de5-kserve-7654cbf8fc-hfwvc"
Sep 11 17:39:15.920461 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:15.920437    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lgf5s\" (UniqueName: \"kubernetes.io/projected/e36986d4-829f-4706-b7ce-a7df8bfee750-kube-api-access-lgf5s\") pod \"e2e-conflict-b-a3b46de5-kserve-7654cbf8fc-hfwvc\" (UID: \"e36986d4-829f-4706-b7ce-a7df8bfee750\") " pod="e2e-models-e2e-worker-w4-144bfa/e2e-conflict-b-a3b46de5-kserve-7654cbf8fc-hfwvc"
Sep 11 17:39:16.016949 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:16.016861    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-144bfa/e2e-conflict-b-a3b46de5-kserve-7654cbf8fc-hfwvc"
Sep 11 17:39:16.310967 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:16.310946    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-144bfa/e2e-conflict-b-a3b46de5-kserve-7654cbf8fc-hfwvc"]
Sep 11 17:39:16.312848 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:39:16.312820    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode36986d4_829f_4706_b7ce_a7df8bfee750.slice/crio-713f80a6006f49f654a2a62154afd4f249c4c766175c40f70b89211b0b1423e7 WatchSource:0}: Error finding container 713f80a6006f49f654a2a62154afd4f249c4c766175c40f70b89211b0b1423e7: Status 404 returned error can't find the container with id 713f80a6006f49f654a2a62154afd4f249c4c766175c40f70b89211b0b1423e7
Sep 11 17:39:16.404137 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:16.404074    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-144bfa/e2e-conflict-b-a3b46de5-kserve-7654cbf8fc-hfwvc" event={"ID":"e36986d4-829f-4706-b7ce-a7df8bfee750","Type":"ContainerStarted","Data":"54b406e1c11c9f7c51545210b08cdae7448b4e868ef90ae0a25e8dc513252fd4"}
Sep 11 17:39:16.404311 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:16.404148    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-144bfa/e2e-conflict-b-a3b46de5-kserve-7654cbf8fc-hfwvc" event={"ID":"e36986d4-829f-4706-b7ce-a7df8bfee750","Type":"ContainerStarted","Data":"713f80a6006f49f654a2a62154afd4f249c4c766175c40f70b89211b0b1423e7"}
Sep 11 17:39:16.420138 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:16.420095    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-6709f1a7/e2e-lifecycle-model-6709f1a7-kserve-7d6d8b6b5f-zmmwx"
Sep 11 17:39:16.420297 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:16.420170    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-6709f1a7/e2e-lifecycle-model-6709f1a7-kserve-7d6d8b6b5f-zmmwx"
Sep 11 17:39:16.430395 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:16.430340    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-144bfa/e2e-conflict-b-a3b46de5-kserve-7654cbf8fc-hfwvc" podStartSLOduration=1.430322015 podStartE2EDuration="1.430322015s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:39:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:39:16.426191012 +0000 UTC m=+1250.947482532" watchObservedRunningTime="2026-09-11 17:39:16.430322015 +0000 UTC m=+1250.951613538"
Sep 11 17:39:16.433677 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:16.433655    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-6709f1a7/e2e-lifecycle-model-6709f1a7-kserve-7d6d8b6b5f-zmmwx"
Sep 11 17:39:17.423076 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:17.423038    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-6709f1a7/e2e-lifecycle-model-6709f1a7-kserve-7d6d8b6b5f-zmmwx"
Sep 11 17:39:18.388822 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:18.388752    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:18.487387 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:18.487351    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:18.586445 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:18.586410    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:18.689589 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:18.689506    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:18.964336 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:18.964232    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:19.075094 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:19.075059    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:19.273797 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:19.273708    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:19.384002 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:19.383967    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:19.591371 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:19.591338    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:19.698635 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:19.698599    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:20.065298 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:20.065214    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:20.963400 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:20.963367    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:21.943421 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:21.943379    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:22.779066 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:22.779027    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:23.490560 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:23.490526    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:24.362514 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:24.362484    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:24.770731 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:24.770632    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:25.111542 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:25.111499    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:25.217661 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:25.217629    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:25.339982 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:25.339949    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-08c6b2-openshift-default-5d4477884-jms7z"]
Sep 11 17:39:25.340304 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:25.340274    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-08c6b2-openshift-default-5d4477884-jms7z" podUID="d886651d-540e-47c7-8df0-7234dd3adca1" containerName="istio-proxy" containerID="cri-o://afc6a587b4f264083ff0f9fba1c6d4f2efc5b6ebd79138260fbeed94de49233e" gracePeriod=30
Sep 11 17:39:25.364201 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:25.364091    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:25.898521 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:25.898493    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:26.017909 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:26.017864    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-144bfa/e2e-conflict-b-a3b46de5-kserve-7654cbf8fc-hfwvc"
Sep 11 17:39:26.018103 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:26.017920    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-144bfa/e2e-conflict-b-a3b46de5-kserve-7654cbf8fc-hfwvc"
Sep 11 17:39:26.030453 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:26.030425    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-144bfa/e2e-conflict-b-a3b46de5-kserve-7654cbf8fc-hfwvc"
Sep 11 17:39:26.488963 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:26.488883    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-144bfa/e2e-conflict-b-a3b46de5-kserve-7654cbf8fc-hfwvc"
Sep 11 17:39:27.284236 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:27.284195    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:28.132886 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:28.132851    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:28.958776 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:28.958739    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:29.853581 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:29.853533    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:30.503013 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:30.502979    2552 generic.go:356] "Generic (PLEG): container finished" podID="d886651d-540e-47c7-8df0-7234dd3adca1" containerID="afc6a587b4f264083ff0f9fba1c6d4f2efc5b6ebd79138260fbeed94de49233e" exitCode=0
Sep 11 17:39:30.503234 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:30.503049    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-08c6b2-openshift-default-5d4477884-jms7z" event={"ID":"d886651d-540e-47c7-8df0-7234dd3adca1","Type":"ContainerDied","Data":"afc6a587b4f264083ff0f9fba1c6d4f2efc5b6ebd79138260fbeed94de49233e"}
Sep 11 17:39:30.503234 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:30.503131    2552 scope.go:117] "RemoveContainer" containerID="7c5c9b36711108494c07db005ca918e97b0b49aeb777d1ff5ccce3bbcd24ec54"
Sep 11 17:39:30.602971 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:30.602912    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-08c6b2-openshift-default-5d4477884-jms7z"
Sep 11 17:39:30.638204 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:30.638173    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:30.762471 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:30.762431    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d886651d-540e-47c7-8df0-7234dd3adca1-istio-data\") pod \"d886651d-540e-47c7-8df0-7234dd3adca1\" (UID: \"d886651d-540e-47c7-8df0-7234dd3adca1\") "
Sep 11 17:39:30.762650 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:30.762488    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8z2gf\" (UniqueName: \"kubernetes.io/projected/d886651d-540e-47c7-8df0-7234dd3adca1-kube-api-access-8z2gf\") pod \"d886651d-540e-47c7-8df0-7234dd3adca1\" (UID: \"d886651d-540e-47c7-8df0-7234dd3adca1\") "
Sep 11 17:39:30.762650 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:30.762589    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d886651d-540e-47c7-8df0-7234dd3adca1-istiod-ca-cert\") pod \"d886651d-540e-47c7-8df0-7234dd3adca1\" (UID: \"d886651d-540e-47c7-8df0-7234dd3adca1\") "
Sep 11 17:39:30.762650 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:30.762617    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d886651d-540e-47c7-8df0-7234dd3adca1-credential-socket\") pod \"d886651d-540e-47c7-8df0-7234dd3adca1\" (UID: \"d886651d-540e-47c7-8df0-7234dd3adca1\") "
Sep 11 17:39:30.762836 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:30.762649    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d886651d-540e-47c7-8df0-7234dd3adca1-istio-envoy\") pod \"d886651d-540e-47c7-8df0-7234dd3adca1\" (UID: \"d886651d-540e-47c7-8df0-7234dd3adca1\") "
Sep 11 17:39:30.762836 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:30.762705    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d886651d-540e-47c7-8df0-7234dd3adca1-workload-socket\") pod \"d886651d-540e-47c7-8df0-7234dd3adca1\" (UID: \"d886651d-540e-47c7-8df0-7234dd3adca1\") "
Sep 11 17:39:30.762836 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:30.762760    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d886651d-540e-47c7-8df0-7234dd3adca1-workload-certs\") pod \"d886651d-540e-47c7-8df0-7234dd3adca1\" (UID: \"d886651d-540e-47c7-8df0-7234dd3adca1\") "
Sep 11 17:39:30.762836 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:30.762792    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d886651d-540e-47c7-8df0-7234dd3adca1-istio-podinfo\") pod \"d886651d-540e-47c7-8df0-7234dd3adca1\" (UID: \"d886651d-540e-47c7-8df0-7234dd3adca1\") "
Sep 11 17:39:30.762836 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:30.762821    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d886651d-540e-47c7-8df0-7234dd3adca1-istio-token\") pod \"d886651d-540e-47c7-8df0-7234dd3adca1\" (UID: \"d886651d-540e-47c7-8df0-7234dd3adca1\") "
Sep 11 17:39:30.763093 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:30.762753    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d886651d-540e-47c7-8df0-7234dd3adca1-istio-data" (OuterVolumeSpecName: "istio-data") pod "d886651d-540e-47c7-8df0-7234dd3adca1" (UID: "d886651d-540e-47c7-8df0-7234dd3adca1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:39:30.763093 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:30.762927    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d886651d-540e-47c7-8df0-7234dd3adca1-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d886651d-540e-47c7-8df0-7234dd3adca1" (UID: "d886651d-540e-47c7-8df0-7234dd3adca1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:39:30.763093 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:30.762961    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d886651d-540e-47c7-8df0-7234dd3adca1-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d886651d-540e-47c7-8df0-7234dd3adca1" (UID: "d886651d-540e-47c7-8df0-7234dd3adca1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:39:30.763093 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:30.762976    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d886651d-540e-47c7-8df0-7234dd3adca1-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d886651d-540e-47c7-8df0-7234dd3adca1" (UID: "d886651d-540e-47c7-8df0-7234dd3adca1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 17:39:30.763093 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:30.763080    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d886651d-540e-47c7-8df0-7234dd3adca1-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d886651d-540e-47c7-8df0-7234dd3adca1" (UID: "d886651d-540e-47c7-8df0-7234dd3adca1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:39:30.763367 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:30.763260    2552 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d886651d-540e-47c7-8df0-7234dd3adca1-istio-data\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:39:30.763367 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:30.763280    2552 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d886651d-540e-47c7-8df0-7234dd3adca1-istiod-ca-cert\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:39:30.763367 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:30.763295    2552 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d886651d-540e-47c7-8df0-7234dd3adca1-credential-socket\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:39:30.763367 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:30.763311    2552 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d886651d-540e-47c7-8df0-7234dd3adca1-workload-socket\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:39:30.763367 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:30.763324    2552 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d886651d-540e-47c7-8df0-7234dd3adca1-workload-certs\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:39:30.765010 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:30.764982    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d886651d-540e-47c7-8df0-7234dd3adca1-kube-api-access-8z2gf" (OuterVolumeSpecName: "kube-api-access-8z2gf") pod "d886651d-540e-47c7-8df0-7234dd3adca1" (UID: "d886651d-540e-47c7-8df0-7234dd3adca1"). InnerVolumeSpecName "kube-api-access-8z2gf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:39:30.765136 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:30.765014    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d886651d-540e-47c7-8df0-7234dd3adca1-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d886651d-540e-47c7-8df0-7234dd3adca1" (UID: "d886651d-540e-47c7-8df0-7234dd3adca1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:39:30.765136 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:30.765032    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d886651d-540e-47c7-8df0-7234dd3adca1-istio-token" (OuterVolumeSpecName: "istio-token") pod "d886651d-540e-47c7-8df0-7234dd3adca1" (UID: "d886651d-540e-47c7-8df0-7234dd3adca1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:39:30.765281 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:30.765261    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d886651d-540e-47c7-8df0-7234dd3adca1-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d886651d-540e-47c7-8df0-7234dd3adca1" (UID: "d886651d-540e-47c7-8df0-7234dd3adca1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 17:39:30.864195 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:30.864095    2552 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d886651d-540e-47c7-8df0-7234dd3adca1-istio-envoy\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:39:30.864195 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:30.864152    2552 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d886651d-540e-47c7-8df0-7234dd3adca1-istio-podinfo\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:39:30.864195 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:30.864162    2552 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d886651d-540e-47c7-8df0-7234dd3adca1-istio-token\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:39:30.864195 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:30.864170    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8z2gf\" (UniqueName: \"kubernetes.io/projected/d886651d-540e-47c7-8df0-7234dd3adca1-kube-api-access-8z2gf\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:39:31.389483 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:31.389449    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:31.511194 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:31.511166    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-08c6b2-openshift-default-5d4477884-jms7z"
Sep 11 17:39:31.511194 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:31.511177    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-08c6b2-openshift-default-5d4477884-jms7z" event={"ID":"d886651d-540e-47c7-8df0-7234dd3adca1","Type":"ContainerDied","Data":"99a0685ca3c87f45c9d79f2b9309f2a7bab7f2fb7840b5eff7ea319e6590982e"}
Sep 11 17:39:31.511423 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:31.511233    2552 scope.go:117] "RemoveContainer" containerID="afc6a587b4f264083ff0f9fba1c6d4f2efc5b6ebd79138260fbeed94de49233e"
Sep 11 17:39:31.536708 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:31.536675    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-08c6b2-openshift-default-5d4477884-jms7z"]
Sep 11 17:39:31.540979 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:31.540955    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-08c6b2-openshift-default-5d4477884-jms7z"]
Sep 11 17:39:31.951880 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:31.951848    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d886651d-540e-47c7-8df0-7234dd3adca1" path="/var/lib/kubelet/pods/d886651d-540e-47c7-8df0-7234dd3adca1/volumes"
Sep 11 17:39:32.155872 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:32.155839    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:36.204315 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:36.204271    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:36.275398 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:36.275363    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:36.377839 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:36.377802    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:36.441444 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:36.441411    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:36.791296 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:36.791262    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:36.886288 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:36.886246    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:37.640058 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:37.640030    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:38.365998 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:38.365960    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:38.611665 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:38.611630    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:38.712036 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:38.711936    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:39.105016 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:39.104988    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:40.001625 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:40.001599    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:40.746137 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:40.746086    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:43.135463 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:43.135423    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:43.302700 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:43.302661    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:43.405385 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:43.405292    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:43.583692 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:43.583663    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-cbbb09-openshift-default-7657d596f6-c5cbc"]
Sep 11 17:39:43.583921 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:43.583901    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-cbbb09-openshift-default-7657d596f6-c5cbc" podUID="979c7e0c-c7e6-4e8a-bdbb-b1e1820989e1" containerName="istio-proxy" containerID="cri-o://d4aa3c470f2ec9edac7d30f766e1feb0ae7fa0cea33d49063d0da23b97c28cbf" gracePeriod=30
Sep 11 17:39:43.598451 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:43.598423    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:44.145807 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:44.145767    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:44.480703 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:44.480618    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:44.606159 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:44.606125    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:47.122391 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:47.122352    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:47.851236 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:47.851198    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:47.955419 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:47.955386    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:48.073016 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:48.072975    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:48.840298 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:48.840268    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-cbbb09-openshift-default-7657d596f6-c5cbc"
Sep 11 17:39:48.946724 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:48.946625    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/979c7e0c-c7e6-4e8a-bdbb-b1e1820989e1-istio-token\") pod \"979c7e0c-c7e6-4e8a-bdbb-b1e1820989e1\" (UID: \"979c7e0c-c7e6-4e8a-bdbb-b1e1820989e1\") "
Sep 11 17:39:48.946724 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:48.946691    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/979c7e0c-c7e6-4e8a-bdbb-b1e1820989e1-workload-certs\") pod \"979c7e0c-c7e6-4e8a-bdbb-b1e1820989e1\" (UID: \"979c7e0c-c7e6-4e8a-bdbb-b1e1820989e1\") "
Sep 11 17:39:48.946724 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:48.946713    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/979c7e0c-c7e6-4e8a-bdbb-b1e1820989e1-istio-data\") pod \"979c7e0c-c7e6-4e8a-bdbb-b1e1820989e1\" (UID: \"979c7e0c-c7e6-4e8a-bdbb-b1e1820989e1\") "
Sep 11 17:39:48.947037 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:48.946742    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/979c7e0c-c7e6-4e8a-bdbb-b1e1820989e1-istio-envoy\") pod \"979c7e0c-c7e6-4e8a-bdbb-b1e1820989e1\" (UID: \"979c7e0c-c7e6-4e8a-bdbb-b1e1820989e1\") "
Sep 11 17:39:48.947037 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:48.946783    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/979c7e0c-c7e6-4e8a-bdbb-b1e1820989e1-workload-socket\") pod \"979c7e0c-c7e6-4e8a-bdbb-b1e1820989e1\" (UID: \"979c7e0c-c7e6-4e8a-bdbb-b1e1820989e1\") "
Sep 11 17:39:48.947037 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:48.946854    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/979c7e0c-c7e6-4e8a-bdbb-b1e1820989e1-credential-socket\") pod \"979c7e0c-c7e6-4e8a-bdbb-b1e1820989e1\" (UID: \"979c7e0c-c7e6-4e8a-bdbb-b1e1820989e1\") "
Sep 11 17:39:48.947037 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:48.946895    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/979c7e0c-c7e6-4e8a-bdbb-b1e1820989e1-istio-podinfo\") pod \"979c7e0c-c7e6-4e8a-bdbb-b1e1820989e1\" (UID: \"979c7e0c-c7e6-4e8a-bdbb-b1e1820989e1\") "
Sep 11 17:39:48.947037 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:48.946925    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zrthp\" (UniqueName: \"kubernetes.io/projected/979c7e0c-c7e6-4e8a-bdbb-b1e1820989e1-kube-api-access-zrthp\") pod \"979c7e0c-c7e6-4e8a-bdbb-b1e1820989e1\" (UID: \"979c7e0c-c7e6-4e8a-bdbb-b1e1820989e1\") "
Sep 11 17:39:48.947037 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:48.946970    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/979c7e0c-c7e6-4e8a-bdbb-b1e1820989e1-istiod-ca-cert\") pod \"979c7e0c-c7e6-4e8a-bdbb-b1e1820989e1\" (UID: \"979c7e0c-c7e6-4e8a-bdbb-b1e1820989e1\") "
Sep 11 17:39:48.947361 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:48.947051    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/979c7e0c-c7e6-4e8a-bdbb-b1e1820989e1-istio-data" (OuterVolumeSpecName: "istio-data") pod "979c7e0c-c7e6-4e8a-bdbb-b1e1820989e1" (UID: "979c7e0c-c7e6-4e8a-bdbb-b1e1820989e1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:39:48.947361 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:48.947070    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/979c7e0c-c7e6-4e8a-bdbb-b1e1820989e1-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "979c7e0c-c7e6-4e8a-bdbb-b1e1820989e1" (UID: "979c7e0c-c7e6-4e8a-bdbb-b1e1820989e1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:39:48.947361 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:48.947197    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/979c7e0c-c7e6-4e8a-bdbb-b1e1820989e1-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "979c7e0c-c7e6-4e8a-bdbb-b1e1820989e1" (UID: "979c7e0c-c7e6-4e8a-bdbb-b1e1820989e1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:39:48.948147 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:48.947420    2552 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/979c7e0c-c7e6-4e8a-bdbb-b1e1820989e1-workload-certs\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:39:48.948147 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:48.947435    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/979c7e0c-c7e6-4e8a-bdbb-b1e1820989e1-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "979c7e0c-c7e6-4e8a-bdbb-b1e1820989e1" (UID: "979c7e0c-c7e6-4e8a-bdbb-b1e1820989e1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:39:48.948147 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:48.947439    2552 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/979c7e0c-c7e6-4e8a-bdbb-b1e1820989e1-istio-data\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:39:48.948147 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:48.947471    2552 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/979c7e0c-c7e6-4e8a-bdbb-b1e1820989e1-workload-socket\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:39:48.948147 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:48.947472    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/979c7e0c-c7e6-4e8a-bdbb-b1e1820989e1-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "979c7e0c-c7e6-4e8a-bdbb-b1e1820989e1" (UID: "979c7e0c-c7e6-4e8a-bdbb-b1e1820989e1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 17:39:48.949415 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:48.949390    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/979c7e0c-c7e6-4e8a-bdbb-b1e1820989e1-istio-token" (OuterVolumeSpecName: "istio-token") pod "979c7e0c-c7e6-4e8a-bdbb-b1e1820989e1" (UID: "979c7e0c-c7e6-4e8a-bdbb-b1e1820989e1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:39:48.949514 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:48.949454    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/979c7e0c-c7e6-4e8a-bdbb-b1e1820989e1-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "979c7e0c-c7e6-4e8a-bdbb-b1e1820989e1" (UID: "979c7e0c-c7e6-4e8a-bdbb-b1e1820989e1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:39:48.949672 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:48.949638    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/979c7e0c-c7e6-4e8a-bdbb-b1e1820989e1-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "979c7e0c-c7e6-4e8a-bdbb-b1e1820989e1" (UID: "979c7e0c-c7e6-4e8a-bdbb-b1e1820989e1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 17:39:48.949672 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:48.949657    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/979c7e0c-c7e6-4e8a-bdbb-b1e1820989e1-kube-api-access-zrthp" (OuterVolumeSpecName: "kube-api-access-zrthp") pod "979c7e0c-c7e6-4e8a-bdbb-b1e1820989e1" (UID: "979c7e0c-c7e6-4e8a-bdbb-b1e1820989e1"). InnerVolumeSpecName "kube-api-access-zrthp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:39:49.048363 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:49.048298    2552 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/979c7e0c-c7e6-4e8a-bdbb-b1e1820989e1-istio-envoy\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:39:49.048363 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:49.048332    2552 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/979c7e0c-c7e6-4e8a-bdbb-b1e1820989e1-credential-socket\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:39:49.048363 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:49.048346    2552 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/979c7e0c-c7e6-4e8a-bdbb-b1e1820989e1-istio-podinfo\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:39:49.048363 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:49.048360    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zrthp\" (UniqueName: \"kubernetes.io/projected/979c7e0c-c7e6-4e8a-bdbb-b1e1820989e1-kube-api-access-zrthp\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:39:49.048363 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:49.048373    2552 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/979c7e0c-c7e6-4e8a-bdbb-b1e1820989e1-istiod-ca-cert\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:39:49.048699 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:49.048385    2552 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/979c7e0c-c7e6-4e8a-bdbb-b1e1820989e1-istio-token\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:39:49.082844 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:49.082804    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:49.630364 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:49.630323    2552 generic.go:356] "Generic (PLEG): container finished" podID="979c7e0c-c7e6-4e8a-bdbb-b1e1820989e1" containerID="d4aa3c470f2ec9edac7d30f766e1feb0ae7fa0cea33d49063d0da23b97c28cbf" exitCode=0
Sep 11 17:39:49.630546 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:49.630389    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-cbbb09-openshift-default-7657d596f6-c5cbc" event={"ID":"979c7e0c-c7e6-4e8a-bdbb-b1e1820989e1","Type":"ContainerDied","Data":"d4aa3c470f2ec9edac7d30f766e1feb0ae7fa0cea33d49063d0da23b97c28cbf"}
Sep 11 17:39:49.630546 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:49.630407    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-cbbb09-openshift-default-7657d596f6-c5cbc"
Sep 11 17:39:49.630546 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:49.630478    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-cbbb09-openshift-default-7657d596f6-c5cbc" event={"ID":"979c7e0c-c7e6-4e8a-bdbb-b1e1820989e1","Type":"ContainerDied","Data":"ba38e1ea14e554b55bd241b0e4cb7590799974db8d345ec0e71d38fcf910bce9"}
Sep 11 17:39:49.630546 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:49.630509    2552 scope.go:117] "RemoveContainer" containerID="d4aa3c470f2ec9edac7d30f766e1feb0ae7fa0cea33d49063d0da23b97c28cbf"
Sep 11 17:39:49.642379 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:49.642354    2552 scope.go:117] "RemoveContainer" containerID="561a56db8923b8fa05fba1f57925e479917969a0619ce3b8a72157f5f69a6942"
Sep 11 17:39:49.651423 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:49.651401    2552 scope.go:117] "RemoveContainer" containerID="d4aa3c470f2ec9edac7d30f766e1feb0ae7fa0cea33d49063d0da23b97c28cbf"
Sep 11 17:39:49.651743 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:39:49.651716    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d4aa3c470f2ec9edac7d30f766e1feb0ae7fa0cea33d49063d0da23b97c28cbf\": container with ID starting with d4aa3c470f2ec9edac7d30f766e1feb0ae7fa0cea33d49063d0da23b97c28cbf not found: ID does not exist" containerID="d4aa3c470f2ec9edac7d30f766e1feb0ae7fa0cea33d49063d0da23b97c28cbf"
Sep 11 17:39:49.651846 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:49.651751    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d4aa3c470f2ec9edac7d30f766e1feb0ae7fa0cea33d49063d0da23b97c28cbf"} err="failed to get container status \"d4aa3c470f2ec9edac7d30f766e1feb0ae7fa0cea33d49063d0da23b97c28cbf\": rpc error: code = NotFound desc = could not find container \"d4aa3c470f2ec9edac7d30f766e1feb0ae7fa0cea33d49063d0da23b97c28cbf\": container with ID starting with d4aa3c470f2ec9edac7d30f766e1feb0ae7fa0cea33d49063d0da23b97c28cbf not found: ID does not exist"
Sep 11 17:39:49.651846 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:49.651777    2552 scope.go:117] "RemoveContainer" containerID="561a56db8923b8fa05fba1f57925e479917969a0619ce3b8a72157f5f69a6942"
Sep 11 17:39:49.652068 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:39:49.652037    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"561a56db8923b8fa05fba1f57925e479917969a0619ce3b8a72157f5f69a6942\": container with ID starting with 561a56db8923b8fa05fba1f57925e479917969a0619ce3b8a72157f5f69a6942 not found: ID does not exist" containerID="561a56db8923b8fa05fba1f57925e479917969a0619ce3b8a72157f5f69a6942"
Sep 11 17:39:49.652145 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:49.652081    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"561a56db8923b8fa05fba1f57925e479917969a0619ce3b8a72157f5f69a6942"} err="failed to get container status \"561a56db8923b8fa05fba1f57925e479917969a0619ce3b8a72157f5f69a6942\": rpc error: code = NotFound desc = could not find container \"561a56db8923b8fa05fba1f57925e479917969a0619ce3b8a72157f5f69a6942\": container with ID starting with 561a56db8923b8fa05fba1f57925e479917969a0619ce3b8a72157f5f69a6942 not found: ID does not exist"
Sep 11 17:39:49.658243 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:49.658219    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-cbbb09-openshift-default-7657d596f6-c5cbc"]
Sep 11 17:39:49.662029 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:49.662006    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-cbbb09-openshift-default-7657d596f6-c5cbc"]
Sep 11 17:39:49.952334 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:49.952254    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="979c7e0c-c7e6-4e8a-bdbb-b1e1820989e1" path="/var/lib/kubelet/pods/979c7e0c-c7e6-4e8a-bdbb-b1e1820989e1/volumes"
Sep 11 17:39:50.012164 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:50.012131    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:50.757766 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:50.757735    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:51.646255 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:51.646222    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:52.440930 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:52.440893    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-2fac54-7965f98cb9-fqn9k"]
Sep 11 17:39:52.441886 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:52.441858    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d886651d-540e-47c7-8df0-7234dd3adca1" containerName="istio-proxy"
Sep 11 17:39:52.441886 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:52.441881    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="d886651d-540e-47c7-8df0-7234dd3adca1" containerName="istio-proxy"
Sep 11 17:39:52.441886 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:52.441892    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d886651d-540e-47c7-8df0-7234dd3adca1" containerName="istio-proxy"
Sep 11 17:39:52.442166 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:52.441897    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="d886651d-540e-47c7-8df0-7234dd3adca1" containerName="istio-proxy"
Sep 11 17:39:52.442166 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:52.441920    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="979c7e0c-c7e6-4e8a-bdbb-b1e1820989e1" containerName="istio-proxy"
Sep 11 17:39:52.442166 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:52.441931    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="979c7e0c-c7e6-4e8a-bdbb-b1e1820989e1" containerName="istio-proxy"
Sep 11 17:39:52.442166 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:52.441943    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="979c7e0c-c7e6-4e8a-bdbb-b1e1820989e1" containerName="istio-proxy"
Sep 11 17:39:52.442166 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:52.441949    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="979c7e0c-c7e6-4e8a-bdbb-b1e1820989e1" containerName="istio-proxy"
Sep 11 17:39:52.442166 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:52.442018    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="d886651d-540e-47c7-8df0-7234dd3adca1" containerName="istio-proxy"
Sep 11 17:39:52.442166 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:52.442027    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="979c7e0c-c7e6-4e8a-bdbb-b1e1820989e1" containerName="istio-proxy"
Sep 11 17:39:52.442166 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:52.442035    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="979c7e0c-c7e6-4e8a-bdbb-b1e1820989e1" containerName="istio-proxy"
Sep 11 17:39:52.446945 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:52.446924    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-2fac54-7965f98cb9-fqn9k"
Sep 11 17:39:52.449048 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:52.449024    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w3-2fac54-serving-cert\""
Sep 11 17:39:52.460943 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:52.460919    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-2fac54-7965f98cb9-fqn9k"]
Sep 11 17:39:52.470480 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:52.470452    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:52.544813 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:52.544771    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:52.583999 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:52.583961    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l7lt8\" (UniqueName: \"kubernetes.io/projected/2b33b5cc-f45a-48f9-8c3f-d2a32db8243d-kube-api-access-l7lt8\") pod \"maas-api-e2e-shared-a-w3-2fac54-7965f98cb9-fqn9k\" (UID: \"2b33b5cc-f45a-48f9-8c3f-d2a32db8243d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-2fac54-7965f98cb9-fqn9k"
Sep 11 17:39:52.584235 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:52.584031    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2b33b5cc-f45a-48f9-8c3f-d2a32db8243d-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-2fac54-7965f98cb9-fqn9k\" (UID: \"2b33b5cc-f45a-48f9-8c3f-d2a32db8243d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-2fac54-7965f98cb9-fqn9k"
Sep 11 17:39:52.584235 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:52.584153    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2b33b5cc-f45a-48f9-8c3f-d2a32db8243d-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-2fac54-7965f98cb9-fqn9k\" (UID: \"2b33b5cc-f45a-48f9-8c3f-d2a32db8243d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-2fac54-7965f98cb9-fqn9k"
Sep 11 17:39:52.584235 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:52.584217    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2b33b5cc-f45a-48f9-8c3f-d2a32db8243d-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-2fac54-7965f98cb9-fqn9k\" (UID: \"2b33b5cc-f45a-48f9-8c3f-d2a32db8243d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-2fac54-7965f98cb9-fqn9k"
Sep 11 17:39:52.671687 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:52.671653    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:52.685355 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:52.685325    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2b33b5cc-f45a-48f9-8c3f-d2a32db8243d-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-2fac54-7965f98cb9-fqn9k\" (UID: \"2b33b5cc-f45a-48f9-8c3f-d2a32db8243d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-2fac54-7965f98cb9-fqn9k"
Sep 11 17:39:52.685509 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:52.685493    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l7lt8\" (UniqueName: \"kubernetes.io/projected/2b33b5cc-f45a-48f9-8c3f-d2a32db8243d-kube-api-access-l7lt8\") pod \"maas-api-e2e-shared-a-w3-2fac54-7965f98cb9-fqn9k\" (UID: \"2b33b5cc-f45a-48f9-8c3f-d2a32db8243d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-2fac54-7965f98cb9-fqn9k"
Sep 11 17:39:52.685587 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:52.685572    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2b33b5cc-f45a-48f9-8c3f-d2a32db8243d-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-2fac54-7965f98cb9-fqn9k\" (UID: \"2b33b5cc-f45a-48f9-8c3f-d2a32db8243d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-2fac54-7965f98cb9-fqn9k"
Sep 11 17:39:52.685643 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:52.685627    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2b33b5cc-f45a-48f9-8c3f-d2a32db8243d-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-2fac54-7965f98cb9-fqn9k\" (UID: \"2b33b5cc-f45a-48f9-8c3f-d2a32db8243d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-2fac54-7965f98cb9-fqn9k"
Sep 11 17:39:52.686198 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:52.686171    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2b33b5cc-f45a-48f9-8c3f-d2a32db8243d-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-2fac54-7965f98cb9-fqn9k\" (UID: \"2b33b5cc-f45a-48f9-8c3f-d2a32db8243d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-2fac54-7965f98cb9-fqn9k"
Sep 11 17:39:52.688043 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:52.688011    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2b33b5cc-f45a-48f9-8c3f-d2a32db8243d-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-2fac54-7965f98cb9-fqn9k\" (UID: \"2b33b5cc-f45a-48f9-8c3f-d2a32db8243d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-2fac54-7965f98cb9-fqn9k"
Sep 11 17:39:52.688310 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:52.688284    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2b33b5cc-f45a-48f9-8c3f-d2a32db8243d-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-2fac54-7965f98cb9-fqn9k\" (UID: \"2b33b5cc-f45a-48f9-8c3f-d2a32db8243d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-2fac54-7965f98cb9-fqn9k"
Sep 11 17:39:52.697807 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:52.697747    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l7lt8\" (UniqueName: \"kubernetes.io/projected/2b33b5cc-f45a-48f9-8c3f-d2a32db8243d-kube-api-access-l7lt8\") pod \"maas-api-e2e-shared-a-w3-2fac54-7965f98cb9-fqn9k\" (UID: \"2b33b5cc-f45a-48f9-8c3f-d2a32db8243d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-2fac54-7965f98cb9-fqn9k"
Sep 11 17:39:52.758264 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:52.758220    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-2fac54-7965f98cb9-fqn9k"
Sep 11 17:39:53.018326 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:53.018242    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:53.049826 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:53.049798    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-2fac54-7965f98cb9-fqn9k"]
Sep 11 17:39:53.052733 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:39:53.052704    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b33b5cc_f45a_48f9_8c3f_d2a32db8243d.slice/crio-36568502b2f6bc4842b417e51dd166cc9d112a7216a1eda74a59d029a8374d50 WatchSource:0}: Error finding container 36568502b2f6bc4842b417e51dd166cc9d112a7216a1eda74a59d029a8374d50: Status 404 returned error can't find the container with id 36568502b2f6bc4842b417e51dd166cc9d112a7216a1eda74a59d029a8374d50
Sep 11 17:39:53.140714 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:53.140670    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:53.668651 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:53.668624    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-2fac54-7965f98cb9-fqn9k" event={"ID":"2b33b5cc-f45a-48f9-8c3f-d2a32db8243d","Type":"ContainerStarted","Data":"36568502b2f6bc4842b417e51dd166cc9d112a7216a1eda74a59d029a8374d50"}
Sep 11 17:39:54.676842 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:54.676811    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-2fac54-7965f98cb9-fqn9k" event={"ID":"2b33b5cc-f45a-48f9-8c3f-d2a32db8243d","Type":"ContainerStarted","Data":"8a92af631272a636349c2c4a127a4cdb0a78443a77090d3e12b53da52a351185"}
Sep 11 17:39:54.677277 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:54.676876    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-2fac54-7965f98cb9-fqn9k"
Sep 11 17:39:54.695656 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:39:54.695601    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-2fac54-7965f98cb9-fqn9k" podStartSLOduration=2.144462323 podStartE2EDuration="2.695583772s" totalImagesPullingTime="551.121449ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:39:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:39:54.693047733 +0000 UTC m=+1289.214339255" watchObservedRunningTime="2026-09-11 17:39:54.695583772 +0000 UTC m=+1289.216875293"
Sep 11 17:40:00.688721 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:40:00.688687    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-2fac54-7965f98cb9-fqn9k"
Sep 11 17:40:04.883075 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:40:04.883036    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:40:04.942311 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:40:04.942264    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:40:15.011330 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:40:15.011273    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:40:15.959989 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:40:15.959950    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:40:16.778626 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:40:16.778561    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:40:17.530455 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:40:17.530408    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:40:17.975271 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:40:17.975237    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:40:18.290326 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:40:18.290233    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:40:18.426742 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:40:18.426704    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:40:18.620435 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:40:18.620392    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:40:18.874665 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:40:18.874582    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:40:19.398612 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:40:19.398582    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:40:20.571899 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:40:20.571868    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:40:21.580130 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:40:21.580091    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:40:22.069403 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:40:22.069369    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:40:22.166316 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:40:22.166283    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:40:22.449824 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:40:22.449776    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:40:23.358633 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:40:23.358594    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:40:24.124494 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:40:24.124468    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:40:24.909812 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:40:24.909781    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:40:25.638660 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:40:25.638598    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:40:26.339581 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:40:26.339550    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:40:27.312505 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:40:27.312470    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-6fc9e6-68fb4f5b48-9tqw5"]
Sep 11 17:40:27.313452 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:40:27.313429    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="d886651d-540e-47c7-8df0-7234dd3adca1" containerName="istio-proxy"
Sep 11 17:40:27.320364 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:40:27.320338    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-6fc9e6-68fb4f5b48-9tqw5"
Sep 11 17:40:27.324048 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:40:27.323462    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w3-6fc9e6-serving-cert\""
Sep 11 17:40:27.329770 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:40:27.329744    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-6fc9e6-68fb4f5b48-9tqw5"]
Sep 11 17:40:27.345932 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:40:27.345904    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:40:27.440252 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:40:27.440212    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/46fa2c22-db29-429f-812f-422ae705d742-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-6fc9e6-68fb4f5b48-9tqw5\" (UID: \"46fa2c22-db29-429f-812f-422ae705d742\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-6fc9e6-68fb4f5b48-9tqw5"
Sep 11 17:40:27.440460 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:40:27.440315    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g74xj\" (UniqueName: \"kubernetes.io/projected/46fa2c22-db29-429f-812f-422ae705d742-kube-api-access-g74xj\") pod \"maas-api-e2e-shared-b-w3-6fc9e6-68fb4f5b48-9tqw5\" (UID: \"46fa2c22-db29-429f-812f-422ae705d742\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-6fc9e6-68fb4f5b48-9tqw5"
Sep 11 17:40:27.440460 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:40:27.440347    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/46fa2c22-db29-429f-812f-422ae705d742-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-6fc9e6-68fb4f5b48-9tqw5\" (UID: \"46fa2c22-db29-429f-812f-422ae705d742\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-6fc9e6-68fb4f5b48-9tqw5"
Sep 11 17:40:27.440460 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:40:27.440439    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/46fa2c22-db29-429f-812f-422ae705d742-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-6fc9e6-68fb4f5b48-9tqw5\" (UID: \"46fa2c22-db29-429f-812f-422ae705d742\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-6fc9e6-68fb4f5b48-9tqw5"
Sep 11 17:40:27.449757 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:40:27.449728    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:40:27.541871 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:40:27.541838    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/46fa2c22-db29-429f-812f-422ae705d742-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-6fc9e6-68fb4f5b48-9tqw5\" (UID: \"46fa2c22-db29-429f-812f-422ae705d742\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-6fc9e6-68fb4f5b48-9tqw5"
Sep 11 17:40:27.542073 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:40:27.541899    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/46fa2c22-db29-429f-812f-422ae705d742-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-6fc9e6-68fb4f5b48-9tqw5\" (UID: \"46fa2c22-db29-429f-812f-422ae705d742\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-6fc9e6-68fb4f5b48-9tqw5"
Sep 11 17:40:27.542073 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:40:27.541953    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/46fa2c22-db29-429f-812f-422ae705d742-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-6fc9e6-68fb4f5b48-9tqw5\" (UID: \"46fa2c22-db29-429f-812f-422ae705d742\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-6fc9e6-68fb4f5b48-9tqw5"
Sep 11 17:40:27.542073 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:40:27.542011    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g74xj\" (UniqueName: \"kubernetes.io/projected/46fa2c22-db29-429f-812f-422ae705d742-kube-api-access-g74xj\") pod \"maas-api-e2e-shared-b-w3-6fc9e6-68fb4f5b48-9tqw5\" (UID: \"46fa2c22-db29-429f-812f-422ae705d742\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-6fc9e6-68fb4f5b48-9tqw5"
Sep 11 17:40:27.542816 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:40:27.542789    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/46fa2c22-db29-429f-812f-422ae705d742-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-6fc9e6-68fb4f5b48-9tqw5\" (UID: \"46fa2c22-db29-429f-812f-422ae705d742\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-6fc9e6-68fb4f5b48-9tqw5"
Sep 11 17:40:27.544542 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:40:27.544512    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/46fa2c22-db29-429f-812f-422ae705d742-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-6fc9e6-68fb4f5b48-9tqw5\" (UID: \"46fa2c22-db29-429f-812f-422ae705d742\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-6fc9e6-68fb4f5b48-9tqw5"
Sep 11 17:40:27.544779 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:40:27.544761    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/46fa2c22-db29-429f-812f-422ae705d742-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-6fc9e6-68fb4f5b48-9tqw5\" (UID: \"46fa2c22-db29-429f-812f-422ae705d742\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-6fc9e6-68fb4f5b48-9tqw5"
Sep 11 17:40:27.547605 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:40:27.547574    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:40:27.549836 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:40:27.549814    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g74xj\" (UniqueName: \"kubernetes.io/projected/46fa2c22-db29-429f-812f-422ae705d742-kube-api-access-g74xj\") pod \"maas-api-e2e-shared-b-w3-6fc9e6-68fb4f5b48-9tqw5\" (UID: \"46fa2c22-db29-429f-812f-422ae705d742\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-6fc9e6-68fb4f5b48-9tqw5"
Sep 11 17:40:27.636530 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:40:27.636501    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-6fc9e6-68fb4f5b48-9tqw5"
Sep 11 17:40:27.914827 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:40:27.914741    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:40:27.917019 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:40:27.916992    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-6fc9e6-68fb4f5b48-9tqw5"]
Sep 11 17:40:27.918601 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:40:27.918448    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46fa2c22_db29_429f_812f_422ae705d742.slice/crio-69b2cfcb977b2b4f8b10ad86ac9b09b2943eff3755fc702900eb07becbe2d103 WatchSource:0}: Error finding container 69b2cfcb977b2b4f8b10ad86ac9b09b2943eff3755fc702900eb07becbe2d103: Status 404 returned error can't find the container with id 69b2cfcb977b2b4f8b10ad86ac9b09b2943eff3755fc702900eb07becbe2d103
Sep 11 17:40:27.920487 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:40:27.920464    2552 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 17:40:28.014580 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:40:28.014530    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:40:28.342303 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:40:28.342267    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:40:28.895966 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:40:28.895927    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-6fc9e6-68fb4f5b48-9tqw5" event={"ID":"46fa2c22-db29-429f-812f-422ae705d742","Type":"ContainerStarted","Data":"fe85f241d9d61781a29630e2d7536694b35a8ef3fb74ce75666b21910a60d8ea"}
Sep 11 17:40:28.896479 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:40:28.895982    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-6fc9e6-68fb4f5b48-9tqw5" event={"ID":"46fa2c22-db29-429f-812f-422ae705d742","Type":"ContainerStarted","Data":"69b2cfcb977b2b4f8b10ad86ac9b09b2943eff3755fc702900eb07becbe2d103"}
Sep 11 17:40:28.896479 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:40:28.896004    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-6fc9e6-68fb4f5b48-9tqw5"
Sep 11 17:40:28.915746 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:40:28.915691    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-6fc9e6-68fb4f5b48-9tqw5" podStartSLOduration=1.42986907 podStartE2EDuration="1.915675684s" totalImagesPullingTime="485.806614ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:40:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:40:28.912482393 +0000 UTC m=+1323.433773924" watchObservedRunningTime="2026-09-11 17:40:28.915675684 +0000 UTC m=+1323.436967205"
Sep 11 17:40:29.356682 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:40:29.356643    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:40:30.101069 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:40:30.101035    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:40:30.808830 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:40:30.808791    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:40:31.644442 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:40:31.644408    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:40:32.522557 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:40:32.522516    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-144bfa/e2e-conflict-b-a3b46de5-kserve-7654cbf8fc-hfwvc"]
Sep 11 17:40:32.558445 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:40:32.558408    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:40:32.646723 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:40:32.646684    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:40:33.546925 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:40:33.546890    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:40:34.275387 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:40:34.275338    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:40:34.442202 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:40:34.442167    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:40:34.906744 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:40:34.906714    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-6fc9e6-68fb4f5b48-9tqw5"
Sep 11 17:40:37.643050 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:40:37.643015    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:40:43.219581 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:40:43.219540    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-6709f1a7-555b6ffbfd-ccd9z"]
Sep 11 17:40:43.220075 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:40:43.219834    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6709f1a7-555b6ffbfd-ccd9z" podUID="3fe17cc3-fdf3-4268-bf76-3ecc00c45bfb" containerName="maas-api" containerID="cri-o://27e7cc98a3bc625b48e442ecb7b54cf154daf7d0ceebd04aabc6f50fbcb36a34" gracePeriod=30
Sep 11 17:40:43.246899 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:40:43.246864    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:40:43.341953 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:40:43.341927    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:40:43.442761 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:40:43.442717    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:40:43.476328 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:40:43.476268    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6709f1a7-555b6ffbfd-ccd9z"
Sep 11 17:40:43.524606 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:40:43.524575    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:40:43.614128 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:40:43.614071    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4nqxq\" (UniqueName: \"kubernetes.io/projected/3fe17cc3-fdf3-4268-bf76-3ecc00c45bfb-kube-api-access-4nqxq\") pod \"3fe17cc3-fdf3-4268-bf76-3ecc00c45bfb\" (UID: \"3fe17cc3-fdf3-4268-bf76-3ecc00c45bfb\") "
Sep 11 17:40:43.614345 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:40:43.614177    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3fe17cc3-fdf3-4268-bf76-3ecc00c45bfb-maas-api-tls\") pod \"3fe17cc3-fdf3-4268-bf76-3ecc00c45bfb\" (UID: \"3fe17cc3-fdf3-4268-bf76-3ecc00c45bfb\") "
Sep 11 17:40:43.614345 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:40:43.614312    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3fe17cc3-fdf3-4268-bf76-3ecc00c45bfb-openshift-service-ca\") pod \"3fe17cc3-fdf3-4268-bf76-3ecc00c45bfb\" (UID: \"3fe17cc3-fdf3-4268-bf76-3ecc00c45bfb\") "
Sep 11 17:40:43.614345 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:40:43.614333    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3fe17cc3-fdf3-4268-bf76-3ecc00c45bfb-metrics-certs\") pod \"3fe17cc3-fdf3-4268-bf76-3ecc00c45bfb\" (UID: \"3fe17cc3-fdf3-4268-bf76-3ecc00c45bfb\") "
Sep 11 17:40:43.615165 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:40:43.615133    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3fe17cc3-fdf3-4268-bf76-3ecc00c45bfb-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "3fe17cc3-fdf3-4268-bf76-3ecc00c45bfb" (UID: "3fe17cc3-fdf3-4268-bf76-3ecc00c45bfb"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 17:40:43.617443 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:40:43.617405    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3fe17cc3-fdf3-4268-bf76-3ecc00c45bfb-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "3fe17cc3-fdf3-4268-bf76-3ecc00c45bfb" (UID: "3fe17cc3-fdf3-4268-bf76-3ecc00c45bfb"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 17:40:43.617586 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:40:43.617557    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3fe17cc3-fdf3-4268-bf76-3ecc00c45bfb-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "3fe17cc3-fdf3-4268-bf76-3ecc00c45bfb" (UID: "3fe17cc3-fdf3-4268-bf76-3ecc00c45bfb"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 17:40:43.617660 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:40:43.617601    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3fe17cc3-fdf3-4268-bf76-3ecc00c45bfb-kube-api-access-4nqxq" (OuterVolumeSpecName: "kube-api-access-4nqxq") pod "3fe17cc3-fdf3-4268-bf76-3ecc00c45bfb" (UID: "3fe17cc3-fdf3-4268-bf76-3ecc00c45bfb"). InnerVolumeSpecName "kube-api-access-4nqxq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:40:43.716274 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:40:43.716246    2552 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3fe17cc3-fdf3-4268-bf76-3ecc00c45bfb-openshift-service-ca\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:40:43.716274 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:40:43.716271    2552 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3fe17cc3-fdf3-4268-bf76-3ecc00c45bfb-metrics-certs\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:40:43.716274 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:40:43.716280    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4nqxq\" (UniqueName: \"kubernetes.io/projected/3fe17cc3-fdf3-4268-bf76-3ecc00c45bfb-kube-api-access-4nqxq\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:40:43.716503 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:40:43.716289    2552 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3fe17cc3-fdf3-4268-bf76-3ecc00c45bfb-maas-api-tls\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:40:43.988838 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:40:43.988804    2552 generic.go:356] "Generic (PLEG): container finished" podID="3fe17cc3-fdf3-4268-bf76-3ecc00c45bfb" containerID="27e7cc98a3bc625b48e442ecb7b54cf154daf7d0ceebd04aabc6f50fbcb36a34" exitCode=0
Sep 11 17:40:43.989034 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:40:43.988878    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6709f1a7-555b6ffbfd-ccd9z"
Sep 11 17:40:43.989034 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:40:43.988891    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6709f1a7-555b6ffbfd-ccd9z" event={"ID":"3fe17cc3-fdf3-4268-bf76-3ecc00c45bfb","Type":"ContainerDied","Data":"27e7cc98a3bc625b48e442ecb7b54cf154daf7d0ceebd04aabc6f50fbcb36a34"}
Sep 11 17:40:43.989034 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:40:43.988944    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6709f1a7-555b6ffbfd-ccd9z" event={"ID":"3fe17cc3-fdf3-4268-bf76-3ecc00c45bfb","Type":"ContainerDied","Data":"2177318dfc0d425619e3a4e5b1bce281cbc7af47e799a151c25cca26ae80f2bb"}
Sep 11 17:40:43.989034 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:40:43.988967    2552 scope.go:117] "RemoveContainer" containerID="27e7cc98a3bc625b48e442ecb7b54cf154daf7d0ceebd04aabc6f50fbcb36a34"
Sep 11 17:40:43.998600 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:40:43.998578    2552 scope.go:117] "RemoveContainer" containerID="27e7cc98a3bc625b48e442ecb7b54cf154daf7d0ceebd04aabc6f50fbcb36a34"
Sep 11 17:40:43.998988 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:40:43.998953    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"27e7cc98a3bc625b48e442ecb7b54cf154daf7d0ceebd04aabc6f50fbcb36a34\": container with ID starting with 27e7cc98a3bc625b48e442ecb7b54cf154daf7d0ceebd04aabc6f50fbcb36a34 not found: ID does not exist" containerID="27e7cc98a3bc625b48e442ecb7b54cf154daf7d0ceebd04aabc6f50fbcb36a34"
Sep 11 17:40:43.999095 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:40:43.999000    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"27e7cc98a3bc625b48e442ecb7b54cf154daf7d0ceebd04aabc6f50fbcb36a34"} err="failed to get container status \"27e7cc98a3bc625b48e442ecb7b54cf154daf7d0ceebd04aabc6f50fbcb36a34\": rpc error: code = NotFound desc = could not find container \"27e7cc98a3bc625b48e442ecb7b54cf154daf7d0ceebd04aabc6f50fbcb36a34\": container with ID starting with 27e7cc98a3bc625b48e442ecb7b54cf154daf7d0ceebd04aabc6f50fbcb36a34 not found: ID does not exist"
Sep 11 17:40:44.008557 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:40:44.008530    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-6709f1a7-555b6ffbfd-ccd9z"]
Sep 11 17:40:44.015895 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:40:44.015874    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-6709f1a7-555b6ffbfd-ccd9z"]
Sep 11 17:40:44.051701 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:40:44.051659    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:40:44.855850 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:40:44.855809    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:40:45.670096 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:40:45.670066    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:40:45.952500 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:40:45.952406    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3fe17cc3-fdf3-4268-bf76-3ecc00c45bfb" path="/var/lib/kubelet/pods/3fe17cc3-fdf3-4268-bf76-3ecc00c45bfb/volumes"
Sep 11 17:40:46.432283 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:40:46.432249    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:40:46.920946 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:40:46.920905    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:40:47.287988 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:40:47.287863    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:40:47.547631 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:40:47.547527    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-144bfa/e2e-conflict-b-a3b46de5-kserve-7654cbf8fc-hfwvc" podUID="e36986d4-829f-4706-b7ce-a7df8bfee750" containerName="main" containerID="cri-o://54b406e1c11c9f7c51545210b08cdae7448b4e868ef90ae0a25e8dc513252fd4" gracePeriod=45
Sep 11 17:40:47.837922 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:40:47.837894    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-144bfa/e2e-conflict-b-a3b46de5-kserve-7654cbf8fc-hfwvc"
Sep 11 17:40:47.852795 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:40:47.852760    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e36986d4-829f-4706-b7ce-a7df8bfee750-tls-certs\") pod \"e36986d4-829f-4706-b7ce-a7df8bfee750\" (UID: \"e36986d4-829f-4706-b7ce-a7df8bfee750\") "
Sep 11 17:40:47.852986 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:40:47.852864    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lgf5s\" (UniqueName: \"kubernetes.io/projected/e36986d4-829f-4706-b7ce-a7df8bfee750-kube-api-access-lgf5s\") pod \"e36986d4-829f-4706-b7ce-a7df8bfee750\" (UID: \"e36986d4-829f-4706-b7ce-a7df8bfee750\") "
Sep 11 17:40:47.852986 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:40:47.852891    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e36986d4-829f-4706-b7ce-a7df8bfee750-tmp-dir\") pod \"e36986d4-829f-4706-b7ce-a7df8bfee750\" (UID: \"e36986d4-829f-4706-b7ce-a7df8bfee750\") "
Sep 11 17:40:47.852986 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:40:47.852908    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e36986d4-829f-4706-b7ce-a7df8bfee750-home\") pod \"e36986d4-829f-4706-b7ce-a7df8bfee750\" (UID: \"e36986d4-829f-4706-b7ce-a7df8bfee750\") "
Sep 11 17:40:47.852986 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:40:47.852933    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e36986d4-829f-4706-b7ce-a7df8bfee750-model-cache\") pod \"e36986d4-829f-4706-b7ce-a7df8bfee750\" (UID: \"e36986d4-829f-4706-b7ce-a7df8bfee750\") "
Sep 11 17:40:47.852986 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:40:47.852972    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e36986d4-829f-4706-b7ce-a7df8bfee750-dshm\") pod \"e36986d4-829f-4706-b7ce-a7df8bfee750\" (UID: \"e36986d4-829f-4706-b7ce-a7df8bfee750\") "
Sep 11 17:40:47.853256 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:40:47.853136    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e36986d4-829f-4706-b7ce-a7df8bfee750-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e36986d4-829f-4706-b7ce-a7df8bfee750" (UID: "e36986d4-829f-4706-b7ce-a7df8bfee750"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:40:47.853307 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:40:47.853264    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e36986d4-829f-4706-b7ce-a7df8bfee750-home" (OuterVolumeSpecName: "home") pod "e36986d4-829f-4706-b7ce-a7df8bfee750" (UID: "e36986d4-829f-4706-b7ce-a7df8bfee750"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:40:47.853361 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:40:47.853299    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e36986d4-829f-4706-b7ce-a7df8bfee750-model-cache" (OuterVolumeSpecName: "model-cache") pod "e36986d4-829f-4706-b7ce-a7df8bfee750" (UID: "e36986d4-829f-4706-b7ce-a7df8bfee750"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:40:47.853361 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:40:47.853311    2552 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e36986d4-829f-4706-b7ce-a7df8bfee750-tmp-dir\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:40:47.857159 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:40:47.856058    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e36986d4-829f-4706-b7ce-a7df8bfee750-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e36986d4-829f-4706-b7ce-a7df8bfee750" (UID: "e36986d4-829f-4706-b7ce-a7df8bfee750"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 17:40:47.857159 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:40:47.856065    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e36986d4-829f-4706-b7ce-a7df8bfee750-dshm" (OuterVolumeSpecName: "dshm") pod "e36986d4-829f-4706-b7ce-a7df8bfee750" (UID: "e36986d4-829f-4706-b7ce-a7df8bfee750"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:40:47.857159 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:40:47.856263    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e36986d4-829f-4706-b7ce-a7df8bfee750-kube-api-access-lgf5s" (OuterVolumeSpecName: "kube-api-access-lgf5s") pod "e36986d4-829f-4706-b7ce-a7df8bfee750" (UID: "e36986d4-829f-4706-b7ce-a7df8bfee750"). InnerVolumeSpecName "kube-api-access-lgf5s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:40:47.954963 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:40:47.954925    2552 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e36986d4-829f-4706-b7ce-a7df8bfee750-tls-certs\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:40:47.954963 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:40:47.954960    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lgf5s\" (UniqueName: \"kubernetes.io/projected/e36986d4-829f-4706-b7ce-a7df8bfee750-kube-api-access-lgf5s\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:40:47.955250 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:40:47.954975    2552 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e36986d4-829f-4706-b7ce-a7df8bfee750-home\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:40:47.955250 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:40:47.954990    2552 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e36986d4-829f-4706-b7ce-a7df8bfee750-model-cache\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:40:47.955250 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:40:47.955004    2552 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e36986d4-829f-4706-b7ce-a7df8bfee750-dshm\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:40:48.019542 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:40:48.019510    2552 generic.go:356] "Generic (PLEG): container finished" podID="e36986d4-829f-4706-b7ce-a7df8bfee750" containerID="54b406e1c11c9f7c51545210b08cdae7448b4e868ef90ae0a25e8dc513252fd4" exitCode=0
Sep 11 17:40:48.019702 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:40:48.019573    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-144bfa/e2e-conflict-b-a3b46de5-kserve-7654cbf8fc-hfwvc"
Sep 11 17:40:48.019702 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:40:48.019578    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-144bfa/e2e-conflict-b-a3b46de5-kserve-7654cbf8fc-hfwvc" event={"ID":"e36986d4-829f-4706-b7ce-a7df8bfee750","Type":"ContainerDied","Data":"54b406e1c11c9f7c51545210b08cdae7448b4e868ef90ae0a25e8dc513252fd4"}
Sep 11 17:40:48.019702 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:40:48.019687    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-144bfa/e2e-conflict-b-a3b46de5-kserve-7654cbf8fc-hfwvc" event={"ID":"e36986d4-829f-4706-b7ce-a7df8bfee750","Type":"ContainerDied","Data":"713f80a6006f49f654a2a62154afd4f249c4c766175c40f70b89211b0b1423e7"}
Sep 11 17:40:48.019702 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:40:48.019703    2552 scope.go:117] "RemoveContainer" containerID="54b406e1c11c9f7c51545210b08cdae7448b4e868ef90ae0a25e8dc513252fd4"
Sep 11 17:40:48.029095 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:40:48.029070    2552 scope.go:117] "RemoveContainer" containerID="54b406e1c11c9f7c51545210b08cdae7448b4e868ef90ae0a25e8dc513252fd4"
Sep 11 17:40:48.029523 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:40:48.029502    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"54b406e1c11c9f7c51545210b08cdae7448b4e868ef90ae0a25e8dc513252fd4\": container with ID starting with 54b406e1c11c9f7c51545210b08cdae7448b4e868ef90ae0a25e8dc513252fd4 not found: ID does not exist" containerID="54b406e1c11c9f7c51545210b08cdae7448b4e868ef90ae0a25e8dc513252fd4"
Sep 11 17:40:48.029630 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:40:48.029529    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"54b406e1c11c9f7c51545210b08cdae7448b4e868ef90ae0a25e8dc513252fd4"} err="failed to get container status \"54b406e1c11c9f7c51545210b08cdae7448b4e868ef90ae0a25e8dc513252fd4\": rpc error: code = NotFound desc = could not find container \"54b406e1c11c9f7c51545210b08cdae7448b4e868ef90ae0a25e8dc513252fd4\": container with ID starting with 54b406e1c11c9f7c51545210b08cdae7448b4e868ef90ae0a25e8dc513252fd4 not found: ID does not exist"
Sep 11 17:40:48.037589 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:40:48.037559    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-144bfa/e2e-conflict-b-a3b46de5-kserve-7654cbf8fc-hfwvc"]
Sep 11 17:40:48.044140 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:40:48.044099    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-144bfa/e2e-conflict-b-a3b46de5-kserve-7654cbf8fc-hfwvc"]
Sep 11 17:40:48.251393 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:40:48.251308    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:40:49.088881 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:40:49.088845    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:40:49.727596 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:40:49.727558    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:40:49.950779 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:40:49.950740    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e36986d4-829f-4706-b7ce-a7df8bfee750" path="/var/lib/kubelet/pods/e36986d4-829f-4706-b7ce-a7df8bfee750/volumes"
Sep 11 17:40:50.423918 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:40:50.423875    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:40:51.191298 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:40:51.191254    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:40:51.913736 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:40:51.913704    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-6709f1a7/e2e-lifecycle-model-6709f1a7-kserve-7d6d8b6b5f-zmmwx"]
Sep 11 17:40:52.009036 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:40:52.009005    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:40:52.109674 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:40:52.109632    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:40:52.207523 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:40:52.207438    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:40:52.959668 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:40:52.959633    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:40:53.927440 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:40:53.927405    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:40:54.624379 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:40:54.624345    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:40:55.332951 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:40:55.332918    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:40:56.115986 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:40:56.115957    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:41:06.933720 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:41:06.933674    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-6709f1a7/e2e-lifecycle-model-6709f1a7-kserve-7d6d8b6b5f-zmmwx" podUID="1fa2f90c-ef64-4ef9-990c-5d7880eeec02" containerName="main" containerID="cri-o://1108ad80d922e3214719ed58ad3488a133e7a7eb019df592bf3ade0391589585" gracePeriod=45
Sep 11 17:41:07.169344 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:41:07.169234    2552 generic.go:356] "Generic (PLEG): container finished" podID="1fa2f90c-ef64-4ef9-990c-5d7880eeec02" containerID="1108ad80d922e3214719ed58ad3488a133e7a7eb019df592bf3ade0391589585" exitCode=0
Sep 11 17:41:07.169344 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:41:07.169307    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-6709f1a7/e2e-lifecycle-model-6709f1a7-kserve-7d6d8b6b5f-zmmwx" event={"ID":"1fa2f90c-ef64-4ef9-990c-5d7880eeec02","Type":"ContainerDied","Data":"1108ad80d922e3214719ed58ad3488a133e7a7eb019df592bf3ade0391589585"}
Sep 11 17:41:07.169483 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:41:07.169347    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-6709f1a7/e2e-lifecycle-model-6709f1a7-kserve-7d6d8b6b5f-zmmwx" event={"ID":"1fa2f90c-ef64-4ef9-990c-5d7880eeec02","Type":"ContainerDied","Data":"4a1dc2a25c37c915b2fbd150990a313de0f648b5c8081a5ee35e5476b863b65a"}
Sep 11 17:41:07.169483 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:41:07.169361    2552 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4a1dc2a25c37c915b2fbd150990a313de0f648b5c8081a5ee35e5476b863b65a"
Sep 11 17:41:07.185664 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:41:07.185601    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-6709f1a7/e2e-lifecycle-model-6709f1a7-kserve-7d6d8b6b5f-zmmwx"
Sep 11 17:41:07.283294 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:41:07.283258    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z7qdv\" (UniqueName: \"kubernetes.io/projected/1fa2f90c-ef64-4ef9-990c-5d7880eeec02-kube-api-access-z7qdv\") pod \"1fa2f90c-ef64-4ef9-990c-5d7880eeec02\" (UID: \"1fa2f90c-ef64-4ef9-990c-5d7880eeec02\") "
Sep 11 17:41:07.283465 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:41:07.283329    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1fa2f90c-ef64-4ef9-990c-5d7880eeec02-home\") pod \"1fa2f90c-ef64-4ef9-990c-5d7880eeec02\" (UID: \"1fa2f90c-ef64-4ef9-990c-5d7880eeec02\") "
Sep 11 17:41:07.283465 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:41:07.283367    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1fa2f90c-ef64-4ef9-990c-5d7880eeec02-dshm\") pod \"1fa2f90c-ef64-4ef9-990c-5d7880eeec02\" (UID: \"1fa2f90c-ef64-4ef9-990c-5d7880eeec02\") "
Sep 11 17:41:07.283465 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:41:07.283388    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1fa2f90c-ef64-4ef9-990c-5d7880eeec02-model-cache\") pod \"1fa2f90c-ef64-4ef9-990c-5d7880eeec02\" (UID: \"1fa2f90c-ef64-4ef9-990c-5d7880eeec02\") "
Sep 11 17:41:07.283465 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:41:07.283416    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1fa2f90c-ef64-4ef9-990c-5d7880eeec02-tmp-dir\") pod \"1fa2f90c-ef64-4ef9-990c-5d7880eeec02\" (UID: \"1fa2f90c-ef64-4ef9-990c-5d7880eeec02\") "
Sep 11 17:41:07.283465 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:41:07.283442    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1fa2f90c-ef64-4ef9-990c-5d7880eeec02-tls-certs\") pod \"1fa2f90c-ef64-4ef9-990c-5d7880eeec02\" (UID: \"1fa2f90c-ef64-4ef9-990c-5d7880eeec02\") "
Sep 11 17:41:07.283667 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:41:07.283629    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1fa2f90c-ef64-4ef9-990c-5d7880eeec02-home" (OuterVolumeSpecName: "home") pod "1fa2f90c-ef64-4ef9-990c-5d7880eeec02" (UID: "1fa2f90c-ef64-4ef9-990c-5d7880eeec02"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:41:07.283725 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:41:07.283674    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1fa2f90c-ef64-4ef9-990c-5d7880eeec02-model-cache" (OuterVolumeSpecName: "model-cache") pod "1fa2f90c-ef64-4ef9-990c-5d7880eeec02" (UID: "1fa2f90c-ef64-4ef9-990c-5d7880eeec02"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:41:07.283853 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:41:07.283810    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1fa2f90c-ef64-4ef9-990c-5d7880eeec02-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "1fa2f90c-ef64-4ef9-990c-5d7880eeec02" (UID: "1fa2f90c-ef64-4ef9-990c-5d7880eeec02"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:41:07.285671 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:41:07.285636    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1fa2f90c-ef64-4ef9-990c-5d7880eeec02-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "1fa2f90c-ef64-4ef9-990c-5d7880eeec02" (UID: "1fa2f90c-ef64-4ef9-990c-5d7880eeec02"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 17:41:07.285671 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:41:07.285651    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1fa2f90c-ef64-4ef9-990c-5d7880eeec02-dshm" (OuterVolumeSpecName: "dshm") pod "1fa2f90c-ef64-4ef9-990c-5d7880eeec02" (UID: "1fa2f90c-ef64-4ef9-990c-5d7880eeec02"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:41:07.285801 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:41:07.285717    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1fa2f90c-ef64-4ef9-990c-5d7880eeec02-kube-api-access-z7qdv" (OuterVolumeSpecName: "kube-api-access-z7qdv") pod "1fa2f90c-ef64-4ef9-990c-5d7880eeec02" (UID: "1fa2f90c-ef64-4ef9-990c-5d7880eeec02"). InnerVolumeSpecName "kube-api-access-z7qdv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:41:07.384448 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:41:07.384414    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z7qdv\" (UniqueName: \"kubernetes.io/projected/1fa2f90c-ef64-4ef9-990c-5d7880eeec02-kube-api-access-z7qdv\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:41:07.384448 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:41:07.384440    2552 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1fa2f90c-ef64-4ef9-990c-5d7880eeec02-home\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:41:07.384448 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:41:07.384449    2552 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1fa2f90c-ef64-4ef9-990c-5d7880eeec02-dshm\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:41:07.384448 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:41:07.384457    2552 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1fa2f90c-ef64-4ef9-990c-5d7880eeec02-model-cache\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:41:07.384725 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:41:07.384465    2552 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1fa2f90c-ef64-4ef9-990c-5d7880eeec02-tmp-dir\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:41:07.384725 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:41:07.384473    2552 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1fa2f90c-ef64-4ef9-990c-5d7880eeec02-tls-certs\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:41:08.175005 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:41:08.174971    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-6709f1a7/e2e-lifecycle-model-6709f1a7-kserve-7d6d8b6b5f-zmmwx"
Sep 11 17:41:08.191446 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:41:08.191415    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-6709f1a7/e2e-lifecycle-model-6709f1a7-kserve-7d6d8b6b5f-zmmwx"]
Sep 11 17:41:08.193945 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:41:08.193917    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-6709f1a7/e2e-lifecycle-model-6709f1a7-kserve-7d6d8b6b5f-zmmwx"]
Sep 11 17:41:09.402494 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:41:09.402458    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-2fac54/auth-test-model-2fac54-kserve-58fc867967-wdj45"]
Sep 11 17:41:09.402890 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:41:09.402866    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3fe17cc3-fdf3-4268-bf76-3ecc00c45bfb" containerName="maas-api"
Sep 11 17:41:09.402890 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:41:09.402878    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="3fe17cc3-fdf3-4268-bf76-3ecc00c45bfb" containerName="maas-api"
Sep 11 17:41:09.402968 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:41:09.402895    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1fa2f90c-ef64-4ef9-990c-5d7880eeec02" containerName="main"
Sep 11 17:41:09.402968 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:41:09.402900    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="1fa2f90c-ef64-4ef9-990c-5d7880eeec02" containerName="main"
Sep 11 17:41:09.402968 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:41:09.402913    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e36986d4-829f-4706-b7ce-a7df8bfee750" containerName="main"
Sep 11 17:41:09.402968 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:41:09.402918    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="e36986d4-829f-4706-b7ce-a7df8bfee750" containerName="main"
Sep 11 17:41:09.403095 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:41:09.402982    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="e36986d4-829f-4706-b7ce-a7df8bfee750" containerName="main"
Sep 11 17:41:09.403095 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:41:09.402988    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="1fa2f90c-ef64-4ef9-990c-5d7880eeec02" containerName="main"
Sep 11 17:41:09.403095 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:41:09.402998    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="3fe17cc3-fdf3-4268-bf76-3ecc00c45bfb" containerName="maas-api"
Sep 11 17:41:09.413858 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:41:09.413827    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-2fac54/auth-test-model-2fac54-kserve-58fc867967-wdj45"
Sep 11 17:41:09.416320 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:41:09.416069    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-2fac54\"/\"auth-test-model-2fac54-kserve-self-signed-certs\""
Sep 11 17:41:09.416682 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:41:09.416661    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-2fac54\"/\"default-dockercfg-wsjxc\""
Sep 11 17:41:09.416819 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:41:09.416751    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-2fac54\"/\"kube-root-ca.crt\""
Sep 11 17:41:09.416890 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:41:09.416711    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-2fac54\"/\"openshift-service-ca.crt\""
Sep 11 17:41:09.417245 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:41:09.417220    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-2fac54/auth-test-model-2fac54-kserve-58fc867967-wdj45"]
Sep 11 17:41:09.511885 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:41:09.511849    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d49b97bb-b28a-480d-8adb-a4d30e863b18-dshm\") pod \"auth-test-model-2fac54-kserve-58fc867967-wdj45\" (UID: \"d49b97bb-b28a-480d-8adb-a4d30e863b18\") " pod="ai-tenant-e2e-shared-a-w3-2fac54/auth-test-model-2fac54-kserve-58fc867967-wdj45"
Sep 11 17:41:09.512053 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:41:09.511896    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d49b97bb-b28a-480d-8adb-a4d30e863b18-home\") pod \"auth-test-model-2fac54-kserve-58fc867967-wdj45\" (UID: \"d49b97bb-b28a-480d-8adb-a4d30e863b18\") " pod="ai-tenant-e2e-shared-a-w3-2fac54/auth-test-model-2fac54-kserve-58fc867967-wdj45"
Sep 11 17:41:09.512053 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:41:09.511964    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d49b97bb-b28a-480d-8adb-a4d30e863b18-tmp-dir\") pod \"auth-test-model-2fac54-kserve-58fc867967-wdj45\" (UID: \"d49b97bb-b28a-480d-8adb-a4d30e863b18\") " pod="ai-tenant-e2e-shared-a-w3-2fac54/auth-test-model-2fac54-kserve-58fc867967-wdj45"
Sep 11 17:41:09.512053 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:41:09.511982    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zt2dl\" (UniqueName: \"kubernetes.io/projected/d49b97bb-b28a-480d-8adb-a4d30e863b18-kube-api-access-zt2dl\") pod \"auth-test-model-2fac54-kserve-58fc867967-wdj45\" (UID: \"d49b97bb-b28a-480d-8adb-a4d30e863b18\") " pod="ai-tenant-e2e-shared-a-w3-2fac54/auth-test-model-2fac54-kserve-58fc867967-wdj45"
Sep 11 17:41:09.512195 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:41:09.512088    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d49b97bb-b28a-480d-8adb-a4d30e863b18-tls-certs\") pod \"auth-test-model-2fac54-kserve-58fc867967-wdj45\" (UID: \"d49b97bb-b28a-480d-8adb-a4d30e863b18\") " pod="ai-tenant-e2e-shared-a-w3-2fac54/auth-test-model-2fac54-kserve-58fc867967-wdj45"
Sep 11 17:41:09.512195 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:41:09.512174    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d49b97bb-b28a-480d-8adb-a4d30e863b18-model-cache\") pod \"auth-test-model-2fac54-kserve-58fc867967-wdj45\" (UID: \"d49b97bb-b28a-480d-8adb-a4d30e863b18\") " pod="ai-tenant-e2e-shared-a-w3-2fac54/auth-test-model-2fac54-kserve-58fc867967-wdj45"
Sep 11 17:41:09.603934 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:41:09.603900    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:41:09.612879 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:41:09.612851    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d49b97bb-b28a-480d-8adb-a4d30e863b18-tmp-dir\") pod \"auth-test-model-2fac54-kserve-58fc867967-wdj45\" (UID: \"d49b97bb-b28a-480d-8adb-a4d30e863b18\") " pod="ai-tenant-e2e-shared-a-w3-2fac54/auth-test-model-2fac54-kserve-58fc867967-wdj45"
Sep 11 17:41:09.613045 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:41:09.612884    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zt2dl\" (UniqueName: \"kubernetes.io/projected/d49b97bb-b28a-480d-8adb-a4d30e863b18-kube-api-access-zt2dl\") pod \"auth-test-model-2fac54-kserve-58fc867967-wdj45\" (UID: \"d49b97bb-b28a-480d-8adb-a4d30e863b18\") " pod="ai-tenant-e2e-shared-a-w3-2fac54/auth-test-model-2fac54-kserve-58fc867967-wdj45"
Sep 11 17:41:09.613045 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:41:09.612934    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d49b97bb-b28a-480d-8adb-a4d30e863b18-tls-certs\") pod \"auth-test-model-2fac54-kserve-58fc867967-wdj45\" (UID: \"d49b97bb-b28a-480d-8adb-a4d30e863b18\") " pod="ai-tenant-e2e-shared-a-w3-2fac54/auth-test-model-2fac54-kserve-58fc867967-wdj45"
Sep 11 17:41:09.613045 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:41:09.612964    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d49b97bb-b28a-480d-8adb-a4d30e863b18-model-cache\") pod \"auth-test-model-2fac54-kserve-58fc867967-wdj45\" (UID: \"d49b97bb-b28a-480d-8adb-a4d30e863b18\") " pod="ai-tenant-e2e-shared-a-w3-2fac54/auth-test-model-2fac54-kserve-58fc867967-wdj45"
Sep 11 17:41:09.613045 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:41:09.613017    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d49b97bb-b28a-480d-8adb-a4d30e863b18-dshm\") pod \"auth-test-model-2fac54-kserve-58fc867967-wdj45\" (UID: \"d49b97bb-b28a-480d-8adb-a4d30e863b18\") " pod="ai-tenant-e2e-shared-a-w3-2fac54/auth-test-model-2fac54-kserve-58fc867967-wdj45"
Sep 11 17:41:09.613296 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:41:09.613079    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d49b97bb-b28a-480d-8adb-a4d30e863b18-home\") pod \"auth-test-model-2fac54-kserve-58fc867967-wdj45\" (UID: \"d49b97bb-b28a-480d-8adb-a4d30e863b18\") " pod="ai-tenant-e2e-shared-a-w3-2fac54/auth-test-model-2fac54-kserve-58fc867967-wdj45"
Sep 11 17:41:09.613359 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:41:09.613344    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d49b97bb-b28a-480d-8adb-a4d30e863b18-tmp-dir\") pod \"auth-test-model-2fac54-kserve-58fc867967-wdj45\" (UID: \"d49b97bb-b28a-480d-8adb-a4d30e863b18\") " pod="ai-tenant-e2e-shared-a-w3-2fac54/auth-test-model-2fac54-kserve-58fc867967-wdj45"
Sep 11 17:41:09.613410 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:41:09.613371    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d49b97bb-b28a-480d-8adb-a4d30e863b18-model-cache\") pod \"auth-test-model-2fac54-kserve-58fc867967-wdj45\" (UID: \"d49b97bb-b28a-480d-8adb-a4d30e863b18\") " pod="ai-tenant-e2e-shared-a-w3-2fac54/auth-test-model-2fac54-kserve-58fc867967-wdj45"
Sep 11 17:41:09.613511 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:41:09.613493    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d49b97bb-b28a-480d-8adb-a4d30e863b18-home\") pod \"auth-test-model-2fac54-kserve-58fc867967-wdj45\" (UID: \"d49b97bb-b28a-480d-8adb-a4d30e863b18\") " pod="ai-tenant-e2e-shared-a-w3-2fac54/auth-test-model-2fac54-kserve-58fc867967-wdj45"
Sep 11 17:41:09.615249 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:41:09.615228    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d49b97bb-b28a-480d-8adb-a4d30e863b18-dshm\") pod \"auth-test-model-2fac54-kserve-58fc867967-wdj45\" (UID: \"d49b97bb-b28a-480d-8adb-a4d30e863b18\") " pod="ai-tenant-e2e-shared-a-w3-2fac54/auth-test-model-2fac54-kserve-58fc867967-wdj45"
Sep 11 17:41:09.615539 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:41:09.615522    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d49b97bb-b28a-480d-8adb-a4d30e863b18-tls-certs\") pod \"auth-test-model-2fac54-kserve-58fc867967-wdj45\" (UID: \"d49b97bb-b28a-480d-8adb-a4d30e863b18\") " pod="ai-tenant-e2e-shared-a-w3-2fac54/auth-test-model-2fac54-kserve-58fc867967-wdj45"
Sep 11 17:41:09.622258 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:41:09.622236    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zt2dl\" (UniqueName: \"kubernetes.io/projected/d49b97bb-b28a-480d-8adb-a4d30e863b18-kube-api-access-zt2dl\") pod \"auth-test-model-2fac54-kserve-58fc867967-wdj45\" (UID: \"d49b97bb-b28a-480d-8adb-a4d30e863b18\") " pod="ai-tenant-e2e-shared-a-w3-2fac54/auth-test-model-2fac54-kserve-58fc867967-wdj45"
Sep 11 17:41:09.724022 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:41:09.723936    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-2fac54/auth-test-model-2fac54-kserve-58fc867967-wdj45"
Sep 11 17:41:09.949935 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:41:09.949898    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1fa2f90c-ef64-4ef9-990c-5d7880eeec02" path="/var/lib/kubelet/pods/1fa2f90c-ef64-4ef9-990c-5d7880eeec02/volumes"
Sep 11 17:41:10.009466 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:41:10.009438    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-2fac54/auth-test-model-2fac54-kserve-58fc867967-wdj45"]
Sep 11 17:41:10.012232 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:41:10.012205    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd49b97bb_b28a_480d_8adb_a4d30e863b18.slice/crio-d247ad4fdf369e49cffaca6f1eb3a8527b3936d328fb6bd829fd625049778361 WatchSource:0}: Error finding container d247ad4fdf369e49cffaca6f1eb3a8527b3936d328fb6bd829fd625049778361: Status 404 returned error can't find the container with id d247ad4fdf369e49cffaca6f1eb3a8527b3936d328fb6bd829fd625049778361
Sep 11 17:41:10.190046 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:41:10.190006    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-2fac54/auth-test-model-2fac54-kserve-58fc867967-wdj45" event={"ID":"d49b97bb-b28a-480d-8adb-a4d30e863b18","Type":"ContainerStarted","Data":"c5dc0e84f9fd5f82092ecf486050445ed69584f47013eda8f381983ee910b20a"}
Sep 11 17:41:10.190046 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:41:10.190050    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-2fac54/auth-test-model-2fac54-kserve-58fc867967-wdj45" event={"ID":"d49b97bb-b28a-480d-8adb-a4d30e863b18","Type":"ContainerStarted","Data":"d247ad4fdf369e49cffaca6f1eb3a8527b3936d328fb6bd829fd625049778361"}
Sep 11 17:41:10.219940 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:41:10.219893    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-2fac54/auth-test-model-2fac54-kserve-58fc867967-wdj45" podStartSLOduration=1.219874357 podStartE2EDuration="1.219874357s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:41:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:41:10.219028581 +0000 UTC m=+1364.740320102" watchObservedRunningTime="2026-09-11 17:41:10.219874357 +0000 UTC m=+1364.741165884"
Sep 11 17:41:19.724370 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:41:19.724333    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-2fac54/auth-test-model-2fac54-kserve-58fc867967-wdj45"
Sep 11 17:41:19.724370 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:41:19.724381    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-2fac54/auth-test-model-2fac54-kserve-58fc867967-wdj45"
Sep 11 17:41:19.736726 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:41:19.736697    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-2fac54/auth-test-model-2fac54-kserve-58fc867967-wdj45"
Sep 11 17:41:20.270217 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:41:20.270190    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-2fac54/auth-test-model-2fac54-kserve-58fc867967-wdj45"
Sep 11 17:41:27.409872 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:41:27.409832    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-144bfa/distinct-2-w4-144bfa-kserve-5cfcc96d78-75f6s"]
Sep 11 17:41:27.416538 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:41:27.416510    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-144bfa/distinct-2-w4-144bfa-kserve-5cfcc96d78-75f6s"
Sep 11 17:41:27.419030 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:41:27.419010    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-144bfa\"/\"distinct-2-w4-144bfa-kserve-self-signed-certs\""
Sep 11 17:41:27.424615 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:41:27.424577    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-144bfa/distinct-2-w4-144bfa-kserve-5cfcc96d78-75f6s"]
Sep 11 17:41:27.492553 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:41:27.492510    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:41:27.495447 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:41:27.495423    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c624a168-2aee-4e51-b69f-106a4c0eedf4-tmp-dir\") pod \"distinct-2-w4-144bfa-kserve-5cfcc96d78-75f6s\" (UID: \"c624a168-2aee-4e51-b69f-106a4c0eedf4\") " pod="e2e-models-e2e-worker-w4-144bfa/distinct-2-w4-144bfa-kserve-5cfcc96d78-75f6s"
Sep 11 17:41:27.495544 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:41:27.495454    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sm7hg\" (UniqueName: \"kubernetes.io/projected/c624a168-2aee-4e51-b69f-106a4c0eedf4-kube-api-access-sm7hg\") pod \"distinct-2-w4-144bfa-kserve-5cfcc96d78-75f6s\" (UID: \"c624a168-2aee-4e51-b69f-106a4c0eedf4\") " pod="e2e-models-e2e-worker-w4-144bfa/distinct-2-w4-144bfa-kserve-5cfcc96d78-75f6s"
Sep 11 17:41:27.495544 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:41:27.495520    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c624a168-2aee-4e51-b69f-106a4c0eedf4-dshm\") pod \"distinct-2-w4-144bfa-kserve-5cfcc96d78-75f6s\" (UID: \"c624a168-2aee-4e51-b69f-106a4c0eedf4\") " pod="e2e-models-e2e-worker-w4-144bfa/distinct-2-w4-144bfa-kserve-5cfcc96d78-75f6s"
Sep 11 17:41:27.495544 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:41:27.495541    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c624a168-2aee-4e51-b69f-106a4c0eedf4-home\") pod \"distinct-2-w4-144bfa-kserve-5cfcc96d78-75f6s\" (UID: \"c624a168-2aee-4e51-b69f-106a4c0eedf4\") " pod="e2e-models-e2e-worker-w4-144bfa/distinct-2-w4-144bfa-kserve-5cfcc96d78-75f6s"
Sep 11 17:41:27.495691 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:41:27.495673    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c624a168-2aee-4e51-b69f-106a4c0eedf4-model-cache\") pod \"distinct-2-w4-144bfa-kserve-5cfcc96d78-75f6s\" (UID: \"c624a168-2aee-4e51-b69f-106a4c0eedf4\") " pod="e2e-models-e2e-worker-w4-144bfa/distinct-2-w4-144bfa-kserve-5cfcc96d78-75f6s"
Sep 11 17:41:27.495733 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:41:27.495706    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c624a168-2aee-4e51-b69f-106a4c0eedf4-tls-certs\") pod \"distinct-2-w4-144bfa-kserve-5cfcc96d78-75f6s\" (UID: \"c624a168-2aee-4e51-b69f-106a4c0eedf4\") " pod="e2e-models-e2e-worker-w4-144bfa/distinct-2-w4-144bfa-kserve-5cfcc96d78-75f6s"
Sep 11 17:41:27.597442 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:41:27.597406    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c624a168-2aee-4e51-b69f-106a4c0eedf4-tls-certs\") pod \"distinct-2-w4-144bfa-kserve-5cfcc96d78-75f6s\" (UID: \"c624a168-2aee-4e51-b69f-106a4c0eedf4\") " pod="e2e-models-e2e-worker-w4-144bfa/distinct-2-w4-144bfa-kserve-5cfcc96d78-75f6s"
Sep 11 17:41:27.597650 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:41:27.597466    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c624a168-2aee-4e51-b69f-106a4c0eedf4-tmp-dir\") pod \"distinct-2-w4-144bfa-kserve-5cfcc96d78-75f6s\" (UID: \"c624a168-2aee-4e51-b69f-106a4c0eedf4\") " pod="e2e-models-e2e-worker-w4-144bfa/distinct-2-w4-144bfa-kserve-5cfcc96d78-75f6s"
Sep 11 17:41:27.597650 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:41:27.597524    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sm7hg\" (UniqueName: \"kubernetes.io/projected/c624a168-2aee-4e51-b69f-106a4c0eedf4-kube-api-access-sm7hg\") pod \"distinct-2-w4-144bfa-kserve-5cfcc96d78-75f6s\" (UID: \"c624a168-2aee-4e51-b69f-106a4c0eedf4\") " pod="e2e-models-e2e-worker-w4-144bfa/distinct-2-w4-144bfa-kserve-5cfcc96d78-75f6s"
Sep 11 17:41:27.597650 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:41:27.597597    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c624a168-2aee-4e51-b69f-106a4c0eedf4-dshm\") pod \"distinct-2-w4-144bfa-kserve-5cfcc96d78-75f6s\" (UID: \"c624a168-2aee-4e51-b69f-106a4c0eedf4\") " pod="e2e-models-e2e-worker-w4-144bfa/distinct-2-w4-144bfa-kserve-5cfcc96d78-75f6s"
Sep 11 17:41:27.597650 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:41:27.597628    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c624a168-2aee-4e51-b69f-106a4c0eedf4-home\") pod \"distinct-2-w4-144bfa-kserve-5cfcc96d78-75f6s\" (UID: \"c624a168-2aee-4e51-b69f-106a4c0eedf4\") " pod="e2e-models-e2e-worker-w4-144bfa/distinct-2-w4-144bfa-kserve-5cfcc96d78-75f6s"
Sep 11 17:41:27.597882 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:41:27.597752    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c624a168-2aee-4e51-b69f-106a4c0eedf4-model-cache\") pod \"distinct-2-w4-144bfa-kserve-5cfcc96d78-75f6s\" (UID: \"c624a168-2aee-4e51-b69f-106a4c0eedf4\") " pod="e2e-models-e2e-worker-w4-144bfa/distinct-2-w4-144bfa-kserve-5cfcc96d78-75f6s"
Sep 11 17:41:27.598236 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:41:27.598209    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c624a168-2aee-4e51-b69f-106a4c0eedf4-home\") pod \"distinct-2-w4-144bfa-kserve-5cfcc96d78-75f6s\" (UID: \"c624a168-2aee-4e51-b69f-106a4c0eedf4\") " pod="e2e-models-e2e-worker-w4-144bfa/distinct-2-w4-144bfa-kserve-5cfcc96d78-75f6s"
Sep 11 17:41:27.598423 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:41:27.598228    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c624a168-2aee-4e51-b69f-106a4c0eedf4-tmp-dir\") pod \"distinct-2-w4-144bfa-kserve-5cfcc96d78-75f6s\" (UID: \"c624a168-2aee-4e51-b69f-106a4c0eedf4\") " pod="e2e-models-e2e-worker-w4-144bfa/distinct-2-w4-144bfa-kserve-5cfcc96d78-75f6s"
Sep 11 17:41:27.598549 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:41:27.598417    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c624a168-2aee-4e51-b69f-106a4c0eedf4-model-cache\") pod \"distinct-2-w4-144bfa-kserve-5cfcc96d78-75f6s\" (UID: \"c624a168-2aee-4e51-b69f-106a4c0eedf4\") " pod="e2e-models-e2e-worker-w4-144bfa/distinct-2-w4-144bfa-kserve-5cfcc96d78-75f6s"
Sep 11 17:41:27.600424 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:41:27.600393    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c624a168-2aee-4e51-b69f-106a4c0eedf4-dshm\") pod \"distinct-2-w4-144bfa-kserve-5cfcc96d78-75f6s\" (UID: \"c624a168-2aee-4e51-b69f-106a4c0eedf4\") " pod="e2e-models-e2e-worker-w4-144bfa/distinct-2-w4-144bfa-kserve-5cfcc96d78-75f6s"
Sep 11 17:41:27.600925 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:41:27.600907    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c624a168-2aee-4e51-b69f-106a4c0eedf4-tls-certs\") pod \"distinct-2-w4-144bfa-kserve-5cfcc96d78-75f6s\" (UID: \"c624a168-2aee-4e51-b69f-106a4c0eedf4\") " pod="e2e-models-e2e-worker-w4-144bfa/distinct-2-w4-144bfa-kserve-5cfcc96d78-75f6s"
Sep 11 17:41:27.611493 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:41:27.611468    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sm7hg\" (UniqueName: \"kubernetes.io/projected/c624a168-2aee-4e51-b69f-106a4c0eedf4-kube-api-access-sm7hg\") pod \"distinct-2-w4-144bfa-kserve-5cfcc96d78-75f6s\" (UID: \"c624a168-2aee-4e51-b69f-106a4c0eedf4\") " pod="e2e-models-e2e-worker-w4-144bfa/distinct-2-w4-144bfa-kserve-5cfcc96d78-75f6s"
Sep 11 17:41:27.726922 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:41:27.726838    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-144bfa/distinct-2-w4-144bfa-kserve-5cfcc96d78-75f6s"
Sep 11 17:41:28.004010 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:41:28.002848    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-144bfa/distinct-2-w4-144bfa-kserve-5cfcc96d78-75f6s"]
Sep 11 17:41:28.004946 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:41:28.004903    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc624a168_2aee_4e51_b69f_106a4c0eedf4.slice/crio-0298f0f8cc2e38a0dba4e7485b89ec5b7067d3cf10769084a91ef6ed73721e86 WatchSource:0}: Error finding container 0298f0f8cc2e38a0dba4e7485b89ec5b7067d3cf10769084a91ef6ed73721e86: Status 404 returned error can't find the container with id 0298f0f8cc2e38a0dba4e7485b89ec5b7067d3cf10769084a91ef6ed73721e86
Sep 11 17:41:28.054645 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:41:28.054610    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:41:28.308308 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:41:28.308224    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-144bfa/distinct-2-w4-144bfa-kserve-5cfcc96d78-75f6s" event={"ID":"c624a168-2aee-4e51-b69f-106a4c0eedf4","Type":"ContainerStarted","Data":"028310bf2fddaaf22dd4a734453feffb7ddd8ee415b285eeaf5bdc420b793ca4"}
Sep 11 17:41:28.308308 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:41:28.308269    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-144bfa/distinct-2-w4-144bfa-kserve-5cfcc96d78-75f6s" event={"ID":"c624a168-2aee-4e51-b69f-106a4c0eedf4","Type":"ContainerStarted","Data":"0298f0f8cc2e38a0dba4e7485b89ec5b7067d3cf10769084a91ef6ed73721e86"}
Sep 11 17:41:28.325476 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:41:28.325426    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-144bfa/distinct-2-w4-144bfa-kserve-5cfcc96d78-75f6s" podStartSLOduration=1.325410003 podStartE2EDuration="1.325410003s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:41:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:41:28.323641815 +0000 UTC m=+1382.844933327" watchObservedRunningTime="2026-09-11 17:41:28.325410003 +0000 UTC m=+1382.846701524"
Sep 11 17:41:28.961895 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:41:28.961861    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:41:34.589565 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:41:34.589527    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:41:34.690626 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:41:34.690594    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:41:34.885518 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:41:34.885436    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:41:36.195546 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:41:36.195509    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:41:36.295296 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:41:36.295258    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:41:36.397086 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:41:36.397046    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:41:37.047368 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:41:37.047337    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:41:37.138918 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:41:37.138887    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:41:37.240178 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:41:37.240134    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:41:37.727239 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:41:37.727205    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-144bfa/distinct-2-w4-144bfa-kserve-5cfcc96d78-75f6s"
Sep 11 17:41:37.727493 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:41:37.727473    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-144bfa/distinct-2-w4-144bfa-kserve-5cfcc96d78-75f6s"
Sep 11 17:41:37.742267 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:41:37.742239    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-144bfa/distinct-2-w4-144bfa-kserve-5cfcc96d78-75f6s"
Sep 11 17:41:38.390770 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:41:38.390729    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-144bfa/distinct-2-w4-144bfa-kserve-5cfcc96d78-75f6s"
Sep 11 17:41:38.481429 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:41:38.481381    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:41:39.368805 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:41:39.368761    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:41:41.317301 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:41:41.317257    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:41:44.668265 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:41:44.668234    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:41:44.768712 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:41:44.768676    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:41:45.325583 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:41:45.325549    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:41:51.574485 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:41:51.574453    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:41:51.687513 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:41:51.687478    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:41:51.776924 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:41:51.776889    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:41:58.337491 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:41:58.337451    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:42:02.295691 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:42:02.295644    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:42:02.540498 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:42:02.540458    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:42:03.830466 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:42:03.830424    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:42:03.930648 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:42:03.930613    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:42:04.597367 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:42:04.597337    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:42:08.859295 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:42:08.859267    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:42:23.421400 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:42:23.421286    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:42:29.465609 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:42:29.465567    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:42:29.588833 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:42:29.588793    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:42:29.797095 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:42:29.797007    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:42:30.086393 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:42:30.086317    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-15f6f2-openshift-default-7c9bf84768-lht27"]
Sep 11 17:42:30.090264 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:42:30.090249    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-15f6f2-openshift-default-7c9bf84768-lht27"
Sep 11 17:42:30.093132 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:42:30.093099    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-15f6f2-openshift-default-dockercfg-whxj6\""
Sep 11 17:42:30.101373 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:42:30.101352    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-15f6f2-openshift-default-7c9bf84768-lht27"]
Sep 11 17:42:30.110713 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:42:30.110687    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:42:30.229872 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:42:30.229839    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tvtg9\" (UniqueName: \"kubernetes.io/projected/39af155e-9284-42e3-b5f4-d6f3c4be1757-kube-api-access-tvtg9\") pod \"e2e-api-a-15f6f2-openshift-default-7c9bf84768-lht27\" (UID: \"39af155e-9284-42e3-b5f4-d6f3c4be1757\") " pod="openshift-ingress/e2e-api-a-15f6f2-openshift-default-7c9bf84768-lht27"
Sep 11 17:42:30.230077 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:42:30.229888    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/39af155e-9284-42e3-b5f4-d6f3c4be1757-istio-podinfo\") pod \"e2e-api-a-15f6f2-openshift-default-7c9bf84768-lht27\" (UID: \"39af155e-9284-42e3-b5f4-d6f3c4be1757\") " pod="openshift-ingress/e2e-api-a-15f6f2-openshift-default-7c9bf84768-lht27"
Sep 11 17:42:30.230077 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:42:30.229911    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/39af155e-9284-42e3-b5f4-d6f3c4be1757-istio-envoy\") pod \"e2e-api-a-15f6f2-openshift-default-7c9bf84768-lht27\" (UID: \"39af155e-9284-42e3-b5f4-d6f3c4be1757\") " pod="openshift-ingress/e2e-api-a-15f6f2-openshift-default-7c9bf84768-lht27"
Sep 11 17:42:30.230077 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:42:30.229987    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/39af155e-9284-42e3-b5f4-d6f3c4be1757-istio-token\") pod \"e2e-api-a-15f6f2-openshift-default-7c9bf84768-lht27\" (UID: \"39af155e-9284-42e3-b5f4-d6f3c4be1757\") " pod="openshift-ingress/e2e-api-a-15f6f2-openshift-default-7c9bf84768-lht27"
Sep 11 17:42:30.230280 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:42:30.230085    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/39af155e-9284-42e3-b5f4-d6f3c4be1757-istio-data\") pod \"e2e-api-a-15f6f2-openshift-default-7c9bf84768-lht27\" (UID: \"39af155e-9284-42e3-b5f4-d6f3c4be1757\") " pod="openshift-ingress/e2e-api-a-15f6f2-openshift-default-7c9bf84768-lht27"
Sep 11 17:42:30.230280 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:42:30.230143    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/39af155e-9284-42e3-b5f4-d6f3c4be1757-credential-socket\") pod \"e2e-api-a-15f6f2-openshift-default-7c9bf84768-lht27\" (UID: \"39af155e-9284-42e3-b5f4-d6f3c4be1757\") " pod="openshift-ingress/e2e-api-a-15f6f2-openshift-default-7c9bf84768-lht27"
Sep 11 17:42:30.230280 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:42:30.230213    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/39af155e-9284-42e3-b5f4-d6f3c4be1757-workload-socket\") pod \"e2e-api-a-15f6f2-openshift-default-7c9bf84768-lht27\" (UID: \"39af155e-9284-42e3-b5f4-d6f3c4be1757\") " pod="openshift-ingress/e2e-api-a-15f6f2-openshift-default-7c9bf84768-lht27"
Sep 11 17:42:30.230280 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:42:30.230243    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/39af155e-9284-42e3-b5f4-d6f3c4be1757-istiod-ca-cert\") pod \"e2e-api-a-15f6f2-openshift-default-7c9bf84768-lht27\" (UID: \"39af155e-9284-42e3-b5f4-d6f3c4be1757\") " pod="openshift-ingress/e2e-api-a-15f6f2-openshift-default-7c9bf84768-lht27"
Sep 11 17:42:30.230280 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:42:30.230275    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/39af155e-9284-42e3-b5f4-d6f3c4be1757-workload-certs\") pod \"e2e-api-a-15f6f2-openshift-default-7c9bf84768-lht27\" (UID: \"39af155e-9284-42e3-b5f4-d6f3c4be1757\") " pod="openshift-ingress/e2e-api-a-15f6f2-openshift-default-7c9bf84768-lht27"
Sep 11 17:42:30.331043 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:42:30.331013    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/39af155e-9284-42e3-b5f4-d6f3c4be1757-workload-socket\") pod \"e2e-api-a-15f6f2-openshift-default-7c9bf84768-lht27\" (UID: \"39af155e-9284-42e3-b5f4-d6f3c4be1757\") " pod="openshift-ingress/e2e-api-a-15f6f2-openshift-default-7c9bf84768-lht27"
Sep 11 17:42:30.331232 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:42:30.331051    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/39af155e-9284-42e3-b5f4-d6f3c4be1757-istiod-ca-cert\") pod \"e2e-api-a-15f6f2-openshift-default-7c9bf84768-lht27\" (UID: \"39af155e-9284-42e3-b5f4-d6f3c4be1757\") " pod="openshift-ingress/e2e-api-a-15f6f2-openshift-default-7c9bf84768-lht27"
Sep 11 17:42:30.331232 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:42:30.331078    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/39af155e-9284-42e3-b5f4-d6f3c4be1757-workload-certs\") pod \"e2e-api-a-15f6f2-openshift-default-7c9bf84768-lht27\" (UID: \"39af155e-9284-42e3-b5f4-d6f3c4be1757\") " pod="openshift-ingress/e2e-api-a-15f6f2-openshift-default-7c9bf84768-lht27"
Sep 11 17:42:30.331232 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:42:30.331177    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tvtg9\" (UniqueName: \"kubernetes.io/projected/39af155e-9284-42e3-b5f4-d6f3c4be1757-kube-api-access-tvtg9\") pod \"e2e-api-a-15f6f2-openshift-default-7c9bf84768-lht27\" (UID: \"39af155e-9284-42e3-b5f4-d6f3c4be1757\") " pod="openshift-ingress/e2e-api-a-15f6f2-openshift-default-7c9bf84768-lht27"
Sep 11 17:42:30.331355 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:42:30.331231    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/39af155e-9284-42e3-b5f4-d6f3c4be1757-istio-podinfo\") pod \"e2e-api-a-15f6f2-openshift-default-7c9bf84768-lht27\" (UID: \"39af155e-9284-42e3-b5f4-d6f3c4be1757\") " pod="openshift-ingress/e2e-api-a-15f6f2-openshift-default-7c9bf84768-lht27"
Sep 11 17:42:30.331355 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:42:30.331260    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/39af155e-9284-42e3-b5f4-d6f3c4be1757-istio-envoy\") pod \"e2e-api-a-15f6f2-openshift-default-7c9bf84768-lht27\" (UID: \"39af155e-9284-42e3-b5f4-d6f3c4be1757\") " pod="openshift-ingress/e2e-api-a-15f6f2-openshift-default-7c9bf84768-lht27"
Sep 11 17:42:30.331355 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:42:30.331298    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/39af155e-9284-42e3-b5f4-d6f3c4be1757-istio-token\") pod \"e2e-api-a-15f6f2-openshift-default-7c9bf84768-lht27\" (UID: \"39af155e-9284-42e3-b5f4-d6f3c4be1757\") " pod="openshift-ingress/e2e-api-a-15f6f2-openshift-default-7c9bf84768-lht27"
Sep 11 17:42:30.331474 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:42:30.331362    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/39af155e-9284-42e3-b5f4-d6f3c4be1757-istio-data\") pod \"e2e-api-a-15f6f2-openshift-default-7c9bf84768-lht27\" (UID: \"39af155e-9284-42e3-b5f4-d6f3c4be1757\") " pod="openshift-ingress/e2e-api-a-15f6f2-openshift-default-7c9bf84768-lht27"
Sep 11 17:42:30.331474 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:42:30.331398    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/39af155e-9284-42e3-b5f4-d6f3c4be1757-credential-socket\") pod \"e2e-api-a-15f6f2-openshift-default-7c9bf84768-lht27\" (UID: \"39af155e-9284-42e3-b5f4-d6f3c4be1757\") " pod="openshift-ingress/e2e-api-a-15f6f2-openshift-default-7c9bf84768-lht27"
Sep 11 17:42:30.331583 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:42:30.331488    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/39af155e-9284-42e3-b5f4-d6f3c4be1757-workload-socket\") pod \"e2e-api-a-15f6f2-openshift-default-7c9bf84768-lht27\" (UID: \"39af155e-9284-42e3-b5f4-d6f3c4be1757\") " pod="openshift-ingress/e2e-api-a-15f6f2-openshift-default-7c9bf84768-lht27"
Sep 11 17:42:30.331673 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:42:30.331648    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/39af155e-9284-42e3-b5f4-d6f3c4be1757-workload-certs\") pod \"e2e-api-a-15f6f2-openshift-default-7c9bf84768-lht27\" (UID: \"39af155e-9284-42e3-b5f4-d6f3c4be1757\") " pod="openshift-ingress/e2e-api-a-15f6f2-openshift-default-7c9bf84768-lht27"
Sep 11 17:42:30.331881 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:42:30.331845    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/39af155e-9284-42e3-b5f4-d6f3c4be1757-credential-socket\") pod \"e2e-api-a-15f6f2-openshift-default-7c9bf84768-lht27\" (UID: \"39af155e-9284-42e3-b5f4-d6f3c4be1757\") " pod="openshift-ingress/e2e-api-a-15f6f2-openshift-default-7c9bf84768-lht27"
Sep 11 17:42:30.331978 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:42:30.331891    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/39af155e-9284-42e3-b5f4-d6f3c4be1757-istio-data\") pod \"e2e-api-a-15f6f2-openshift-default-7c9bf84768-lht27\" (UID: \"39af155e-9284-42e3-b5f4-d6f3c4be1757\") " pod="openshift-ingress/e2e-api-a-15f6f2-openshift-default-7c9bf84768-lht27"
Sep 11 17:42:30.331978 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:42:30.331902    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/39af155e-9284-42e3-b5f4-d6f3c4be1757-istiod-ca-cert\") pod \"e2e-api-a-15f6f2-openshift-default-7c9bf84768-lht27\" (UID: \"39af155e-9284-42e3-b5f4-d6f3c4be1757\") " pod="openshift-ingress/e2e-api-a-15f6f2-openshift-default-7c9bf84768-lht27"
Sep 11 17:42:30.333754 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:42:30.333734    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/39af155e-9284-42e3-b5f4-d6f3c4be1757-istio-envoy\") pod \"e2e-api-a-15f6f2-openshift-default-7c9bf84768-lht27\" (UID: \"39af155e-9284-42e3-b5f4-d6f3c4be1757\") " pod="openshift-ingress/e2e-api-a-15f6f2-openshift-default-7c9bf84768-lht27"
Sep 11 17:42:30.333907 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:42:30.333872    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/39af155e-9284-42e3-b5f4-d6f3c4be1757-istio-podinfo\") pod \"e2e-api-a-15f6f2-openshift-default-7c9bf84768-lht27\" (UID: \"39af155e-9284-42e3-b5f4-d6f3c4be1757\") " pod="openshift-ingress/e2e-api-a-15f6f2-openshift-default-7c9bf84768-lht27"
Sep 11 17:42:30.340124 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:42:30.340079    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tvtg9\" (UniqueName: \"kubernetes.io/projected/39af155e-9284-42e3-b5f4-d6f3c4be1757-kube-api-access-tvtg9\") pod \"e2e-api-a-15f6f2-openshift-default-7c9bf84768-lht27\" (UID: \"39af155e-9284-42e3-b5f4-d6f3c4be1757\") " pod="openshift-ingress/e2e-api-a-15f6f2-openshift-default-7c9bf84768-lht27"
Sep 11 17:42:30.340245 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:42:30.340183    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/39af155e-9284-42e3-b5f4-d6f3c4be1757-istio-token\") pod \"e2e-api-a-15f6f2-openshift-default-7c9bf84768-lht27\" (UID: \"39af155e-9284-42e3-b5f4-d6f3c4be1757\") " pod="openshift-ingress/e2e-api-a-15f6f2-openshift-default-7c9bf84768-lht27"
Sep 11 17:42:30.401642 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:42:30.401616    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-15f6f2-openshift-default-7c9bf84768-lht27"
Sep 11 17:42:30.636813 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:42:30.636720    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:42:30.686728 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:42:30.686694    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-15f6f2-openshift-default-7c9bf84768-lht27"]
Sep 11 17:42:30.687035 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:42:30.687011    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39af155e_9284_42e3_b5f4_d6f3c4be1757.slice/crio-2acbc433b43533ad33b805362f28325cefc82ab3b51085cda4cb388d25a8d8de WatchSource:0}: Error finding container 2acbc433b43533ad33b805362f28325cefc82ab3b51085cda4cb388d25a8d8de: Status 404 returned error can't find the container with id 2acbc433b43533ad33b805362f28325cefc82ab3b51085cda4cb388d25a8d8de
Sep 11 17:42:30.693193 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:42:30.693158    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 17:42:30.693281 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:42:30.693238    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 17:42:30.693322 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:42:30.693279    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 17:42:30.712686 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:42:30.712660    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-15f6f2-openshift-default-7c9bf84768-lht27" event={"ID":"39af155e-9284-42e3-b5f4-d6f3c4be1757","Type":"ContainerStarted","Data":"2acbc433b43533ad33b805362f28325cefc82ab3b51085cda4cb388d25a8d8de"}
Sep 11 17:42:31.727746 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:42:31.727708    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-15f6f2-openshift-default-7c9bf84768-lht27" event={"ID":"39af155e-9284-42e3-b5f4-d6f3c4be1757","Type":"ContainerStarted","Data":"3ca0b0285a89f72db6c8e6b2a5cdbbe5a21e418e6a6e266c5ca54516ea5d90ad"}
Sep 11 17:42:31.752732 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:42:31.752681    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-15f6f2-openshift-default-7c9bf84768-lht27" podStartSLOduration=1.752664011 podStartE2EDuration="1.752664011s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:42:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:42:31.750299301 +0000 UTC m=+1446.271590860" watchObservedRunningTime="2026-09-11 17:42:31.752664011 +0000 UTC m=+1446.273955532"
Sep 11 17:42:32.402594 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:42:32.402556    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-15f6f2-openshift-default-7c9bf84768-lht27"
Sep 11 17:42:32.407597 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:42:32.407574    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-15f6f2-openshift-default-7c9bf84768-lht27"
Sep 11 17:42:32.733983 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:42:32.733892    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-15f6f2-openshift-default-7c9bf84768-lht27"
Sep 11 17:42:32.734947 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:42:32.734925    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-15f6f2-openshift-default-7c9bf84768-lht27"
Sep 11 17:42:33.122694 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:42:33.122638    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:42:33.706331 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:42:33.706265    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:42:33.815818 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:42:33.815778    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:42:34.225418 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:42:34.225373    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:42:35.151597 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:42:35.151563    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:42:36.075494 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:42:36.075460    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:42:37.025432 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:42:37.025402    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:42:37.950350 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:42:37.950311    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:42:38.740628 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:42:38.740589    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:42:39.706694 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:42:39.706660    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-15f6f2-f9df89b46-4mhkh"]
Sep 11 17:42:39.711464 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:42:39.711391    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-15f6f2-f9df89b46-4mhkh"
Sep 11 17:42:39.713559 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:42:39.713536    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-15f6f2-serving-cert\""
Sep 11 17:42:39.725661 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:42:39.725639    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-15f6f2-f9df89b46-4mhkh"]
Sep 11 17:42:39.727264 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:42:39.727239    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f4b1c65a-dc4e-40f4-8b6e-799eb9479bc5-metrics-certs\") pod \"maas-api-e2e-api-a-15f6f2-f9df89b46-4mhkh\" (UID: \"f4b1c65a-dc4e-40f4-8b6e-799eb9479bc5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-15f6f2-f9df89b46-4mhkh"
Sep 11 17:42:39.727394 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:42:39.727278    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-whl95\" (UniqueName: \"kubernetes.io/projected/f4b1c65a-dc4e-40f4-8b6e-799eb9479bc5-kube-api-access-whl95\") pod \"maas-api-e2e-api-a-15f6f2-f9df89b46-4mhkh\" (UID: \"f4b1c65a-dc4e-40f4-8b6e-799eb9479bc5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-15f6f2-f9df89b46-4mhkh"
Sep 11 17:42:39.727394 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:42:39.727307    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f4b1c65a-dc4e-40f4-8b6e-799eb9479bc5-maas-api-tls\") pod \"maas-api-e2e-api-a-15f6f2-f9df89b46-4mhkh\" (UID: \"f4b1c65a-dc4e-40f4-8b6e-799eb9479bc5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-15f6f2-f9df89b46-4mhkh"
Sep 11 17:42:39.727516 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:42:39.727469    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f4b1c65a-dc4e-40f4-8b6e-799eb9479bc5-openshift-service-ca\") pod \"maas-api-e2e-api-a-15f6f2-f9df89b46-4mhkh\" (UID: \"f4b1c65a-dc4e-40f4-8b6e-799eb9479bc5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-15f6f2-f9df89b46-4mhkh"
Sep 11 17:42:39.748039 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:42:39.748015    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:42:39.829893 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:42:39.829854    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f4b1c65a-dc4e-40f4-8b6e-799eb9479bc5-openshift-service-ca\") pod \"maas-api-e2e-api-a-15f6f2-f9df89b46-4mhkh\" (UID: \"f4b1c65a-dc4e-40f4-8b6e-799eb9479bc5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-15f6f2-f9df89b46-4mhkh"
Sep 11 17:42:39.830193 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:42:39.829937    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f4b1c65a-dc4e-40f4-8b6e-799eb9479bc5-metrics-certs\") pod \"maas-api-e2e-api-a-15f6f2-f9df89b46-4mhkh\" (UID: \"f4b1c65a-dc4e-40f4-8b6e-799eb9479bc5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-15f6f2-f9df89b46-4mhkh"
Sep 11 17:42:39.830193 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:42:39.829970    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-whl95\" (UniqueName: \"kubernetes.io/projected/f4b1c65a-dc4e-40f4-8b6e-799eb9479bc5-kube-api-access-whl95\") pod \"maas-api-e2e-api-a-15f6f2-f9df89b46-4mhkh\" (UID: \"f4b1c65a-dc4e-40f4-8b6e-799eb9479bc5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-15f6f2-f9df89b46-4mhkh"
Sep 11 17:42:39.830193 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:42:39.830001    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f4b1c65a-dc4e-40f4-8b6e-799eb9479bc5-maas-api-tls\") pod \"maas-api-e2e-api-a-15f6f2-f9df89b46-4mhkh\" (UID: \"f4b1c65a-dc4e-40f4-8b6e-799eb9479bc5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-15f6f2-f9df89b46-4mhkh"
Sep 11 17:42:39.830452 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:42:39.830216    2552 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-15f6f2-serving-cert: secret "maas-api-e2e-api-a-15f6f2-serving-cert" not found
Sep 11 17:42:39.830452 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:42:39.830294    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f4b1c65a-dc4e-40f4-8b6e-799eb9479bc5-maas-api-tls podName:f4b1c65a-dc4e-40f4-8b6e-799eb9479bc5 nodeName:}" failed. No retries permitted until 2026-09-11 17:42:40.3302727 +0000 UTC m=+1454.851564218 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/f4b1c65a-dc4e-40f4-8b6e-799eb9479bc5-maas-api-tls") pod "maas-api-e2e-api-a-15f6f2-f9df89b46-4mhkh" (UID: "f4b1c65a-dc4e-40f4-8b6e-799eb9479bc5") : secret "maas-api-e2e-api-a-15f6f2-serving-cert" not found
Sep 11 17:42:39.831675 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:42:39.831382    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f4b1c65a-dc4e-40f4-8b6e-799eb9479bc5-openshift-service-ca\") pod \"maas-api-e2e-api-a-15f6f2-f9df89b46-4mhkh\" (UID: \"f4b1c65a-dc4e-40f4-8b6e-799eb9479bc5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-15f6f2-f9df89b46-4mhkh"
Sep 11 17:42:39.833930 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:42:39.833904    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f4b1c65a-dc4e-40f4-8b6e-799eb9479bc5-metrics-certs\") pod \"maas-api-e2e-api-a-15f6f2-f9df89b46-4mhkh\" (UID: \"f4b1c65a-dc4e-40f4-8b6e-799eb9479bc5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-15f6f2-f9df89b46-4mhkh"
Sep 11 17:42:39.841508 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:42:39.841484    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-whl95\" (UniqueName: \"kubernetes.io/projected/f4b1c65a-dc4e-40f4-8b6e-799eb9479bc5-kube-api-access-whl95\") pod \"maas-api-e2e-api-a-15f6f2-f9df89b46-4mhkh\" (UID: \"f4b1c65a-dc4e-40f4-8b6e-799eb9479bc5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-15f6f2-f9df89b46-4mhkh"
Sep 11 17:42:39.848219 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:42:39.848184    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:42:40.008200 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:42:40.008064    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:42:40.335597 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:42:40.335379    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f4b1c65a-dc4e-40f4-8b6e-799eb9479bc5-maas-api-tls\") pod \"maas-api-e2e-api-a-15f6f2-f9df89b46-4mhkh\" (UID: \"f4b1c65a-dc4e-40f4-8b6e-799eb9479bc5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-15f6f2-f9df89b46-4mhkh"
Sep 11 17:42:40.338321 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:42:40.338294    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f4b1c65a-dc4e-40f4-8b6e-799eb9479bc5-maas-api-tls\") pod \"maas-api-e2e-api-a-15f6f2-f9df89b46-4mhkh\" (UID: \"f4b1c65a-dc4e-40f4-8b6e-799eb9479bc5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-15f6f2-f9df89b46-4mhkh"
Sep 11 17:42:40.375627 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:42:40.375601    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:42:40.473225 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:42:40.473195    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:42:40.623661 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:42:40.623584    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-15f6f2-f9df89b46-4mhkh"
Sep 11 17:42:40.907225 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:42:40.907195    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-15f6f2-f9df89b46-4mhkh"]
Sep 11 17:42:40.908197 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:42:40.908172    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4b1c65a_dc4e_40f4_8b6e_799eb9479bc5.slice/crio-1b939d99a47e71375aa68af3143b363890ef3455f31f4f964cb6a1de9afcb251 WatchSource:0}: Error finding container 1b939d99a47e71375aa68af3143b363890ef3455f31f4f964cb6a1de9afcb251: Status 404 returned error can't find the container with id 1b939d99a47e71375aa68af3143b363890ef3455f31f4f964cb6a1de9afcb251
Sep 11 17:42:41.791902 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:42:41.791865    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-15f6f2-f9df89b46-4mhkh" event={"ID":"f4b1c65a-dc4e-40f4-8b6e-799eb9479bc5","Type":"ContainerStarted","Data":"ac435397d1330864997ca6f1112e9a82e583c70dd86ef73049e298da09e52ba0"}
Sep 11 17:42:41.792098 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:42:41.791924    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-15f6f2-f9df89b46-4mhkh" event={"ID":"f4b1c65a-dc4e-40f4-8b6e-799eb9479bc5","Type":"ContainerStarted","Data":"1b939d99a47e71375aa68af3143b363890ef3455f31f4f964cb6a1de9afcb251"}
Sep 11 17:42:41.792098 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:42:41.791951    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-15f6f2-f9df89b46-4mhkh"
Sep 11 17:42:41.812612 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:42:41.812564    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-15f6f2-f9df89b46-4mhkh" podStartSLOduration=2.384159522 podStartE2EDuration="2.812549355s" totalImagesPullingTime="428.389833ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:42:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:42:41.810845149 +0000 UTC m=+1456.332136670" watchObservedRunningTime="2026-09-11 17:42:41.812549355 +0000 UTC m=+1456.333840875"
Sep 11 17:42:47.802875 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:42:47.802849    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-15f6f2-f9df89b46-4mhkh"
Sep 11 17:42:50.458450 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:42:50.458414    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:43:00.886160 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:43:00.886102    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:43:02.366979 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:43:02.366943    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:43:03.663434 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:43:03.663384    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:43:03.922820 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:43:03.922735    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:43:04.333092 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:43:04.332999    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:43:04.811917 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:43:04.811885    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:43:05.116134 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:43:05.116075    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:43:06.135671 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:43:06.135638    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:43:06.943611 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:43:06.943577    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:43:07.948485 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:43:07.948446    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:43:08.120838 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:43:08.120800    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:43:08.242995 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:43:08.242903    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:43:09.051094 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:43:09.051054    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:43:10.179624 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:43:10.179583    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:43:10.945735 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:43:10.945697    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:43:11.682554 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:43:11.682512    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-83a2bc-785c75d74b-p5pqd"]
Sep 11 17:43:11.688330 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:43:11.688302    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-83a2bc-785c75d74b-p5pqd"
Sep 11 17:43:11.690681 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:43:11.690655    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-83a2bc-serving-cert\""
Sep 11 17:43:11.697003 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:43:11.696981    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-83a2bc-785c75d74b-p5pqd"]
Sep 11 17:43:11.718789 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:43:11.718758    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:43:11.761450 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:43:11.761412    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e4f646e8-e7dc-4a53-9385-cb391d0e658d-metrics-certs\") pod \"maas-api-e2e-api-b-83a2bc-785c75d74b-p5pqd\" (UID: \"e4f646e8-e7dc-4a53-9385-cb391d0e658d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-83a2bc-785c75d74b-p5pqd"
Sep 11 17:43:11.761659 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:43:11.761487    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e4f646e8-e7dc-4a53-9385-cb391d0e658d-maas-api-tls\") pod \"maas-api-e2e-api-b-83a2bc-785c75d74b-p5pqd\" (UID: \"e4f646e8-e7dc-4a53-9385-cb391d0e658d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-83a2bc-785c75d74b-p5pqd"
Sep 11 17:43:11.761659 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:43:11.761605    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e4f646e8-e7dc-4a53-9385-cb391d0e658d-openshift-service-ca\") pod \"maas-api-e2e-api-b-83a2bc-785c75d74b-p5pqd\" (UID: \"e4f646e8-e7dc-4a53-9385-cb391d0e658d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-83a2bc-785c75d74b-p5pqd"
Sep 11 17:43:11.761659 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:43:11.761632    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zn8cg\" (UniqueName: \"kubernetes.io/projected/e4f646e8-e7dc-4a53-9385-cb391d0e658d-kube-api-access-zn8cg\") pod \"maas-api-e2e-api-b-83a2bc-785c75d74b-p5pqd\" (UID: \"e4f646e8-e7dc-4a53-9385-cb391d0e658d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-83a2bc-785c75d74b-p5pqd"
Sep 11 17:43:11.821368 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:43:11.821340    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:43:11.862289 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:43:11.862245    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e4f646e8-e7dc-4a53-9385-cb391d0e658d-openshift-service-ca\") pod \"maas-api-e2e-api-b-83a2bc-785c75d74b-p5pqd\" (UID: \"e4f646e8-e7dc-4a53-9385-cb391d0e658d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-83a2bc-785c75d74b-p5pqd"
Sep 11 17:43:11.862289 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:43:11.862290    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zn8cg\" (UniqueName: \"kubernetes.io/projected/e4f646e8-e7dc-4a53-9385-cb391d0e658d-kube-api-access-zn8cg\") pod \"maas-api-e2e-api-b-83a2bc-785c75d74b-p5pqd\" (UID: \"e4f646e8-e7dc-4a53-9385-cb391d0e658d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-83a2bc-785c75d74b-p5pqd"
Sep 11 17:43:11.862554 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:43:11.862326    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e4f646e8-e7dc-4a53-9385-cb391d0e658d-metrics-certs\") pod \"maas-api-e2e-api-b-83a2bc-785c75d74b-p5pqd\" (UID: \"e4f646e8-e7dc-4a53-9385-cb391d0e658d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-83a2bc-785c75d74b-p5pqd"
Sep 11 17:43:11.862554 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:43:11.862390    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e4f646e8-e7dc-4a53-9385-cb391d0e658d-maas-api-tls\") pod \"maas-api-e2e-api-b-83a2bc-785c75d74b-p5pqd\" (UID: \"e4f646e8-e7dc-4a53-9385-cb391d0e658d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-83a2bc-785c75d74b-p5pqd"
Sep 11 17:43:11.862683 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:43:11.862558    2552 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-83a2bc-serving-cert: secret "maas-api-e2e-api-b-83a2bc-serving-cert" not found
Sep 11 17:43:11.862683 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:43:11.862632    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e4f646e8-e7dc-4a53-9385-cb391d0e658d-maas-api-tls podName:e4f646e8-e7dc-4a53-9385-cb391d0e658d nodeName:}" failed. No retries permitted until 2026-09-11 17:43:12.362610905 +0000 UTC m=+1486.883902408 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e4f646e8-e7dc-4a53-9385-cb391d0e658d-maas-api-tls") pod "maas-api-e2e-api-b-83a2bc-785c75d74b-p5pqd" (UID: "e4f646e8-e7dc-4a53-9385-cb391d0e658d") : secret "maas-api-e2e-api-b-83a2bc-serving-cert" not found
Sep 11 17:43:11.863647 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:43:11.863620    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e4f646e8-e7dc-4a53-9385-cb391d0e658d-openshift-service-ca\") pod \"maas-api-e2e-api-b-83a2bc-785c75d74b-p5pqd\" (UID: \"e4f646e8-e7dc-4a53-9385-cb391d0e658d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-83a2bc-785c75d74b-p5pqd"
Sep 11 17:43:11.865762 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:43:11.865737    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e4f646e8-e7dc-4a53-9385-cb391d0e658d-metrics-certs\") pod \"maas-api-e2e-api-b-83a2bc-785c75d74b-p5pqd\" (UID: \"e4f646e8-e7dc-4a53-9385-cb391d0e658d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-83a2bc-785c75d74b-p5pqd"
Sep 11 17:43:11.878460 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:43:11.878432    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zn8cg\" (UniqueName: \"kubernetes.io/projected/e4f646e8-e7dc-4a53-9385-cb391d0e658d-kube-api-access-zn8cg\") pod \"maas-api-e2e-api-b-83a2bc-785c75d74b-p5pqd\" (UID: \"e4f646e8-e7dc-4a53-9385-cb391d0e658d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-83a2bc-785c75d74b-p5pqd"
Sep 11 17:43:12.181304 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:43:12.181266    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:43:12.335022 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:43:12.334978    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:43:12.369459 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:43:12.368840    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e4f646e8-e7dc-4a53-9385-cb391d0e658d-maas-api-tls\") pod \"maas-api-e2e-api-b-83a2bc-785c75d74b-p5pqd\" (UID: \"e4f646e8-e7dc-4a53-9385-cb391d0e658d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-83a2bc-785c75d74b-p5pqd"
Sep 11 17:43:12.373311 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:43:12.373277    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e4f646e8-e7dc-4a53-9385-cb391d0e658d-maas-api-tls\") pod \"maas-api-e2e-api-b-83a2bc-785c75d74b-p5pqd\" (UID: \"e4f646e8-e7dc-4a53-9385-cb391d0e658d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-83a2bc-785c75d74b-p5pqd"
Sep 11 17:43:12.431607 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:43:12.431527    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:43:12.603967 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:43:12.603935    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-83a2bc-785c75d74b-p5pqd"
Sep 11 17:43:12.852044 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:43:12.851964    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:43:12.905659 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:43:12.905636    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-83a2bc-785c75d74b-p5pqd"]
Sep 11 17:43:12.908018 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:43:12.907985    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4f646e8_e7dc_4a53_9385_cb391d0e658d.slice/crio-ff4359e25ed59e8999579c232443d6df5bc548ed3be10212cab8e62ce4b959eb WatchSource:0}: Error finding container ff4359e25ed59e8999579c232443d6df5bc548ed3be10212cab8e62ce4b959eb: Status 404 returned error can't find the container with id ff4359e25ed59e8999579c232443d6df5bc548ed3be10212cab8e62ce4b959eb
Sep 11 17:43:13.000854 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:43:13.000772    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-83a2bc-785c75d74b-p5pqd" event={"ID":"e4f646e8-e7dc-4a53-9385-cb391d0e658d","Type":"ContainerStarted","Data":"ff4359e25ed59e8999579c232443d6df5bc548ed3be10212cab8e62ce4b959eb"}
Sep 11 17:43:13.726020 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:43:13.725983    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:43:14.009145 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:43:14.009030    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-83a2bc-785c75d74b-p5pqd" event={"ID":"e4f646e8-e7dc-4a53-9385-cb391d0e658d","Type":"ContainerStarted","Data":"93537275e4ace6e9424d2d9c8029a3fd779bcd933233b6267529ce8916e57919"}
Sep 11 17:43:14.009529 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:43:14.009167    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-83a2bc-785c75d74b-p5pqd"
Sep 11 17:43:14.026807 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:43:14.026762    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-83a2bc-785c75d74b-p5pqd" podStartSLOduration=2.653739948 podStartE2EDuration="3.026747829s" totalImagesPullingTime="373.007881ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:43:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:43:14.026204073 +0000 UTC m=+1488.547495596" watchObservedRunningTime="2026-09-11 17:43:14.026747829 +0000 UTC m=+1488.548039350"
Sep 11 17:43:14.544260 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:43:14.544222    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:43:15.252511 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:43:15.252467    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:43:15.970992 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:43:15.970954    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:43:18.774337 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:43:18.774284    2552 patch_prober.go:28] interesting pod/e2e-api-a-15f6f2-openshift-default-7c9bf84768-lht27 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 11 17:43:18.774807 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:43:18.774354    2552 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-15f6f2-openshift-default-7c9bf84768-lht27" podUID="39af155e-9284-42e3-b5f4-d6f3c4be1757" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 11 17:43:20.020952 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:43:20.020872    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-83a2bc-785c75d74b-p5pqd"
Sep 11 17:43:20.050253 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:43:20.050221    2552 generic.go:356] "Generic (PLEG): container finished" podID="39af155e-9284-42e3-b5f4-d6f3c4be1757" containerID="3ca0b0285a89f72db6c8e6b2a5cdbbe5a21e418e6a6e266c5ca54516ea5d90ad" exitCode=0
Sep 11 17:43:20.050432 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:43:20.050307    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-15f6f2-openshift-default-7c9bf84768-lht27" event={"ID":"39af155e-9284-42e3-b5f4-d6f3c4be1757","Type":"ContainerDied","Data":"3ca0b0285a89f72db6c8e6b2a5cdbbe5a21e418e6a6e266c5ca54516ea5d90ad"}
Sep 11 17:43:20.050847 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:43:20.050829    2552 scope.go:117] "RemoveContainer" containerID="3ca0b0285a89f72db6c8e6b2a5cdbbe5a21e418e6a6e266c5ca54516ea5d90ad"
Sep 11 17:43:20.052430 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:43:20.052397    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 17:43:20.052532 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:43:20.052510    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 17:43:20.052592 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:43:20.052579    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 17:43:21.064085 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:43:21.064046    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-15f6f2-openshift-default-7c9bf84768-lht27" event={"ID":"39af155e-9284-42e3-b5f4-d6f3c4be1757","Type":"ContainerStarted","Data":"c2ca97b271ea5dc160eadeab8e4194f09365c73e4d4f7b76712087d55cade4f7"}
Sep 11 17:43:21.402390 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:43:21.402330    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-15f6f2-openshift-default-7c9bf84768-lht27"
Sep 11 17:43:21.402670 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:43:21.402645    2552 patch_prober.go:28] interesting pod/e2e-api-a-15f6f2-openshift-default-7c9bf84768-lht27 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" start-of-body=
Sep 11 17:43:21.402731 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:43:21.402711    2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-15f6f2-openshift-default-7c9bf84768-lht27" podUID="39af155e-9284-42e3-b5f4-d6f3c4be1757" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused"
Sep 11 17:43:22.402181 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:43:22.402140    2552 patch_prober.go:28] interesting pod/e2e-api-a-15f6f2-openshift-default-7c9bf84768-lht27 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" start-of-body=
Sep 11 17:43:22.402577 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:43:22.402211    2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-15f6f2-openshift-default-7c9bf84768-lht27" podUID="39af155e-9284-42e3-b5f4-d6f3c4be1757" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused"
Sep 11 17:43:23.402067 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:43:23.402019    2552 patch_prober.go:28] interesting pod/e2e-api-a-15f6f2-openshift-default-7c9bf84768-lht27 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" start-of-body=
Sep 11 17:43:23.402276 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:43:23.402089    2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-15f6f2-openshift-default-7c9bf84768-lht27" podUID="39af155e-9284-42e3-b5f4-d6f3c4be1757" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused"
Sep 11 17:43:24.402412 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:43:24.402376    2552 patch_prober.go:28] interesting pod/e2e-api-a-15f6f2-openshift-default-7c9bf84768-lht27 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" start-of-body=
Sep 11 17:43:24.402799 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:43:24.402444    2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-15f6f2-openshift-default-7c9bf84768-lht27" podUID="39af155e-9284-42e3-b5f4-d6f3c4be1757" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused"
Sep 11 17:43:25.402493 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:43:25.402459    2552 patch_prober.go:28] interesting pod/e2e-api-a-15f6f2-openshift-default-7c9bf84768-lht27 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" start-of-body=
Sep 11 17:43:25.402920 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:43:25.402818    2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-15f6f2-openshift-default-7c9bf84768-lht27" podUID="39af155e-9284-42e3-b5f4-d6f3c4be1757" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused"
Sep 11 17:43:26.065213 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:43:26.065184    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-hg7p8_eb5f3be1-2e36-40ba-b3a0-3bec413056e8/console-operator/1.log"
Sep 11 17:43:26.071063 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:43:26.071042    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6k9tr_0620c0a0-8070-4307-86d9-c69a76ea14a9/ovn-acl-logging/0.log"
Sep 11 17:43:26.075719 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:43:26.075704    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-hg7p8_eb5f3be1-2e36-40ba-b3a0-3bec413056e8/console-operator/1.log"
Sep 11 17:43:26.079078 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:43:26.079060    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6k9tr_0620c0a0-8070-4307-86d9-c69a76ea14a9/ovn-acl-logging/0.log"
Sep 11 17:43:26.402316 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:43:26.402281    2552 patch_prober.go:28] interesting pod/e2e-api-a-15f6f2-openshift-default-7c9bf84768-lht27 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" start-of-body=
Sep 11 17:43:26.402563 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:43:26.402335    2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-15f6f2-openshift-default-7c9bf84768-lht27" podUID="39af155e-9284-42e3-b5f4-d6f3c4be1757" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused"
Sep 11 17:43:27.405844 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:43:27.405814    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-15f6f2-openshift-default-7c9bf84768-lht27"
Sep 11 17:43:27.406245 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:43:27.406106    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-15f6f2-openshift-default-7c9bf84768-lht27"
Sep 11 17:43:27.406828 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:43:27.406811    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-15f6f2-openshift-default-7c9bf84768-lht27"
Sep 11 17:43:34.517582 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:43:34.517550    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:43:35.610406 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:43:35.610372    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:43:36.418615 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:43:36.418578    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:43:37.553046 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:43:37.553012    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:43:38.091685 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:43:38.091636    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:43:38.501862 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:43:38.501741    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:43:39.679202 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:43:39.679169    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:43:40.588331 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:43:40.588258    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:43:41.570079 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:43:41.570042    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:43:42.456150 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:43:42.456090    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:43:43.497418 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:43:43.497384    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:43:44.431774 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:43:44.431738    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:43:45.478657 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:43:45.478618    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:43:46.331324 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:43:46.331291    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:43:47.290324 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:43:47.290291    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:43:48.225766 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:43:48.225677    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:43:49.341820 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:43:49.341783    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:43:53.269565 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:43:53.269532    2552 generic.go:356] "Generic (PLEG): container finished" podID="3ebd01e5-f712-42b8-9e89-deff14ec577a" containerID="18dcd4d7b55049a0bcd1aea91d80e7b4a1f7409a1631ba55d4369637dab0bbdb" exitCode=137
Sep 11 17:43:53.270010 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:43:53.269606    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-fzs45" event={"ID":"3ebd01e5-f712-42b8-9e89-deff14ec577a","Type":"ContainerDied","Data":"18dcd4d7b55049a0bcd1aea91d80e7b4a1f7409a1631ba55d4369637dab0bbdb"}
Sep 11 17:43:53.270010 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:43:53.269946    2552 scope.go:117] "RemoveContainer" containerID="18dcd4d7b55049a0bcd1aea91d80e7b4a1f7409a1631ba55d4369637dab0bbdb"
Sep 11 17:43:54.277477 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:43:54.277448    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-fzs45" event={"ID":"3ebd01e5-f712-42b8-9e89-deff14ec577a","Type":"ContainerStarted","Data":"abed4f1ed03e5825bf40ce82670fbe15fcf82e5905d1419576c22e170d5104bb"}
Sep 11 17:43:54.277845 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:43:54.277655    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-fzs45"
Sep 11 17:43:56.543255 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:43:56.543216    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-15f6f2-f9df89b46-4mhkh"]
Sep 11 17:43:56.543712 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:43:56.543447    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-15f6f2-f9df89b46-4mhkh" podUID="f4b1c65a-dc4e-40f4-8b6e-799eb9479bc5" containerName="maas-api" containerID="cri-o://ac435397d1330864997ca6f1112e9a82e583c70dd86ef73049e298da09e52ba0" gracePeriod=30
Sep 11 17:43:56.581058 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:43:56.581032    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:43:56.704925 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:43:56.704886    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:43:56.791434 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:43:56.791407    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-15f6f2-f9df89b46-4mhkh"
Sep 11 17:43:56.803480 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:43:56.803400    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:43:56.827923 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:43:56.827893    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f4b1c65a-dc4e-40f4-8b6e-799eb9479bc5-openshift-service-ca\") pod \"f4b1c65a-dc4e-40f4-8b6e-799eb9479bc5\" (UID: \"f4b1c65a-dc4e-40f4-8b6e-799eb9479bc5\") "
Sep 11 17:43:56.828061 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:43:56.827960    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f4b1c65a-dc4e-40f4-8b6e-799eb9479bc5-metrics-certs\") pod \"f4b1c65a-dc4e-40f4-8b6e-799eb9479bc5\" (UID: \"f4b1c65a-dc4e-40f4-8b6e-799eb9479bc5\") "
Sep 11 17:43:56.828061 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:43:56.828058    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f4b1c65a-dc4e-40f4-8b6e-799eb9479bc5-maas-api-tls\") pod \"f4b1c65a-dc4e-40f4-8b6e-799eb9479bc5\" (UID: \"f4b1c65a-dc4e-40f4-8b6e-799eb9479bc5\") "
Sep 11 17:43:56.828207 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:43:56.828101    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-whl95\" (UniqueName: \"kubernetes.io/projected/f4b1c65a-dc4e-40f4-8b6e-799eb9479bc5-kube-api-access-whl95\") pod \"f4b1c65a-dc4e-40f4-8b6e-799eb9479bc5\" (UID: \"f4b1c65a-dc4e-40f4-8b6e-799eb9479bc5\") "
Sep 11 17:43:56.828301 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:43:56.828279    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f4b1c65a-dc4e-40f4-8b6e-799eb9479bc5-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f4b1c65a-dc4e-40f4-8b6e-799eb9479bc5" (UID: "f4b1c65a-dc4e-40f4-8b6e-799eb9479bc5"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 17:43:56.828539 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:43:56.828517    2552 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f4b1c65a-dc4e-40f4-8b6e-799eb9479bc5-openshift-service-ca\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:43:56.830127 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:43:56.830094    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f4b1c65a-dc4e-40f4-8b6e-799eb9479bc5-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f4b1c65a-dc4e-40f4-8b6e-799eb9479bc5" (UID: "f4b1c65a-dc4e-40f4-8b6e-799eb9479bc5"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 17:43:56.830199 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:43:56.830172    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f4b1c65a-dc4e-40f4-8b6e-799eb9479bc5-kube-api-access-whl95" (OuterVolumeSpecName: "kube-api-access-whl95") pod "f4b1c65a-dc4e-40f4-8b6e-799eb9479bc5" (UID: "f4b1c65a-dc4e-40f4-8b6e-799eb9479bc5"). InnerVolumeSpecName "kube-api-access-whl95". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:43:56.830416 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:43:56.830402    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f4b1c65a-dc4e-40f4-8b6e-799eb9479bc5-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "f4b1c65a-dc4e-40f4-8b6e-799eb9479bc5" (UID: "f4b1c65a-dc4e-40f4-8b6e-799eb9479bc5"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 17:43:56.886286 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:43:56.886254    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:43:56.929621 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:43:56.929591    2552 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f4b1c65a-dc4e-40f4-8b6e-799eb9479bc5-metrics-certs\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:43:56.929621 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:43:56.929617    2552 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f4b1c65a-dc4e-40f4-8b6e-799eb9479bc5-maas-api-tls\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:43:56.929765 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:43:56.929626    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-whl95\" (UniqueName: \"kubernetes.io/projected/f4b1c65a-dc4e-40f4-8b6e-799eb9479bc5-kube-api-access-whl95\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:43:56.978971 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:43:56.978884    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:43:57.243282 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:43:57.243246    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:43:57.296098 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:43:57.296070    2552 generic.go:356] "Generic (PLEG): container finished" podID="f4b1c65a-dc4e-40f4-8b6e-799eb9479bc5" containerID="ac435397d1330864997ca6f1112e9a82e583c70dd86ef73049e298da09e52ba0" exitCode=0
Sep 11 17:43:57.296288 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:43:57.296128    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-15f6f2-f9df89b46-4mhkh"
Sep 11 17:43:57.296288 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:43:57.296167    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-15f6f2-f9df89b46-4mhkh" event={"ID":"f4b1c65a-dc4e-40f4-8b6e-799eb9479bc5","Type":"ContainerDied","Data":"ac435397d1330864997ca6f1112e9a82e583c70dd86ef73049e298da09e52ba0"}
Sep 11 17:43:57.296288 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:43:57.296228    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-15f6f2-f9df89b46-4mhkh" event={"ID":"f4b1c65a-dc4e-40f4-8b6e-799eb9479bc5","Type":"ContainerDied","Data":"1b939d99a47e71375aa68af3143b363890ef3455f31f4f964cb6a1de9afcb251"}
Sep 11 17:43:57.296288 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:43:57.296257    2552 scope.go:117] "RemoveContainer" containerID="ac435397d1330864997ca6f1112e9a82e583c70dd86ef73049e298da09e52ba0"
Sep 11 17:43:57.306187 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:43:57.306159    2552 scope.go:117] "RemoveContainer" containerID="ac435397d1330864997ca6f1112e9a82e583c70dd86ef73049e298da09e52ba0"
Sep 11 17:43:57.306448 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:43:57.306427    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ac435397d1330864997ca6f1112e9a82e583c70dd86ef73049e298da09e52ba0\": container with ID starting with ac435397d1330864997ca6f1112e9a82e583c70dd86ef73049e298da09e52ba0 not found: ID does not exist" containerID="ac435397d1330864997ca6f1112e9a82e583c70dd86ef73049e298da09e52ba0"
Sep 11 17:43:57.306547 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:43:57.306453    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ac435397d1330864997ca6f1112e9a82e583c70dd86ef73049e298da09e52ba0"} err="failed to get container status \"ac435397d1330864997ca6f1112e9a82e583c70dd86ef73049e298da09e52ba0\": rpc error: code = NotFound desc = could not find container \"ac435397d1330864997ca6f1112e9a82e583c70dd86ef73049e298da09e52ba0\": container with ID starting with ac435397d1330864997ca6f1112e9a82e583c70dd86ef73049e298da09e52ba0 not found: ID does not exist"
Sep 11 17:43:57.322663 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:43:57.322638    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-15f6f2-f9df89b46-4mhkh"]
Sep 11 17:43:57.329868 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:43:57.329848    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-15f6f2-f9df89b46-4mhkh"]
Sep 11 17:43:57.369604 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:43:57.369571    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:43:57.949583 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:43:57.949543    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f4b1c65a-dc4e-40f4-8b6e-799eb9479bc5" path="/var/lib/kubelet/pods/f4b1c65a-dc4e-40f4-8b6e-799eb9479bc5/volumes"
Sep 11 17:43:58.317482 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:43:58.317392    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:43:59.260914 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:43:59.260886    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:44:00.230790 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:00.230757    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:44:00.975517 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:00.975484    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:44:01.983733 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:01.983699    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:44:02.819449 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:02.819419    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:44:03.763884 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:03.763849    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:44:04.003977 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:04.003944    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:44:04.143584 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:04.143544    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:44:04.198770 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:04.198741    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-15f6f2-openshift-default-7c9bf84768-lht27"]
Sep 11 17:44:04.198984 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:04.198960    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-15f6f2-openshift-default-7c9bf84768-lht27" podUID="39af155e-9284-42e3-b5f4-d6f3c4be1757" containerName="istio-proxy" containerID="cri-o://c2ca97b271ea5dc160eadeab8e4194f09365c73e4d4f7b76712087d55cade4f7" gracePeriod=30
Sep 11 17:44:04.246753 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:04.246716    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:44:05.040343 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:05.040312    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:44:05.285439 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:05.285410    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-fzs45"
Sep 11 17:44:05.964610 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:05.964578    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:44:07.014390 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:07.014352    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:44:07.989206 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:07.989173    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:44:08.782954 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:08.782920    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:44:09.374886 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:09.374854    2552 generic.go:356] "Generic (PLEG): container finished" podID="39af155e-9284-42e3-b5f4-d6f3c4be1757" containerID="c2ca97b271ea5dc160eadeab8e4194f09365c73e4d4f7b76712087d55cade4f7" exitCode=0
Sep 11 17:44:09.375014 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:09.374916    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-15f6f2-openshift-default-7c9bf84768-lht27" event={"ID":"39af155e-9284-42e3-b5f4-d6f3c4be1757","Type":"ContainerDied","Data":"c2ca97b271ea5dc160eadeab8e4194f09365c73e4d4f7b76712087d55cade4f7"}
Sep 11 17:44:09.375014 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:09.374977    2552 scope.go:117] "RemoveContainer" containerID="3ca0b0285a89f72db6c8e6b2a5cdbbe5a21e418e6a6e266c5ca54516ea5d90ad"
Sep 11 17:44:09.456551 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:09.456524    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-15f6f2-openshift-default-7c9bf84768-lht27"
Sep 11 17:44:09.557958 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:09.557889    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/39af155e-9284-42e3-b5f4-d6f3c4be1757-istio-token\") pod \"39af155e-9284-42e3-b5f4-d6f3c4be1757\" (UID: \"39af155e-9284-42e3-b5f4-d6f3c4be1757\") "
Sep 11 17:44:09.557958 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:09.557941    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/39af155e-9284-42e3-b5f4-d6f3c4be1757-workload-certs\") pod \"39af155e-9284-42e3-b5f4-d6f3c4be1757\" (UID: \"39af155e-9284-42e3-b5f4-d6f3c4be1757\") "
Sep 11 17:44:09.558207 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:09.557987    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/39af155e-9284-42e3-b5f4-d6f3c4be1757-istio-data\") pod \"39af155e-9284-42e3-b5f4-d6f3c4be1757\" (UID: \"39af155e-9284-42e3-b5f4-d6f3c4be1757\") "
Sep 11 17:44:09.558207 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:09.558008    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tvtg9\" (UniqueName: \"kubernetes.io/projected/39af155e-9284-42e3-b5f4-d6f3c4be1757-kube-api-access-tvtg9\") pod \"39af155e-9284-42e3-b5f4-d6f3c4be1757\" (UID: \"39af155e-9284-42e3-b5f4-d6f3c4be1757\") "
Sep 11 17:44:09.558207 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:09.558051    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/39af155e-9284-42e3-b5f4-d6f3c4be1757-credential-socket\") pod \"39af155e-9284-42e3-b5f4-d6f3c4be1757\" (UID: \"39af155e-9284-42e3-b5f4-d6f3c4be1757\") "
Sep 11 17:44:09.558207 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:09.558098    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/39af155e-9284-42e3-b5f4-d6f3c4be1757-istiod-ca-cert\") pod \"39af155e-9284-42e3-b5f4-d6f3c4be1757\" (UID: \"39af155e-9284-42e3-b5f4-d6f3c4be1757\") "
Sep 11 17:44:09.558207 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:09.558168    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/39af155e-9284-42e3-b5f4-d6f3c4be1757-istio-podinfo\") pod \"39af155e-9284-42e3-b5f4-d6f3c4be1757\" (UID: \"39af155e-9284-42e3-b5f4-d6f3c4be1757\") "
Sep 11 17:44:09.558475 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:09.558251    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/39af155e-9284-42e3-b5f4-d6f3c4be1757-istio-envoy\") pod \"39af155e-9284-42e3-b5f4-d6f3c4be1757\" (UID: \"39af155e-9284-42e3-b5f4-d6f3c4be1757\") "
Sep 11 17:44:09.558475 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:09.558291    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/39af155e-9284-42e3-b5f4-d6f3c4be1757-workload-socket\") pod \"39af155e-9284-42e3-b5f4-d6f3c4be1757\" (UID: \"39af155e-9284-42e3-b5f4-d6f3c4be1757\") "
Sep 11 17:44:09.558475 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:09.558322    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/39af155e-9284-42e3-b5f4-d6f3c4be1757-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "39af155e-9284-42e3-b5f4-d6f3c4be1757" (UID: "39af155e-9284-42e3-b5f4-d6f3c4be1757"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:44:09.558475 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:09.558335    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/39af155e-9284-42e3-b5f4-d6f3c4be1757-istio-data" (OuterVolumeSpecName: "istio-data") pod "39af155e-9284-42e3-b5f4-d6f3c4be1757" (UID: "39af155e-9284-42e3-b5f4-d6f3c4be1757"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:44:09.558475 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:09.558378    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/39af155e-9284-42e3-b5f4-d6f3c4be1757-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "39af155e-9284-42e3-b5f4-d6f3c4be1757" (UID: "39af155e-9284-42e3-b5f4-d6f3c4be1757"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:44:09.558734 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:09.558570    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/39af155e-9284-42e3-b5f4-d6f3c4be1757-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "39af155e-9284-42e3-b5f4-d6f3c4be1757" (UID: "39af155e-9284-42e3-b5f4-d6f3c4be1757"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:44:09.558734 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:09.558707    2552 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/39af155e-9284-42e3-b5f4-d6f3c4be1757-istio-data\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:44:09.558734 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:09.558715    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/39af155e-9284-42e3-b5f4-d6f3c4be1757-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "39af155e-9284-42e3-b5f4-d6f3c4be1757" (UID: "39af155e-9284-42e3-b5f4-d6f3c4be1757"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 17:44:09.558734 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:09.558729    2552 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/39af155e-9284-42e3-b5f4-d6f3c4be1757-credential-socket\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:44:09.558931 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:09.558758    2552 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/39af155e-9284-42e3-b5f4-d6f3c4be1757-workload-socket\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:44:09.558931 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:09.558777    2552 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/39af155e-9284-42e3-b5f4-d6f3c4be1757-workload-certs\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:44:09.560192 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:09.560164    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/39af155e-9284-42e3-b5f4-d6f3c4be1757-istio-token" (OuterVolumeSpecName: "istio-token") pod "39af155e-9284-42e3-b5f4-d6f3c4be1757" (UID: "39af155e-9284-42e3-b5f4-d6f3c4be1757"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:44:09.560294 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:09.560222    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/39af155e-9284-42e3-b5f4-d6f3c4be1757-kube-api-access-tvtg9" (OuterVolumeSpecName: "kube-api-access-tvtg9") pod "39af155e-9284-42e3-b5f4-d6f3c4be1757" (UID: "39af155e-9284-42e3-b5f4-d6f3c4be1757"). InnerVolumeSpecName "kube-api-access-tvtg9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:44:09.560609 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:09.560593    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/39af155e-9284-42e3-b5f4-d6f3c4be1757-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "39af155e-9284-42e3-b5f4-d6f3c4be1757" (UID: "39af155e-9284-42e3-b5f4-d6f3c4be1757"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:44:09.560703 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:09.560679    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/39af155e-9284-42e3-b5f4-d6f3c4be1757-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "39af155e-9284-42e3-b5f4-d6f3c4be1757" (UID: "39af155e-9284-42e3-b5f4-d6f3c4be1757"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 17:44:09.659752 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:09.659722    2552 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/39af155e-9284-42e3-b5f4-d6f3c4be1757-istio-envoy\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:44:09.659752 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:09.659747    2552 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/39af155e-9284-42e3-b5f4-d6f3c4be1757-istio-token\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:44:09.659752 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:09.659756    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tvtg9\" (UniqueName: \"kubernetes.io/projected/39af155e-9284-42e3-b5f4-d6f3c4be1757-kube-api-access-tvtg9\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:44:09.659985 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:09.659765    2552 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/39af155e-9284-42e3-b5f4-d6f3c4be1757-istiod-ca-cert\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:44:09.659985 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:09.659772    2552 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/39af155e-9284-42e3-b5f4-d6f3c4be1757-istio-podinfo\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:44:09.953795 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:09.953755    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:44:10.382294 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:10.382269    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-15f6f2-openshift-default-7c9bf84768-lht27"
Sep 11 17:44:10.382294 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:10.382285    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-15f6f2-openshift-default-7c9bf84768-lht27" event={"ID":"39af155e-9284-42e3-b5f4-d6f3c4be1757","Type":"ContainerDied","Data":"2acbc433b43533ad33b805362f28325cefc82ab3b51085cda4cb388d25a8d8de"}
Sep 11 17:44:10.382560 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:10.382338    2552 scope.go:117] "RemoveContainer" containerID="c2ca97b271ea5dc160eadeab8e4194f09365c73e4d4f7b76712087d55cade4f7"
Sep 11 17:44:10.401436 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:10.401409    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-15f6f2-openshift-default-7c9bf84768-lht27"]
Sep 11 17:44:10.404442 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:10.404422    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-15f6f2-openshift-default-7c9bf84768-lht27"]
Sep 11 17:44:10.960373 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:10.960335    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:44:11.747551 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:11.747510    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:44:11.951031 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:11.950995    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="39af155e-9284-42e3-b5f4-d6f3c4be1757" path="/var/lib/kubelet/pods/39af155e-9284-42e3-b5f4-d6f3c4be1757/volumes"
Sep 11 17:44:14.820499 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:14.820467    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:44:20.704839 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:20.704801    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-83a2bc-785c75d74b-p5pqd"]
Sep 11 17:44:20.705306 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:20.705046    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-83a2bc-785c75d74b-p5pqd" podUID="e4f646e8-e7dc-4a53-9385-cb391d0e658d" containerName="maas-api" containerID="cri-o://93537275e4ace6e9424d2d9c8029a3fd779bcd933233b6267529ce8916e57919" gracePeriod=30
Sep 11 17:44:20.742788 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:20.742761    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:44:20.848251 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:20.848218    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:44:20.945749 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:20.945719    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:44:20.964600 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:20.964550    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-83a2bc-785c75d74b-p5pqd"
Sep 11 17:44:21.049885 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:21.049851    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:44:21.069929 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:21.069901    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e4f646e8-e7dc-4a53-9385-cb391d0e658d-openshift-service-ca\") pod \"e4f646e8-e7dc-4a53-9385-cb391d0e658d\" (UID: \"e4f646e8-e7dc-4a53-9385-cb391d0e658d\") "
Sep 11 17:44:21.070105 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:21.069977    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e4f646e8-e7dc-4a53-9385-cb391d0e658d-maas-api-tls\") pod \"e4f646e8-e7dc-4a53-9385-cb391d0e658d\" (UID: \"e4f646e8-e7dc-4a53-9385-cb391d0e658d\") "
Sep 11 17:44:21.070105 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:21.070008    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zn8cg\" (UniqueName: \"kubernetes.io/projected/e4f646e8-e7dc-4a53-9385-cb391d0e658d-kube-api-access-zn8cg\") pod \"e4f646e8-e7dc-4a53-9385-cb391d0e658d\" (UID: \"e4f646e8-e7dc-4a53-9385-cb391d0e658d\") "
Sep 11 17:44:21.070105 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:21.070038    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e4f646e8-e7dc-4a53-9385-cb391d0e658d-metrics-certs\") pod \"e4f646e8-e7dc-4a53-9385-cb391d0e658d\" (UID: \"e4f646e8-e7dc-4a53-9385-cb391d0e658d\") "
Sep 11 17:44:21.070331 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:21.070305    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e4f646e8-e7dc-4a53-9385-cb391d0e658d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e4f646e8-e7dc-4a53-9385-cb391d0e658d" (UID: "e4f646e8-e7dc-4a53-9385-cb391d0e658d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 17:44:21.070416 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:21.070399    2552 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e4f646e8-e7dc-4a53-9385-cb391d0e658d-openshift-service-ca\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:44:21.072144 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:21.072121    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e4f646e8-e7dc-4a53-9385-cb391d0e658d-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "e4f646e8-e7dc-4a53-9385-cb391d0e658d" (UID: "e4f646e8-e7dc-4a53-9385-cb391d0e658d"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 17:44:21.072474 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:21.072452    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e4f646e8-e7dc-4a53-9385-cb391d0e658d-kube-api-access-zn8cg" (OuterVolumeSpecName: "kube-api-access-zn8cg") pod "e4f646e8-e7dc-4a53-9385-cb391d0e658d" (UID: "e4f646e8-e7dc-4a53-9385-cb391d0e658d"). InnerVolumeSpecName "kube-api-access-zn8cg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:44:21.072580 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:21.072514    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e4f646e8-e7dc-4a53-9385-cb391d0e658d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e4f646e8-e7dc-4a53-9385-cb391d0e658d" (UID: "e4f646e8-e7dc-4a53-9385-cb391d0e658d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 17:44:21.157503 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:21.157464    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:44:21.171624 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:21.171598    2552 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e4f646e8-e7dc-4a53-9385-cb391d0e658d-maas-api-tls\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:44:21.171624 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:21.171621    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zn8cg\" (UniqueName: \"kubernetes.io/projected/e4f646e8-e7dc-4a53-9385-cb391d0e658d-kube-api-access-zn8cg\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:44:21.171624 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:21.171630    2552 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e4f646e8-e7dc-4a53-9385-cb391d0e658d-metrics-certs\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:44:21.454084 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:21.454044    2552 generic.go:356] "Generic (PLEG): container finished" podID="e4f646e8-e7dc-4a53-9385-cb391d0e658d" containerID="93537275e4ace6e9424d2d9c8029a3fd779bcd933233b6267529ce8916e57919" exitCode=0
Sep 11 17:44:21.454289 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:21.454129    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-83a2bc-785c75d74b-p5pqd"
Sep 11 17:44:21.454289 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:21.454137    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-83a2bc-785c75d74b-p5pqd" event={"ID":"e4f646e8-e7dc-4a53-9385-cb391d0e658d","Type":"ContainerDied","Data":"93537275e4ace6e9424d2d9c8029a3fd779bcd933233b6267529ce8916e57919"}
Sep 11 17:44:21.454289 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:21.454194    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-83a2bc-785c75d74b-p5pqd" event={"ID":"e4f646e8-e7dc-4a53-9385-cb391d0e658d","Type":"ContainerDied","Data":"ff4359e25ed59e8999579c232443d6df5bc548ed3be10212cab8e62ce4b959eb"}
Sep 11 17:44:21.454289 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:21.454224    2552 scope.go:117] "RemoveContainer" containerID="93537275e4ace6e9424d2d9c8029a3fd779bcd933233b6267529ce8916e57919"
Sep 11 17:44:21.456896 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:21.456870    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:44:21.464990 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:21.464975    2552 scope.go:117] "RemoveContainer" containerID="93537275e4ace6e9424d2d9c8029a3fd779bcd933233b6267529ce8916e57919"
Sep 11 17:44:21.465296 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:44:21.465277    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"93537275e4ace6e9424d2d9c8029a3fd779bcd933233b6267529ce8916e57919\": container with ID starting with 93537275e4ace6e9424d2d9c8029a3fd779bcd933233b6267529ce8916e57919 not found: ID does not exist" containerID="93537275e4ace6e9424d2d9c8029a3fd779bcd933233b6267529ce8916e57919"
Sep 11 17:44:21.465364 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:21.465304    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"93537275e4ace6e9424d2d9c8029a3fd779bcd933233b6267529ce8916e57919"} err="failed to get container status \"93537275e4ace6e9424d2d9c8029a3fd779bcd933233b6267529ce8916e57919\": rpc error: code = NotFound desc = could not find container \"93537275e4ace6e9424d2d9c8029a3fd779bcd933233b6267529ce8916e57919\": container with ID starting with 93537275e4ace6e9424d2d9c8029a3fd779bcd933233b6267529ce8916e57919 not found: ID does not exist"
Sep 11 17:44:21.477054 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:21.477028    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-83a2bc-785c75d74b-p5pqd"]
Sep 11 17:44:21.479622 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:21.479600    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-83a2bc-785c75d74b-p5pqd"]
Sep 11 17:44:21.554495 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:21.554466    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:44:21.951502 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:21.951473    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e4f646e8-e7dc-4a53-9385-cb391d0e658d" path="/var/lib/kubelet/pods/e4f646e8-e7dc-4a53-9385-cb391d0e658d/volumes"
Sep 11 17:44:22.534468 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:22.534427    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:44:24.063202 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:24.063173    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:44:25.482201 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:25.482173    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:44:26.532416 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:26.532320    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:44:27.302426 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:27.302372    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:44:28.396715 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:28.396680    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:44:29.108521 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:29.108488    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:44:29.210148 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:29.210091    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:44:29.457504 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:29.457429    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:44:30.133419 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:30.133387    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:44:31.172568 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:31.172536    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:44:31.900665 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:31.900631    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:44:32.715839 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:32.715804    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:44:33.577396 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:33.577361    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:44:34.677264 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:34.677236    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:44:35.467494 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:35.467461    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:44:36.166191 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:36.166149    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:44:39.155308 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:39.155277    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:44:40.280567 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:40.280533    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-290683-74d74bd8b7-qpj8k"]
Sep 11 17:44:40.281009 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:40.280764    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-290683-74d74bd8b7-qpj8k" podUID="9e9cb1a4-4a5f-417d-9ad4-9769dd1ae1fc" containerName="maas-api" containerID="cri-o://7610238f9f5881c24a588050cf2460416f4020e498fa06792cddfe633600c9d4" gracePeriod=30
Sep 11 17:44:40.310070 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:40.310049    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:44:40.356340 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:40.356304    2552 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-290683-74d74bd8b7-qpj8k" podUID="9e9cb1a4-4a5f-417d-9ad4-9769dd1ae1fc" containerName="maas-api" probeResult="failure" output="Get \"https://10.132.0.69:8443/health\": dial tcp 10.132.0.69:8443: connect: connection refused"
Sep 11 17:44:40.376519 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:40.376486    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:44:40.476750 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:40.476718    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:44:40.528393 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:40.528371    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-290683-74d74bd8b7-qpj8k"
Sep 11 17:44:40.551998 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:40.551915    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:44:40.575681 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:40.575649    2552 generic.go:356] "Generic (PLEG): container finished" podID="9e9cb1a4-4a5f-417d-9ad4-9769dd1ae1fc" containerID="7610238f9f5881c24a588050cf2460416f4020e498fa06792cddfe633600c9d4" exitCode=0
Sep 11 17:44:40.575796 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:40.575684    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-290683-74d74bd8b7-qpj8k"
Sep 11 17:44:40.575796 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:40.575717    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-290683-74d74bd8b7-qpj8k" event={"ID":"9e9cb1a4-4a5f-417d-9ad4-9769dd1ae1fc","Type":"ContainerDied","Data":"7610238f9f5881c24a588050cf2460416f4020e498fa06792cddfe633600c9d4"}
Sep 11 17:44:40.575796 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:40.575775    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-290683-74d74bd8b7-qpj8k" event={"ID":"9e9cb1a4-4a5f-417d-9ad4-9769dd1ae1fc","Type":"ContainerDied","Data":"cec81e42b19d1a5ec52e4a4b193cd1bd2a8eb7c32c208f81400e0a04c09600de"}
Sep 11 17:44:40.575923 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:40.575809    2552 scope.go:117] "RemoveContainer" containerID="7610238f9f5881c24a588050cf2460416f4020e498fa06792cddfe633600c9d4"
Sep 11 17:44:40.586035 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:40.586012    2552 scope.go:117] "RemoveContainer" containerID="7610238f9f5881c24a588050cf2460416f4020e498fa06792cddfe633600c9d4"
Sep 11 17:44:40.586318 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:44:40.586296    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7610238f9f5881c24a588050cf2460416f4020e498fa06792cddfe633600c9d4\": container with ID starting with 7610238f9f5881c24a588050cf2460416f4020e498fa06792cddfe633600c9d4 not found: ID does not exist" containerID="7610238f9f5881c24a588050cf2460416f4020e498fa06792cddfe633600c9d4"
Sep 11 17:44:40.586407 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:40.586325    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7610238f9f5881c24a588050cf2460416f4020e498fa06792cddfe633600c9d4"} err="failed to get container status \"7610238f9f5881c24a588050cf2460416f4020e498fa06792cddfe633600c9d4\": rpc error: code = NotFound desc = could not find container \"7610238f9f5881c24a588050cf2460416f4020e498fa06792cddfe633600c9d4\": container with ID starting with 7610238f9f5881c24a588050cf2460416f4020e498fa06792cddfe633600c9d4 not found: ID does not exist"
Sep 11 17:44:40.663689 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:40.663661    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9e9cb1a4-4a5f-417d-9ad4-9769dd1ae1fc-maas-api-tls\") pod \"9e9cb1a4-4a5f-417d-9ad4-9769dd1ae1fc\" (UID: \"9e9cb1a4-4a5f-417d-9ad4-9769dd1ae1fc\") "
Sep 11 17:44:40.663885 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:40.663809    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9e9cb1a4-4a5f-417d-9ad4-9769dd1ae1fc-metrics-certs\") pod \"9e9cb1a4-4a5f-417d-9ad4-9769dd1ae1fc\" (UID: \"9e9cb1a4-4a5f-417d-9ad4-9769dd1ae1fc\") "
Sep 11 17:44:40.663885 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:40.663852    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nh7md\" (UniqueName: \"kubernetes.io/projected/9e9cb1a4-4a5f-417d-9ad4-9769dd1ae1fc-kube-api-access-nh7md\") pod \"9e9cb1a4-4a5f-417d-9ad4-9769dd1ae1fc\" (UID: \"9e9cb1a4-4a5f-417d-9ad4-9769dd1ae1fc\") "
Sep 11 17:44:40.664013 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:40.663901    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e9cb1a4-4a5f-417d-9ad4-9769dd1ae1fc-openshift-service-ca\") pod \"9e9cb1a4-4a5f-417d-9ad4-9769dd1ae1fc\" (UID: \"9e9cb1a4-4a5f-417d-9ad4-9769dd1ae1fc\") "
Sep 11 17:44:40.664582 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:40.664545    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9e9cb1a4-4a5f-417d-9ad4-9769dd1ae1fc-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "9e9cb1a4-4a5f-417d-9ad4-9769dd1ae1fc" (UID: "9e9cb1a4-4a5f-417d-9ad4-9769dd1ae1fc"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 17:44:40.667667 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:40.667641    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9e9cb1a4-4a5f-417d-9ad4-9769dd1ae1fc-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "9e9cb1a4-4a5f-417d-9ad4-9769dd1ae1fc" (UID: "9e9cb1a4-4a5f-417d-9ad4-9769dd1ae1fc"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 17:44:40.668245 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:40.668217    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9e9cb1a4-4a5f-417d-9ad4-9769dd1ae1fc-kube-api-access-nh7md" (OuterVolumeSpecName: "kube-api-access-nh7md") pod "9e9cb1a4-4a5f-417d-9ad4-9769dd1ae1fc" (UID: "9e9cb1a4-4a5f-417d-9ad4-9769dd1ae1fc"). InnerVolumeSpecName "kube-api-access-nh7md". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:44:40.668611 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:40.668590    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9e9cb1a4-4a5f-417d-9ad4-9769dd1ae1fc-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "9e9cb1a4-4a5f-417d-9ad4-9769dd1ae1fc" (UID: "9e9cb1a4-4a5f-417d-9ad4-9769dd1ae1fc"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 17:44:40.765633 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:40.765601    2552 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9e9cb1a4-4a5f-417d-9ad4-9769dd1ae1fc-metrics-certs\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:44:40.765633 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:40.765629    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nh7md\" (UniqueName: \"kubernetes.io/projected/9e9cb1a4-4a5f-417d-9ad4-9769dd1ae1fc-kube-api-access-nh7md\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:44:40.765633 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:40.765641    2552 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e9cb1a4-4a5f-417d-9ad4-9769dd1ae1fc-openshift-service-ca\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:44:40.765864 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:40.765649    2552 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9e9cb1a4-4a5f-417d-9ad4-9769dd1ae1fc-maas-api-tls\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:44:40.920093 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:40.920062    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-290683-74d74bd8b7-qpj8k"]
Sep 11 17:44:40.934264 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:40.934242    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-290683-74d74bd8b7-qpj8k"]
Sep 11 17:44:40.972315 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:40.972282    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:44:41.168741 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:41.168663    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:44:41.861223 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:41.861187    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:44:41.950060 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:41.950024    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9e9cb1a4-4a5f-417d-9ad4-9769dd1ae1fc" path="/var/lib/kubelet/pods/9e9cb1a4-4a5f-417d-9ad4-9769dd1ae1fc/volumes"
Sep 11 17:44:42.060727 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:42.060696    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:44:42.965711 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:42.965677    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:44:43.948071 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:43.948025    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:44:44.724346 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:44.724314    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:44:45.545919 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:45.545881    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:44:46.336187 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:46.336151    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:44:46.986774 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:46.986743    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:44:47.389359 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:47.389325    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:44:48.551738 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:48.551700    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:44:48.672277 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:48.672241    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:44:48.825513 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:48.825424    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-290683-openshift-default-57c7b78dd6-q77mh"]
Sep 11 17:44:48.825678 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:48.825658    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w2-290683-openshift-default-57c7b78dd6-q77mh" podUID="34c43407-b150-4315-b2d4-b1413944bdd3" containerName="istio-proxy" containerID="cri-o://a05ccad1ecbfdbc40d7efea851903347f1edad657b01f11184ec50f50488f722" gracePeriod=30
Sep 11 17:44:48.863373 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:48.863329    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:44:49.473047 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:49.473013    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:44:50.773747 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:50.773713    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:44:51.752734 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:51.752698    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:44:51.880054 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:51.880007    2552 patch_prober.go:28] interesting pod/e2e-shared-a-w2-290683-openshift-default-57c7b78dd6-q77mh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 11 17:44:51.880437 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:51.880069    2552 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w2-290683-openshift-default-57c7b78dd6-q77mh" podUID="34c43407-b150-4315-b2d4-b1413944bdd3" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 11 17:44:52.772876 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:52.772846    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:44:53.762657 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:53.762623    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:44:54.098031 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:54.098003    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-290683-openshift-default-57c7b78dd6-q77mh"
Sep 11 17:44:54.202550 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:54.202515    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/34c43407-b150-4315-b2d4-b1413944bdd3-workload-socket\") pod \"34c43407-b150-4315-b2d4-b1413944bdd3\" (UID: \"34c43407-b150-4315-b2d4-b1413944bdd3\") "
Sep 11 17:44:54.202736 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:54.202590    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7gclm\" (UniqueName: \"kubernetes.io/projected/34c43407-b150-4315-b2d4-b1413944bdd3-kube-api-access-7gclm\") pod \"34c43407-b150-4315-b2d4-b1413944bdd3\" (UID: \"34c43407-b150-4315-b2d4-b1413944bdd3\") "
Sep 11 17:44:54.202736 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:54.202609    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/34c43407-b150-4315-b2d4-b1413944bdd3-istio-podinfo\") pod \"34c43407-b150-4315-b2d4-b1413944bdd3\" (UID: \"34c43407-b150-4315-b2d4-b1413944bdd3\") "
Sep 11 17:44:54.202736 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:54.202643    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/34c43407-b150-4315-b2d4-b1413944bdd3-istio-data\") pod \"34c43407-b150-4315-b2d4-b1413944bdd3\" (UID: \"34c43407-b150-4315-b2d4-b1413944bdd3\") "
Sep 11 17:44:54.202736 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:54.202693    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/34c43407-b150-4315-b2d4-b1413944bdd3-istio-token\") pod \"34c43407-b150-4315-b2d4-b1413944bdd3\" (UID: \"34c43407-b150-4315-b2d4-b1413944bdd3\") "
Sep 11 17:44:54.202736 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:54.202724    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/34c43407-b150-4315-b2d4-b1413944bdd3-credential-socket\") pod \"34c43407-b150-4315-b2d4-b1413944bdd3\" (UID: \"34c43407-b150-4315-b2d4-b1413944bdd3\") "
Sep 11 17:44:54.202985 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:54.202751    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/34c43407-b150-4315-b2d4-b1413944bdd3-workload-certs\") pod \"34c43407-b150-4315-b2d4-b1413944bdd3\" (UID: \"34c43407-b150-4315-b2d4-b1413944bdd3\") "
Sep 11 17:44:54.202985 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:54.202770    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/34c43407-b150-4315-b2d4-b1413944bdd3-istio-envoy\") pod \"34c43407-b150-4315-b2d4-b1413944bdd3\" (UID: \"34c43407-b150-4315-b2d4-b1413944bdd3\") "
Sep 11 17:44:54.202985 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:54.202801    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/34c43407-b150-4315-b2d4-b1413944bdd3-istiod-ca-cert\") pod \"34c43407-b150-4315-b2d4-b1413944bdd3\" (UID: \"34c43407-b150-4315-b2d4-b1413944bdd3\") "
Sep 11 17:44:54.202985 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:54.202842    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/34c43407-b150-4315-b2d4-b1413944bdd3-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "34c43407-b150-4315-b2d4-b1413944bdd3" (UID: "34c43407-b150-4315-b2d4-b1413944bdd3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:44:54.203221 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:54.203049    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/34c43407-b150-4315-b2d4-b1413944bdd3-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "34c43407-b150-4315-b2d4-b1413944bdd3" (UID: "34c43407-b150-4315-b2d4-b1413944bdd3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:44:54.203221 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:54.203068    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/34c43407-b150-4315-b2d4-b1413944bdd3-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "34c43407-b150-4315-b2d4-b1413944bdd3" (UID: "34c43407-b150-4315-b2d4-b1413944bdd3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:44:54.203221 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:54.203078    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/34c43407-b150-4315-b2d4-b1413944bdd3-istio-data" (OuterVolumeSpecName: "istio-data") pod "34c43407-b150-4315-b2d4-b1413944bdd3" (UID: "34c43407-b150-4315-b2d4-b1413944bdd3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:44:54.203221 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:54.203137    2552 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/34c43407-b150-4315-b2d4-b1413944bdd3-workload-socket\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:44:54.203444 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:54.203366    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/34c43407-b150-4315-b2d4-b1413944bdd3-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "34c43407-b150-4315-b2d4-b1413944bdd3" (UID: "34c43407-b150-4315-b2d4-b1413944bdd3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 17:44:54.205094 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:54.205063    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/34c43407-b150-4315-b2d4-b1413944bdd3-istio-token" (OuterVolumeSpecName: "istio-token") pod "34c43407-b150-4315-b2d4-b1413944bdd3" (UID: "34c43407-b150-4315-b2d4-b1413944bdd3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:44:54.205224 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:54.205137    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/34c43407-b150-4315-b2d4-b1413944bdd3-kube-api-access-7gclm" (OuterVolumeSpecName: "kube-api-access-7gclm") pod "34c43407-b150-4315-b2d4-b1413944bdd3" (UID: "34c43407-b150-4315-b2d4-b1413944bdd3"). InnerVolumeSpecName "kube-api-access-7gclm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:44:54.205415 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:54.205401    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/34c43407-b150-4315-b2d4-b1413944bdd3-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "34c43407-b150-4315-b2d4-b1413944bdd3" (UID: "34c43407-b150-4315-b2d4-b1413944bdd3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 17:44:54.205525 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:54.205505    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/34c43407-b150-4315-b2d4-b1413944bdd3-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "34c43407-b150-4315-b2d4-b1413944bdd3" (UID: "34c43407-b150-4315-b2d4-b1413944bdd3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:44:54.304316 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:54.304278    2552 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/34c43407-b150-4315-b2d4-b1413944bdd3-istio-data\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:44:54.304316 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:54.304308    2552 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/34c43407-b150-4315-b2d4-b1413944bdd3-istio-token\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:44:54.304316 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:54.304318    2552 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/34c43407-b150-4315-b2d4-b1413944bdd3-credential-socket\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:44:54.304316 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:54.304327    2552 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/34c43407-b150-4315-b2d4-b1413944bdd3-workload-certs\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:44:54.304580 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:54.304336    2552 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/34c43407-b150-4315-b2d4-b1413944bdd3-istio-envoy\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:44:54.304580 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:54.304343    2552 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/34c43407-b150-4315-b2d4-b1413944bdd3-istiod-ca-cert\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:44:54.304580 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:54.304351    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7gclm\" (UniqueName: \"kubernetes.io/projected/34c43407-b150-4315-b2d4-b1413944bdd3-kube-api-access-7gclm\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:44:54.304580 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:54.304359    2552 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/34c43407-b150-4315-b2d4-b1413944bdd3-istio-podinfo\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:44:54.554984 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:54.554947    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:44:54.666162 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:54.666099    2552 generic.go:356] "Generic (PLEG): container finished" podID="34c43407-b150-4315-b2d4-b1413944bdd3" containerID="a05ccad1ecbfdbc40d7efea851903347f1edad657b01f11184ec50f50488f722" exitCode=0
Sep 11 17:44:54.666349 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:54.666170    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-290683-openshift-default-57c7b78dd6-q77mh"
Sep 11 17:44:54.666349 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:54.666180    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-290683-openshift-default-57c7b78dd6-q77mh" event={"ID":"34c43407-b150-4315-b2d4-b1413944bdd3","Type":"ContainerDied","Data":"a05ccad1ecbfdbc40d7efea851903347f1edad657b01f11184ec50f50488f722"}
Sep 11 17:44:54.666349 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:54.666251    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-290683-openshift-default-57c7b78dd6-q77mh" event={"ID":"34c43407-b150-4315-b2d4-b1413944bdd3","Type":"ContainerDied","Data":"7f76ee46a11348295eaaa9ea561905984d6adb6efb1349274cecdc4aa8edb60c"}
Sep 11 17:44:54.666349 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:54.666284    2552 scope.go:117] "RemoveContainer" containerID="a05ccad1ecbfdbc40d7efea851903347f1edad657b01f11184ec50f50488f722"
Sep 11 17:44:54.681217 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:54.681192    2552 scope.go:117] "RemoveContainer" containerID="9f5858a971d9e8f8f91f9b412e2797ba213b59c52ec2d31e9d9d5e236461056c"
Sep 11 17:44:54.691999 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:54.691947    2552 scope.go:117] "RemoveContainer" containerID="a05ccad1ecbfdbc40d7efea851903347f1edad657b01f11184ec50f50488f722"
Sep 11 17:44:54.692584 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:44:54.692423    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a05ccad1ecbfdbc40d7efea851903347f1edad657b01f11184ec50f50488f722\": container with ID starting with a05ccad1ecbfdbc40d7efea851903347f1edad657b01f11184ec50f50488f722 not found: ID does not exist" containerID="a05ccad1ecbfdbc40d7efea851903347f1edad657b01f11184ec50f50488f722"
Sep 11 17:44:54.692584 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:54.692467    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a05ccad1ecbfdbc40d7efea851903347f1edad657b01f11184ec50f50488f722"} err="failed to get container status \"a05ccad1ecbfdbc40d7efea851903347f1edad657b01f11184ec50f50488f722\": rpc error: code = NotFound desc = could not find container \"a05ccad1ecbfdbc40d7efea851903347f1edad657b01f11184ec50f50488f722\": container with ID starting with a05ccad1ecbfdbc40d7efea851903347f1edad657b01f11184ec50f50488f722 not found: ID does not exist"
Sep 11 17:44:54.692584 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:54.692492    2552 scope.go:117] "RemoveContainer" containerID="9f5858a971d9e8f8f91f9b412e2797ba213b59c52ec2d31e9d9d5e236461056c"
Sep 11 17:44:54.692868 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:54.692851    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-290683-openshift-default-57c7b78dd6-q77mh"]
Sep 11 17:44:54.692919 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:44:54.692864    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9f5858a971d9e8f8f91f9b412e2797ba213b59c52ec2d31e9d9d5e236461056c\": container with ID starting with 9f5858a971d9e8f8f91f9b412e2797ba213b59c52ec2d31e9d9d5e236461056c not found: ID does not exist" containerID="9f5858a971d9e8f8f91f9b412e2797ba213b59c52ec2d31e9d9d5e236461056c"
Sep 11 17:44:54.692919 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:54.692898    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f5858a971d9e8f8f91f9b412e2797ba213b59c52ec2d31e9d9d5e236461056c"} err="failed to get container status \"9f5858a971d9e8f8f91f9b412e2797ba213b59c52ec2d31e9d9d5e236461056c\": rpc error: code = NotFound desc = could not find container \"9f5858a971d9e8f8f91f9b412e2797ba213b59c52ec2d31e9d9d5e236461056c\": container with ID starting with 9f5858a971d9e8f8f91f9b412e2797ba213b59c52ec2d31e9d9d5e236461056c not found: ID does not exist"
Sep 11 17:44:54.695783 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:54.695763    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-290683-openshift-default-57c7b78dd6-q77mh"]
Sep 11 17:44:55.951089 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:55.951051    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="34c43407-b150-4315-b2d4-b1413944bdd3" path="/var/lib/kubelet/pods/34c43407-b150-4315-b2d4-b1413944bdd3/volumes"
Sep 11 17:44:57.560149 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:57.560103    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:44:57.661300 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:57.661266    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:44:57.764616 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:57.764588    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:44:57.860665 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:57.860633    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:44:58.270898 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:58.270806    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:44:58.371896 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:58.371857    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:44:59.140008 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:44:59.139969    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:45:00.026798 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:00.026750    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-2f0a78-ccb9cd454-8h8gh"]
Sep 11 17:45:00.027107 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:00.027072    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-2f0a78-ccb9cd454-8h8gh" podUID="474d5c3f-d5e3-40bf-bc71-c9584b4efeec" containerName="maas-api" containerID="cri-o://7ced25eb7e951ae19a69c03b411e4feee1f99b0ffedc049d4436ea583a51f0b6" gracePeriod=30
Sep 11 17:45:00.072753 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:00.072703    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:45:00.154394 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:00.154355    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-6fc9e6-29819145-szflz"]
Sep 11 17:45:00.154825 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:00.154761    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f4b1c65a-dc4e-40f4-8b6e-799eb9479bc5" containerName="maas-api"
Sep 11 17:45:00.154825 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:00.154772    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="f4b1c65a-dc4e-40f4-8b6e-799eb9479bc5" containerName="maas-api"
Sep 11 17:45:00.154825 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:00.154781    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="34c43407-b150-4315-b2d4-b1413944bdd3" containerName="istio-proxy"
Sep 11 17:45:00.154825 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:00.154787    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="34c43407-b150-4315-b2d4-b1413944bdd3" containerName="istio-proxy"
Sep 11 17:45:00.154825 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:00.154795    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="34c43407-b150-4315-b2d4-b1413944bdd3" containerName="istio-proxy"
Sep 11 17:45:00.154825 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:00.154800    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="34c43407-b150-4315-b2d4-b1413944bdd3" containerName="istio-proxy"
Sep 11 17:45:00.154825 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:00.154808    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39af155e-9284-42e3-b5f4-d6f3c4be1757" containerName="istio-proxy"
Sep 11 17:45:00.154825 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:00.154813    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="39af155e-9284-42e3-b5f4-d6f3c4be1757" containerName="istio-proxy"
Sep 11 17:45:00.154825 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:00.154828    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9e9cb1a4-4a5f-417d-9ad4-9769dd1ae1fc" containerName="maas-api"
Sep 11 17:45:00.154825 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:00.154833    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e9cb1a4-4a5f-417d-9ad4-9769dd1ae1fc" containerName="maas-api"
Sep 11 17:45:00.155281 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:00.154840    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e4f646e8-e7dc-4a53-9385-cb391d0e658d" containerName="maas-api"
Sep 11 17:45:00.155281 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:00.154845    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="e4f646e8-e7dc-4a53-9385-cb391d0e658d" containerName="maas-api"
Sep 11 17:45:00.155281 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:00.154853    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39af155e-9284-42e3-b5f4-d6f3c4be1757" containerName="istio-proxy"
Sep 11 17:45:00.155281 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:00.154858    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="39af155e-9284-42e3-b5f4-d6f3c4be1757" containerName="istio-proxy"
Sep 11 17:45:00.155281 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:00.154921    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="e4f646e8-e7dc-4a53-9385-cb391d0e658d" containerName="maas-api"
Sep 11 17:45:00.155281 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:00.154930    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="f4b1c65a-dc4e-40f4-8b6e-799eb9479bc5" containerName="maas-api"
Sep 11 17:45:00.155281 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:00.154935    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="39af155e-9284-42e3-b5f4-d6f3c4be1757" containerName="istio-proxy"
Sep 11 17:45:00.155281 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:00.154941    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="34c43407-b150-4315-b2d4-b1413944bdd3" containerName="istio-proxy"
Sep 11 17:45:00.155281 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:00.154947    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="9e9cb1a4-4a5f-417d-9ad4-9769dd1ae1fc" containerName="maas-api"
Sep 11 17:45:00.155281 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:00.154954    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="34c43407-b150-4315-b2d4-b1413944bdd3" containerName="istio-proxy"
Sep 11 17:45:00.158190 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:00.158171    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-6fc9e6-29819145-szflz"
Sep 11 17:45:00.162430 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:00.162411    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9n8gf\""
Sep 11 17:45:00.168975 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:00.168956    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-6fc9e6-29819145-szflz"
Sep 11 17:45:00.176917 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:00.176843    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-6fc9e6-29819145-szflz"]
Sep 11 17:45:00.194696 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:00.194300    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:45:00.287579 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:00.287528    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:45:00.314890 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:00.314861    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-2f0a78-ccb9cd454-8h8gh"
Sep 11 17:45:00.366838 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:00.366809    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/474d5c3f-d5e3-40bf-bc71-c9584b4efeec-openshift-service-ca\") pod \"474d5c3f-d5e3-40bf-bc71-c9584b4efeec\" (UID: \"474d5c3f-d5e3-40bf-bc71-c9584b4efeec\") "
Sep 11 17:45:00.367023 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:00.366881    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/474d5c3f-d5e3-40bf-bc71-c9584b4efeec-maas-api-tls\") pod \"474d5c3f-d5e3-40bf-bc71-c9584b4efeec\" (UID: \"474d5c3f-d5e3-40bf-bc71-c9584b4efeec\") "
Sep 11 17:45:00.367023 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:00.366906    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t4c8t\" (UniqueName: \"kubernetes.io/projected/474d5c3f-d5e3-40bf-bc71-c9584b4efeec-kube-api-access-t4c8t\") pod \"474d5c3f-d5e3-40bf-bc71-c9584b4efeec\" (UID: \"474d5c3f-d5e3-40bf-bc71-c9584b4efeec\") "
Sep 11 17:45:00.367023 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:00.366939    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/474d5c3f-d5e3-40bf-bc71-c9584b4efeec-metrics-certs\") pod \"474d5c3f-d5e3-40bf-bc71-c9584b4efeec\" (UID: \"474d5c3f-d5e3-40bf-bc71-c9584b4efeec\") "
Sep 11 17:45:00.367284 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:00.367261    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/474d5c3f-d5e3-40bf-bc71-c9584b4efeec-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "474d5c3f-d5e3-40bf-bc71-c9584b4efeec" (UID: "474d5c3f-d5e3-40bf-bc71-c9584b4efeec"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 17:45:00.369066 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:00.369039    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/474d5c3f-d5e3-40bf-bc71-c9584b4efeec-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "474d5c3f-d5e3-40bf-bc71-c9584b4efeec" (UID: "474d5c3f-d5e3-40bf-bc71-c9584b4efeec"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 17:45:00.369197 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:00.369136    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/474d5c3f-d5e3-40bf-bc71-c9584b4efeec-kube-api-access-t4c8t" (OuterVolumeSpecName: "kube-api-access-t4c8t") pod "474d5c3f-d5e3-40bf-bc71-c9584b4efeec" (UID: "474d5c3f-d5e3-40bf-bc71-c9584b4efeec"). InnerVolumeSpecName "kube-api-access-t4c8t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:45:00.369197 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:00.369152    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/474d5c3f-d5e3-40bf-bc71-c9584b4efeec-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "474d5c3f-d5e3-40bf-bc71-c9584b4efeec" (UID: "474d5c3f-d5e3-40bf-bc71-c9584b4efeec"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 17:45:00.389089 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:00.389057    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:45:00.465035 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:00.465003    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-6fc9e6-29819145-szflz"]
Sep 11 17:45:00.467463 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:45:00.467435    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddcd7f9a8_1a7a_4eb8_abf0_0945972796a9.slice/crio-44bf026f9aba8d05b5c78c8f72040e084aa76e144f5c9a415d080f855c7f1e94 WatchSource:0}: Error finding container 44bf026f9aba8d05b5c78c8f72040e084aa76e144f5c9a415d080f855c7f1e94: Status 404 returned error can't find the container with id 44bf026f9aba8d05b5c78c8f72040e084aa76e144f5c9a415d080f855c7f1e94
Sep 11 17:45:00.467846 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:00.467813    2552 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/474d5c3f-d5e3-40bf-bc71-c9584b4efeec-metrics-certs\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:45:00.467925 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:00.467852    2552 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/474d5c3f-d5e3-40bf-bc71-c9584b4efeec-openshift-service-ca\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:45:00.467925 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:00.467867    2552 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/474d5c3f-d5e3-40bf-bc71-c9584b4efeec-maas-api-tls\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:45:00.467925 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:00.467880    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t4c8t\" (UniqueName: \"kubernetes.io/projected/474d5c3f-d5e3-40bf-bc71-c9584b4efeec-kube-api-access-t4c8t\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:45:00.470983 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:00.470916    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:45:00.639013 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:00.638946    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:45:00.706037 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:00.705999    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-6fc9e6-29819145-szflz" event={"ID":"dcd7f9a8-1a7a-4eb8-abf0-0945972796a9","Type":"ContainerStarted","Data":"44bf026f9aba8d05b5c78c8f72040e084aa76e144f5c9a415d080f855c7f1e94"}
Sep 11 17:45:00.707307 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:00.707280    2552 generic.go:356] "Generic (PLEG): container finished" podID="474d5c3f-d5e3-40bf-bc71-c9584b4efeec" containerID="7ced25eb7e951ae19a69c03b411e4feee1f99b0ffedc049d4436ea583a51f0b6" exitCode=0
Sep 11 17:45:00.707412 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:00.707321    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-2f0a78-ccb9cd454-8h8gh"
Sep 11 17:45:00.707412 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:00.707328    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-2f0a78-ccb9cd454-8h8gh" event={"ID":"474d5c3f-d5e3-40bf-bc71-c9584b4efeec","Type":"ContainerDied","Data":"7ced25eb7e951ae19a69c03b411e4feee1f99b0ffedc049d4436ea583a51f0b6"}
Sep 11 17:45:00.707412 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:00.707364    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-2f0a78-ccb9cd454-8h8gh" event={"ID":"474d5c3f-d5e3-40bf-bc71-c9584b4efeec","Type":"ContainerDied","Data":"ad64bb06798477a3162ba1a6db40fb90982f0a08568dc4ad0dc3c260c03ca9d9"}
Sep 11 17:45:00.707412 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:00.707392    2552 scope.go:117] "RemoveContainer" containerID="7ced25eb7e951ae19a69c03b411e4feee1f99b0ffedc049d4436ea583a51f0b6"
Sep 11 17:45:00.717884 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:00.717867    2552 scope.go:117] "RemoveContainer" containerID="7ced25eb7e951ae19a69c03b411e4feee1f99b0ffedc049d4436ea583a51f0b6"
Sep 11 17:45:00.718163 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:45:00.718146    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7ced25eb7e951ae19a69c03b411e4feee1f99b0ffedc049d4436ea583a51f0b6\": container with ID starting with 7ced25eb7e951ae19a69c03b411e4feee1f99b0ffedc049d4436ea583a51f0b6 not found: ID does not exist" containerID="7ced25eb7e951ae19a69c03b411e4feee1f99b0ffedc049d4436ea583a51f0b6"
Sep 11 17:45:00.718217 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:00.718172    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7ced25eb7e951ae19a69c03b411e4feee1f99b0ffedc049d4436ea583a51f0b6"} err="failed to get container status \"7ced25eb7e951ae19a69c03b411e4feee1f99b0ffedc049d4436ea583a51f0b6\": rpc error: code = NotFound desc = could not find container \"7ced25eb7e951ae19a69c03b411e4feee1f99b0ffedc049d4436ea583a51f0b6\": container with ID starting with 7ced25eb7e951ae19a69c03b411e4feee1f99b0ffedc049d4436ea583a51f0b6 not found: ID does not exist"
Sep 11 17:45:00.733227 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:00.733203    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-2f0a78-ccb9cd454-8h8gh"]
Sep 11 17:45:00.741163 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:00.741139    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-2f0a78-ccb9cd454-8h8gh"]
Sep 11 17:45:00.809928 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:00.809896    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:45:01.554319 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:01.554280    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:45:01.716223 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:01.716185    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-6fc9e6-29819145-szflz" event={"ID":"dcd7f9a8-1a7a-4eb8-abf0-0945972796a9","Type":"ContainerStarted","Data":"6f93d3c8e8be11597da774ab24e38506d8b4cea5eb0d2ddc769fe3d18c2454fb"}
Sep 11 17:45:01.730735 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:01.730685    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-6fc9e6-29819145-szflz" podStartSLOduration=1.7306672330000001 podStartE2EDuration="1.730667233s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:45:01.730450796 +0000 UTC m=+1596.251742316" watchObservedRunningTime="2026-09-11 17:45:01.730667233 +0000 UTC m=+1596.251958735"
Sep 11 17:45:01.950089 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:01.950056    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="474d5c3f-d5e3-40bf-bc71-c9584b4efeec" path="/var/lib/kubelet/pods/474d5c3f-d5e3-40bf-bc71-c9584b4efeec/volumes"
Sep 11 17:45:02.623011 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:02.622975    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:45:02.724039 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:02.724003    2552 generic.go:356] "Generic (PLEG): container finished" podID="dcd7f9a8-1a7a-4eb8-abf0-0945972796a9" containerID="6f93d3c8e8be11597da774ab24e38506d8b4cea5eb0d2ddc769fe3d18c2454fb" exitCode=0
Sep 11 17:45:02.724222 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:02.724063    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-6fc9e6-29819145-szflz" event={"ID":"dcd7f9a8-1a7a-4eb8-abf0-0945972796a9","Type":"ContainerDied","Data":"6f93d3c8e8be11597da774ab24e38506d8b4cea5eb0d2ddc769fe3d18c2454fb"}
Sep 11 17:45:03.874037 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:03.874013    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-6fc9e6-29819145-szflz"
Sep 11 17:45:04.063542 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:04.063432    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:45:04.738296 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:04.738266    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-6fc9e6-29819145-szflz"
Sep 11 17:45:04.738473 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:04.738301    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-6fc9e6-29819145-szflz" event={"ID":"dcd7f9a8-1a7a-4eb8-abf0-0945972796a9","Type":"ContainerDied","Data":"44bf026f9aba8d05b5c78c8f72040e084aa76e144f5c9a415d080f855c7f1e94"}
Sep 11 17:45:04.738473 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:04.738361    2552 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="44bf026f9aba8d05b5c78c8f72040e084aa76e144f5c9a415d080f855c7f1e94"
Sep 11 17:45:04.854661 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:04.854600    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:45:07.066352 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:07.066315    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:45:07.347202 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:07.347164    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:45:07.648881 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:07.648789    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:45:09.372469 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:09.372438    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:45:09.492288 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:09.492256    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:45:09.670398 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:09.670306    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-2f0a78-openshift-default-5f8fffc99b-4rrv5"]
Sep 11 17:45:09.670570 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:09.670541    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w2-2f0a78-openshift-default-5f8fffc99b-4rrv5" podUID="bb3e4bcb-fefa-4488-be59-1ac3b5b56bd4" containerName="istio-proxy" containerID="cri-o://da49bff43604ec2aa054bd0dda6ed030e07a2a927f1baa12ac4f7a2ce014633b" gracePeriod=30
Sep 11 17:45:09.689423 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:09.689379    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:45:14.071632 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:14.071601    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:45:14.807778 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:14.807749    2552 generic.go:356] "Generic (PLEG): container finished" podID="bb3e4bcb-fefa-4488-be59-1ac3b5b56bd4" containerID="da49bff43604ec2aa054bd0dda6ed030e07a2a927f1baa12ac4f7a2ce014633b" exitCode=0
Sep 11 17:45:14.807954 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:14.807828    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-2f0a78-openshift-default-5f8fffc99b-4rrv5" event={"ID":"bb3e4bcb-fefa-4488-be59-1ac3b5b56bd4","Type":"ContainerDied","Data":"da49bff43604ec2aa054bd0dda6ed030e07a2a927f1baa12ac4f7a2ce014633b"}
Sep 11 17:45:14.807954 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:14.807904    2552 scope.go:117] "RemoveContainer" containerID="a7e64f6828bdcccf84f762dcb31cff7fd1c20bbc0f4298312fd6c95c3b7099b4"
Sep 11 17:45:14.940947 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:14.940881    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-2f0a78-openshift-default-5f8fffc99b-4rrv5"
Sep 11 17:45:15.013075 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:15.013039    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb3e4bcb-fefa-4488-be59-1ac3b5b56bd4-credential-socket\") pod \"bb3e4bcb-fefa-4488-be59-1ac3b5b56bd4\" (UID: \"bb3e4bcb-fefa-4488-be59-1ac3b5b56bd4\") "
Sep 11 17:45:15.013075 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:15.013076    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb3e4bcb-fefa-4488-be59-1ac3b5b56bd4-workload-socket\") pod \"bb3e4bcb-fefa-4488-be59-1ac3b5b56bd4\" (UID: \"bb3e4bcb-fefa-4488-be59-1ac3b5b56bd4\") "
Sep 11 17:45:15.013345 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:15.013098    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bb3e4bcb-fefa-4488-be59-1ac3b5b56bd4-istio-data\") pod \"bb3e4bcb-fefa-4488-be59-1ac3b5b56bd4\" (UID: \"bb3e4bcb-fefa-4488-be59-1ac3b5b56bd4\") "
Sep 11 17:45:15.013345 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:15.013144    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bb3e4bcb-fefa-4488-be59-1ac3b5b56bd4-istio-envoy\") pod \"bb3e4bcb-fefa-4488-be59-1ac3b5b56bd4\" (UID: \"bb3e4bcb-fefa-4488-be59-1ac3b5b56bd4\") "
Sep 11 17:45:15.013345 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:15.013166    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bb3e4bcb-fefa-4488-be59-1ac3b5b56bd4-istiod-ca-cert\") pod \"bb3e4bcb-fefa-4488-be59-1ac3b5b56bd4\" (UID: \"bb3e4bcb-fefa-4488-be59-1ac3b5b56bd4\") "
Sep 11 17:45:15.013345 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:15.013245    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-85kgq\" (UniqueName: \"kubernetes.io/projected/bb3e4bcb-fefa-4488-be59-1ac3b5b56bd4-kube-api-access-85kgq\") pod \"bb3e4bcb-fefa-4488-be59-1ac3b5b56bd4\" (UID: \"bb3e4bcb-fefa-4488-be59-1ac3b5b56bd4\") "
Sep 11 17:45:15.013345 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:15.013283    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bb3e4bcb-fefa-4488-be59-1ac3b5b56bd4-istio-podinfo\") pod \"bb3e4bcb-fefa-4488-be59-1ac3b5b56bd4\" (UID: \"bb3e4bcb-fefa-4488-be59-1ac3b5b56bd4\") "
Sep 11 17:45:15.013345 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:15.013313    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bb3e4bcb-fefa-4488-be59-1ac3b5b56bd4-istio-token\") pod \"bb3e4bcb-fefa-4488-be59-1ac3b5b56bd4\" (UID: \"bb3e4bcb-fefa-4488-be59-1ac3b5b56bd4\") "
Sep 11 17:45:15.013653 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:15.013377    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bb3e4bcb-fefa-4488-be59-1ac3b5b56bd4-workload-certs\") pod \"bb3e4bcb-fefa-4488-be59-1ac3b5b56bd4\" (UID: \"bb3e4bcb-fefa-4488-be59-1ac3b5b56bd4\") "
Sep 11 17:45:15.013653 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:15.013412    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb3e4bcb-fefa-4488-be59-1ac3b5b56bd4-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "bb3e4bcb-fefa-4488-be59-1ac3b5b56bd4" (UID: "bb3e4bcb-fefa-4488-be59-1ac3b5b56bd4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:45:15.013653 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:15.013482    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb3e4bcb-fefa-4488-be59-1ac3b5b56bd4-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "bb3e4bcb-fefa-4488-be59-1ac3b5b56bd4" (UID: "bb3e4bcb-fefa-4488-be59-1ac3b5b56bd4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:45:15.013653 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:15.013488    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb3e4bcb-fefa-4488-be59-1ac3b5b56bd4-istio-data" (OuterVolumeSpecName: "istio-data") pod "bb3e4bcb-fefa-4488-be59-1ac3b5b56bd4" (UID: "bb3e4bcb-fefa-4488-be59-1ac3b5b56bd4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:45:15.013845 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:15.013719    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb3e4bcb-fefa-4488-be59-1ac3b5b56bd4-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "bb3e4bcb-fefa-4488-be59-1ac3b5b56bd4" (UID: "bb3e4bcb-fefa-4488-be59-1ac3b5b56bd4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:45:15.013845 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:15.013754    2552 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb3e4bcb-fefa-4488-be59-1ac3b5b56bd4-credential-socket\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:45:15.013845 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:15.013775    2552 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb3e4bcb-fefa-4488-be59-1ac3b5b56bd4-workload-socket\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:45:15.013845 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:15.013790    2552 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bb3e4bcb-fefa-4488-be59-1ac3b5b56bd4-istio-data\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:45:15.014079 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:15.013892    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bb3e4bcb-fefa-4488-be59-1ac3b5b56bd4-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "bb3e4bcb-fefa-4488-be59-1ac3b5b56bd4" (UID: "bb3e4bcb-fefa-4488-be59-1ac3b5b56bd4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 17:45:15.016027 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:15.016003    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb3e4bcb-fefa-4488-be59-1ac3b5b56bd4-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "bb3e4bcb-fefa-4488-be59-1ac3b5b56bd4" (UID: "bb3e4bcb-fefa-4488-be59-1ac3b5b56bd4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:45:15.016027 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:15.016021    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bb3e4bcb-fefa-4488-be59-1ac3b5b56bd4-istio-token" (OuterVolumeSpecName: "istio-token") pod "bb3e4bcb-fefa-4488-be59-1ac3b5b56bd4" (UID: "bb3e4bcb-fefa-4488-be59-1ac3b5b56bd4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:45:15.016206 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:15.016032    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/bb3e4bcb-fefa-4488-be59-1ac3b5b56bd4-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "bb3e4bcb-fefa-4488-be59-1ac3b5b56bd4" (UID: "bb3e4bcb-fefa-4488-be59-1ac3b5b56bd4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 17:45:15.016206 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:15.016046    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bb3e4bcb-fefa-4488-be59-1ac3b5b56bd4-kube-api-access-85kgq" (OuterVolumeSpecName: "kube-api-access-85kgq") pod "bb3e4bcb-fefa-4488-be59-1ac3b5b56bd4" (UID: "bb3e4bcb-fefa-4488-be59-1ac3b5b56bd4"). InnerVolumeSpecName "kube-api-access-85kgq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:45:15.114491 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:15.114460    2552 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bb3e4bcb-fefa-4488-be59-1ac3b5b56bd4-istio-envoy\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:45:15.114491 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:15.114488    2552 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bb3e4bcb-fefa-4488-be59-1ac3b5b56bd4-istiod-ca-cert\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:45:15.114491 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:15.114497    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-85kgq\" (UniqueName: \"kubernetes.io/projected/bb3e4bcb-fefa-4488-be59-1ac3b5b56bd4-kube-api-access-85kgq\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:45:15.114928 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:15.114506    2552 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bb3e4bcb-fefa-4488-be59-1ac3b5b56bd4-istio-podinfo\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:45:15.114928 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:15.114514    2552 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bb3e4bcb-fefa-4488-be59-1ac3b5b56bd4-istio-token\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:45:15.114928 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:15.114523    2552 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bb3e4bcb-fefa-4488-be59-1ac3b5b56bd4-workload-certs\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:45:15.816514 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:15.816469    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-2f0a78-openshift-default-5f8fffc99b-4rrv5"
Sep 11 17:45:15.816514 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:15.816509    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-2f0a78-openshift-default-5f8fffc99b-4rrv5" event={"ID":"bb3e4bcb-fefa-4488-be59-1ac3b5b56bd4","Type":"ContainerDied","Data":"2b2214b3532b31b0d169d196187b45d941b03f79af654e317cd012cafcdd36b9"}
Sep 11 17:45:15.816789 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:15.816571    2552 scope.go:117] "RemoveContainer" containerID="da49bff43604ec2aa054bd0dda6ed030e07a2a927f1baa12ac4f7a2ce014633b"
Sep 11 17:45:15.839500 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:15.839472    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-2f0a78-openshift-default-5f8fffc99b-4rrv5"]
Sep 11 17:45:15.846923 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:15.846895    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-2f0a78-openshift-default-5f8fffc99b-4rrv5"]
Sep 11 17:45:15.949548 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:15.949510    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bb3e4bcb-fefa-4488-be59-1ac3b5b56bd4" path="/var/lib/kubelet/pods/bb3e4bcb-fefa-4488-be59-1ac3b5b56bd4/volumes"
Sep 11 17:45:23.050569 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:23.050465    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:45:23.149081 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:23.149043    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:45:24.803393 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:24.803360    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:45:26.259356 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:26.259321    2552 scope.go:117] "RemoveContainer" containerID="1108ad80d922e3214719ed58ad3488a133e7a7eb019df592bf3ade0391589585"
Sep 11 17:45:28.034538 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:28.034509    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:45:28.524300 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:28.524264    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:45:29.551219 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:29.551175    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:45:30.355187 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:30.355153    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:45:31.297486 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:31.297451    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:45:32.443606 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:32.443565    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:45:33.038328 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:33.038287    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-144bfa/distinct-2-w4-144bfa-kserve-5cfcc96d78-75f6s"]
Sep 11 17:45:33.059907 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:33.059877    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-144bfa/premium-w4-144bfa-kserve-5b5658659d-7v9nm"]
Sep 11 17:45:33.161192 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:33.161161    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:45:40.416381 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:40.416348    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:45:43.924091 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:43.924052    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:45:44.009201 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:44.009164    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:45:44.763068 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:44.763033    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:45:44.877202 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:44.877167    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:45:45.683995 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:45.683961    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:45:46.491372 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:46.491337    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:45:48.058040 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:48.058000    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-144bfa/distinct-2-w4-144bfa-kserve-5cfcc96d78-75f6s" podUID="c624a168-2aee-4e51-b69f-106a4c0eedf4" containerName="main" containerID="cri-o://028310bf2fddaaf22dd4a734453feffb7ddd8ee415b285eeaf5bdc420b793ca4" gracePeriod=45
Sep 11 17:45:48.080198 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:48.080167    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-144bfa/premium-w4-144bfa-kserve-5b5658659d-7v9nm" podUID="01c753eb-e3ca-4a2b-8b9b-c3cc7b672d7d" containerName="main" containerID="cri-o://a174e2857e9e118fd25945086e1a141898e7c83254e39debfb06d0c48f793f2b" gracePeriod=45
Sep 11 17:45:48.327772 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:48.327747    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-144bfa/distinct-2-w4-144bfa-kserve-5cfcc96d78-75f6s"
Sep 11 17:45:48.363187 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:48.363163    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-144bfa/premium-w4-144bfa-kserve-5b5658659d-7v9nm"
Sep 11 17:45:48.441089 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:48.441043    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/01c753eb-e3ca-4a2b-8b9b-c3cc7b672d7d-tls-certs\") pod \"01c753eb-e3ca-4a2b-8b9b-c3cc7b672d7d\" (UID: \"01c753eb-e3ca-4a2b-8b9b-c3cc7b672d7d\") "
Sep 11 17:45:48.441305 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:48.441103    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/01c753eb-e3ca-4a2b-8b9b-c3cc7b672d7d-model-cache\") pod \"01c753eb-e3ca-4a2b-8b9b-c3cc7b672d7d\" (UID: \"01c753eb-e3ca-4a2b-8b9b-c3cc7b672d7d\") "
Sep 11 17:45:48.441305 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:48.441185    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wkrps\" (UniqueName: \"kubernetes.io/projected/01c753eb-e3ca-4a2b-8b9b-c3cc7b672d7d-kube-api-access-wkrps\") pod \"01c753eb-e3ca-4a2b-8b9b-c3cc7b672d7d\" (UID: \"01c753eb-e3ca-4a2b-8b9b-c3cc7b672d7d\") "
Sep 11 17:45:48.441305 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:48.441209    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c624a168-2aee-4e51-b69f-106a4c0eedf4-tmp-dir\") pod \"c624a168-2aee-4e51-b69f-106a4c0eedf4\" (UID: \"c624a168-2aee-4e51-b69f-106a4c0eedf4\") "
Sep 11 17:45:48.441503 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:48.441475    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/01c753eb-e3ca-4a2b-8b9b-c3cc7b672d7d-model-cache" (OuterVolumeSpecName: "model-cache") pod "01c753eb-e3ca-4a2b-8b9b-c3cc7b672d7d" (UID: "01c753eb-e3ca-4a2b-8b9b-c3cc7b672d7d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:45:48.441648 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:48.441621    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c624a168-2aee-4e51-b69f-106a4c0eedf4-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "c624a168-2aee-4e51-b69f-106a4c0eedf4" (UID: "c624a168-2aee-4e51-b69f-106a4c0eedf4"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:45:48.441900 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:48.441858    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sm7hg\" (UniqueName: \"kubernetes.io/projected/c624a168-2aee-4e51-b69f-106a4c0eedf4-kube-api-access-sm7hg\") pod \"c624a168-2aee-4e51-b69f-106a4c0eedf4\" (UID: \"c624a168-2aee-4e51-b69f-106a4c0eedf4\") "
Sep 11 17:45:48.442239 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:48.442216    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c624a168-2aee-4e51-b69f-106a4c0eedf4-home\") pod \"c624a168-2aee-4e51-b69f-106a4c0eedf4\" (UID: \"c624a168-2aee-4e51-b69f-106a4c0eedf4\") "
Sep 11 17:45:48.442491 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:48.442468    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c624a168-2aee-4e51-b69f-106a4c0eedf4-home" (OuterVolumeSpecName: "home") pod "c624a168-2aee-4e51-b69f-106a4c0eedf4" (UID: "c624a168-2aee-4e51-b69f-106a4c0eedf4"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:45:48.442595 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:48.442525    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/01c753eb-e3ca-4a2b-8b9b-c3cc7b672d7d-tmp-dir\") pod \"01c753eb-e3ca-4a2b-8b9b-c3cc7b672d7d\" (UID: \"01c753eb-e3ca-4a2b-8b9b-c3cc7b672d7d\") "
Sep 11 17:45:48.443386 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:48.442740    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/01c753eb-e3ca-4a2b-8b9b-c3cc7b672d7d-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "01c753eb-e3ca-4a2b-8b9b-c3cc7b672d7d" (UID: "01c753eb-e3ca-4a2b-8b9b-c3cc7b672d7d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:45:48.443386 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:48.442781    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c624a168-2aee-4e51-b69f-106a4c0eedf4-dshm\") pod \"c624a168-2aee-4e51-b69f-106a4c0eedf4\" (UID: \"c624a168-2aee-4e51-b69f-106a4c0eedf4\") "
Sep 11 17:45:48.443386 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:48.442828    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/01c753eb-e3ca-4a2b-8b9b-c3cc7b672d7d-home\") pod \"01c753eb-e3ca-4a2b-8b9b-c3cc7b672d7d\" (UID: \"01c753eb-e3ca-4a2b-8b9b-c3cc7b672d7d\") "
Sep 11 17:45:48.443386 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:48.442855    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/01c753eb-e3ca-4a2b-8b9b-c3cc7b672d7d-dshm\") pod \"01c753eb-e3ca-4a2b-8b9b-c3cc7b672d7d\" (UID: \"01c753eb-e3ca-4a2b-8b9b-c3cc7b672d7d\") "
Sep 11 17:45:48.443386 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:48.443037    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/01c753eb-e3ca-4a2b-8b9b-c3cc7b672d7d-home" (OuterVolumeSpecName: "home") pod "01c753eb-e3ca-4a2b-8b9b-c3cc7b672d7d" (UID: "01c753eb-e3ca-4a2b-8b9b-c3cc7b672d7d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:45:48.443386 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:48.443076    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c624a168-2aee-4e51-b69f-106a4c0eedf4-model-cache\") pod \"c624a168-2aee-4e51-b69f-106a4c0eedf4\" (UID: \"c624a168-2aee-4e51-b69f-106a4c0eedf4\") "
Sep 11 17:45:48.443386 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:48.443161    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c624a168-2aee-4e51-b69f-106a4c0eedf4-tls-certs\") pod \"c624a168-2aee-4e51-b69f-106a4c0eedf4\" (UID: \"c624a168-2aee-4e51-b69f-106a4c0eedf4\") "
Sep 11 17:45:48.443386 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:48.443333    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c624a168-2aee-4e51-b69f-106a4c0eedf4-model-cache" (OuterVolumeSpecName: "model-cache") pod "c624a168-2aee-4e51-b69f-106a4c0eedf4" (UID: "c624a168-2aee-4e51-b69f-106a4c0eedf4"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:45:48.443926 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:48.443900    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/01c753eb-e3ca-4a2b-8b9b-c3cc7b672d7d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "01c753eb-e3ca-4a2b-8b9b-c3cc7b672d7d" (UID: "01c753eb-e3ca-4a2b-8b9b-c3cc7b672d7d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 17:45:48.444063 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:48.444004    2552 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c624a168-2aee-4e51-b69f-106a4c0eedf4-home\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:45:48.444063 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:48.444033    2552 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/01c753eb-e3ca-4a2b-8b9b-c3cc7b672d7d-tmp-dir\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:45:48.444063 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:48.444046    2552 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/01c753eb-e3ca-4a2b-8b9b-c3cc7b672d7d-home\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:45:48.444063 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:48.444058    2552 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c624a168-2aee-4e51-b69f-106a4c0eedf4-model-cache\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:45:48.444330 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:48.444078    2552 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/01c753eb-e3ca-4a2b-8b9b-c3cc7b672d7d-tls-certs\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:45:48.444330 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:48.444090    2552 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/01c753eb-e3ca-4a2b-8b9b-c3cc7b672d7d-model-cache\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:45:48.444330 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:48.444102    2552 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c624a168-2aee-4e51-b69f-106a4c0eedf4-tmp-dir\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:45:48.446661 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:48.446557    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c624a168-2aee-4e51-b69f-106a4c0eedf4-kube-api-access-sm7hg" (OuterVolumeSpecName: "kube-api-access-sm7hg") pod "c624a168-2aee-4e51-b69f-106a4c0eedf4" (UID: "c624a168-2aee-4e51-b69f-106a4c0eedf4"). InnerVolumeSpecName "kube-api-access-sm7hg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:45:48.446661 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:48.446637    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/01c753eb-e3ca-4a2b-8b9b-c3cc7b672d7d-kube-api-access-wkrps" (OuterVolumeSpecName: "kube-api-access-wkrps") pod "01c753eb-e3ca-4a2b-8b9b-c3cc7b672d7d" (UID: "01c753eb-e3ca-4a2b-8b9b-c3cc7b672d7d"). InnerVolumeSpecName "kube-api-access-wkrps". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:45:48.446661 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:48.446652    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c624a168-2aee-4e51-b69f-106a4c0eedf4-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "c624a168-2aee-4e51-b69f-106a4c0eedf4" (UID: "c624a168-2aee-4e51-b69f-106a4c0eedf4"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 17:45:48.447082 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:48.447048    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c624a168-2aee-4e51-b69f-106a4c0eedf4-dshm" (OuterVolumeSpecName: "dshm") pod "c624a168-2aee-4e51-b69f-106a4c0eedf4" (UID: "c624a168-2aee-4e51-b69f-106a4c0eedf4"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:45:48.448102 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:48.448074    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/01c753eb-e3ca-4a2b-8b9b-c3cc7b672d7d-dshm" (OuterVolumeSpecName: "dshm") pod "01c753eb-e3ca-4a2b-8b9b-c3cc7b672d7d" (UID: "01c753eb-e3ca-4a2b-8b9b-c3cc7b672d7d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:45:48.545209 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:48.545172    2552 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c624a168-2aee-4e51-b69f-106a4c0eedf4-tls-certs\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:45:48.545209 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:48.545202    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wkrps\" (UniqueName: \"kubernetes.io/projected/01c753eb-e3ca-4a2b-8b9b-c3cc7b672d7d-kube-api-access-wkrps\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:45:48.545209 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:48.545212    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sm7hg\" (UniqueName: \"kubernetes.io/projected/c624a168-2aee-4e51-b69f-106a4c0eedf4-kube-api-access-sm7hg\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:45:48.545448 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:48.545223    2552 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c624a168-2aee-4e51-b69f-106a4c0eedf4-dshm\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:45:48.545448 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:48.545232    2552 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/01c753eb-e3ca-4a2b-8b9b-c3cc7b672d7d-dshm\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:45:49.020677 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:49.020634    2552 generic.go:356] "Generic (PLEG): container finished" podID="c624a168-2aee-4e51-b69f-106a4c0eedf4" containerID="028310bf2fddaaf22dd4a734453feffb7ddd8ee415b285eeaf5bdc420b793ca4" exitCode=0
Sep 11 17:45:49.020677 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:49.020680    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-144bfa/distinct-2-w4-144bfa-kserve-5cfcc96d78-75f6s"
Sep 11 17:45:49.020884 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:49.020720    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-144bfa/distinct-2-w4-144bfa-kserve-5cfcc96d78-75f6s" event={"ID":"c624a168-2aee-4e51-b69f-106a4c0eedf4","Type":"ContainerDied","Data":"028310bf2fddaaf22dd4a734453feffb7ddd8ee415b285eeaf5bdc420b793ca4"}
Sep 11 17:45:49.020884 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:49.020764    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-144bfa/distinct-2-w4-144bfa-kserve-5cfcc96d78-75f6s" event={"ID":"c624a168-2aee-4e51-b69f-106a4c0eedf4","Type":"ContainerDied","Data":"0298f0f8cc2e38a0dba4e7485b89ec5b7067d3cf10769084a91ef6ed73721e86"}
Sep 11 17:45:49.020884 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:49.020783    2552 scope.go:117] "RemoveContainer" containerID="028310bf2fddaaf22dd4a734453feffb7ddd8ee415b285eeaf5bdc420b793ca4"
Sep 11 17:45:49.022394 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:49.022371    2552 generic.go:356] "Generic (PLEG): container finished" podID="01c753eb-e3ca-4a2b-8b9b-c3cc7b672d7d" containerID="a174e2857e9e118fd25945086e1a141898e7c83254e39debfb06d0c48f793f2b" exitCode=0
Sep 11 17:45:49.022508 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:49.022417    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-144bfa/premium-w4-144bfa-kserve-5b5658659d-7v9nm"
Sep 11 17:45:49.022508 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:49.022452    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-144bfa/premium-w4-144bfa-kserve-5b5658659d-7v9nm" event={"ID":"01c753eb-e3ca-4a2b-8b9b-c3cc7b672d7d","Type":"ContainerDied","Data":"a174e2857e9e118fd25945086e1a141898e7c83254e39debfb06d0c48f793f2b"}
Sep 11 17:45:49.022508 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:49.022491    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-144bfa/premium-w4-144bfa-kserve-5b5658659d-7v9nm" event={"ID":"01c753eb-e3ca-4a2b-8b9b-c3cc7b672d7d","Type":"ContainerDied","Data":"c70aeaa5ae45a5528062b573f8f05866690b36c0ab2eddbc143baeb569a0ba59"}
Sep 11 17:45:49.031968 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:49.031928    2552 scope.go:117] "RemoveContainer" containerID="028310bf2fddaaf22dd4a734453feffb7ddd8ee415b285eeaf5bdc420b793ca4"
Sep 11 17:45:49.032374 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:45:49.032346    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"028310bf2fddaaf22dd4a734453feffb7ddd8ee415b285eeaf5bdc420b793ca4\": container with ID starting with 028310bf2fddaaf22dd4a734453feffb7ddd8ee415b285eeaf5bdc420b793ca4 not found: ID does not exist" containerID="028310bf2fddaaf22dd4a734453feffb7ddd8ee415b285eeaf5bdc420b793ca4"
Sep 11 17:45:49.032460 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:49.032381    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"028310bf2fddaaf22dd4a734453feffb7ddd8ee415b285eeaf5bdc420b793ca4"} err="failed to get container status \"028310bf2fddaaf22dd4a734453feffb7ddd8ee415b285eeaf5bdc420b793ca4\": rpc error: code = NotFound desc = could not find container \"028310bf2fddaaf22dd4a734453feffb7ddd8ee415b285eeaf5bdc420b793ca4\": container with ID starting with 028310bf2fddaaf22dd4a734453feffb7ddd8ee415b285eeaf5bdc420b793ca4 not found: ID does not exist"
Sep 11 17:45:49.032460 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:49.032399    2552 scope.go:117] "RemoveContainer" containerID="a174e2857e9e118fd25945086e1a141898e7c83254e39debfb06d0c48f793f2b"
Sep 11 17:45:49.042302 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:49.042280    2552 scope.go:117] "RemoveContainer" containerID="a174e2857e9e118fd25945086e1a141898e7c83254e39debfb06d0c48f793f2b"
Sep 11 17:45:49.042619 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:45:49.042598    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a174e2857e9e118fd25945086e1a141898e7c83254e39debfb06d0c48f793f2b\": container with ID starting with a174e2857e9e118fd25945086e1a141898e7c83254e39debfb06d0c48f793f2b not found: ID does not exist" containerID="a174e2857e9e118fd25945086e1a141898e7c83254e39debfb06d0c48f793f2b"
Sep 11 17:45:49.042693 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:49.042630    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a174e2857e9e118fd25945086e1a141898e7c83254e39debfb06d0c48f793f2b"} err="failed to get container status \"a174e2857e9e118fd25945086e1a141898e7c83254e39debfb06d0c48f793f2b\": rpc error: code = NotFound desc = could not find container \"a174e2857e9e118fd25945086e1a141898e7c83254e39debfb06d0c48f793f2b\": container with ID starting with a174e2857e9e118fd25945086e1a141898e7c83254e39debfb06d0c48f793f2b not found: ID does not exist"
Sep 11 17:45:49.043511 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:49.043486    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-144bfa/distinct-2-w4-144bfa-kserve-5cfcc96d78-75f6s"]
Sep 11 17:45:49.054134 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:49.051035    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-144bfa/distinct-2-w4-144bfa-kserve-5cfcc96d78-75f6s"]
Sep 11 17:45:49.066897 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:49.066869    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-144bfa/premium-w4-144bfa-kserve-5b5658659d-7v9nm"]
Sep 11 17:45:49.069857 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:49.069829    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-144bfa/premium-w4-144bfa-kserve-5b5658659d-7v9nm"]
Sep 11 17:45:49.950673 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:49.950635    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="01c753eb-e3ca-4a2b-8b9b-c3cc7b672d7d" path="/var/lib/kubelet/pods/01c753eb-e3ca-4a2b-8b9b-c3cc7b672d7d/volumes"
Sep 11 17:45:49.951217 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:49.951197    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c624a168-2aee-4e51-b69f-106a4c0eedf4" path="/var/lib/kubelet/pods/c624a168-2aee-4e51-b69f-106a4c0eedf4/volumes"
Sep 11 17:45:50.456176 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:50.456138    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:45:50.557062 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:50.557025    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:45:50.718106 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:50.718009    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-144bfa-openshift-default-5dc996f7dd-ws7l7"]
Sep 11 17:45:50.718310 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:50.718246    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-144bfa-openshift-default-5dc996f7dd-ws7l7" podUID="ae170805-c6a4-469f-b280-8e7629749557" containerName="istio-proxy" containerID="cri-o://47b6c8b94c1c604285b5d8c2992077d7960c23b7a0b7905f9560b34ab9521935" gracePeriod=30
Sep 11 17:45:50.729569 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:50.729544    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:45:54.879253 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:54.879214    2552 patch_prober.go:28] interesting pod/e2e-worker-w4-144bfa-openshift-default-5dc996f7dd-ws7l7 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 11 17:45:54.879623 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:54.879270    2552 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-144bfa-openshift-default-5dc996f7dd-ws7l7" podUID="ae170805-c6a4-469f-b280-8e7629749557" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 11 17:45:55.984832 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:55.984771    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-144bfa-openshift-default-5dc996f7dd-ws7l7"
Sep 11 17:45:56.072657 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:56.072621    2552 generic.go:356] "Generic (PLEG): container finished" podID="ae170805-c6a4-469f-b280-8e7629749557" containerID="47b6c8b94c1c604285b5d8c2992077d7960c23b7a0b7905f9560b34ab9521935" exitCode=0
Sep 11 17:45:56.072859 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:56.072678    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-144bfa-openshift-default-5dc996f7dd-ws7l7"
Sep 11 17:45:56.072859 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:56.072676    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-144bfa-openshift-default-5dc996f7dd-ws7l7" event={"ID":"ae170805-c6a4-469f-b280-8e7629749557","Type":"ContainerDied","Data":"47b6c8b94c1c604285b5d8c2992077d7960c23b7a0b7905f9560b34ab9521935"}
Sep 11 17:45:56.072859 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:56.072747    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-144bfa-openshift-default-5dc996f7dd-ws7l7" event={"ID":"ae170805-c6a4-469f-b280-8e7629749557","Type":"ContainerDied","Data":"ac60dedbc966c4fe7d95c93ccbab657ff50f4cc0839423f40f4c072ee981130d"}
Sep 11 17:45:56.072859 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:56.072784    2552 scope.go:117] "RemoveContainer" containerID="47b6c8b94c1c604285b5d8c2992077d7960c23b7a0b7905f9560b34ab9521935"
Sep 11 17:45:56.082144 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:56.082127    2552 scope.go:117] "RemoveContainer" containerID="99222f6f532e8093632159f0abb4c3c2f4ebac614a270467e0e7e71cea3735c8"
Sep 11 17:45:56.090314 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:56.090298    2552 scope.go:117] "RemoveContainer" containerID="47b6c8b94c1c604285b5d8c2992077d7960c23b7a0b7905f9560b34ab9521935"
Sep 11 17:45:56.090593 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:45:56.090571    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"47b6c8b94c1c604285b5d8c2992077d7960c23b7a0b7905f9560b34ab9521935\": container with ID starting with 47b6c8b94c1c604285b5d8c2992077d7960c23b7a0b7905f9560b34ab9521935 not found: ID does not exist" containerID="47b6c8b94c1c604285b5d8c2992077d7960c23b7a0b7905f9560b34ab9521935"
Sep 11 17:45:56.090657 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:56.090601    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"47b6c8b94c1c604285b5d8c2992077d7960c23b7a0b7905f9560b34ab9521935"} err="failed to get container status \"47b6c8b94c1c604285b5d8c2992077d7960c23b7a0b7905f9560b34ab9521935\": rpc error: code = NotFound desc = could not find container \"47b6c8b94c1c604285b5d8c2992077d7960c23b7a0b7905f9560b34ab9521935\": container with ID starting with 47b6c8b94c1c604285b5d8c2992077d7960c23b7a0b7905f9560b34ab9521935 not found: ID does not exist"
Sep 11 17:45:56.090657 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:56.090618    2552 scope.go:117] "RemoveContainer" containerID="99222f6f532e8093632159f0abb4c3c2f4ebac614a270467e0e7e71cea3735c8"
Sep 11 17:45:56.090850 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:45:56.090834    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"99222f6f532e8093632159f0abb4c3c2f4ebac614a270467e0e7e71cea3735c8\": container with ID starting with 99222f6f532e8093632159f0abb4c3c2f4ebac614a270467e0e7e71cea3735c8 not found: ID does not exist" containerID="99222f6f532e8093632159f0abb4c3c2f4ebac614a270467e0e7e71cea3735c8"
Sep 11 17:45:56.090899 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:56.090854    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"99222f6f532e8093632159f0abb4c3c2f4ebac614a270467e0e7e71cea3735c8"} err="failed to get container status \"99222f6f532e8093632159f0abb4c3c2f4ebac614a270467e0e7e71cea3735c8\": rpc error: code = NotFound desc = could not find container \"99222f6f532e8093632159f0abb4c3c2f4ebac614a270467e0e7e71cea3735c8\": container with ID starting with 99222f6f532e8093632159f0abb4c3c2f4ebac614a270467e0e7e71cea3735c8 not found: ID does not exist"
Sep 11 17:45:56.119233 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:56.119200    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ae170805-c6a4-469f-b280-8e7629749557-istio-token\") pod \"ae170805-c6a4-469f-b280-8e7629749557\" (UID: \"ae170805-c6a4-469f-b280-8e7629749557\") "
Sep 11 17:45:56.119363 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:56.119259    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae170805-c6a4-469f-b280-8e7629749557-credential-socket\") pod \"ae170805-c6a4-469f-b280-8e7629749557\" (UID: \"ae170805-c6a4-469f-b280-8e7629749557\") "
Sep 11 17:45:56.119363 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:56.119292    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rkzcg\" (UniqueName: \"kubernetes.io/projected/ae170805-c6a4-469f-b280-8e7629749557-kube-api-access-rkzcg\") pod \"ae170805-c6a4-469f-b280-8e7629749557\" (UID: \"ae170805-c6a4-469f-b280-8e7629749557\") "
Sep 11 17:45:56.119363 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:56.119320    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ae170805-c6a4-469f-b280-8e7629749557-istio-data\") pod \"ae170805-c6a4-469f-b280-8e7629749557\" (UID: \"ae170805-c6a4-469f-b280-8e7629749557\") "
Sep 11 17:45:56.119525 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:56.119385    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ae170805-c6a4-469f-b280-8e7629749557-istio-podinfo\") pod \"ae170805-c6a4-469f-b280-8e7629749557\" (UID: \"ae170805-c6a4-469f-b280-8e7629749557\") "
Sep 11 17:45:56.119525 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:56.119417    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ae170805-c6a4-469f-b280-8e7629749557-istiod-ca-cert\") pod \"ae170805-c6a4-469f-b280-8e7629749557\" (UID: \"ae170805-c6a4-469f-b280-8e7629749557\") "
Sep 11 17:45:56.119525 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:56.119461    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae170805-c6a4-469f-b280-8e7629749557-workload-socket\") pod \"ae170805-c6a4-469f-b280-8e7629749557\" (UID: \"ae170805-c6a4-469f-b280-8e7629749557\") "
Sep 11 17:45:56.119525 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:56.119502    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ae170805-c6a4-469f-b280-8e7629749557-workload-certs\") pod \"ae170805-c6a4-469f-b280-8e7629749557\" (UID: \"ae170805-c6a4-469f-b280-8e7629749557\") "
Sep 11 17:45:56.119719 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:56.119544    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ae170805-c6a4-469f-b280-8e7629749557-istio-envoy\") pod \"ae170805-c6a4-469f-b280-8e7629749557\" (UID: \"ae170805-c6a4-469f-b280-8e7629749557\") "
Sep 11 17:45:56.119719 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:56.119602    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae170805-c6a4-469f-b280-8e7629749557-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ae170805-c6a4-469f-b280-8e7629749557" (UID: "ae170805-c6a4-469f-b280-8e7629749557"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:45:56.119719 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:56.119660    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae170805-c6a4-469f-b280-8e7629749557-istio-data" (OuterVolumeSpecName: "istio-data") pod "ae170805-c6a4-469f-b280-8e7629749557" (UID: "ae170805-c6a4-469f-b280-8e7629749557"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:45:56.119938 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:56.119907    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae170805-c6a4-469f-b280-8e7629749557-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ae170805-c6a4-469f-b280-8e7629749557" (UID: "ae170805-c6a4-469f-b280-8e7629749557"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:45:56.120023 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:56.119938    2552 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae170805-c6a4-469f-b280-8e7629749557-credential-socket\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:45:56.120023 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:56.119959    2552 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ae170805-c6a4-469f-b280-8e7629749557-istio-data\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:45:56.120023 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:56.119977    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ae170805-c6a4-469f-b280-8e7629749557-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ae170805-c6a4-469f-b280-8e7629749557" (UID: "ae170805-c6a4-469f-b280-8e7629749557"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 17:45:56.120281 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:56.120041    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae170805-c6a4-469f-b280-8e7629749557-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ae170805-c6a4-469f-b280-8e7629749557" (UID: "ae170805-c6a4-469f-b280-8e7629749557"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:45:56.121599 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:56.121549    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ae170805-c6a4-469f-b280-8e7629749557-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ae170805-c6a4-469f-b280-8e7629749557" (UID: "ae170805-c6a4-469f-b280-8e7629749557"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 17:45:56.121599 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:56.121557    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ae170805-c6a4-469f-b280-8e7629749557-istio-token" (OuterVolumeSpecName: "istio-token") pod "ae170805-c6a4-469f-b280-8e7629749557" (UID: "ae170805-c6a4-469f-b280-8e7629749557"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:45:56.121761 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:56.121742    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ae170805-c6a4-469f-b280-8e7629749557-kube-api-access-rkzcg" (OuterVolumeSpecName: "kube-api-access-rkzcg") pod "ae170805-c6a4-469f-b280-8e7629749557" (UID: "ae170805-c6a4-469f-b280-8e7629749557"). InnerVolumeSpecName "kube-api-access-rkzcg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:45:56.121803 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:56.121747    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae170805-c6a4-469f-b280-8e7629749557-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ae170805-c6a4-469f-b280-8e7629749557" (UID: "ae170805-c6a4-469f-b280-8e7629749557"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:45:56.220859 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:56.220828    2552 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ae170805-c6a4-469f-b280-8e7629749557-workload-certs\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:45:56.220859 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:56.220855    2552 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ae170805-c6a4-469f-b280-8e7629749557-istio-envoy\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:45:56.220859 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:56.220866    2552 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ae170805-c6a4-469f-b280-8e7629749557-istio-token\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:45:56.221098 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:56.220874    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rkzcg\" (UniqueName: \"kubernetes.io/projected/ae170805-c6a4-469f-b280-8e7629749557-kube-api-access-rkzcg\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:45:56.221098 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:56.220883    2552 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ae170805-c6a4-469f-b280-8e7629749557-istio-podinfo\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:45:56.221098 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:56.220891    2552 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ae170805-c6a4-469f-b280-8e7629749557-istiod-ca-cert\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:45:56.221098 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:56.220898    2552 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae170805-c6a4-469f-b280-8e7629749557-workload-socket\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:45:56.396377 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:56.396349    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-144bfa-openshift-default-5dc996f7dd-ws7l7"]
Sep 11 17:45:56.398761 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:56.398739    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-144bfa-openshift-default-5dc996f7dd-ws7l7"]
Sep 11 17:45:57.949715 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:45:57.949681    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ae170805-c6a4-469f-b280-8e7629749557" path="/var/lib/kubelet/pods/ae170805-c6a4-469f-b280-8e7629749557/volumes"
Sep 11 17:46:09.246593 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:46:09.246559    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:46:09.346266 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:46:09.346227    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:46:11.054343 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:46:11.054313    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:46:16.790665 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:46:16.790631    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:46:24.450996 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:46:24.450955    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7689-c71de7"]
Sep 11 17:46:24.451546 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:46:24.451524    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae170805-c6a4-469f-b280-8e7629749557" containerName="istio-proxy"
Sep 11 17:46:24.451643 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:46:24.451549    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae170805-c6a4-469f-b280-8e7629749557" containerName="istio-proxy"
Sep 11 17:46:24.451643 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:46:24.451560    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb3e4bcb-fefa-4488-be59-1ac3b5b56bd4" containerName="istio-proxy"
Sep 11 17:46:24.451643 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:46:24.451569    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb3e4bcb-fefa-4488-be59-1ac3b5b56bd4" containerName="istio-proxy"
Sep 11 17:46:24.451643 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:46:24.451591    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="01c753eb-e3ca-4a2b-8b9b-c3cc7b672d7d" containerName="main"
Sep 11 17:46:24.451643 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:46:24.451601    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="01c753eb-e3ca-4a2b-8b9b-c3cc7b672d7d" containerName="main"
Sep 11 17:46:24.451643 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:46:24.451622    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="474d5c3f-d5e3-40bf-bc71-c9584b4efeec" containerName="maas-api"
Sep 11 17:46:24.451643 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:46:24.451631    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="474d5c3f-d5e3-40bf-bc71-c9584b4efeec" containerName="maas-api"
Sep 11 17:46:24.451643 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:46:24.451640    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dcd7f9a8-1a7a-4eb8-abf0-0945972796a9" containerName="cleanup"
Sep 11 17:46:24.452053 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:46:24.451650    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="dcd7f9a8-1a7a-4eb8-abf0-0945972796a9" containerName="cleanup"
Sep 11 17:46:24.452053 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:46:24.451673    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae170805-c6a4-469f-b280-8e7629749557" containerName="istio-proxy"
Sep 11 17:46:24.452053 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:46:24.451683    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae170805-c6a4-469f-b280-8e7629749557" containerName="istio-proxy"
Sep 11 17:46:24.452053 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:46:24.451691    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c624a168-2aee-4e51-b69f-106a4c0eedf4" containerName="main"
Sep 11 17:46:24.452053 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:46:24.451699    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="c624a168-2aee-4e51-b69f-106a4c0eedf4" containerName="main"
Sep 11 17:46:24.452053 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:46:24.451794    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="474d5c3f-d5e3-40bf-bc71-c9584b4efeec" containerName="maas-api"
Sep 11 17:46:24.452053 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:46:24.451818    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="ae170805-c6a4-469f-b280-8e7629749557" containerName="istio-proxy"
Sep 11 17:46:24.452053 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:46:24.451827    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="c624a168-2aee-4e51-b69f-106a4c0eedf4" containerName="main"
Sep 11 17:46:24.452053 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:46:24.451836    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="39af155e-9284-42e3-b5f4-d6f3c4be1757" containerName="istio-proxy"
Sep 11 17:46:24.452053 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:46:24.451847    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="01c753eb-e3ca-4a2b-8b9b-c3cc7b672d7d" containerName="main"
Sep 11 17:46:24.452053 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:46:24.451854    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="bb3e4bcb-fefa-4488-be59-1ac3b5b56bd4" containerName="istio-proxy"
Sep 11 17:46:24.452053 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:46:24.451864    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="ae170805-c6a4-469f-b280-8e7629749557" containerName="istio-proxy"
Sep 11 17:46:24.452053 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:46:24.451875    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="dcd7f9a8-1a7a-4eb8-abf0-0945972796a9" containerName="cleanup"
Sep 11 17:46:24.452053 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:46:24.451886    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="bb3e4bcb-fefa-4488-be59-1ac3b5b56bd4" containerName="istio-proxy"
Sep 11 17:46:24.455042 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:46:24.455022    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7689-c71de7"
Sep 11 17:46:24.457559 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:46:24.457541    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-6ptn5\""
Sep 11 17:46:24.461060 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:46:24.461030    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7689-c71de7"]
Sep 11 17:46:24.481287 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:46:24.481246    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ggw87\" (UniqueName: \"kubernetes.io/projected/ec258467-5d81-4def-b2a8-edf54906d337-kube-api-access-ggw87\") pod \"test-curl-7689-c71de7\" (UID: \"ec258467-5d81-4def-b2a8-edf54906d337\") " pod="opendatahub/test-curl-7689-c71de7"
Sep 11 17:46:24.581817 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:46:24.581781    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ggw87\" (UniqueName: \"kubernetes.io/projected/ec258467-5d81-4def-b2a8-edf54906d337-kube-api-access-ggw87\") pod \"test-curl-7689-c71de7\" (UID: \"ec258467-5d81-4def-b2a8-edf54906d337\") " pod="opendatahub/test-curl-7689-c71de7"
Sep 11 17:46:24.590529 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:46:24.590502    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ggw87\" (UniqueName: \"kubernetes.io/projected/ec258467-5d81-4def-b2a8-edf54906d337-kube-api-access-ggw87\") pod \"test-curl-7689-c71de7\" (UID: \"ec258467-5d81-4def-b2a8-edf54906d337\") " pod="opendatahub/test-curl-7689-c71de7"
Sep 11 17:46:24.766397 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:46:24.766300    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7689-c71de7"
Sep 11 17:46:25.039996 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:46:25.039972    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7689-c71de7"]
Sep 11 17:46:25.041822 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:46:25.041780    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podec258467_5d81_4def_b2a8_edf54906d337.slice/crio-ede370dbafa7df90d7d727fa7f23147d4881d33ef418c75816edecc9f6f1b04d WatchSource:0}: Error finding container ede370dbafa7df90d7d727fa7f23147d4881d33ef418c75816edecc9f6f1b04d: Status 404 returned error can't find the container with id ede370dbafa7df90d7d727fa7f23147d4881d33ef418c75816edecc9f6f1b04d
Sep 11 17:46:25.244900 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:46:25.244859    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7689-c71de7" event={"ID":"ec258467-5d81-4def-b2a8-edf54906d337","Type":"ContainerStarted","Data":"eb4c2d87877ecf9f31cf0ee80a109e07b3f5e98607d02ab0e86549317e3d8f6a"}
Sep 11 17:46:25.245081 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:46:25.244909    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7689-c71de7" event={"ID":"ec258467-5d81-4def-b2a8-edf54906d337","Type":"ContainerStarted","Data":"ede370dbafa7df90d7d727fa7f23147d4881d33ef418c75816edecc9f6f1b04d"}
Sep 11 17:46:25.273919 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:46:25.273869    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7689-c71de7" podStartSLOduration=1.2738532 podStartE2EDuration="1.2738532s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:46:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:46:25.272433925 +0000 UTC m=+1679.793725444" watchObservedRunningTime="2026-09-11 17:46:25.2738532 +0000 UTC m=+1679.795144720"
Sep 11 17:46:25.956127 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:46:25.956008    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7689-c71de7"]
Sep 11 17:46:25.960019 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:46:25.959990    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7689-c71de7"]
Sep 11 17:46:26.251478 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:46:26.251402    2552 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7689-c71de7" container="test-curl-7689-c71de7"
Sep 11 17:46:27.256975 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:46:27.256938    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7689-c71de7" podUID="ec258467-5d81-4def-b2a8-edf54906d337" containerName="test-curl-7689-c71de7" containerID="cri-o://eb4c2d87877ecf9f31cf0ee80a109e07b3f5e98607d02ab0e86549317e3d8f6a" gracePeriod=2
Sep 11 17:46:27.258782 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:46:27.258747    2552 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7689-c71de7\" is forbidden: User \"system:node:ip-10-0-140-51.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-51.ec2.internal' and this object" podUID="ec258467-5d81-4def-b2a8-edf54906d337" pod="opendatahub/test-curl-7689-c71de7"
Sep 11 17:46:27.303387 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:46:27.303350    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-2fac54/auth-test-model-2fac54-kserve-58fc867967-wdj45"]
Sep 11 17:46:27.305468 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:46:27.305433    2552 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7689-c71de7\" is forbidden: User \"system:node:ip-10-0-140-51.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-51.ec2.internal' and this object" podUID="ec258467-5d81-4def-b2a8-edf54906d337" pod="opendatahub/test-curl-7689-c71de7"
Sep 11 17:46:27.317121 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:46:27.317075    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:46:28.307568 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:46:28.307530    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:46:29.433752 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:46:29.433728    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7689-c71de7"
Sep 11 17:46:29.520049 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:46:29.519949    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ggw87\" (UniqueName: \"kubernetes.io/projected/ec258467-5d81-4def-b2a8-edf54906d337-kube-api-access-ggw87\") pod \"ec258467-5d81-4def-b2a8-edf54906d337\" (UID: \"ec258467-5d81-4def-b2a8-edf54906d337\") "
Sep 11 17:46:29.522047 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:46:29.522020    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ec258467-5d81-4def-b2a8-edf54906d337-kube-api-access-ggw87" (OuterVolumeSpecName: "kube-api-access-ggw87") pod "ec258467-5d81-4def-b2a8-edf54906d337" (UID: "ec258467-5d81-4def-b2a8-edf54906d337"). InnerVolumeSpecName "kube-api-access-ggw87". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:46:29.621280 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:46:29.621241    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ggw87\" (UniqueName: \"kubernetes.io/projected/ec258467-5d81-4def-b2a8-edf54906d337-kube-api-access-ggw87\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:46:29.949840 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:46:29.949808    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ec258467-5d81-4def-b2a8-edf54906d337" path="/var/lib/kubelet/pods/ec258467-5d81-4def-b2a8-edf54906d337/volumes"
Sep 11 17:46:30.276087 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:46:30.275999    2552 generic.go:356] "Generic (PLEG): container finished" podID="ec258467-5d81-4def-b2a8-edf54906d337" containerID="eb4c2d87877ecf9f31cf0ee80a109e07b3f5e98607d02ab0e86549317e3d8f6a" exitCode=137
Sep 11 17:46:30.276087 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:46:30.276037    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7689-c71de7"
Sep 11 17:46:30.276338 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:46:30.276092    2552 scope.go:117] "RemoveContainer" containerID="eb4c2d87877ecf9f31cf0ee80a109e07b3f5e98607d02ab0e86549317e3d8f6a"
Sep 11 17:46:30.285226 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:46:30.285209    2552 scope.go:117] "RemoveContainer" containerID="eb4c2d87877ecf9f31cf0ee80a109e07b3f5e98607d02ab0e86549317e3d8f6a"
Sep 11 17:46:30.285501 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:46:30.285481    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eb4c2d87877ecf9f31cf0ee80a109e07b3f5e98607d02ab0e86549317e3d8f6a\": container with ID starting with eb4c2d87877ecf9f31cf0ee80a109e07b3f5e98607d02ab0e86549317e3d8f6a not found: ID does not exist" containerID="eb4c2d87877ecf9f31cf0ee80a109e07b3f5e98607d02ab0e86549317e3d8f6a"
Sep 11 17:46:30.285558 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:46:30.285509    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eb4c2d87877ecf9f31cf0ee80a109e07b3f5e98607d02ab0e86549317e3d8f6a"} err="failed to get container status \"eb4c2d87877ecf9f31cf0ee80a109e07b3f5e98607d02ab0e86549317e3d8f6a\": rpc error: code = NotFound desc = could not find container \"eb4c2d87877ecf9f31cf0ee80a109e07b3f5e98607d02ab0e86549317e3d8f6a\": container with ID starting with eb4c2d87877ecf9f31cf0ee80a109e07b3f5e98607d02ab0e86549317e3d8f6a not found: ID does not exist"
Sep 11 17:46:35.129477 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:46:35.129441    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:46:35.231123 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:46:35.231066    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:46:35.328329 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:46:35.328297    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:46:42.322583 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:46:42.322542    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-2fac54/auth-test-model-2fac54-kserve-58fc867967-wdj45" podUID="d49b97bb-b28a-480d-8adb-a4d30e863b18" containerName="main" containerID="cri-o://c5dc0e84f9fd5f82092ecf486050445ed69584f47013eda8f381983ee910b20a" gracePeriod=45
Sep 11 17:46:42.586413 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:46:42.586355    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-2fac54/auth-test-model-2fac54-kserve-58fc867967-wdj45"
Sep 11 17:46:42.641310 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:46:42.641274    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d49b97bb-b28a-480d-8adb-a4d30e863b18-tls-certs\") pod \"d49b97bb-b28a-480d-8adb-a4d30e863b18\" (UID: \"d49b97bb-b28a-480d-8adb-a4d30e863b18\") "
Sep 11 17:46:42.641486 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:46:42.641381    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zt2dl\" (UniqueName: \"kubernetes.io/projected/d49b97bb-b28a-480d-8adb-a4d30e863b18-kube-api-access-zt2dl\") pod \"d49b97bb-b28a-480d-8adb-a4d30e863b18\" (UID: \"d49b97bb-b28a-480d-8adb-a4d30e863b18\") "
Sep 11 17:46:42.641486 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:46:42.641445    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d49b97bb-b28a-480d-8adb-a4d30e863b18-tmp-dir\") pod \"d49b97bb-b28a-480d-8adb-a4d30e863b18\" (UID: \"d49b97bb-b28a-480d-8adb-a4d30e863b18\") "
Sep 11 17:46:42.641612 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:46:42.641558    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d49b97bb-b28a-480d-8adb-a4d30e863b18-dshm\") pod \"d49b97bb-b28a-480d-8adb-a4d30e863b18\" (UID: \"d49b97bb-b28a-480d-8adb-a4d30e863b18\") "
Sep 11 17:46:42.641612 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:46:42.641603    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d49b97bb-b28a-480d-8adb-a4d30e863b18-home\") pod \"d49b97bb-b28a-480d-8adb-a4d30e863b18\" (UID: \"d49b97bb-b28a-480d-8adb-a4d30e863b18\") "
Sep 11 17:46:42.641724 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:46:42.641641    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d49b97bb-b28a-480d-8adb-a4d30e863b18-model-cache\") pod \"d49b97bb-b28a-480d-8adb-a4d30e863b18\" (UID: \"d49b97bb-b28a-480d-8adb-a4d30e863b18\") "
Sep 11 17:46:42.641724 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:46:42.641690    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d49b97bb-b28a-480d-8adb-a4d30e863b18-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d49b97bb-b28a-480d-8adb-a4d30e863b18" (UID: "d49b97bb-b28a-480d-8adb-a4d30e863b18"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:46:42.641909 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:46:42.641886    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d49b97bb-b28a-480d-8adb-a4d30e863b18-home" (OuterVolumeSpecName: "home") pod "d49b97bb-b28a-480d-8adb-a4d30e863b18" (UID: "d49b97bb-b28a-480d-8adb-a4d30e863b18"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:46:42.641998 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:46:42.641971    2552 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d49b97bb-b28a-480d-8adb-a4d30e863b18-tmp-dir\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:46:42.641998 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:46:42.641987    2552 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d49b97bb-b28a-480d-8adb-a4d30e863b18-home\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:46:42.642174 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:46:42.642096    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d49b97bb-b28a-480d-8adb-a4d30e863b18-model-cache" (OuterVolumeSpecName: "model-cache") pod "d49b97bb-b28a-480d-8adb-a4d30e863b18" (UID: "d49b97bb-b28a-480d-8adb-a4d30e863b18"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:46:42.643673 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:46:42.643643    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d49b97bb-b28a-480d-8adb-a4d30e863b18-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d49b97bb-b28a-480d-8adb-a4d30e863b18" (UID: "d49b97bb-b28a-480d-8adb-a4d30e863b18"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 17:46:42.643958 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:46:42.643935    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d49b97bb-b28a-480d-8adb-a4d30e863b18-dshm" (OuterVolumeSpecName: "dshm") pod "d49b97bb-b28a-480d-8adb-a4d30e863b18" (UID: "d49b97bb-b28a-480d-8adb-a4d30e863b18"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:46:42.644013 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:46:42.643959    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d49b97bb-b28a-480d-8adb-a4d30e863b18-kube-api-access-zt2dl" (OuterVolumeSpecName: "kube-api-access-zt2dl") pod "d49b97bb-b28a-480d-8adb-a4d30e863b18" (UID: "d49b97bb-b28a-480d-8adb-a4d30e863b18"). InnerVolumeSpecName "kube-api-access-zt2dl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:46:42.742538 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:46:42.742495    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zt2dl\" (UniqueName: \"kubernetes.io/projected/d49b97bb-b28a-480d-8adb-a4d30e863b18-kube-api-access-zt2dl\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:46:42.742538 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:46:42.742536    2552 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d49b97bb-b28a-480d-8adb-a4d30e863b18-dshm\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:46:42.742538 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:46:42.742546    2552 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d49b97bb-b28a-480d-8adb-a4d30e863b18-model-cache\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:46:42.742766 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:46:42.742556    2552 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d49b97bb-b28a-480d-8adb-a4d30e863b18-tls-certs\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:46:43.356058 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:46:43.356023    2552 generic.go:356] "Generic (PLEG): container finished" podID="d49b97bb-b28a-480d-8adb-a4d30e863b18" containerID="c5dc0e84f9fd5f82092ecf486050445ed69584f47013eda8f381983ee910b20a" exitCode=0
Sep 11 17:46:43.356570 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:46:43.356064    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-2fac54/auth-test-model-2fac54-kserve-58fc867967-wdj45"
Sep 11 17:46:43.356570 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:46:43.356121    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-2fac54/auth-test-model-2fac54-kserve-58fc867967-wdj45" event={"ID":"d49b97bb-b28a-480d-8adb-a4d30e863b18","Type":"ContainerDied","Data":"c5dc0e84f9fd5f82092ecf486050445ed69584f47013eda8f381983ee910b20a"}
Sep 11 17:46:43.356570 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:46:43.356173    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-2fac54/auth-test-model-2fac54-kserve-58fc867967-wdj45" event={"ID":"d49b97bb-b28a-480d-8adb-a4d30e863b18","Type":"ContainerDied","Data":"d247ad4fdf369e49cffaca6f1eb3a8527b3936d328fb6bd829fd625049778361"}
Sep 11 17:46:43.356570 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:46:43.356196    2552 scope.go:117] "RemoveContainer" containerID="c5dc0e84f9fd5f82092ecf486050445ed69584f47013eda8f381983ee910b20a"
Sep 11 17:46:43.365691 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:46:43.365674    2552 scope.go:117] "RemoveContainer" containerID="c5dc0e84f9fd5f82092ecf486050445ed69584f47013eda8f381983ee910b20a"
Sep 11 17:46:43.365971 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:46:43.365953    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c5dc0e84f9fd5f82092ecf486050445ed69584f47013eda8f381983ee910b20a\": container with ID starting with c5dc0e84f9fd5f82092ecf486050445ed69584f47013eda8f381983ee910b20a not found: ID does not exist" containerID="c5dc0e84f9fd5f82092ecf486050445ed69584f47013eda8f381983ee910b20a"
Sep 11 17:46:43.366049 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:46:43.365983    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c5dc0e84f9fd5f82092ecf486050445ed69584f47013eda8f381983ee910b20a"} err="failed to get container status \"c5dc0e84f9fd5f82092ecf486050445ed69584f47013eda8f381983ee910b20a\": rpc error: code = NotFound desc = could not find container \"c5dc0e84f9fd5f82092ecf486050445ed69584f47013eda8f381983ee910b20a\": container with ID starting with c5dc0e84f9fd5f82092ecf486050445ed69584f47013eda8f381983ee910b20a not found: ID does not exist"
Sep 11 17:46:43.377702 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:46:43.377672    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-2fac54/auth-test-model-2fac54-kserve-58fc867967-wdj45"]
Sep 11 17:46:43.382337 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:46:43.382317    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-2fac54/auth-test-model-2fac54-kserve-58fc867967-wdj45"]
Sep 11 17:46:43.949836 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:46:43.949806    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d49b97bb-b28a-480d-8adb-a4d30e863b18" path="/var/lib/kubelet/pods/d49b97bb-b28a-480d-8adb-a4d30e863b18/volumes"
Sep 11 17:46:55.881227 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:46:55.881187    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:46:55.975908 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:46:55.975856    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:47:12.874670 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:47:12.874638    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:47:41.417104 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:47:41.417064    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7689-df3710"]
Sep 11 17:47:41.417507 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:47:41.417491    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d49b97bb-b28a-480d-8adb-a4d30e863b18" containerName="main"
Sep 11 17:47:41.417553 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:47:41.417509    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="d49b97bb-b28a-480d-8adb-a4d30e863b18" containerName="main"
Sep 11 17:47:41.417553 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:47:41.417520    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb3e4bcb-fefa-4488-be59-1ac3b5b56bd4" containerName="istio-proxy"
Sep 11 17:47:41.417553 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:47:41.417526    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb3e4bcb-fefa-4488-be59-1ac3b5b56bd4" containerName="istio-proxy"
Sep 11 17:47:41.417553 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:47:41.417533    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ec258467-5d81-4def-b2a8-edf54906d337" containerName="test-curl-7689-c71de7"
Sep 11 17:47:41.417553 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:47:41.417539    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="ec258467-5d81-4def-b2a8-edf54906d337" containerName="test-curl-7689-c71de7"
Sep 11 17:47:41.417726 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:47:41.417602    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="d49b97bb-b28a-480d-8adb-a4d30e863b18" containerName="main"
Sep 11 17:47:41.417726 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:47:41.417611    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="ec258467-5d81-4def-b2a8-edf54906d337" containerName="test-curl-7689-c71de7"
Sep 11 17:47:41.420659 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:47:41.420641    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7689-df3710"
Sep 11 17:47:41.423188 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:47:41.423166    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-6ptn5\""
Sep 11 17:47:41.428732 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:47:41.428710    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7689-df3710"]
Sep 11 17:47:41.583338 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:47:41.583295    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-grb9q\" (UniqueName: \"kubernetes.io/projected/e1bad88b-55a0-45e1-a7fb-26c4f5bc3c71-kube-api-access-grb9q\") pod \"test-curl-7689-df3710\" (UID: \"e1bad88b-55a0-45e1-a7fb-26c4f5bc3c71\") " pod="opendatahub/test-curl-7689-df3710"
Sep 11 17:47:41.684841 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:47:41.684733    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-grb9q\" (UniqueName: \"kubernetes.io/projected/e1bad88b-55a0-45e1-a7fb-26c4f5bc3c71-kube-api-access-grb9q\") pod \"test-curl-7689-df3710\" (UID: \"e1bad88b-55a0-45e1-a7fb-26c4f5bc3c71\") " pod="opendatahub/test-curl-7689-df3710"
Sep 11 17:47:41.695898 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:47:41.695871    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-grb9q\" (UniqueName: \"kubernetes.io/projected/e1bad88b-55a0-45e1-a7fb-26c4f5bc3c71-kube-api-access-grb9q\") pod \"test-curl-7689-df3710\" (UID: \"e1bad88b-55a0-45e1-a7fb-26c4f5bc3c71\") " pod="opendatahub/test-curl-7689-df3710"
Sep 11 17:47:41.730220 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:47:41.730177    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7689-df3710"
Sep 11 17:47:42.004274 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:47:42.004249    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7689-df3710"]
Sep 11 17:47:42.005938 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:47:42.005911    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode1bad88b_55a0_45e1_a7fb_26c4f5bc3c71.slice/crio-c94860c4c82871116379371fedf33d9b8f63c57b9c8074d771bba7e6980acd0d WatchSource:0}: Error finding container c94860c4c82871116379371fedf33d9b8f63c57b9c8074d771bba7e6980acd0d: Status 404 returned error can't find the container with id c94860c4c82871116379371fedf33d9b8f63c57b9c8074d771bba7e6980acd0d
Sep 11 17:47:42.710119 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:47:42.710073    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7689-df3710" event={"ID":"e1bad88b-55a0-45e1-a7fb-26c4f5bc3c71","Type":"ContainerStarted","Data":"f1934738a7ff42ff51f75043c769a3d2054bc31faa06551460d9a069d33ce81d"}
Sep 11 17:47:42.710539 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:47:42.710135    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7689-df3710" event={"ID":"e1bad88b-55a0-45e1-a7fb-26c4f5bc3c71","Type":"ContainerStarted","Data":"c94860c4c82871116379371fedf33d9b8f63c57b9c8074d771bba7e6980acd0d"}
Sep 11 17:47:42.725047 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:47:42.725003    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7689-df3710" podStartSLOduration=1.724989543 podStartE2EDuration="1.724989543s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:47:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:47:42.723970249 +0000 UTC m=+1757.245261782" watchObservedRunningTime="2026-09-11 17:47:42.724989543 +0000 UTC m=+1757.246281079"
Sep 11 17:47:43.291504 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:47:43.291465    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7689-df3710"]
Sep 11 17:47:43.294763 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:47:43.294735    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7689-df3710"]
Sep 11 17:47:43.404806 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:47:43.404773    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7689-e589f5"]
Sep 11 17:47:43.405210 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:47:43.405194    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e1bad88b-55a0-45e1-a7fb-26c4f5bc3c71" containerName="test-curl-7689-df3710"
Sep 11 17:47:43.405270 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:47:43.405211    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="e1bad88b-55a0-45e1-a7fb-26c4f5bc3c71" containerName="test-curl-7689-df3710"
Sep 11 17:47:43.405307 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:47:43.405277    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="e1bad88b-55a0-45e1-a7fb-26c4f5bc3c71" containerName="test-curl-7689-df3710"
Sep 11 17:47:43.408174 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:47:43.408157    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7689-e589f5"
Sep 11 17:47:43.413072 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:47:43.413039    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7689-e589f5"]
Sep 11 17:47:43.499920 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:47:43.499877    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hg68n\" (UniqueName: \"kubernetes.io/projected/db8b92ab-6e7b-4026-9f70-80220203081d-kube-api-access-hg68n\") pod \"test-curl-7689-e589f5\" (UID: \"db8b92ab-6e7b-4026-9f70-80220203081d\") " pod="opendatahub/test-curl-7689-e589f5"
Sep 11 17:47:43.600914 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:47:43.600863    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hg68n\" (UniqueName: \"kubernetes.io/projected/db8b92ab-6e7b-4026-9f70-80220203081d-kube-api-access-hg68n\") pod \"test-curl-7689-e589f5\" (UID: \"db8b92ab-6e7b-4026-9f70-80220203081d\") " pod="opendatahub/test-curl-7689-e589f5"
Sep 11 17:47:43.610180 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:47:43.610147    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hg68n\" (UniqueName: \"kubernetes.io/projected/db8b92ab-6e7b-4026-9f70-80220203081d-kube-api-access-hg68n\") pod \"test-curl-7689-e589f5\" (UID: \"db8b92ab-6e7b-4026-9f70-80220203081d\") " pod="opendatahub/test-curl-7689-e589f5"
Sep 11 17:47:43.715548 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:47:43.715472    2552 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7689-df3710" container="test-curl-7689-df3710"
Sep 11 17:47:43.719268 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:47:43.719242    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7689-e589f5"
Sep 11 17:47:43.989340 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:47:43.989315    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7689-e589f5"]
Sep 11 17:47:43.991372 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:47:43.991342    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddb8b92ab_6e7b_4026_9f70_80220203081d.slice/crio-412d44a9e1d95854a5fe968218b20e5bbbe2807a61c5bb9355f0c5c1bfaaf025 WatchSource:0}: Error finding container 412d44a9e1d95854a5fe968218b20e5bbbe2807a61c5bb9355f0c5c1bfaaf025: Status 404 returned error can't find the container with id 412d44a9e1d95854a5fe968218b20e5bbbe2807a61c5bb9355f0c5c1bfaaf025
Sep 11 17:47:44.722420 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:47:44.722377    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7689-e589f5" event={"ID":"db8b92ab-6e7b-4026-9f70-80220203081d","Type":"ContainerStarted","Data":"759e220b739b22f0f44c43f3301fdd27ae68fc6bfc9a0b0baae03f3d517fd951"}
Sep 11 17:47:44.722828 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:47:44.722432    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7689-e589f5" event={"ID":"db8b92ab-6e7b-4026-9f70-80220203081d","Type":"ContainerStarted","Data":"412d44a9e1d95854a5fe968218b20e5bbbe2807a61c5bb9355f0c5c1bfaaf025"}
Sep 11 17:47:44.722828 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:47:44.722520    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7689-df3710" podUID="e1bad88b-55a0-45e1-a7fb-26c4f5bc3c71" containerName="test-curl-7689-df3710" containerID="cri-o://f1934738a7ff42ff51f75043c769a3d2054bc31faa06551460d9a069d33ce81d" gracePeriod=2
Sep 11 17:47:44.738162 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:47:44.738093    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7689-e589f5" podStartSLOduration=1.738072858 podStartE2EDuration="1.738072858s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:47:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:47:44.735716148 +0000 UTC m=+1759.257007671" watchObservedRunningTime="2026-09-11 17:47:44.738072858 +0000 UTC m=+1759.259364379"
Sep 11 17:47:45.309211 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:47:45.309176    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7689-e589f5"]
Sep 11 17:47:45.313397 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:47:45.313369    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7689-e589f5"]
Sep 11 17:47:45.728028 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:47:45.727998    2552 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7689-e589f5" container="test-curl-7689-e589f5"
Sep 11 17:47:46.733473 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:47:46.733440    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7689-e589f5" podUID="db8b92ab-6e7b-4026-9f70-80220203081d" containerName="test-curl-7689-e589f5" containerID="cri-o://759e220b739b22f0f44c43f3301fdd27ae68fc6bfc9a0b0baae03f3d517fd951" gracePeriod=2
Sep 11 17:47:46.888361 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:47:46.888336    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7689-df3710"
Sep 11 17:47:47.033882 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:47:47.033797    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-grb9q\" (UniqueName: \"kubernetes.io/projected/e1bad88b-55a0-45e1-a7fb-26c4f5bc3c71-kube-api-access-grb9q\") pod \"e1bad88b-55a0-45e1-a7fb-26c4f5bc3c71\" (UID: \"e1bad88b-55a0-45e1-a7fb-26c4f5bc3c71\") "
Sep 11 17:47:47.035819 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:47:47.035784    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e1bad88b-55a0-45e1-a7fb-26c4f5bc3c71-kube-api-access-grb9q" (OuterVolumeSpecName: "kube-api-access-grb9q") pod "e1bad88b-55a0-45e1-a7fb-26c4f5bc3c71" (UID: "e1bad88b-55a0-45e1-a7fb-26c4f5bc3c71"). InnerVolumeSpecName "kube-api-access-grb9q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:47:47.134843 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:47:47.134804    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-grb9q\" (UniqueName: \"kubernetes.io/projected/e1bad88b-55a0-45e1-a7fb-26c4f5bc3c71-kube-api-access-grb9q\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:47:47.378227 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:47:47.378192    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:47:47.742686 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:47:47.742596    2552 generic.go:356] "Generic (PLEG): container finished" podID="e1bad88b-55a0-45e1-a7fb-26c4f5bc3c71" containerID="f1934738a7ff42ff51f75043c769a3d2054bc31faa06551460d9a069d33ce81d" exitCode=137
Sep 11 17:47:47.742686 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:47:47.742658    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7689-df3710"
Sep 11 17:47:47.743252 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:47:47.742694    2552 scope.go:117] "RemoveContainer" containerID="f1934738a7ff42ff51f75043c769a3d2054bc31faa06551460d9a069d33ce81d"
Sep 11 17:47:47.751985 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:47:47.751970    2552 scope.go:117] "RemoveContainer" containerID="f1934738a7ff42ff51f75043c769a3d2054bc31faa06551460d9a069d33ce81d"
Sep 11 17:47:47.752284 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:47:47.752265    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f1934738a7ff42ff51f75043c769a3d2054bc31faa06551460d9a069d33ce81d\": container with ID starting with f1934738a7ff42ff51f75043c769a3d2054bc31faa06551460d9a069d33ce81d not found: ID does not exist" containerID="f1934738a7ff42ff51f75043c769a3d2054bc31faa06551460d9a069d33ce81d"
Sep 11 17:47:47.752365 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:47:47.752291    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f1934738a7ff42ff51f75043c769a3d2054bc31faa06551460d9a069d33ce81d"} err="failed to get container status \"f1934738a7ff42ff51f75043c769a3d2054bc31faa06551460d9a069d33ce81d\": rpc error: code = NotFound desc = could not find container \"f1934738a7ff42ff51f75043c769a3d2054bc31faa06551460d9a069d33ce81d\": container with ID starting with f1934738a7ff42ff51f75043c769a3d2054bc31faa06551460d9a069d33ce81d not found: ID does not exist"
Sep 11 17:47:47.752608 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:47:47.752590    2552 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7689-df3710\" is forbidden: User \"system:node:ip-10-0-140-51.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-51.ec2.internal' and this object" podUID="e1bad88b-55a0-45e1-a7fb-26c4f5bc3c71" pod="opendatahub/test-curl-7689-df3710"
Sep 11 17:47:47.949545 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:47:47.949509    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e1bad88b-55a0-45e1-a7fb-26c4f5bc3c71" path="/var/lib/kubelet/pods/e1bad88b-55a0-45e1-a7fb-26c4f5bc3c71/volumes"
Sep 11 17:47:48.324004 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:47:48.323972    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:47:48.911987 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:47:48.911965    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7689-e589f5"
Sep 11 17:47:49.052449 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:47:49.052366    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hg68n\" (UniqueName: \"kubernetes.io/projected/db8b92ab-6e7b-4026-9f70-80220203081d-kube-api-access-hg68n\") pod \"db8b92ab-6e7b-4026-9f70-80220203081d\" (UID: \"db8b92ab-6e7b-4026-9f70-80220203081d\") "
Sep 11 17:47:49.054386 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:47:49.054364    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/db8b92ab-6e7b-4026-9f70-80220203081d-kube-api-access-hg68n" (OuterVolumeSpecName: "kube-api-access-hg68n") pod "db8b92ab-6e7b-4026-9f70-80220203081d" (UID: "db8b92ab-6e7b-4026-9f70-80220203081d"). InnerVolumeSpecName "kube-api-access-hg68n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:47:49.153088 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:47:49.153057    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hg68n\" (UniqueName: \"kubernetes.io/projected/db8b92ab-6e7b-4026-9f70-80220203081d-kube-api-access-hg68n\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:47:49.757568 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:47:49.757530    2552 generic.go:356] "Generic (PLEG): container finished" podID="db8b92ab-6e7b-4026-9f70-80220203081d" containerID="759e220b739b22f0f44c43f3301fdd27ae68fc6bfc9a0b0baae03f3d517fd951" exitCode=137
Sep 11 17:47:49.757766 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:47:49.757576    2552 scope.go:117] "RemoveContainer" containerID="759e220b739b22f0f44c43f3301fdd27ae68fc6bfc9a0b0baae03f3d517fd951"
Sep 11 17:47:49.757766 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:47:49.757592    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7689-e589f5"
Sep 11 17:47:49.766696 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:47:49.766678    2552 scope.go:117] "RemoveContainer" containerID="759e220b739b22f0f44c43f3301fdd27ae68fc6bfc9a0b0baae03f3d517fd951"
Sep 11 17:47:49.766962 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:47:49.766944    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"759e220b739b22f0f44c43f3301fdd27ae68fc6bfc9a0b0baae03f3d517fd951\": container with ID starting with 759e220b739b22f0f44c43f3301fdd27ae68fc6bfc9a0b0baae03f3d517fd951 not found: ID does not exist" containerID="759e220b739b22f0f44c43f3301fdd27ae68fc6bfc9a0b0baae03f3d517fd951"
Sep 11 17:47:49.766998 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:47:49.766972    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"759e220b739b22f0f44c43f3301fdd27ae68fc6bfc9a0b0baae03f3d517fd951"} err="failed to get container status \"759e220b739b22f0f44c43f3301fdd27ae68fc6bfc9a0b0baae03f3d517fd951\": rpc error: code = NotFound desc = could not find container \"759e220b739b22f0f44c43f3301fdd27ae68fc6bfc9a0b0baae03f3d517fd951\": container with ID starting with 759e220b739b22f0f44c43f3301fdd27ae68fc6bfc9a0b0baae03f3d517fd951 not found: ID does not exist"
Sep 11 17:47:49.767265 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:47:49.767241    2552 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7689-e589f5\" is forbidden: User \"system:node:ip-10-0-140-51.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-51.ec2.internal' and this object" podUID="db8b92ab-6e7b-4026-9f70-80220203081d" pod="opendatahub/test-curl-7689-e589f5"
Sep 11 17:47:49.950377 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:47:49.950348    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="db8b92ab-6e7b-4026-9f70-80220203081d" path="/var/lib/kubelet/pods/db8b92ab-6e7b-4026-9f70-80220203081d/volumes"
Sep 11 17:48:11.936515 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:48:11.936480    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:48:13.541902 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:48:13.541869    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:48:26.107662 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:48:26.107630    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-hg7p8_eb5f3be1-2e36-40ba-b3a0-3bec413056e8/console-operator/1.log"
Sep 11 17:48:26.111521 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:48:26.111494    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6k9tr_0620c0a0-8070-4307-86d9-c69a76ea14a9/ovn-acl-logging/0.log"
Sep 11 17:48:26.117590 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:48:26.117568    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-hg7p8_eb5f3be1-2e36-40ba-b3a0-3bec413056e8/console-operator/1.log"
Sep 11 17:48:26.120966 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:48:26.120946    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6k9tr_0620c0a0-8070-4307-86d9-c69a76ea14a9/ovn-acl-logging/0.log"
Sep 11 17:48:47.862751 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:48:47.862716    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:49:18.631523 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:49:18.631484    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:50:08.466424 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:08.466389    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:50:09.410163 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:09.410131    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:50:09.850548 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:09.850512    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:50:10.122588 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:10.122513    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-027edd-openshift-default-6f84795f8b-46ktf"]
Sep 11 17:50:10.122902 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:10.122890    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="db8b92ab-6e7b-4026-9f70-80220203081d" containerName="test-curl-7689-e589f5"
Sep 11 17:50:10.122947 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:10.122905    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="db8b92ab-6e7b-4026-9f70-80220203081d" containerName="test-curl-7689-e589f5"
Sep 11 17:50:10.122989 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:10.122979    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="db8b92ab-6e7b-4026-9f70-80220203081d" containerName="test-curl-7689-e589f5"
Sep 11 17:50:10.126101 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:10.126082    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-027edd-openshift-default-6f84795f8b-46ktf"
Sep 11 17:50:10.134823 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:10.134805    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-027edd-openshift-default-dockercfg-f889v\""
Sep 11 17:50:10.154241 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:10.154208    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-027edd-openshift-default-6f84795f8b-46ktf"]
Sep 11 17:50:10.176209 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:10.176178    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:50:10.223347 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:10.223320    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/96210882-8a70-4795-910a-01c409c73c78-istio-token\") pod \"e2e-ipp-a-027edd-openshift-default-6f84795f8b-46ktf\" (UID: \"96210882-8a70-4795-910a-01c409c73c78\") " pod="openshift-ingress/e2e-ipp-a-027edd-openshift-default-6f84795f8b-46ktf"
Sep 11 17:50:10.223347 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:10.223349    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/96210882-8a70-4795-910a-01c409c73c78-istiod-ca-cert\") pod \"e2e-ipp-a-027edd-openshift-default-6f84795f8b-46ktf\" (UID: \"96210882-8a70-4795-910a-01c409c73c78\") " pod="openshift-ingress/e2e-ipp-a-027edd-openshift-default-6f84795f8b-46ktf"
Sep 11 17:50:10.223566 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:10.223432    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/96210882-8a70-4795-910a-01c409c73c78-credential-socket\") pod \"e2e-ipp-a-027edd-openshift-default-6f84795f8b-46ktf\" (UID: \"96210882-8a70-4795-910a-01c409c73c78\") " pod="openshift-ingress/e2e-ipp-a-027edd-openshift-default-6f84795f8b-46ktf"
Sep 11 17:50:10.223566 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:10.223487    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/96210882-8a70-4795-910a-01c409c73c78-istio-podinfo\") pod \"e2e-ipp-a-027edd-openshift-default-6f84795f8b-46ktf\" (UID: \"96210882-8a70-4795-910a-01c409c73c78\") " pod="openshift-ingress/e2e-ipp-a-027edd-openshift-default-6f84795f8b-46ktf"
Sep 11 17:50:10.223566 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:10.223518    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/96210882-8a70-4795-910a-01c409c73c78-istio-data\") pod \"e2e-ipp-a-027edd-openshift-default-6f84795f8b-46ktf\" (UID: \"96210882-8a70-4795-910a-01c409c73c78\") " pod="openshift-ingress/e2e-ipp-a-027edd-openshift-default-6f84795f8b-46ktf"
Sep 11 17:50:10.223727 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:10.223619    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/96210882-8a70-4795-910a-01c409c73c78-istio-envoy\") pod \"e2e-ipp-a-027edd-openshift-default-6f84795f8b-46ktf\" (UID: \"96210882-8a70-4795-910a-01c409c73c78\") " pod="openshift-ingress/e2e-ipp-a-027edd-openshift-default-6f84795f8b-46ktf"
Sep 11 17:50:10.223727 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:10.223680    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pl9r5\" (UniqueName: \"kubernetes.io/projected/96210882-8a70-4795-910a-01c409c73c78-kube-api-access-pl9r5\") pod \"e2e-ipp-a-027edd-openshift-default-6f84795f8b-46ktf\" (UID: \"96210882-8a70-4795-910a-01c409c73c78\") " pod="openshift-ingress/e2e-ipp-a-027edd-openshift-default-6f84795f8b-46ktf"
Sep 11 17:50:10.223727 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:10.223704    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/96210882-8a70-4795-910a-01c409c73c78-workload-certs\") pod \"e2e-ipp-a-027edd-openshift-default-6f84795f8b-46ktf\" (UID: \"96210882-8a70-4795-910a-01c409c73c78\") " pod="openshift-ingress/e2e-ipp-a-027edd-openshift-default-6f84795f8b-46ktf"
Sep 11 17:50:10.223840 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:10.223786    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/96210882-8a70-4795-910a-01c409c73c78-workload-socket\") pod \"e2e-ipp-a-027edd-openshift-default-6f84795f8b-46ktf\" (UID: \"96210882-8a70-4795-910a-01c409c73c78\") " pod="openshift-ingress/e2e-ipp-a-027edd-openshift-default-6f84795f8b-46ktf"
Sep 11 17:50:10.324377 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:10.324343    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/96210882-8a70-4795-910a-01c409c73c78-workload-socket\") pod \"e2e-ipp-a-027edd-openshift-default-6f84795f8b-46ktf\" (UID: \"96210882-8a70-4795-910a-01c409c73c78\") " pod="openshift-ingress/e2e-ipp-a-027edd-openshift-default-6f84795f8b-46ktf"
Sep 11 17:50:10.324580 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:10.324386    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/96210882-8a70-4795-910a-01c409c73c78-istio-token\") pod \"e2e-ipp-a-027edd-openshift-default-6f84795f8b-46ktf\" (UID: \"96210882-8a70-4795-910a-01c409c73c78\") " pod="openshift-ingress/e2e-ipp-a-027edd-openshift-default-6f84795f8b-46ktf"
Sep 11 17:50:10.324580 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:10.324403    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/96210882-8a70-4795-910a-01c409c73c78-istiod-ca-cert\") pod \"e2e-ipp-a-027edd-openshift-default-6f84795f8b-46ktf\" (UID: \"96210882-8a70-4795-910a-01c409c73c78\") " pod="openshift-ingress/e2e-ipp-a-027edd-openshift-default-6f84795f8b-46ktf"
Sep 11 17:50:10.324580 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:10.324524    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/96210882-8a70-4795-910a-01c409c73c78-credential-socket\") pod \"e2e-ipp-a-027edd-openshift-default-6f84795f8b-46ktf\" (UID: \"96210882-8a70-4795-910a-01c409c73c78\") " pod="openshift-ingress/e2e-ipp-a-027edd-openshift-default-6f84795f8b-46ktf"
Sep 11 17:50:10.324580 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:10.324571    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/96210882-8a70-4795-910a-01c409c73c78-istio-podinfo\") pod \"e2e-ipp-a-027edd-openshift-default-6f84795f8b-46ktf\" (UID: \"96210882-8a70-4795-910a-01c409c73c78\") " pod="openshift-ingress/e2e-ipp-a-027edd-openshift-default-6f84795f8b-46ktf"
Sep 11 17:50:10.324819 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:10.324599    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/96210882-8a70-4795-910a-01c409c73c78-istio-data\") pod \"e2e-ipp-a-027edd-openshift-default-6f84795f8b-46ktf\" (UID: \"96210882-8a70-4795-910a-01c409c73c78\") " pod="openshift-ingress/e2e-ipp-a-027edd-openshift-default-6f84795f8b-46ktf"
Sep 11 17:50:10.324819 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:10.324651    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/96210882-8a70-4795-910a-01c409c73c78-istio-envoy\") pod \"e2e-ipp-a-027edd-openshift-default-6f84795f8b-46ktf\" (UID: \"96210882-8a70-4795-910a-01c409c73c78\") " pod="openshift-ingress/e2e-ipp-a-027edd-openshift-default-6f84795f8b-46ktf"
Sep 11 17:50:10.324819 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:10.324708    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pl9r5\" (UniqueName: \"kubernetes.io/projected/96210882-8a70-4795-910a-01c409c73c78-kube-api-access-pl9r5\") pod \"e2e-ipp-a-027edd-openshift-default-6f84795f8b-46ktf\" (UID: \"96210882-8a70-4795-910a-01c409c73c78\") " pod="openshift-ingress/e2e-ipp-a-027edd-openshift-default-6f84795f8b-46ktf"
Sep 11 17:50:10.324819 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:10.324733    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/96210882-8a70-4795-910a-01c409c73c78-workload-certs\") pod \"e2e-ipp-a-027edd-openshift-default-6f84795f8b-46ktf\" (UID: \"96210882-8a70-4795-910a-01c409c73c78\") " pod="openshift-ingress/e2e-ipp-a-027edd-openshift-default-6f84795f8b-46ktf"
Sep 11 17:50:10.325005 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:10.324823    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/96210882-8a70-4795-910a-01c409c73c78-workload-socket\") pod \"e2e-ipp-a-027edd-openshift-default-6f84795f8b-46ktf\" (UID: \"96210882-8a70-4795-910a-01c409c73c78\") " pod="openshift-ingress/e2e-ipp-a-027edd-openshift-default-6f84795f8b-46ktf"
Sep 11 17:50:10.325005 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:10.324870    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/96210882-8a70-4795-910a-01c409c73c78-credential-socket\") pod \"e2e-ipp-a-027edd-openshift-default-6f84795f8b-46ktf\" (UID: \"96210882-8a70-4795-910a-01c409c73c78\") " pod="openshift-ingress/e2e-ipp-a-027edd-openshift-default-6f84795f8b-46ktf"
Sep 11 17:50:10.325005 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:10.324976    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/96210882-8a70-4795-910a-01c409c73c78-istio-data\") pod \"e2e-ipp-a-027edd-openshift-default-6f84795f8b-46ktf\" (UID: \"96210882-8a70-4795-910a-01c409c73c78\") " pod="openshift-ingress/e2e-ipp-a-027edd-openshift-default-6f84795f8b-46ktf"
Sep 11 17:50:10.325196 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:10.325139    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/96210882-8a70-4795-910a-01c409c73c78-workload-certs\") pod \"e2e-ipp-a-027edd-openshift-default-6f84795f8b-46ktf\" (UID: \"96210882-8a70-4795-910a-01c409c73c78\") " pod="openshift-ingress/e2e-ipp-a-027edd-openshift-default-6f84795f8b-46ktf"
Sep 11 17:50:10.325356 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:10.325339    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/96210882-8a70-4795-910a-01c409c73c78-istiod-ca-cert\") pod \"e2e-ipp-a-027edd-openshift-default-6f84795f8b-46ktf\" (UID: \"96210882-8a70-4795-910a-01c409c73c78\") " pod="openshift-ingress/e2e-ipp-a-027edd-openshift-default-6f84795f8b-46ktf"
Sep 11 17:50:10.326815 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:10.326794    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/96210882-8a70-4795-910a-01c409c73c78-istio-envoy\") pod \"e2e-ipp-a-027edd-openshift-default-6f84795f8b-46ktf\" (UID: \"96210882-8a70-4795-910a-01c409c73c78\") " pod="openshift-ingress/e2e-ipp-a-027edd-openshift-default-6f84795f8b-46ktf"
Sep 11 17:50:10.327062 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:10.327045    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/96210882-8a70-4795-910a-01c409c73c78-istio-podinfo\") pod \"e2e-ipp-a-027edd-openshift-default-6f84795f8b-46ktf\" (UID: \"96210882-8a70-4795-910a-01c409c73c78\") " pod="openshift-ingress/e2e-ipp-a-027edd-openshift-default-6f84795f8b-46ktf"
Sep 11 17:50:10.333877 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:10.333851    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/96210882-8a70-4795-910a-01c409c73c78-istio-token\") pod \"e2e-ipp-a-027edd-openshift-default-6f84795f8b-46ktf\" (UID: \"96210882-8a70-4795-910a-01c409c73c78\") " pod="openshift-ingress/e2e-ipp-a-027edd-openshift-default-6f84795f8b-46ktf"
Sep 11 17:50:10.339437 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:10.339415    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pl9r5\" (UniqueName: \"kubernetes.io/projected/96210882-8a70-4795-910a-01c409c73c78-kube-api-access-pl9r5\") pod \"e2e-ipp-a-027edd-openshift-default-6f84795f8b-46ktf\" (UID: \"96210882-8a70-4795-910a-01c409c73c78\") " pod="openshift-ingress/e2e-ipp-a-027edd-openshift-default-6f84795f8b-46ktf"
Sep 11 17:50:10.437148 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:10.437053    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-027edd-openshift-default-6f84795f8b-46ktf"
Sep 11 17:50:10.569614 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:10.569565    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:50:10.714070 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:10.714046    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-027edd-openshift-default-6f84795f8b-46ktf"]
Sep 11 17:50:10.715763 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:50:10.715736    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96210882_8a70_4795_910a_01c409c73c78.slice/crio-c6e456e7f1a7c29312456c8fe0ea4ff3a378e0aaaada2e8dd9288165637f4339 WatchSource:0}: Error finding container c6e456e7f1a7c29312456c8fe0ea4ff3a378e0aaaada2e8dd9288165637f4339: Status 404 returned error can't find the container with id c6e456e7f1a7c29312456c8fe0ea4ff3a378e0aaaada2e8dd9288165637f4339
Sep 11 17:50:10.717910 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:10.717878    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 17:50:10.718022 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:10.717957    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 17:50:10.718022 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:10.717998    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 17:50:11.580878 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:11.580844    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-027edd-openshift-default-6f84795f8b-46ktf" event={"ID":"96210882-8a70-4795-910a-01c409c73c78","Type":"ContainerStarted","Data":"4930fd338c02f15925a8207cc8330a7c4c7230ea437f22a474f4bc9fbfe3e88f"}
Sep 11 17:50:11.581309 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:11.580894    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-027edd-openshift-default-6f84795f8b-46ktf" event={"ID":"96210882-8a70-4795-910a-01c409c73c78","Type":"ContainerStarted","Data":"c6e456e7f1a7c29312456c8fe0ea4ff3a378e0aaaada2e8dd9288165637f4339"}
Sep 11 17:50:11.604162 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:11.604093    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-027edd-openshift-default-6f84795f8b-46ktf" podStartSLOduration=1.6040757110000001 podStartE2EDuration="1.604075711s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:50:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:50:11.60292778 +0000 UTC m=+1906.124219302" watchObservedRunningTime="2026-09-11 17:50:11.604075711 +0000 UTC m=+1906.125367288"
Sep 11 17:50:12.437456 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:12.437413    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-027edd-openshift-default-6f84795f8b-46ktf"
Sep 11 17:50:12.442617 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:12.442592    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-027edd-openshift-default-6f84795f8b-46ktf"
Sep 11 17:50:12.586627 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:12.586585    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-027edd-openshift-default-6f84795f8b-46ktf"
Sep 11 17:50:12.587543 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:12.587523    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-027edd-openshift-default-6f84795f8b-46ktf"
Sep 11 17:50:12.936849 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:12.936811    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:50:13.595889 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:13.595854    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:50:13.685075 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:13.685033    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:50:13.791502 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:13.791462    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:50:14.042687 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:14.042598    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:50:14.144474 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:14.144438    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:50:14.651626 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:14.651581    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:50:15.468820 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:15.468782    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-027edd-6d646f7489-f5c8z"]
Sep 11 17:50:15.472767 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:15.472741    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-027edd-6d646f7489-f5c8z"
Sep 11 17:50:15.473478 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:15.473450    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/250eabc9-048b-4de1-a4f1-a07773b7c393-maas-api-tls\") pod \"maas-api-e2e-ipp-a-027edd-6d646f7489-f5c8z\" (UID: \"250eabc9-048b-4de1-a4f1-a07773b7c393\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-027edd-6d646f7489-f5c8z"
Sep 11 17:50:15.473618 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:15.473491    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-66rkn\" (UniqueName: \"kubernetes.io/projected/250eabc9-048b-4de1-a4f1-a07773b7c393-kube-api-access-66rkn\") pod \"maas-api-e2e-ipp-a-027edd-6d646f7489-f5c8z\" (UID: \"250eabc9-048b-4de1-a4f1-a07773b7c393\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-027edd-6d646f7489-f5c8z"
Sep 11 17:50:15.473618 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:15.473556    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/250eabc9-048b-4de1-a4f1-a07773b7c393-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-027edd-6d646f7489-f5c8z\" (UID: \"250eabc9-048b-4de1-a4f1-a07773b7c393\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-027edd-6d646f7489-f5c8z"
Sep 11 17:50:15.473618 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:15.473602    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/250eabc9-048b-4de1-a4f1-a07773b7c393-metrics-certs\") pod \"maas-api-e2e-ipp-a-027edd-6d646f7489-f5c8z\" (UID: \"250eabc9-048b-4de1-a4f1-a07773b7c393\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-027edd-6d646f7489-f5c8z"
Sep 11 17:50:15.474779 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:15.474754    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-027edd-serving-cert\""
Sep 11 17:50:15.486688 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:15.486658    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-027edd-6d646f7489-f5c8z"]
Sep 11 17:50:15.531502 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:15.531470    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:50:15.574730 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:15.574691    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/250eabc9-048b-4de1-a4f1-a07773b7c393-metrics-certs\") pod \"maas-api-e2e-ipp-a-027edd-6d646f7489-f5c8z\" (UID: \"250eabc9-048b-4de1-a4f1-a07773b7c393\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-027edd-6d646f7489-f5c8z"
Sep 11 17:50:15.574914 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:15.574749    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/250eabc9-048b-4de1-a4f1-a07773b7c393-maas-api-tls\") pod \"maas-api-e2e-ipp-a-027edd-6d646f7489-f5c8z\" (UID: \"250eabc9-048b-4de1-a4f1-a07773b7c393\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-027edd-6d646f7489-f5c8z"
Sep 11 17:50:15.574914 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:15.574771    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-66rkn\" (UniqueName: \"kubernetes.io/projected/250eabc9-048b-4de1-a4f1-a07773b7c393-kube-api-access-66rkn\") pod \"maas-api-e2e-ipp-a-027edd-6d646f7489-f5c8z\" (UID: \"250eabc9-048b-4de1-a4f1-a07773b7c393\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-027edd-6d646f7489-f5c8z"
Sep 11 17:50:15.574914 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:15.574808    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/250eabc9-048b-4de1-a4f1-a07773b7c393-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-027edd-6d646f7489-f5c8z\" (UID: \"250eabc9-048b-4de1-a4f1-a07773b7c393\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-027edd-6d646f7489-f5c8z"
Sep 11 17:50:15.574914 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:50:15.574884    2552 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-a-027edd-serving-cert: secret "maas-api-e2e-ipp-a-027edd-serving-cert" not found
Sep 11 17:50:15.575086 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:50:15.574947    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/250eabc9-048b-4de1-a4f1-a07773b7c393-maas-api-tls podName:250eabc9-048b-4de1-a4f1-a07773b7c393 nodeName:}" failed. No retries permitted until 2026-09-11 17:50:16.074929401 +0000 UTC m=+1910.596220899 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/250eabc9-048b-4de1-a4f1-a07773b7c393-maas-api-tls") pod "maas-api-e2e-ipp-a-027edd-6d646f7489-f5c8z" (UID: "250eabc9-048b-4de1-a4f1-a07773b7c393") : secret "maas-api-e2e-ipp-a-027edd-serving-cert" not found
Sep 11 17:50:15.575491 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:15.575468    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/250eabc9-048b-4de1-a4f1-a07773b7c393-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-027edd-6d646f7489-f5c8z\" (UID: \"250eabc9-048b-4de1-a4f1-a07773b7c393\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-027edd-6d646f7489-f5c8z"
Sep 11 17:50:15.577434 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:15.577415    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/250eabc9-048b-4de1-a4f1-a07773b7c393-metrics-certs\") pod \"maas-api-e2e-ipp-a-027edd-6d646f7489-f5c8z\" (UID: \"250eabc9-048b-4de1-a4f1-a07773b7c393\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-027edd-6d646f7489-f5c8z"
Sep 11 17:50:15.587506 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:15.587482    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-66rkn\" (UniqueName: \"kubernetes.io/projected/250eabc9-048b-4de1-a4f1-a07773b7c393-kube-api-access-66rkn\") pod \"maas-api-e2e-ipp-a-027edd-6d646f7489-f5c8z\" (UID: \"250eabc9-048b-4de1-a4f1-a07773b7c393\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-027edd-6d646f7489-f5c8z"
Sep 11 17:50:15.630783 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:15.630756    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:50:15.982656 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:15.982612    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:50:16.079730 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:16.079674    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/250eabc9-048b-4de1-a4f1-a07773b7c393-maas-api-tls\") pod \"maas-api-e2e-ipp-a-027edd-6d646f7489-f5c8z\" (UID: \"250eabc9-048b-4de1-a4f1-a07773b7c393\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-027edd-6d646f7489-f5c8z"
Sep 11 17:50:16.082044 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:16.082023    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/250eabc9-048b-4de1-a4f1-a07773b7c393-maas-api-tls\") pod \"maas-api-e2e-ipp-a-027edd-6d646f7489-f5c8z\" (UID: \"250eabc9-048b-4de1-a4f1-a07773b7c393\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-027edd-6d646f7489-f5c8z"
Sep 11 17:50:16.085943 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:16.085926    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-027edd-6d646f7489-f5c8z"
Sep 11 17:50:16.136849 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:16.136643    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:50:16.237490 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:16.237399    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:50:16.380361 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:16.380317    2552 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 17:50:16.380879 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:16.380851    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-027edd-6d646f7489-f5c8z"]
Sep 11 17:50:16.569221 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:16.569134    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:50:16.610929 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:16.610894    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-027edd-6d646f7489-f5c8z" event={"ID":"250eabc9-048b-4de1-a4f1-a07773b7c393","Type":"ContainerStarted","Data":"f02aa66b0dc87f79b6a714f6687b680e083a249c810970acf03d28aa84236189"}
Sep 11 17:50:17.601101 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:17.601065    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:50:17.618508 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:17.618480    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-027edd-6d646f7489-f5c8z" event={"ID":"250eabc9-048b-4de1-a4f1-a07773b7c393","Type":"ContainerStarted","Data":"9937e2118dabfe7654728b8314c5a4b76ec9e81d7503d7193215088451bd63c8"}
Sep 11 17:50:17.618639 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:17.618609    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-027edd-6d646f7489-f5c8z"
Sep 11 17:50:17.636221 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:17.636180    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-027edd-6d646f7489-f5c8z" podStartSLOduration=2.071887478 podStartE2EDuration="2.636165085s" totalImagesPullingTime="564.277607ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:50:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:50:17.634262205 +0000 UTC m=+1912.155553750" watchObservedRunningTime="2026-09-11 17:50:17.636165085 +0000 UTC m=+1912.157456606"
Sep 11 17:50:23.629445 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:23.629414    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-027edd-6d646f7489-f5c8z"
Sep 11 17:50:28.628278 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:28.628234    2552 patch_prober.go:28] interesting pod/e2e-ipp-a-027edd-openshift-default-6f84795f8b-46ktf container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.90:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 11 17:50:28.628767 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:28.628300    2552 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-a-027edd-openshift-default-6f84795f8b-46ktf" podUID="96210882-8a70-4795-910a-01c409c73c78" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.90:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 11 17:50:29.284656 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:29.284618    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:50:29.345866 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:29.345830    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:50:30.701698 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:30.701609    2552 generic.go:356] "Generic (PLEG): container finished" podID="96210882-8a70-4795-910a-01c409c73c78" containerID="4930fd338c02f15925a8207cc8330a7c4c7230ea437f22a474f4bc9fbfe3e88f" exitCode=0
Sep 11 17:50:30.702187 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:30.701688    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-027edd-openshift-default-6f84795f8b-46ktf" event={"ID":"96210882-8a70-4795-910a-01c409c73c78","Type":"ContainerDied","Data":"4930fd338c02f15925a8207cc8330a7c4c7230ea437f22a474f4bc9fbfe3e88f"}
Sep 11 17:50:30.702187 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:30.702046    2552 scope.go:117] "RemoveContainer" containerID="4930fd338c02f15925a8207cc8330a7c4c7230ea437f22a474f4bc9fbfe3e88f"
Sep 11 17:50:30.703096 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:30.703072    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 17:50:30.703180 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:30.703162    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 17:50:30.703219 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:30.703198    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 17:50:31.709760 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:31.709722    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-027edd-openshift-default-6f84795f8b-46ktf" event={"ID":"96210882-8a70-4795-910a-01c409c73c78","Type":"ContainerStarted","Data":"dbc584f29989dde3fb1ecb25254b55fa42d2e2a7aa87da14e753c9b1d86b4e2a"}
Sep 11 17:50:32.438153 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:32.438083    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-027edd-openshift-default-6f84795f8b-46ktf"
Sep 11 17:50:32.438412 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:32.438226    2552 patch_prober.go:28] interesting pod/e2e-ipp-a-027edd-openshift-default-6f84795f8b-46ktf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused" start-of-body=
Sep 11 17:50:32.438412 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:32.438274    2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-027edd-openshift-default-6f84795f8b-46ktf" podUID="96210882-8a70-4795-910a-01c409c73c78" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused"
Sep 11 17:50:33.438230 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:33.437787    2552 patch_prober.go:28] interesting pod/e2e-ipp-a-027edd-openshift-default-6f84795f8b-46ktf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused" start-of-body=
Sep 11 17:50:33.438230 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:33.437850    2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-027edd-openshift-default-6f84795f8b-46ktf" podUID="96210882-8a70-4795-910a-01c409c73c78" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused"
Sep 11 17:50:34.437937 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:34.437896    2552 patch_prober.go:28] interesting pod/e2e-ipp-a-027edd-openshift-default-6f84795f8b-46ktf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused" start-of-body=
Sep 11 17:50:34.438276 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:34.438245    2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-027edd-openshift-default-6f84795f8b-46ktf" podUID="96210882-8a70-4795-910a-01c409c73c78" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused"
Sep 11 17:50:35.438144 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:35.438088    2552 patch_prober.go:28] interesting pod/e2e-ipp-a-027edd-openshift-default-6f84795f8b-46ktf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused" start-of-body=
Sep 11 17:50:35.438983 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:35.438943    2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-027edd-openshift-default-6f84795f8b-46ktf" podUID="96210882-8a70-4795-910a-01c409c73c78" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused"
Sep 11 17:50:36.438157 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:36.438102    2552 patch_prober.go:28] interesting pod/e2e-ipp-a-027edd-openshift-default-6f84795f8b-46ktf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused" start-of-body=
Sep 11 17:50:36.438436 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:36.438409    2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-027edd-openshift-default-6f84795f8b-46ktf" podUID="96210882-8a70-4795-910a-01c409c73c78" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused"
Sep 11 17:50:37.441026 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:37.440995    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-027edd-openshift-default-6f84795f8b-46ktf"
Sep 11 17:50:37.441517 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:37.441497    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-027edd-openshift-default-6f84795f8b-46ktf"
Sep 11 17:50:37.442085 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:37.442066    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-027edd-openshift-default-6f84795f8b-46ktf"
Sep 11 17:50:37.658674 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:37.658638    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:50:38.383738 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:38.383700    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:50:38.549513 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:38.549471    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:50:39.258313 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:39.258284    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:50:39.356863 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:39.356830    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:50:39.584650 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:39.584573    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-10bbea-openshift-default-f67547c5d-hnlvj"]
Sep 11 17:50:39.587384 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:39.587366    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-10bbea-openshift-default-f67547c5d-hnlvj"
Sep 11 17:50:39.589616 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:39.589588    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-b-10bbea-openshift-default-dockercfg-n7jtp\""
Sep 11 17:50:39.604095 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:39.604075    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-10bbea-openshift-default-f67547c5d-hnlvj"]
Sep 11 17:50:39.619361 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:39.619332    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:50:39.676913 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:39.676878    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/69a4dccc-5986-4985-9494-9c2acf232e41-istio-token\") pod \"e2e-ipp-b-10bbea-openshift-default-f67547c5d-hnlvj\" (UID: \"69a4dccc-5986-4985-9494-9c2acf232e41\") " pod="openshift-ingress/e2e-ipp-b-10bbea-openshift-default-f67547c5d-hnlvj"
Sep 11 17:50:39.676913 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:39.676927    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/69a4dccc-5986-4985-9494-9c2acf232e41-credential-socket\") pod \"e2e-ipp-b-10bbea-openshift-default-f67547c5d-hnlvj\" (UID: \"69a4dccc-5986-4985-9494-9c2acf232e41\") " pod="openshift-ingress/e2e-ipp-b-10bbea-openshift-default-f67547c5d-hnlvj"
Sep 11 17:50:39.677195 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:39.676962    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n6bhx\" (UniqueName: \"kubernetes.io/projected/69a4dccc-5986-4985-9494-9c2acf232e41-kube-api-access-n6bhx\") pod \"e2e-ipp-b-10bbea-openshift-default-f67547c5d-hnlvj\" (UID: \"69a4dccc-5986-4985-9494-9c2acf232e41\") " pod="openshift-ingress/e2e-ipp-b-10bbea-openshift-default-f67547c5d-hnlvj"
Sep 11 17:50:39.677195 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:39.677034    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/69a4dccc-5986-4985-9494-9c2acf232e41-istio-podinfo\") pod \"e2e-ipp-b-10bbea-openshift-default-f67547c5d-hnlvj\" (UID: \"69a4dccc-5986-4985-9494-9c2acf232e41\") " pod="openshift-ingress/e2e-ipp-b-10bbea-openshift-default-f67547c5d-hnlvj"
Sep 11 17:50:39.677195 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:39.677067    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/69a4dccc-5986-4985-9494-9c2acf232e41-workload-socket\") pod \"e2e-ipp-b-10bbea-openshift-default-f67547c5d-hnlvj\" (UID: \"69a4dccc-5986-4985-9494-9c2acf232e41\") " pod="openshift-ingress/e2e-ipp-b-10bbea-openshift-default-f67547c5d-hnlvj"
Sep 11 17:50:39.677195 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:39.677089    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/69a4dccc-5986-4985-9494-9c2acf232e41-istiod-ca-cert\") pod \"e2e-ipp-b-10bbea-openshift-default-f67547c5d-hnlvj\" (UID: \"69a4dccc-5986-4985-9494-9c2acf232e41\") " pod="openshift-ingress/e2e-ipp-b-10bbea-openshift-default-f67547c5d-hnlvj"
Sep 11 17:50:39.677195 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:39.677136    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/69a4dccc-5986-4985-9494-9c2acf232e41-workload-certs\") pod \"e2e-ipp-b-10bbea-openshift-default-f67547c5d-hnlvj\" (UID: \"69a4dccc-5986-4985-9494-9c2acf232e41\") " pod="openshift-ingress/e2e-ipp-b-10bbea-openshift-default-f67547c5d-hnlvj"
Sep 11 17:50:39.677195 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:39.677185    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/69a4dccc-5986-4985-9494-9c2acf232e41-istio-envoy\") pod \"e2e-ipp-b-10bbea-openshift-default-f67547c5d-hnlvj\" (UID: \"69a4dccc-5986-4985-9494-9c2acf232e41\") " pod="openshift-ingress/e2e-ipp-b-10bbea-openshift-default-f67547c5d-hnlvj"
Sep 11 17:50:39.677195 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:39.677198    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/69a4dccc-5986-4985-9494-9c2acf232e41-istio-data\") pod \"e2e-ipp-b-10bbea-openshift-default-f67547c5d-hnlvj\" (UID: \"69a4dccc-5986-4985-9494-9c2acf232e41\") " pod="openshift-ingress/e2e-ipp-b-10bbea-openshift-default-f67547c5d-hnlvj"
Sep 11 17:50:39.740844 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:39.740781    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:50:39.778444 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:39.778411    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/69a4dccc-5986-4985-9494-9c2acf232e41-istio-podinfo\") pod \"e2e-ipp-b-10bbea-openshift-default-f67547c5d-hnlvj\" (UID: \"69a4dccc-5986-4985-9494-9c2acf232e41\") " pod="openshift-ingress/e2e-ipp-b-10bbea-openshift-default-f67547c5d-hnlvj"
Sep 11 17:50:39.778444 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:39.778444    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/69a4dccc-5986-4985-9494-9c2acf232e41-workload-socket\") pod \"e2e-ipp-b-10bbea-openshift-default-f67547c5d-hnlvj\" (UID: \"69a4dccc-5986-4985-9494-9c2acf232e41\") " pod="openshift-ingress/e2e-ipp-b-10bbea-openshift-default-f67547c5d-hnlvj"
Sep 11 17:50:39.778664 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:39.778465    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/69a4dccc-5986-4985-9494-9c2acf232e41-istiod-ca-cert\") pod \"e2e-ipp-b-10bbea-openshift-default-f67547c5d-hnlvj\" (UID: \"69a4dccc-5986-4985-9494-9c2acf232e41\") " pod="openshift-ingress/e2e-ipp-b-10bbea-openshift-default-f67547c5d-hnlvj"
Sep 11 17:50:39.778664 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:39.778584    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/69a4dccc-5986-4985-9494-9c2acf232e41-workload-certs\") pod \"e2e-ipp-b-10bbea-openshift-default-f67547c5d-hnlvj\" (UID: \"69a4dccc-5986-4985-9494-9c2acf232e41\") " pod="openshift-ingress/e2e-ipp-b-10bbea-openshift-default-f67547c5d-hnlvj"
Sep 11 17:50:39.778664 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:39.778629    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/69a4dccc-5986-4985-9494-9c2acf232e41-istio-envoy\") pod \"e2e-ipp-b-10bbea-openshift-default-f67547c5d-hnlvj\" (UID: \"69a4dccc-5986-4985-9494-9c2acf232e41\") " pod="openshift-ingress/e2e-ipp-b-10bbea-openshift-default-f67547c5d-hnlvj"
Sep 11 17:50:39.778664 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:39.778644    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/69a4dccc-5986-4985-9494-9c2acf232e41-istio-data\") pod \"e2e-ipp-b-10bbea-openshift-default-f67547c5d-hnlvj\" (UID: \"69a4dccc-5986-4985-9494-9c2acf232e41\") " pod="openshift-ingress/e2e-ipp-b-10bbea-openshift-default-f67547c5d-hnlvj"
Sep 11 17:50:39.778865 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:39.778725    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/69a4dccc-5986-4985-9494-9c2acf232e41-istio-token\") pod \"e2e-ipp-b-10bbea-openshift-default-f67547c5d-hnlvj\" (UID: \"69a4dccc-5986-4985-9494-9c2acf232e41\") " pod="openshift-ingress/e2e-ipp-b-10bbea-openshift-default-f67547c5d-hnlvj"
Sep 11 17:50:39.778865 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:39.778768    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/69a4dccc-5986-4985-9494-9c2acf232e41-credential-socket\") pod \"e2e-ipp-b-10bbea-openshift-default-f67547c5d-hnlvj\" (UID: \"69a4dccc-5986-4985-9494-9c2acf232e41\") " pod="openshift-ingress/e2e-ipp-b-10bbea-openshift-default-f67547c5d-hnlvj"
Sep 11 17:50:39.778865 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:39.778818    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n6bhx\" (UniqueName: \"kubernetes.io/projected/69a4dccc-5986-4985-9494-9c2acf232e41-kube-api-access-n6bhx\") pod \"e2e-ipp-b-10bbea-openshift-default-f67547c5d-hnlvj\" (UID: \"69a4dccc-5986-4985-9494-9c2acf232e41\") " pod="openshift-ingress/e2e-ipp-b-10bbea-openshift-default-f67547c5d-hnlvj"
Sep 11 17:50:39.778865 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:39.778833    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/69a4dccc-5986-4985-9494-9c2acf232e41-workload-socket\") pod \"e2e-ipp-b-10bbea-openshift-default-f67547c5d-hnlvj\" (UID: \"69a4dccc-5986-4985-9494-9c2acf232e41\") " pod="openshift-ingress/e2e-ipp-b-10bbea-openshift-default-f67547c5d-hnlvj"
Sep 11 17:50:39.779165 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:39.779141    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/69a4dccc-5986-4985-9494-9c2acf232e41-istio-data\") pod \"e2e-ipp-b-10bbea-openshift-default-f67547c5d-hnlvj\" (UID: \"69a4dccc-5986-4985-9494-9c2acf232e41\") " pod="openshift-ingress/e2e-ipp-b-10bbea-openshift-default-f67547c5d-hnlvj"
Sep 11 17:50:39.779259 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:39.779168    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/69a4dccc-5986-4985-9494-9c2acf232e41-credential-socket\") pod \"e2e-ipp-b-10bbea-openshift-default-f67547c5d-hnlvj\" (UID: \"69a4dccc-5986-4985-9494-9c2acf232e41\") " pod="openshift-ingress/e2e-ipp-b-10bbea-openshift-default-f67547c5d-hnlvj"
Sep 11 17:50:39.779322 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:39.779300    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/69a4dccc-5986-4985-9494-9c2acf232e41-istiod-ca-cert\") pod \"e2e-ipp-b-10bbea-openshift-default-f67547c5d-hnlvj\" (UID: \"69a4dccc-5986-4985-9494-9c2acf232e41\") " pod="openshift-ingress/e2e-ipp-b-10bbea-openshift-default-f67547c5d-hnlvj"
Sep 11 17:50:39.779372 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:39.779304    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/69a4dccc-5986-4985-9494-9c2acf232e41-workload-certs\") pod \"e2e-ipp-b-10bbea-openshift-default-f67547c5d-hnlvj\" (UID: \"69a4dccc-5986-4985-9494-9c2acf232e41\") " pod="openshift-ingress/e2e-ipp-b-10bbea-openshift-default-f67547c5d-hnlvj"
Sep 11 17:50:39.780909 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:39.780884    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/69a4dccc-5986-4985-9494-9c2acf232e41-istio-envoy\") pod \"e2e-ipp-b-10bbea-openshift-default-f67547c5d-hnlvj\" (UID: \"69a4dccc-5986-4985-9494-9c2acf232e41\") " pod="openshift-ingress/e2e-ipp-b-10bbea-openshift-default-f67547c5d-hnlvj"
Sep 11 17:50:39.781182 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:39.781166    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/69a4dccc-5986-4985-9494-9c2acf232e41-istio-podinfo\") pod \"e2e-ipp-b-10bbea-openshift-default-f67547c5d-hnlvj\" (UID: \"69a4dccc-5986-4985-9494-9c2acf232e41\") " pod="openshift-ingress/e2e-ipp-b-10bbea-openshift-default-f67547c5d-hnlvj"
Sep 11 17:50:39.787083 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:39.787061    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/69a4dccc-5986-4985-9494-9c2acf232e41-istio-token\") pod \"e2e-ipp-b-10bbea-openshift-default-f67547c5d-hnlvj\" (UID: \"69a4dccc-5986-4985-9494-9c2acf232e41\") " pod="openshift-ingress/e2e-ipp-b-10bbea-openshift-default-f67547c5d-hnlvj"
Sep 11 17:50:39.787185 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:39.787131    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n6bhx\" (UniqueName: \"kubernetes.io/projected/69a4dccc-5986-4985-9494-9c2acf232e41-kube-api-access-n6bhx\") pod \"e2e-ipp-b-10bbea-openshift-default-f67547c5d-hnlvj\" (UID: \"69a4dccc-5986-4985-9494-9c2acf232e41\") " pod="openshift-ingress/e2e-ipp-b-10bbea-openshift-default-f67547c5d-hnlvj"
Sep 11 17:50:39.898284 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:39.898249    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-10bbea-openshift-default-f67547c5d-hnlvj"
Sep 11 17:50:40.162080 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:40.161997    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:50:40.184893 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:40.184868    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-10bbea-openshift-default-f67547c5d-hnlvj"]
Sep 11 17:50:40.187873 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:50:40.187845    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69a4dccc_5986_4985_9494_9c2acf232e41.slice/crio-0e5c68623d61e28a880122b95f3ab8acee562d3a54f5f284b2d77ea5d3954ce2 WatchSource:0}: Error finding container 0e5c68623d61e28a880122b95f3ab8acee562d3a54f5f284b2d77ea5d3954ce2: Status 404 returned error can't find the container with id 0e5c68623d61e28a880122b95f3ab8acee562d3a54f5f284b2d77ea5d3954ce2
Sep 11 17:50:40.190147 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:40.190099    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 17:50:40.190230 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:40.190181    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 17:50:40.190230 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:40.190208    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 17:50:40.770855 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:40.770815    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-10bbea-openshift-default-f67547c5d-hnlvj" event={"ID":"69a4dccc-5986-4985-9494-9c2acf232e41","Type":"ContainerStarted","Data":"d86ce15950010f5ee5b96118157cc8019c677ffe17fd1574fc3fe081aac88b0f"}
Sep 11 17:50:40.771274 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:40.770880    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-10bbea-openshift-default-f67547c5d-hnlvj" event={"ID":"69a4dccc-5986-4985-9494-9c2acf232e41","Type":"ContainerStarted","Data":"0e5c68623d61e28a880122b95f3ab8acee562d3a54f5f284b2d77ea5d3954ce2"}
Sep 11 17:50:40.792127 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:40.792074    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-10bbea-openshift-default-f67547c5d-hnlvj" podStartSLOduration=1.79205462 podStartE2EDuration="1.79205462s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:50:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:50:40.79089749 +0000 UTC m=+1935.312189012" watchObservedRunningTime="2026-09-11 17:50:40.79205462 +0000 UTC m=+1935.313346142"
Sep 11 17:50:40.898924 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:40.898876    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-10bbea-openshift-default-f67547c5d-hnlvj"
Sep 11 17:50:41.903945 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:41.903915    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-10bbea-openshift-default-f67547c5d-hnlvj"
Sep 11 17:50:42.032043 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:42.032007    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:50:42.731039 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:42.731007    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:50:42.783176 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:42.783148    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-10bbea-openshift-default-f67547c5d-hnlvj"
Sep 11 17:50:42.784123 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:42.784088    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-10bbea-openshift-default-f67547c5d-hnlvj"
Sep 11 17:50:42.830972 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:42.830938    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:50:43.674092 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:43.674056    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:50:44.487470 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:44.487435    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:50:45.528416 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:45.528382    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-10bbea-5b6d6b7df9-fd74t"]
Sep 11 17:50:45.531578 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:45.531557    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-10bbea-5b6d6b7df9-fd74t"
Sep 11 17:50:45.533833 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:45.533806    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-10bbea-serving-cert\""
Sep 11 17:50:45.544581 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:45.544548    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-10bbea-5b6d6b7df9-fd74t"]
Sep 11 17:50:45.544837 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:45.544807    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:50:45.632003 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:45.631963    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/287b139c-ae79-4112-8188-031eb9bb66a1-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-10bbea-5b6d6b7df9-fd74t\" (UID: \"287b139c-ae79-4112-8188-031eb9bb66a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-10bbea-5b6d6b7df9-fd74t"
Sep 11 17:50:45.632207 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:45.632086    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/287b139c-ae79-4112-8188-031eb9bb66a1-metrics-certs\") pod \"maas-api-e2e-ipp-b-10bbea-5b6d6b7df9-fd74t\" (UID: \"287b139c-ae79-4112-8188-031eb9bb66a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-10bbea-5b6d6b7df9-fd74t"
Sep 11 17:50:45.632207 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:45.632188    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6znjf\" (UniqueName: \"kubernetes.io/projected/287b139c-ae79-4112-8188-031eb9bb66a1-kube-api-access-6znjf\") pod \"maas-api-e2e-ipp-b-10bbea-5b6d6b7df9-fd74t\" (UID: \"287b139c-ae79-4112-8188-031eb9bb66a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-10bbea-5b6d6b7df9-fd74t"
Sep 11 17:50:45.632319 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:45.632224    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/287b139c-ae79-4112-8188-031eb9bb66a1-maas-api-tls\") pod \"maas-api-e2e-ipp-b-10bbea-5b6d6b7df9-fd74t\" (UID: \"287b139c-ae79-4112-8188-031eb9bb66a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-10bbea-5b6d6b7df9-fd74t"
Sep 11 17:50:45.642891 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:45.642844    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:50:45.733010 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:45.732974    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/287b139c-ae79-4112-8188-031eb9bb66a1-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-10bbea-5b6d6b7df9-fd74t\" (UID: \"287b139c-ae79-4112-8188-031eb9bb66a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-10bbea-5b6d6b7df9-fd74t"
Sep 11 17:50:45.733232 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:45.733082    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/287b139c-ae79-4112-8188-031eb9bb66a1-metrics-certs\") pod \"maas-api-e2e-ipp-b-10bbea-5b6d6b7df9-fd74t\" (UID: \"287b139c-ae79-4112-8188-031eb9bb66a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-10bbea-5b6d6b7df9-fd74t"
Sep 11 17:50:45.733232 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:45.733163    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6znjf\" (UniqueName: \"kubernetes.io/projected/287b139c-ae79-4112-8188-031eb9bb66a1-kube-api-access-6znjf\") pod \"maas-api-e2e-ipp-b-10bbea-5b6d6b7df9-fd74t\" (UID: \"287b139c-ae79-4112-8188-031eb9bb66a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-10bbea-5b6d6b7df9-fd74t"
Sep 11 17:50:45.733232 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:45.733201    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/287b139c-ae79-4112-8188-031eb9bb66a1-maas-api-tls\") pod \"maas-api-e2e-ipp-b-10bbea-5b6d6b7df9-fd74t\" (UID: \"287b139c-ae79-4112-8188-031eb9bb66a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-10bbea-5b6d6b7df9-fd74t"
Sep 11 17:50:45.733770 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:45.733744    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/287b139c-ae79-4112-8188-031eb9bb66a1-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-10bbea-5b6d6b7df9-fd74t\" (UID: \"287b139c-ae79-4112-8188-031eb9bb66a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-10bbea-5b6d6b7df9-fd74t"
Sep 11 17:50:45.735637 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:45.735615    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/287b139c-ae79-4112-8188-031eb9bb66a1-maas-api-tls\") pod \"maas-api-e2e-ipp-b-10bbea-5b6d6b7df9-fd74t\" (UID: \"287b139c-ae79-4112-8188-031eb9bb66a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-10bbea-5b6d6b7df9-fd74t"
Sep 11 17:50:45.735849 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:45.735826    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/287b139c-ae79-4112-8188-031eb9bb66a1-metrics-certs\") pod \"maas-api-e2e-ipp-b-10bbea-5b6d6b7df9-fd74t\" (UID: \"287b139c-ae79-4112-8188-031eb9bb66a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-10bbea-5b6d6b7df9-fd74t"
Sep 11 17:50:45.741327 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:45.741302    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:50:45.741975 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:45.741949    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6znjf\" (UniqueName: \"kubernetes.io/projected/287b139c-ae79-4112-8188-031eb9bb66a1-kube-api-access-6znjf\") pod \"maas-api-e2e-ipp-b-10bbea-5b6d6b7df9-fd74t\" (UID: \"287b139c-ae79-4112-8188-031eb9bb66a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-10bbea-5b6d6b7df9-fd74t"
Sep 11 17:50:45.844860 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:45.844830    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-10bbea-5b6d6b7df9-fd74t"
Sep 11 17:50:46.028901 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:46.028870    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:50:46.118411 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:46.118385    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-10bbea-5b6d6b7df9-fd74t"]
Sep 11 17:50:46.120013 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:50:46.119977    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod287b139c_ae79_4112_8188_031eb9bb66a1.slice/crio-9a90fc59bc2e429dad10ef65b9f8e9f29e6154e73bf31731710dbc470c3d4ccb WatchSource:0}: Error finding container 9a90fc59bc2e429dad10ef65b9f8e9f29e6154e73bf31731710dbc470c3d4ccb: Status 404 returned error can't find the container with id 9a90fc59bc2e429dad10ef65b9f8e9f29e6154e73bf31731710dbc470c3d4ccb
Sep 11 17:50:46.122247 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:46.122217    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:50:46.808713 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:46.808620    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-10bbea-5b6d6b7df9-fd74t" event={"ID":"287b139c-ae79-4112-8188-031eb9bb66a1","Type":"ContainerStarted","Data":"7d8c3154a59a83a3b6fe952af8623f6f146a4b50992dddd8f672f7154fa039a0"}
Sep 11 17:50:46.808713 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:46.808673    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-10bbea-5b6d6b7df9-fd74t" event={"ID":"287b139c-ae79-4112-8188-031eb9bb66a1","Type":"ContainerStarted","Data":"9a90fc59bc2e429dad10ef65b9f8e9f29e6154e73bf31731710dbc470c3d4ccb"}
Sep 11 17:50:46.809194 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:46.808786    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-10bbea-5b6d6b7df9-fd74t"
Sep 11 17:50:46.827252 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:46.827205    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-10bbea-5b6d6b7df9-fd74t" podStartSLOduration=1.4784988829999999 podStartE2EDuration="1.82718859s" totalImagesPullingTime="348.689707ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:50:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:50:46.82468888 +0000 UTC m=+1941.345980400" watchObservedRunningTime="2026-09-11 17:50:46.82718859 +0000 UTC m=+1941.348480112"
Sep 11 17:50:50.638516 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:50.638479    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:50:52.820045 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:52.820011    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-10bbea-5b6d6b7df9-fd74t"
Sep 11 17:50:58.825242 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:58.825207    2552 patch_prober.go:28] interesting pod/e2e-ipp-b-10bbea-openshift-default-f67547c5d-hnlvj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.92:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 11 17:50:58.825660 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:50:58.825264    2552 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-b-10bbea-openshift-default-f67547c5d-hnlvj" podUID="69a4dccc-5986-4985-9494-9c2acf232e41" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.92:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 11 17:51:00.914971 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:51:00.914888    2552 generic.go:356] "Generic (PLEG): container finished" podID="69a4dccc-5986-4985-9494-9c2acf232e41" containerID="d86ce15950010f5ee5b96118157cc8019c677ffe17fd1574fc3fe081aac88b0f" exitCode=0
Sep 11 17:51:00.915438 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:51:00.914961    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-10bbea-openshift-default-f67547c5d-hnlvj" event={"ID":"69a4dccc-5986-4985-9494-9c2acf232e41","Type":"ContainerDied","Data":"d86ce15950010f5ee5b96118157cc8019c677ffe17fd1574fc3fe081aac88b0f"}
Sep 11 17:51:00.915438 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:51:00.915409    2552 scope.go:117] "RemoveContainer" containerID="d86ce15950010f5ee5b96118157cc8019c677ffe17fd1574fc3fe081aac88b0f"
Sep 11 17:51:00.916511 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:51:00.916483    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 17:51:00.916613 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:51:00.916564    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 17:51:00.916675 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:51:00.916611    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 17:51:01.923609 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:51:01.923574    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-10bbea-openshift-default-f67547c5d-hnlvj" event={"ID":"69a4dccc-5986-4985-9494-9c2acf232e41","Type":"ContainerStarted","Data":"9695cc959d50b5f78d30ba436193051734594f08cbb756cbe2cc368d5996626a"}
Sep 11 17:51:02.898808 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:51:02.898716    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-10bbea-openshift-default-f67547c5d-hnlvj"
Sep 11 17:51:02.899013 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:51:02.898986    2552 patch_prober.go:28] interesting pod/e2e-ipp-b-10bbea-openshift-default-f67547c5d-hnlvj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.92:15021/healthz/ready\": dial tcp 10.132.0.92:15021: connect: connection refused" start-of-body=
Sep 11 17:51:02.899100 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:51:02.899045    2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-10bbea-openshift-default-f67547c5d-hnlvj" podUID="69a4dccc-5986-4985-9494-9c2acf232e41" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.92:15021/healthz/ready\": dial tcp 10.132.0.92:15021: connect: connection refused"
Sep 11 17:51:03.899347 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:51:03.899303    2552 patch_prober.go:28] interesting pod/e2e-ipp-b-10bbea-openshift-default-f67547c5d-hnlvj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.92:15021/healthz/ready\": dial tcp 10.132.0.92:15021: connect: connection refused" start-of-body=
Sep 11 17:51:03.899790 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:51:03.899369    2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-10bbea-openshift-default-f67547c5d-hnlvj" podUID="69a4dccc-5986-4985-9494-9c2acf232e41" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.92:15021/healthz/ready\": dial tcp 10.132.0.92:15021: connect: connection refused"
Sep 11 17:51:04.898981 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:51:04.898946    2552 patch_prober.go:28] interesting pod/e2e-ipp-b-10bbea-openshift-default-f67547c5d-hnlvj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.92:15021/healthz/ready\": dial tcp 10.132.0.92:15021: connect: connection refused" start-of-body=
Sep 11 17:51:04.899195 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:51:04.899018    2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-10bbea-openshift-default-f67547c5d-hnlvj" podUID="69a4dccc-5986-4985-9494-9c2acf232e41" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.92:15021/healthz/ready\": dial tcp 10.132.0.92:15021: connect: connection refused"
Sep 11 17:51:05.898928 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:51:05.898888    2552 patch_prober.go:28] interesting pod/e2e-ipp-b-10bbea-openshift-default-f67547c5d-hnlvj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.92:15021/healthz/ready\": dial tcp 10.132.0.92:15021: connect: connection refused" start-of-body=
Sep 11 17:51:05.899380 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:51:05.898947    2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-10bbea-openshift-default-f67547c5d-hnlvj" podUID="69a4dccc-5986-4985-9494-9c2acf232e41" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.92:15021/healthz/ready\": dial tcp 10.132.0.92:15021: connect: connection refused"
Sep 11 17:51:06.426623 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:51:06.426588    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:06.898922 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:51:06.898879    2552 patch_prober.go:28] interesting pod/e2e-ipp-b-10bbea-openshift-default-f67547c5d-hnlvj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.92:15021/healthz/ready\": dial tcp 10.132.0.92:15021: connect: connection refused" start-of-body=
Sep 11 17:51:06.899149 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:51:06.898940    2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-10bbea-openshift-default-f67547c5d-hnlvj" podUID="69a4dccc-5986-4985-9494-9c2acf232e41" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.92:15021/healthz/ready\": dial tcp 10.132.0.92:15021: connect: connection refused"
Sep 11 17:51:07.682610 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:51:07.682576    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:07.785919 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:51:07.785886    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:07.902359 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:51:07.902329    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-10bbea-openshift-default-f67547c5d-hnlvj"
Sep 11 17:51:07.902834 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:51:07.902736    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-10bbea-openshift-default-f67547c5d-hnlvj"
Sep 11 17:51:07.903344 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:51:07.903326    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-10bbea-openshift-default-f67547c5d-hnlvj"
Sep 11 17:51:08.726091 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:51:08.726056    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:10.015326 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:51:10.015280    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:10.744756 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:51:10.744723    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:11.523822 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:51:11.523794    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:14.332628 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:51:14.332590    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:44.218814 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:51:44.218724    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:44.546829 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:51:44.546739    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:45.041967 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:51:45.041896    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:46.836214 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:51:46.836171    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:47.631939 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:51:47.631896    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:47.728267 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:51:47.728225    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:47.936665 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:51:47.936570    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:48.938889 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:51:48.938851    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:50.148752 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:51:50.148705    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:51.262302 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:51:51.262237    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:52.059538 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:51:52.059500    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-574f8b-79c56865f6-p5cjb"]
Sep 11 17:51:52.062450 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:51:52.062429    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-574f8b-79c56865f6-p5cjb"
Sep 11 17:51:52.065546 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:51:52.065523    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-574f8b-serving-cert\""
Sep 11 17:51:52.077947 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:51:52.077918    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-574f8b-79c56865f6-p5cjb"]
Sep 11 17:51:52.121683 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:51:52.121648    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:52.171036 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:51:52.170991    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/222814af-b6d3-474b-bf27-f6ba4d77dc84-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-574f8b-79c56865f6-p5cjb\" (UID: \"222814af-b6d3-474b-bf27-f6ba4d77dc84\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-574f8b-79c56865f6-p5cjb"
Sep 11 17:51:52.171228 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:51:52.171038    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-th8f8\" (UniqueName: \"kubernetes.io/projected/222814af-b6d3-474b-bf27-f6ba4d77dc84-kube-api-access-th8f8\") pod \"maas-api-e2e-ipp-cleanup-574f8b-79c56865f6-p5cjb\" (UID: \"222814af-b6d3-474b-bf27-f6ba4d77dc84\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-574f8b-79c56865f6-p5cjb"
Sep 11 17:51:52.171228 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:51:52.171165    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/222814af-b6d3-474b-bf27-f6ba4d77dc84-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-574f8b-79c56865f6-p5cjb\" (UID: \"222814af-b6d3-474b-bf27-f6ba4d77dc84\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-574f8b-79c56865f6-p5cjb"
Sep 11 17:51:52.171325 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:51:52.171297    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/222814af-b6d3-474b-bf27-f6ba4d77dc84-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-574f8b-79c56865f6-p5cjb\" (UID: \"222814af-b6d3-474b-bf27-f6ba4d77dc84\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-574f8b-79c56865f6-p5cjb"
Sep 11 17:51:52.231287 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:51:52.231251    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:52.272368 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:51:52.272329    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/222814af-b6d3-474b-bf27-f6ba4d77dc84-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-574f8b-79c56865f6-p5cjb\" (UID: \"222814af-b6d3-474b-bf27-f6ba4d77dc84\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-574f8b-79c56865f6-p5cjb"
Sep 11 17:51:52.272829 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:51:52.272407    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/222814af-b6d3-474b-bf27-f6ba4d77dc84-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-574f8b-79c56865f6-p5cjb\" (UID: \"222814af-b6d3-474b-bf27-f6ba4d77dc84\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-574f8b-79c56865f6-p5cjb"
Sep 11 17:51:52.272829 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:51:52.272427    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-th8f8\" (UniqueName: \"kubernetes.io/projected/222814af-b6d3-474b-bf27-f6ba4d77dc84-kube-api-access-th8f8\") pod \"maas-api-e2e-ipp-cleanup-574f8b-79c56865f6-p5cjb\" (UID: \"222814af-b6d3-474b-bf27-f6ba4d77dc84\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-574f8b-79c56865f6-p5cjb"
Sep 11 17:51:52.272829 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:51:52.272463    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/222814af-b6d3-474b-bf27-f6ba4d77dc84-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-574f8b-79c56865f6-p5cjb\" (UID: \"222814af-b6d3-474b-bf27-f6ba4d77dc84\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-574f8b-79c56865f6-p5cjb"
Sep 11 17:51:52.272829 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:51:52.272617    2552 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-574f8b-serving-cert: secret "maas-api-e2e-ipp-cleanup-574f8b-serving-cert" not found
Sep 11 17:51:52.272829 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:51:52.272673    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/222814af-b6d3-474b-bf27-f6ba4d77dc84-maas-api-tls podName:222814af-b6d3-474b-bf27-f6ba4d77dc84 nodeName:}" failed. No retries permitted until 2026-09-11 17:51:52.77265486 +0000 UTC m=+2007.293946361 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/222814af-b6d3-474b-bf27-f6ba4d77dc84-maas-api-tls") pod "maas-api-e2e-ipp-cleanup-574f8b-79c56865f6-p5cjb" (UID: "222814af-b6d3-474b-bf27-f6ba4d77dc84") : secret "maas-api-e2e-ipp-cleanup-574f8b-serving-cert" not found
Sep 11 17:51:52.273129 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:51:52.273089    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/222814af-b6d3-474b-bf27-f6ba4d77dc84-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-574f8b-79c56865f6-p5cjb\" (UID: \"222814af-b6d3-474b-bf27-f6ba4d77dc84\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-574f8b-79c56865f6-p5cjb"
Sep 11 17:51:52.274926 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:51:52.274907    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/222814af-b6d3-474b-bf27-f6ba4d77dc84-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-574f8b-79c56865f6-p5cjb\" (UID: \"222814af-b6d3-474b-bf27-f6ba4d77dc84\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-574f8b-79c56865f6-p5cjb"
Sep 11 17:51:52.284244 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:51:52.284216    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-th8f8\" (UniqueName: \"kubernetes.io/projected/222814af-b6d3-474b-bf27-f6ba4d77dc84-kube-api-access-th8f8\") pod \"maas-api-e2e-ipp-cleanup-574f8b-79c56865f6-p5cjb\" (UID: \"222814af-b6d3-474b-bf27-f6ba4d77dc84\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-574f8b-79c56865f6-p5cjb"
Sep 11 17:51:52.440934 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:51:52.440894    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:52.747883 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:51:52.747801    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:52.778949 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:51:52.778914    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/222814af-b6d3-474b-bf27-f6ba4d77dc84-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-574f8b-79c56865f6-p5cjb\" (UID: \"222814af-b6d3-474b-bf27-f6ba4d77dc84\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-574f8b-79c56865f6-p5cjb"
Sep 11 17:51:52.781566 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:51:52.781532    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/222814af-b6d3-474b-bf27-f6ba4d77dc84-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-574f8b-79c56865f6-p5cjb\" (UID: \"222814af-b6d3-474b-bf27-f6ba4d77dc84\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-574f8b-79c56865f6-p5cjb"
Sep 11 17:51:52.846184 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:51:52.846148    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:52.975992 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:51:52.975958    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-574f8b-79c56865f6-p5cjb"
Sep 11 17:51:53.268851 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:51:53.268820    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-574f8b-79c56865f6-p5cjb"]
Sep 11 17:51:53.269456 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:51:53.269430    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod222814af_b6d3_474b_bf27_f6ba4d77dc84.slice/crio-5294e5d3a51795d8bf070ba4242738e1b429ecf65c61b04f9b579c0b2d6b9d4f WatchSource:0}: Error finding container 5294e5d3a51795d8bf070ba4242738e1b429ecf65c61b04f9b579c0b2d6b9d4f: Status 404 returned error can't find the container with id 5294e5d3a51795d8bf070ba4242738e1b429ecf65c61b04f9b579c0b2d6b9d4f
Sep 11 17:51:54.254254 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:51:54.254223    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-574f8b-79c56865f6-p5cjb" event={"ID":"222814af-b6d3-474b-bf27-f6ba4d77dc84","Type":"ContainerStarted","Data":"389e1ae6021c3ac8f6210e9beeeba21a3ee80cdf51a4ab5baa54eb04bc3d9e85"}
Sep 11 17:51:54.254763 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:51:54.254280    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-574f8b-79c56865f6-p5cjb" event={"ID":"222814af-b6d3-474b-bf27-f6ba4d77dc84","Type":"ContainerStarted","Data":"5294e5d3a51795d8bf070ba4242738e1b429ecf65c61b04f9b579c0b2d6b9d4f"}
Sep 11 17:51:54.254763 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:51:54.254346    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-574f8b-79c56865f6-p5cjb"
Sep 11 17:51:54.283573 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:51:54.283515    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-574f8b-79c56865f6-p5cjb" podStartSLOduration=1.814205967 podStartE2EDuration="2.283498047s" totalImagesPullingTime="469.29208ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:51:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:51:54.281969075 +0000 UTC m=+2008.803260596" watchObservedRunningTime="2026-09-11 17:51:54.283498047 +0000 UTC m=+2008.804789568"
Sep 11 17:52:00.266201 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:52:00.266168    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-574f8b-79c56865f6-p5cjb"
Sep 11 17:52:15.740672 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:52:15.740633    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:52:16.545698 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:52:16.545665    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:52:17.590563 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:52:17.590526    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:52:17.742039 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:52:17.742006    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:52:18.452198 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:52:18.452152    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:52:19.467296 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:52:19.467263    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:52:20.358534 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:52:20.358498    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:52:21.112556 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:52:21.112525    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:52:21.981738 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:52:21.981709    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:52:22.753974 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:52:22.753939    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:52:23.466824 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:52:23.466792    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:52:28.949736 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:52:28.949698    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:52:29.597788 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:52:29.597752    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-574f8b-79c56865f6-p5cjb"]
Sep 11 17:52:29.598056 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:52:29.598021    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-574f8b-79c56865f6-p5cjb" podUID="222814af-b6d3-474b-bf27-f6ba4d77dc84" containerName="maas-api" containerID="cri-o://389e1ae6021c3ac8f6210e9beeeba21a3ee80cdf51a4ab5baa54eb04bc3d9e85" gracePeriod=30
Sep 11 17:52:29.636319 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:52:29.636264    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:52:29.717729 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:52:29.717674    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:52:29.816956 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:52:29.816922    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:52:29.857057 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:52:29.856990    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-574f8b-79c56865f6-p5cjb"
Sep 11 17:52:29.917920 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:52:29.917880    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:52:29.946852 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:52:29.946825    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/222814af-b6d3-474b-bf27-f6ba4d77dc84-maas-api-tls\") pod \"222814af-b6d3-474b-bf27-f6ba4d77dc84\" (UID: \"222814af-b6d3-474b-bf27-f6ba4d77dc84\") "
Sep 11 17:52:29.947032 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:52:29.946871    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/222814af-b6d3-474b-bf27-f6ba4d77dc84-metrics-certs\") pod \"222814af-b6d3-474b-bf27-f6ba4d77dc84\" (UID: \"222814af-b6d3-474b-bf27-f6ba4d77dc84\") "
Sep 11 17:52:29.947032 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:52:29.946898    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-th8f8\" (UniqueName: \"kubernetes.io/projected/222814af-b6d3-474b-bf27-f6ba4d77dc84-kube-api-access-th8f8\") pod \"222814af-b6d3-474b-bf27-f6ba4d77dc84\" (UID: \"222814af-b6d3-474b-bf27-f6ba4d77dc84\") "
Sep 11 17:52:29.947032 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:52:29.946936    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/222814af-b6d3-474b-bf27-f6ba4d77dc84-openshift-service-ca\") pod \"222814af-b6d3-474b-bf27-f6ba4d77dc84\" (UID: \"222814af-b6d3-474b-bf27-f6ba4d77dc84\") "
Sep 11 17:52:29.947430 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:52:29.947399    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/222814af-b6d3-474b-bf27-f6ba4d77dc84-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "222814af-b6d3-474b-bf27-f6ba4d77dc84" (UID: "222814af-b6d3-474b-bf27-f6ba4d77dc84"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 17:52:29.949253 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:52:29.949227    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/222814af-b6d3-474b-bf27-f6ba4d77dc84-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "222814af-b6d3-474b-bf27-f6ba4d77dc84" (UID: "222814af-b6d3-474b-bf27-f6ba4d77dc84"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 17:52:29.949253 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:52:29.949228    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/222814af-b6d3-474b-bf27-f6ba4d77dc84-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "222814af-b6d3-474b-bf27-f6ba4d77dc84" (UID: "222814af-b6d3-474b-bf27-f6ba4d77dc84"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 17:52:29.949622 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:52:29.949598    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/222814af-b6d3-474b-bf27-f6ba4d77dc84-kube-api-access-th8f8" (OuterVolumeSpecName: "kube-api-access-th8f8") pod "222814af-b6d3-474b-bf27-f6ba4d77dc84" (UID: "222814af-b6d3-474b-bf27-f6ba4d77dc84"). InnerVolumeSpecName "kube-api-access-th8f8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:52:30.048313 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:52:30.048277    2552 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/222814af-b6d3-474b-bf27-f6ba4d77dc84-metrics-certs\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:52:30.048313 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:52:30.048307    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-th8f8\" (UniqueName: \"kubernetes.io/projected/222814af-b6d3-474b-bf27-f6ba4d77dc84-kube-api-access-th8f8\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:52:30.048313 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:52:30.048317    2552 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/222814af-b6d3-474b-bf27-f6ba4d77dc84-openshift-service-ca\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:52:30.048763 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:52:30.048326    2552 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/222814af-b6d3-474b-bf27-f6ba4d77dc84-maas-api-tls\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:52:30.242555 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:52:30.242467    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:52:30.353600 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:52:30.353556    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:52:30.477106 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:52:30.477073    2552 generic.go:356] "Generic (PLEG): container finished" podID="222814af-b6d3-474b-bf27-f6ba4d77dc84" containerID="389e1ae6021c3ac8f6210e9beeeba21a3ee80cdf51a4ab5baa54eb04bc3d9e85" exitCode=0
Sep 11 17:52:30.477299 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:52:30.477135    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-574f8b-79c56865f6-p5cjb"
Sep 11 17:52:30.477299 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:52:30.477165    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-574f8b-79c56865f6-p5cjb" event={"ID":"222814af-b6d3-474b-bf27-f6ba4d77dc84","Type":"ContainerDied","Data":"389e1ae6021c3ac8f6210e9beeeba21a3ee80cdf51a4ab5baa54eb04bc3d9e85"}
Sep 11 17:52:30.477299 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:52:30.477218    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-574f8b-79c56865f6-p5cjb" event={"ID":"222814af-b6d3-474b-bf27-f6ba4d77dc84","Type":"ContainerDied","Data":"5294e5d3a51795d8bf070ba4242738e1b429ecf65c61b04f9b579c0b2d6b9d4f"}
Sep 11 17:52:30.477299 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:52:30.477240    2552 scope.go:117] "RemoveContainer" containerID="389e1ae6021c3ac8f6210e9beeeba21a3ee80cdf51a4ab5baa54eb04bc3d9e85"
Sep 11 17:52:30.487371 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:52:30.487351    2552 scope.go:117] "RemoveContainer" containerID="389e1ae6021c3ac8f6210e9beeeba21a3ee80cdf51a4ab5baa54eb04bc3d9e85"
Sep 11 17:52:30.487670 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:52:30.487647    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"389e1ae6021c3ac8f6210e9beeeba21a3ee80cdf51a4ab5baa54eb04bc3d9e85\": container with ID starting with 389e1ae6021c3ac8f6210e9beeeba21a3ee80cdf51a4ab5baa54eb04bc3d9e85 not found: ID does not exist" containerID="389e1ae6021c3ac8f6210e9beeeba21a3ee80cdf51a4ab5baa54eb04bc3d9e85"
Sep 11 17:52:30.487738 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:52:30.487682    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"389e1ae6021c3ac8f6210e9beeeba21a3ee80cdf51a4ab5baa54eb04bc3d9e85"} err="failed to get container status \"389e1ae6021c3ac8f6210e9beeeba21a3ee80cdf51a4ab5baa54eb04bc3d9e85\": rpc error: code = NotFound desc = could not find container \"389e1ae6021c3ac8f6210e9beeeba21a3ee80cdf51a4ab5baa54eb04bc3d9e85\": container with ID starting with 389e1ae6021c3ac8f6210e9beeeba21a3ee80cdf51a4ab5baa54eb04bc3d9e85 not found: ID does not exist"
Sep 11 17:52:30.495238 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:52:30.495176    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-574f8b-79c56865f6-p5cjb"]
Sep 11 17:52:30.498737 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:52:30.498715    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-574f8b-79c56865f6-p5cjb"]
Sep 11 17:52:31.100010 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:52:31.099980    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:52:31.946510 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:52:31.946469    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:52:31.949896 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:52:31.949870    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="222814af-b6d3-474b-bf27-f6ba4d77dc84" path="/var/lib/kubelet/pods/222814af-b6d3-474b-bf27-f6ba4d77dc84/volumes"
Sep 11 17:52:32.661525 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:52:32.661488    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:52:33.369720 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:52:33.369685    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:52:36.290687 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:52:36.290649    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:52:36.356317 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:52:36.356287    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:52:36.547550 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:52:36.547466    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:52:40.292350 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:52:40.292321    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:52:41.113355 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:52:41.113324    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:52:41.930764 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:52:41.930726    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:52:42.831765 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:52:42.831721    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:52:43.591532 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:52:43.591489    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:52:44.304520 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:52:44.304474    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:52:55.715372 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:52:55.715338    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-027edd-6d646f7489-f5c8z"]
Sep 11 17:52:55.715979 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:52:55.715592    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-027edd-6d646f7489-f5c8z" podUID="250eabc9-048b-4de1-a4f1-a07773b7c393" containerName="maas-api" containerID="cri-o://9937e2118dabfe7654728b8314c5a4b76ec9e81d7503d7193215088451bd63c8" gracePeriod=30
Sep 11 17:52:55.746963 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:52:55.746915    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:52:55.848048 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:52:55.848014    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:52:55.957479 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:52:55.957438    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:52:55.994739 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:52:55.994719    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-027edd-6d646f7489-f5c8z"
Sep 11 17:52:56.016400 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:52:56.016346    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:52:56.085398 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:52:56.085369    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/250eabc9-048b-4de1-a4f1-a07773b7c393-openshift-service-ca\") pod \"250eabc9-048b-4de1-a4f1-a07773b7c393\" (UID: \"250eabc9-048b-4de1-a4f1-a07773b7c393\") "
Sep 11 17:52:56.085398 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:52:56.085403    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/250eabc9-048b-4de1-a4f1-a07773b7c393-metrics-certs\") pod \"250eabc9-048b-4de1-a4f1-a07773b7c393\" (UID: \"250eabc9-048b-4de1-a4f1-a07773b7c393\") "
Sep 11 17:52:56.085603 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:52:56.085445    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-66rkn\" (UniqueName: \"kubernetes.io/projected/250eabc9-048b-4de1-a4f1-a07773b7c393-kube-api-access-66rkn\") pod \"250eabc9-048b-4de1-a4f1-a07773b7c393\" (UID: \"250eabc9-048b-4de1-a4f1-a07773b7c393\") "
Sep 11 17:52:56.085603 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:52:56.085492    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/250eabc9-048b-4de1-a4f1-a07773b7c393-maas-api-tls\") pod \"250eabc9-048b-4de1-a4f1-a07773b7c393\" (UID: \"250eabc9-048b-4de1-a4f1-a07773b7c393\") "
Sep 11 17:52:56.085755 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:52:56.085729    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/250eabc9-048b-4de1-a4f1-a07773b7c393-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "250eabc9-048b-4de1-a4f1-a07773b7c393" (UID: "250eabc9-048b-4de1-a4f1-a07773b7c393"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 17:52:56.086088 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:52:56.086051    2552 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/250eabc9-048b-4de1-a4f1-a07773b7c393-openshift-service-ca\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:52:56.087733 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:52:56.087705    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/250eabc9-048b-4de1-a4f1-a07773b7c393-kube-api-access-66rkn" (OuterVolumeSpecName: "kube-api-access-66rkn") pod "250eabc9-048b-4de1-a4f1-a07773b7c393" (UID: "250eabc9-048b-4de1-a4f1-a07773b7c393"). InnerVolumeSpecName "kube-api-access-66rkn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:52:56.087865 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:52:56.087728    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/250eabc9-048b-4de1-a4f1-a07773b7c393-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "250eabc9-048b-4de1-a4f1-a07773b7c393" (UID: "250eabc9-048b-4de1-a4f1-a07773b7c393"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 17:52:56.087865 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:52:56.087749    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/250eabc9-048b-4de1-a4f1-a07773b7c393-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "250eabc9-048b-4de1-a4f1-a07773b7c393" (UID: "250eabc9-048b-4de1-a4f1-a07773b7c393"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 17:52:56.186661 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:52:56.186625    2552 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/250eabc9-048b-4de1-a4f1-a07773b7c393-metrics-certs\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:52:56.186661 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:52:56.186655    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-66rkn\" (UniqueName: \"kubernetes.io/projected/250eabc9-048b-4de1-a4f1-a07773b7c393-kube-api-access-66rkn\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:52:56.186661 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:52:56.186666    2552 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/250eabc9-048b-4de1-a4f1-a07773b7c393-maas-api-tls\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:52:56.314003 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:52:56.313920    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:52:56.436578 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:52:56.436530    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:52:56.628865 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:52:56.628829    2552 generic.go:356] "Generic (PLEG): container finished" podID="250eabc9-048b-4de1-a4f1-a07773b7c393" containerID="9937e2118dabfe7654728b8314c5a4b76ec9e81d7503d7193215088451bd63c8" exitCode=0
Sep 11 17:52:56.629067 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:52:56.628877    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-027edd-6d646f7489-f5c8z"
Sep 11 17:52:56.629067 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:52:56.628916    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-027edd-6d646f7489-f5c8z" event={"ID":"250eabc9-048b-4de1-a4f1-a07773b7c393","Type":"ContainerDied","Data":"9937e2118dabfe7654728b8314c5a4b76ec9e81d7503d7193215088451bd63c8"}
Sep 11 17:52:56.629067 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:52:56.628980    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-027edd-6d646f7489-f5c8z" event={"ID":"250eabc9-048b-4de1-a4f1-a07773b7c393","Type":"ContainerDied","Data":"f02aa66b0dc87f79b6a714f6687b680e083a249c810970acf03d28aa84236189"}
Sep 11 17:52:56.629067 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:52:56.629012    2552 scope.go:117] "RemoveContainer" containerID="9937e2118dabfe7654728b8314c5a4b76ec9e81d7503d7193215088451bd63c8"
Sep 11 17:52:56.640038 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:52:56.640017    2552 scope.go:117] "RemoveContainer" containerID="9937e2118dabfe7654728b8314c5a4b76ec9e81d7503d7193215088451bd63c8"
Sep 11 17:52:56.640372 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:52:56.640348    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9937e2118dabfe7654728b8314c5a4b76ec9e81d7503d7193215088451bd63c8\": container with ID starting with 9937e2118dabfe7654728b8314c5a4b76ec9e81d7503d7193215088451bd63c8 not found: ID does not exist" containerID="9937e2118dabfe7654728b8314c5a4b76ec9e81d7503d7193215088451bd63c8"
Sep 11 17:52:56.640430 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:52:56.640385    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9937e2118dabfe7654728b8314c5a4b76ec9e81d7503d7193215088451bd63c8"} err="failed to get container status \"9937e2118dabfe7654728b8314c5a4b76ec9e81d7503d7193215088451bd63c8\": rpc error: code = NotFound desc = could not find container \"9937e2118dabfe7654728b8314c5a4b76ec9e81d7503d7193215088451bd63c8\": container with ID starting with 9937e2118dabfe7654728b8314c5a4b76ec9e81d7503d7193215088451bd63c8 not found: ID does not exist"
Sep 11 17:52:56.653916 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:52:56.653891    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-027edd-6d646f7489-f5c8z"]
Sep 11 17:52:56.660212 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:52:56.660194    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-027edd-6d646f7489-f5c8z"]
Sep 11 17:52:57.156881 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:52:57.156843    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:52:57.950083 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:52:57.950049    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="250eabc9-048b-4de1-a4f1-a07773b7c393" path="/var/lib/kubelet/pods/250eabc9-048b-4de1-a4f1-a07773b7c393/volumes"
Sep 11 17:52:57.999436 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:52:57.999395    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:52:58.714573 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:52:58.714538    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:53:01.344031 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:53:01.343990    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:53:03.004128 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:53:03.004018    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:53:12.813572 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:53:12.813531    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:53:26.147819 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:53:26.147776    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-hg7p8_eb5f3be1-2e36-40ba-b3a0-3bec413056e8/console-operator/1.log"
Sep 11 17:53:26.151295 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:53:26.151268    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6k9tr_0620c0a0-8070-4307-86d9-c69a76ea14a9/ovn-acl-logging/0.log"
Sep 11 17:53:26.158229 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:53:26.158209    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-hg7p8_eb5f3be1-2e36-40ba-b3a0-3bec413056e8/console-operator/1.log"
Sep 11 17:53:26.161926 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:53:26.161908    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6k9tr_0620c0a0-8070-4307-86d9-c69a76ea14a9/ovn-acl-logging/0.log"
Sep 11 17:53:38.123864 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:53:38.123819    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:53:38.832396 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:53:38.832362    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:53:44.626956 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:53:44.626893    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:53:50.319889 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:53:50.319847    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:53:50.575677 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:53:50.575597    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-027edd-openshift-default-6f84795f8b-46ktf"]
Sep 11 17:53:50.575907 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:53:50.575863    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-027edd-openshift-default-6f84795f8b-46ktf" podUID="96210882-8a70-4795-910a-01c409c73c78" containerName="istio-proxy" containerID="cri-o://dbc584f29989dde3fb1ecb25254b55fa42d2e2a7aa87da14e753c9b1d86b4e2a" gracePeriod=30
Sep 11 17:53:51.956026 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:53:51.955992    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:53:52.139626 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:53:52.139569    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:53:52.444318 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:53:52.444279    2552 patch_prober.go:28] interesting pod/e2e-ipp-a-027edd-openshift-default-6f84795f8b-46ktf container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 11 17:53:52.444484 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:53:52.444347    2552 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-a-027edd-openshift-default-6f84795f8b-46ktf" podUID="96210882-8a70-4795-910a-01c409c73c78" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 11 17:53:53.146521 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:53:53.146487    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:53:53.841336 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:53:53.841300    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:53:54.614254 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:53:54.614219    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:53:55.824450 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:53:55.824427    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-027edd-openshift-default-6f84795f8b-46ktf"
Sep 11 17:53:55.922064 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:53:55.922021    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/96210882-8a70-4795-910a-01c409c73c78-credential-socket\") pod \"96210882-8a70-4795-910a-01c409c73c78\" (UID: \"96210882-8a70-4795-910a-01c409c73c78\") "
Sep 11 17:53:55.922064 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:53:55.922071    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/96210882-8a70-4795-910a-01c409c73c78-istiod-ca-cert\") pod \"96210882-8a70-4795-910a-01c409c73c78\" (UID: \"96210882-8a70-4795-910a-01c409c73c78\") "
Sep 11 17:53:55.922437 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:53:55.922099    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/96210882-8a70-4795-910a-01c409c73c78-istio-podinfo\") pod \"96210882-8a70-4795-910a-01c409c73c78\" (UID: \"96210882-8a70-4795-910a-01c409c73c78\") "
Sep 11 17:53:55.922437 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:53:55.922153    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/96210882-8a70-4795-910a-01c409c73c78-istio-envoy\") pod \"96210882-8a70-4795-910a-01c409c73c78\" (UID: \"96210882-8a70-4795-910a-01c409c73c78\") "
Sep 11 17:53:55.922437 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:53:55.922227    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/96210882-8a70-4795-910a-01c409c73c78-istio-data\") pod \"96210882-8a70-4795-910a-01c409c73c78\" (UID: \"96210882-8a70-4795-910a-01c409c73c78\") "
Sep 11 17:53:55.922437 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:53:55.922256    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/96210882-8a70-4795-910a-01c409c73c78-workload-certs\") pod \"96210882-8a70-4795-910a-01c409c73c78\" (UID: \"96210882-8a70-4795-910a-01c409c73c78\") "
Sep 11 17:53:55.922437 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:53:55.922288    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/96210882-8a70-4795-910a-01c409c73c78-istio-token\") pod \"96210882-8a70-4795-910a-01c409c73c78\" (UID: \"96210882-8a70-4795-910a-01c409c73c78\") "
Sep 11 17:53:55.922770 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:53:55.922339    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96210882-8a70-4795-910a-01c409c73c78-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "96210882-8a70-4795-910a-01c409c73c78" (UID: "96210882-8a70-4795-910a-01c409c73c78"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:53:55.922844 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:53:55.922754    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pl9r5\" (UniqueName: \"kubernetes.io/projected/96210882-8a70-4795-910a-01c409c73c78-kube-api-access-pl9r5\") pod \"96210882-8a70-4795-910a-01c409c73c78\" (UID: \"96210882-8a70-4795-910a-01c409c73c78\") "
Sep 11 17:53:55.922844 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:53:55.922824    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/96210882-8a70-4795-910a-01c409c73c78-workload-socket\") pod \"96210882-8a70-4795-910a-01c409c73c78\" (UID: \"96210882-8a70-4795-910a-01c409c73c78\") "
Sep 11 17:53:55.922967 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:53:55.922505    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/96210882-8a70-4795-910a-01c409c73c78-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "96210882-8a70-4795-910a-01c409c73c78" (UID: "96210882-8a70-4795-910a-01c409c73c78"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 17:53:55.922967 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:53:55.922635    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96210882-8a70-4795-910a-01c409c73c78-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "96210882-8a70-4795-910a-01c409c73c78" (UID: "96210882-8a70-4795-910a-01c409c73c78"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:53:55.922967 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:53:55.922700    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96210882-8a70-4795-910a-01c409c73c78-istio-data" (OuterVolumeSpecName: "istio-data") pod "96210882-8a70-4795-910a-01c409c73c78" (UID: "96210882-8a70-4795-910a-01c409c73c78"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:53:55.923104 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:53:55.923075    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96210882-8a70-4795-910a-01c409c73c78-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "96210882-8a70-4795-910a-01c409c73c78" (UID: "96210882-8a70-4795-910a-01c409c73c78"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:53:55.923296 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:53:55.923276    2552 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/96210882-8a70-4795-910a-01c409c73c78-credential-socket\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:53:55.923340 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:53:55.923304    2552 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/96210882-8a70-4795-910a-01c409c73c78-istiod-ca-cert\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:53:55.923340 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:53:55.923319    2552 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/96210882-8a70-4795-910a-01c409c73c78-istio-data\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:53:55.923340 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:53:55.923332    2552 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/96210882-8a70-4795-910a-01c409c73c78-workload-certs\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:53:55.923442 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:53:55.923345    2552 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/96210882-8a70-4795-910a-01c409c73c78-workload-socket\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:53:55.924541 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:53:55.924501    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/96210882-8a70-4795-910a-01c409c73c78-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "96210882-8a70-4795-910a-01c409c73c78" (UID: "96210882-8a70-4795-910a-01c409c73c78"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 17:53:55.924624 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:53:55.924532    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/96210882-8a70-4795-910a-01c409c73c78-istio-token" (OuterVolumeSpecName: "istio-token") pod "96210882-8a70-4795-910a-01c409c73c78" (UID: "96210882-8a70-4795-910a-01c409c73c78"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:53:55.924810 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:53:55.924789    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96210882-8a70-4795-910a-01c409c73c78-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "96210882-8a70-4795-910a-01c409c73c78" (UID: "96210882-8a70-4795-910a-01c409c73c78"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:53:55.924884 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:53:55.924865    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/96210882-8a70-4795-910a-01c409c73c78-kube-api-access-pl9r5" (OuterVolumeSpecName: "kube-api-access-pl9r5") pod "96210882-8a70-4795-910a-01c409c73c78" (UID: "96210882-8a70-4795-910a-01c409c73c78"). InnerVolumeSpecName "kube-api-access-pl9r5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:53:55.983890 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:53:55.983856    2552 generic.go:356] "Generic (PLEG): container finished" podID="96210882-8a70-4795-910a-01c409c73c78" containerID="dbc584f29989dde3fb1ecb25254b55fa42d2e2a7aa87da14e753c9b1d86b4e2a" exitCode=0
Sep 11 17:53:55.984032 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:53:55.983897    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-027edd-openshift-default-6f84795f8b-46ktf" event={"ID":"96210882-8a70-4795-910a-01c409c73c78","Type":"ContainerDied","Data":"dbc584f29989dde3fb1ecb25254b55fa42d2e2a7aa87da14e753c9b1d86b4e2a"}
Sep 11 17:53:55.984032 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:53:55.983920    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-027edd-openshift-default-6f84795f8b-46ktf"
Sep 11 17:53:55.984032 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:53:55.983951    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-027edd-openshift-default-6f84795f8b-46ktf" event={"ID":"96210882-8a70-4795-910a-01c409c73c78","Type":"ContainerDied","Data":"c6e456e7f1a7c29312456c8fe0ea4ff3a378e0aaaada2e8dd9288165637f4339"}
Sep 11 17:53:55.984032 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:53:55.983975    2552 scope.go:117] "RemoveContainer" containerID="dbc584f29989dde3fb1ecb25254b55fa42d2e2a7aa87da14e753c9b1d86b4e2a"
Sep 11 17:53:55.993610 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:53:55.993592    2552 scope.go:117] "RemoveContainer" containerID="4930fd338c02f15925a8207cc8330a7c4c7230ea437f22a474f4bc9fbfe3e88f"
Sep 11 17:53:56.001370 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:53:56.001347    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-027edd-openshift-default-6f84795f8b-46ktf"]
Sep 11 17:53:56.002302 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:53:56.002283    2552 scope.go:117] "RemoveContainer" containerID="dbc584f29989dde3fb1ecb25254b55fa42d2e2a7aa87da14e753c9b1d86b4e2a"
Sep 11 17:53:56.002602 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:53:56.002582    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dbc584f29989dde3fb1ecb25254b55fa42d2e2a7aa87da14e753c9b1d86b4e2a\": container with ID starting with dbc584f29989dde3fb1ecb25254b55fa42d2e2a7aa87da14e753c9b1d86b4e2a not found: ID does not exist" containerID="dbc584f29989dde3fb1ecb25254b55fa42d2e2a7aa87da14e753c9b1d86b4e2a"
Sep 11 17:53:56.002672 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:53:56.002612    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dbc584f29989dde3fb1ecb25254b55fa42d2e2a7aa87da14e753c9b1d86b4e2a"} err="failed to get container status \"dbc584f29989dde3fb1ecb25254b55fa42d2e2a7aa87da14e753c9b1d86b4e2a\": rpc error: code = NotFound desc = could not find container \"dbc584f29989dde3fb1ecb25254b55fa42d2e2a7aa87da14e753c9b1d86b4e2a\": container with ID starting with dbc584f29989dde3fb1ecb25254b55fa42d2e2a7aa87da14e753c9b1d86b4e2a not found: ID does not exist"
Sep 11 17:53:56.002672 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:53:56.002628    2552 scope.go:117] "RemoveContainer" containerID="4930fd338c02f15925a8207cc8330a7c4c7230ea437f22a474f4bc9fbfe3e88f"
Sep 11 17:53:56.002880 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:53:56.002858    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4930fd338c02f15925a8207cc8330a7c4c7230ea437f22a474f4bc9fbfe3e88f\": container with ID starting with 4930fd338c02f15925a8207cc8330a7c4c7230ea437f22a474f4bc9fbfe3e88f not found: ID does not exist" containerID="4930fd338c02f15925a8207cc8330a7c4c7230ea437f22a474f4bc9fbfe3e88f"
Sep 11 17:53:56.002940 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:53:56.002889    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4930fd338c02f15925a8207cc8330a7c4c7230ea437f22a474f4bc9fbfe3e88f"} err="failed to get container status \"4930fd338c02f15925a8207cc8330a7c4c7230ea437f22a474f4bc9fbfe3e88f\": rpc error: code = NotFound desc = could not find container \"4930fd338c02f15925a8207cc8330a7c4c7230ea437f22a474f4bc9fbfe3e88f\": container with ID starting with 4930fd338c02f15925a8207cc8330a7c4c7230ea437f22a474f4bc9fbfe3e88f not found: ID does not exist"
Sep 11 17:53:56.007761 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:53:56.007742    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-027edd-openshift-default-6f84795f8b-46ktf"]
Sep 11 17:53:56.024322 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:53:56.024300    2552 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/96210882-8a70-4795-910a-01c409c73c78-istio-podinfo\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:53:56.024322 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:53:56.024319    2552 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/96210882-8a70-4795-910a-01c409c73c78-istio-envoy\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:53:56.024322 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:53:56.024326    2552 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/96210882-8a70-4795-910a-01c409c73c78-istio-token\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:53:56.024476 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:53:56.024334    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pl9r5\" (UniqueName: \"kubernetes.io/projected/96210882-8a70-4795-910a-01c409c73c78-kube-api-access-pl9r5\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:53:57.949834 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:53:57.949802    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="96210882-8a70-4795-910a-01c409c73c78" path="/var/lib/kubelet/pods/96210882-8a70-4795-910a-01c409c73c78/volumes"
Sep 11 17:54:01.431636 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:01.431595    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-10bbea-5b6d6b7df9-fd74t"]
Sep 11 17:54:01.432148 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:01.431874    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-10bbea-5b6d6b7df9-fd74t" podUID="287b139c-ae79-4112-8188-031eb9bb66a1" containerName="maas-api" containerID="cri-o://7d8c3154a59a83a3b6fe952af8623f6f146a4b50992dddd8f672f7154fa039a0" gracePeriod=30
Sep 11 17:54:01.456990 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:01.456953    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:54:01.529266 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:01.529228    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:54:01.600332 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:01.600305    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:54:01.689871 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:01.689802    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-10bbea-5b6d6b7df9-fd74t"
Sep 11 17:54:01.777125 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:01.777090    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:54:01.780084 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:01.780061    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/287b139c-ae79-4112-8188-031eb9bb66a1-maas-api-tls\") pod \"287b139c-ae79-4112-8188-031eb9bb66a1\" (UID: \"287b139c-ae79-4112-8188-031eb9bb66a1\") "
Sep 11 17:54:01.780186 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:01.780103    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/287b139c-ae79-4112-8188-031eb9bb66a1-metrics-certs\") pod \"287b139c-ae79-4112-8188-031eb9bb66a1\" (UID: \"287b139c-ae79-4112-8188-031eb9bb66a1\") "
Sep 11 17:54:01.780306 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:01.780287    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/287b139c-ae79-4112-8188-031eb9bb66a1-openshift-service-ca\") pod \"287b139c-ae79-4112-8188-031eb9bb66a1\" (UID: \"287b139c-ae79-4112-8188-031eb9bb66a1\") "
Sep 11 17:54:01.780357 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:01.780321    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6znjf\" (UniqueName: \"kubernetes.io/projected/287b139c-ae79-4112-8188-031eb9bb66a1-kube-api-access-6znjf\") pod \"287b139c-ae79-4112-8188-031eb9bb66a1\" (UID: \"287b139c-ae79-4112-8188-031eb9bb66a1\") "
Sep 11 17:54:01.780670 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:01.780638    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/287b139c-ae79-4112-8188-031eb9bb66a1-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "287b139c-ae79-4112-8188-031eb9bb66a1" (UID: "287b139c-ae79-4112-8188-031eb9bb66a1"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 17:54:01.780790 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:01.780714    2552 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/287b139c-ae79-4112-8188-031eb9bb66a1-openshift-service-ca\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:54:01.782436 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:01.782409    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/287b139c-ae79-4112-8188-031eb9bb66a1-kube-api-access-6znjf" (OuterVolumeSpecName: "kube-api-access-6znjf") pod "287b139c-ae79-4112-8188-031eb9bb66a1" (UID: "287b139c-ae79-4112-8188-031eb9bb66a1"). InnerVolumeSpecName "kube-api-access-6znjf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:54:01.782713 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:01.782690    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/287b139c-ae79-4112-8188-031eb9bb66a1-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "287b139c-ae79-4112-8188-031eb9bb66a1" (UID: "287b139c-ae79-4112-8188-031eb9bb66a1"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 17:54:01.782765 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:01.782699    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/287b139c-ae79-4112-8188-031eb9bb66a1-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "287b139c-ae79-4112-8188-031eb9bb66a1" (UID: "287b139c-ae79-4112-8188-031eb9bb66a1"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 17:54:01.881528 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:01.881488    2552 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/287b139c-ae79-4112-8188-031eb9bb66a1-maas-api-tls\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:54:01.881528 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:01.881521    2552 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/287b139c-ae79-4112-8188-031eb9bb66a1-metrics-certs\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:54:01.881528 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:01.881532    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6znjf\" (UniqueName: \"kubernetes.io/projected/287b139c-ae79-4112-8188-031eb9bb66a1-kube-api-access-6znjf\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:54:02.023510 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:02.023475    2552 generic.go:356] "Generic (PLEG): container finished" podID="287b139c-ae79-4112-8188-031eb9bb66a1" containerID="7d8c3154a59a83a3b6fe952af8623f6f146a4b50992dddd8f672f7154fa039a0" exitCode=0
Sep 11 17:54:02.023719 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:02.023517    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-10bbea-5b6d6b7df9-fd74t"
Sep 11 17:54:02.023719 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:02.023562    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-10bbea-5b6d6b7df9-fd74t" event={"ID":"287b139c-ae79-4112-8188-031eb9bb66a1","Type":"ContainerDied","Data":"7d8c3154a59a83a3b6fe952af8623f6f146a4b50992dddd8f672f7154fa039a0"}
Sep 11 17:54:02.023719 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:02.023625    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-10bbea-5b6d6b7df9-fd74t" event={"ID":"287b139c-ae79-4112-8188-031eb9bb66a1","Type":"ContainerDied","Data":"9a90fc59bc2e429dad10ef65b9f8e9f29e6154e73bf31731710dbc470c3d4ccb"}
Sep 11 17:54:02.023719 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:02.023657    2552 scope.go:117] "RemoveContainer" containerID="7d8c3154a59a83a3b6fe952af8623f6f146a4b50992dddd8f672f7154fa039a0"
Sep 11 17:54:02.033634 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:02.033613    2552 scope.go:117] "RemoveContainer" containerID="7d8c3154a59a83a3b6fe952af8623f6f146a4b50992dddd8f672f7154fa039a0"
Sep 11 17:54:02.033977 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:54:02.033949    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7d8c3154a59a83a3b6fe952af8623f6f146a4b50992dddd8f672f7154fa039a0\": container with ID starting with 7d8c3154a59a83a3b6fe952af8623f6f146a4b50992dddd8f672f7154fa039a0 not found: ID does not exist" containerID="7d8c3154a59a83a3b6fe952af8623f6f146a4b50992dddd8f672f7154fa039a0"
Sep 11 17:54:02.034034 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:02.033990    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d8c3154a59a83a3b6fe952af8623f6f146a4b50992dddd8f672f7154fa039a0"} err="failed to get container status \"7d8c3154a59a83a3b6fe952af8623f6f146a4b50992dddd8f672f7154fa039a0\": rpc error: code = NotFound desc = could not find container \"7d8c3154a59a83a3b6fe952af8623f6f146a4b50992dddd8f672f7154fa039a0\": container with ID starting with 7d8c3154a59a83a3b6fe952af8623f6f146a4b50992dddd8f672f7154fa039a0 not found: ID does not exist"
Sep 11 17:54:02.040687 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:02.040656    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-10bbea-5b6d6b7df9-fd74t"]
Sep 11 17:54:02.046756 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:02.046727    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-10bbea-5b6d6b7df9-fd74t"]
Sep 11 17:54:02.114572 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:02.114535    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:54:02.228315 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:02.228207    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:54:03.042731 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:03.042616    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:54:03.796168 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:03.796136    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:54:03.950187 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:03.950155    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="287b139c-ae79-4112-8188-031eb9bb66a1" path="/var/lib/kubelet/pods/287b139c-ae79-4112-8188-031eb9bb66a1/volumes"
Sep 11 17:54:08.320682 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:08.320640    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:54:08.423715 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:08.423673    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:54:08.607006 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:08.606971    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-10bbea-openshift-default-f67547c5d-hnlvj"]
Sep 11 17:54:08.607256 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:08.607232    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-10bbea-openshift-default-f67547c5d-hnlvj" podUID="69a4dccc-5986-4985-9494-9c2acf232e41" containerName="istio-proxy" containerID="cri-o://9695cc959d50b5f78d30ba436193051734594f08cbb756cbe2cc368d5996626a" gracePeriod=30
Sep 11 17:54:08.642038 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:08.642010    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:54:09.604495 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:09.604461    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:54:10.331559 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:10.331516    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:54:11.151422 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:11.151392    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:54:11.253743 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:11.253706    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:54:12.996122 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:12.996085    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:54:13.868275 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:13.868252    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-10bbea-openshift-default-f67547c5d-hnlvj"
Sep 11 17:54:13.998702 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:13.998604    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/69a4dccc-5986-4985-9494-9c2acf232e41-workload-socket\") pod \"69a4dccc-5986-4985-9494-9c2acf232e41\" (UID: \"69a4dccc-5986-4985-9494-9c2acf232e41\") "
Sep 11 17:54:13.998702 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:13.998662    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/69a4dccc-5986-4985-9494-9c2acf232e41-istiod-ca-cert\") pod \"69a4dccc-5986-4985-9494-9c2acf232e41\" (UID: \"69a4dccc-5986-4985-9494-9c2acf232e41\") "
Sep 11 17:54:13.998702 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:13.998703    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/69a4dccc-5986-4985-9494-9c2acf232e41-istio-token\") pod \"69a4dccc-5986-4985-9494-9c2acf232e41\" (UID: \"69a4dccc-5986-4985-9494-9c2acf232e41\") "
Sep 11 17:54:13.999310 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:13.998725    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/69a4dccc-5986-4985-9494-9c2acf232e41-istio-data\") pod \"69a4dccc-5986-4985-9494-9c2acf232e41\" (UID: \"69a4dccc-5986-4985-9494-9c2acf232e41\") "
Sep 11 17:54:13.999310 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:13.998749    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/69a4dccc-5986-4985-9494-9c2acf232e41-credential-socket\") pod \"69a4dccc-5986-4985-9494-9c2acf232e41\" (UID: \"69a4dccc-5986-4985-9494-9c2acf232e41\") "
Sep 11 17:54:13.999310 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:13.998772    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n6bhx\" (UniqueName: \"kubernetes.io/projected/69a4dccc-5986-4985-9494-9c2acf232e41-kube-api-access-n6bhx\") pod \"69a4dccc-5986-4985-9494-9c2acf232e41\" (UID: \"69a4dccc-5986-4985-9494-9c2acf232e41\") "
Sep 11 17:54:13.999310 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:13.998808    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/69a4dccc-5986-4985-9494-9c2acf232e41-istio-podinfo\") pod \"69a4dccc-5986-4985-9494-9c2acf232e41\" (UID: \"69a4dccc-5986-4985-9494-9c2acf232e41\") "
Sep 11 17:54:13.999310 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:13.998860    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/69a4dccc-5986-4985-9494-9c2acf232e41-workload-certs\") pod \"69a4dccc-5986-4985-9494-9c2acf232e41\" (UID: \"69a4dccc-5986-4985-9494-9c2acf232e41\") "
Sep 11 17:54:13.999310 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:13.998897    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/69a4dccc-5986-4985-9494-9c2acf232e41-istio-envoy\") pod \"69a4dccc-5986-4985-9494-9c2acf232e41\" (UID: \"69a4dccc-5986-4985-9494-9c2acf232e41\") "
Sep 11 17:54:13.999310 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:13.998944    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/69a4dccc-5986-4985-9494-9c2acf232e41-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "69a4dccc-5986-4985-9494-9c2acf232e41" (UID: "69a4dccc-5986-4985-9494-9c2acf232e41"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:54:13.999310 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:13.999131    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/69a4dccc-5986-4985-9494-9c2acf232e41-istio-data" (OuterVolumeSpecName: "istio-data") pod "69a4dccc-5986-4985-9494-9c2acf232e41" (UID: "69a4dccc-5986-4985-9494-9c2acf232e41"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:54:13.999310 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:13.999160    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/69a4dccc-5986-4985-9494-9c2acf232e41-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "69a4dccc-5986-4985-9494-9c2acf232e41" (UID: "69a4dccc-5986-4985-9494-9c2acf232e41"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 17:54:13.999310 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:13.999199    2552 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/69a4dccc-5986-4985-9494-9c2acf232e41-workload-socket\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:54:13.999310 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:13.999238    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/69a4dccc-5986-4985-9494-9c2acf232e41-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "69a4dccc-5986-4985-9494-9c2acf232e41" (UID: "69a4dccc-5986-4985-9494-9c2acf232e41"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:54:13.999842 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:13.999341    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/69a4dccc-5986-4985-9494-9c2acf232e41-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "69a4dccc-5986-4985-9494-9c2acf232e41" (UID: "69a4dccc-5986-4985-9494-9c2acf232e41"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:54:14.001102 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:14.001081    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/69a4dccc-5986-4985-9494-9c2acf232e41-istio-token" (OuterVolumeSpecName: "istio-token") pod "69a4dccc-5986-4985-9494-9c2acf232e41" (UID: "69a4dccc-5986-4985-9494-9c2acf232e41"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:54:14.001298 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:14.001276    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/69a4dccc-5986-4985-9494-9c2acf232e41-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "69a4dccc-5986-4985-9494-9c2acf232e41" (UID: "69a4dccc-5986-4985-9494-9c2acf232e41"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 17:54:14.001535 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:14.001516    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/69a4dccc-5986-4985-9494-9c2acf232e41-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "69a4dccc-5986-4985-9494-9c2acf232e41" (UID: "69a4dccc-5986-4985-9494-9c2acf232e41"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:54:14.001535 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:14.001523    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/69a4dccc-5986-4985-9494-9c2acf232e41-kube-api-access-n6bhx" (OuterVolumeSpecName: "kube-api-access-n6bhx") pod "69a4dccc-5986-4985-9494-9c2acf232e41" (UID: "69a4dccc-5986-4985-9494-9c2acf232e41"). InnerVolumeSpecName "kube-api-access-n6bhx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:54:14.098355 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:14.098314    2552 generic.go:356] "Generic (PLEG): container finished" podID="69a4dccc-5986-4985-9494-9c2acf232e41" containerID="9695cc959d50b5f78d30ba436193051734594f08cbb756cbe2cc368d5996626a" exitCode=0
Sep 11 17:54:14.098543 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:14.098363    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-10bbea-openshift-default-f67547c5d-hnlvj"
Sep 11 17:54:14.098543 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:14.098393    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-10bbea-openshift-default-f67547c5d-hnlvj" event={"ID":"69a4dccc-5986-4985-9494-9c2acf232e41","Type":"ContainerDied","Data":"9695cc959d50b5f78d30ba436193051734594f08cbb756cbe2cc368d5996626a"}
Sep 11 17:54:14.098543 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:14.098454    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-10bbea-openshift-default-f67547c5d-hnlvj" event={"ID":"69a4dccc-5986-4985-9494-9c2acf232e41","Type":"ContainerDied","Data":"0e5c68623d61e28a880122b95f3ab8acee562d3a54f5f284b2d77ea5d3954ce2"}
Sep 11 17:54:14.098543 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:14.098487    2552 scope.go:117] "RemoveContainer" containerID="9695cc959d50b5f78d30ba436193051734594f08cbb756cbe2cc368d5996626a"
Sep 11 17:54:14.099734 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:14.099713    2552 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/69a4dccc-5986-4985-9494-9c2acf232e41-istio-envoy\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:54:14.099836 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:14.099739    2552 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/69a4dccc-5986-4985-9494-9c2acf232e41-istiod-ca-cert\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:54:14.099836 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:14.099753    2552 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/69a4dccc-5986-4985-9494-9c2acf232e41-istio-token\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:54:14.099836 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:14.099764    2552 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/69a4dccc-5986-4985-9494-9c2acf232e41-istio-data\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:54:14.099836 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:14.099777    2552 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/69a4dccc-5986-4985-9494-9c2acf232e41-credential-socket\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:54:14.099836 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:14.099798    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n6bhx\" (UniqueName: \"kubernetes.io/projected/69a4dccc-5986-4985-9494-9c2acf232e41-kube-api-access-n6bhx\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:54:14.099836 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:14.099811    2552 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/69a4dccc-5986-4985-9494-9c2acf232e41-istio-podinfo\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:54:14.100075 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:14.099842    2552 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/69a4dccc-5986-4985-9494-9c2acf232e41-workload-certs\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:54:14.108658 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:14.108638    2552 scope.go:117] "RemoveContainer" containerID="d86ce15950010f5ee5b96118157cc8019c677ffe17fd1574fc3fe081aac88b0f"
Sep 11 17:54:14.117673 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:14.117650    2552 scope.go:117] "RemoveContainer" containerID="9695cc959d50b5f78d30ba436193051734594f08cbb756cbe2cc368d5996626a"
Sep 11 17:54:14.118017 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:54:14.117992    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9695cc959d50b5f78d30ba436193051734594f08cbb756cbe2cc368d5996626a\": container with ID starting with 9695cc959d50b5f78d30ba436193051734594f08cbb756cbe2cc368d5996626a not found: ID does not exist" containerID="9695cc959d50b5f78d30ba436193051734594f08cbb756cbe2cc368d5996626a"
Sep 11 17:54:14.118150 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:14.118023    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9695cc959d50b5f78d30ba436193051734594f08cbb756cbe2cc368d5996626a"} err="failed to get container status \"9695cc959d50b5f78d30ba436193051734594f08cbb756cbe2cc368d5996626a\": rpc error: code = NotFound desc = could not find container \"9695cc959d50b5f78d30ba436193051734594f08cbb756cbe2cc368d5996626a\": container with ID starting with 9695cc959d50b5f78d30ba436193051734594f08cbb756cbe2cc368d5996626a not found: ID does not exist"
Sep 11 17:54:14.118150 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:14.118043    2552 scope.go:117] "RemoveContainer" containerID="d86ce15950010f5ee5b96118157cc8019c677ffe17fd1574fc3fe081aac88b0f"
Sep 11 17:54:14.118406 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:54:14.118380    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d86ce15950010f5ee5b96118157cc8019c677ffe17fd1574fc3fe081aac88b0f\": container with ID starting with d86ce15950010f5ee5b96118157cc8019c677ffe17fd1574fc3fe081aac88b0f not found: ID does not exist" containerID="d86ce15950010f5ee5b96118157cc8019c677ffe17fd1574fc3fe081aac88b0f"
Sep 11 17:54:14.118536 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:14.118422    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d86ce15950010f5ee5b96118157cc8019c677ffe17fd1574fc3fe081aac88b0f"} err="failed to get container status \"d86ce15950010f5ee5b96118157cc8019c677ffe17fd1574fc3fe081aac88b0f\": rpc error: code = NotFound desc = could not find container \"d86ce15950010f5ee5b96118157cc8019c677ffe17fd1574fc3fe081aac88b0f\": container with ID starting with d86ce15950010f5ee5b96118157cc8019c677ffe17fd1574fc3fe081aac88b0f not found: ID does not exist"
Sep 11 17:54:14.121376 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:14.121354    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-10bbea-openshift-default-f67547c5d-hnlvj"]
Sep 11 17:54:14.126192 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:14.126167    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-10bbea-openshift-default-f67547c5d-hnlvj"]
Sep 11 17:54:15.949712 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:15.949666    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="69a4dccc-5986-4985-9494-9c2acf232e41" path="/var/lib/kubelet/pods/69a4dccc-5986-4985-9494-9c2acf232e41/volumes"
Sep 11 17:54:19.440511 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:19.440296    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-2fac54-7965f98cb9-fqn9k"]
Sep 11 17:54:19.441325 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:19.441284    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-2fac54-7965f98cb9-fqn9k" podUID="2b33b5cc-f45a-48f9-8c3f-d2a32db8243d" containerName="maas-api" containerID="cri-o://8a92af631272a636349c2c4a127a4cdb0a78443a77090d3e12b53da52a351185" gracePeriod=30
Sep 11 17:54:19.467674 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:19.467640    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:54:19.572049 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:19.572001    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:54:19.669284 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:19.669242    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:54:19.700235 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:19.700147    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-2fac54-7965f98cb9-fqn9k"
Sep 11 17:54:19.768409 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:19.768356    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:54:19.851661 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:19.851626    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2b33b5cc-f45a-48f9-8c3f-d2a32db8243d-maas-api-tls\") pod \"2b33b5cc-f45a-48f9-8c3f-d2a32db8243d\" (UID: \"2b33b5cc-f45a-48f9-8c3f-d2a32db8243d\") "
Sep 11 17:54:19.851876 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:19.851736    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l7lt8\" (UniqueName: \"kubernetes.io/projected/2b33b5cc-f45a-48f9-8c3f-d2a32db8243d-kube-api-access-l7lt8\") pod \"2b33b5cc-f45a-48f9-8c3f-d2a32db8243d\" (UID: \"2b33b5cc-f45a-48f9-8c3f-d2a32db8243d\") "
Sep 11 17:54:19.851876 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:19.851762    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2b33b5cc-f45a-48f9-8c3f-d2a32db8243d-openshift-service-ca\") pod \"2b33b5cc-f45a-48f9-8c3f-d2a32db8243d\" (UID: \"2b33b5cc-f45a-48f9-8c3f-d2a32db8243d\") "
Sep 11 17:54:19.851876 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:19.851816    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2b33b5cc-f45a-48f9-8c3f-d2a32db8243d-metrics-certs\") pod \"2b33b5cc-f45a-48f9-8c3f-d2a32db8243d\" (UID: \"2b33b5cc-f45a-48f9-8c3f-d2a32db8243d\") "
Sep 11 17:54:19.852172 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:19.852144    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2b33b5cc-f45a-48f9-8c3f-d2a32db8243d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "2b33b5cc-f45a-48f9-8c3f-d2a32db8243d" (UID: "2b33b5cc-f45a-48f9-8c3f-d2a32db8243d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 17:54:19.854032 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:19.853996    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2b33b5cc-f45a-48f9-8c3f-d2a32db8243d-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "2b33b5cc-f45a-48f9-8c3f-d2a32db8243d" (UID: "2b33b5cc-f45a-48f9-8c3f-d2a32db8243d"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 17:54:19.854032 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:19.854027    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2b33b5cc-f45a-48f9-8c3f-d2a32db8243d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "2b33b5cc-f45a-48f9-8c3f-d2a32db8243d" (UID: "2b33b5cc-f45a-48f9-8c3f-d2a32db8243d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 17:54:19.854239 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:19.854040    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2b33b5cc-f45a-48f9-8c3f-d2a32db8243d-kube-api-access-l7lt8" (OuterVolumeSpecName: "kube-api-access-l7lt8") pod "2b33b5cc-f45a-48f9-8c3f-d2a32db8243d" (UID: "2b33b5cc-f45a-48f9-8c3f-d2a32db8243d"). InnerVolumeSpecName "kube-api-access-l7lt8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:54:19.953342 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:19.953248    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l7lt8\" (UniqueName: \"kubernetes.io/projected/2b33b5cc-f45a-48f9-8c3f-d2a32db8243d-kube-api-access-l7lt8\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:54:19.953342 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:19.953275    2552 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2b33b5cc-f45a-48f9-8c3f-d2a32db8243d-openshift-service-ca\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:54:19.953342 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:19.953285    2552 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2b33b5cc-f45a-48f9-8c3f-d2a32db8243d-metrics-certs\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:54:19.953342 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:19.953293    2552 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2b33b5cc-f45a-48f9-8c3f-d2a32db8243d-maas-api-tls\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:54:20.055425 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:20.055387    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:54:20.137825 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:20.137782    2552 generic.go:356] "Generic (PLEG): container finished" podID="2b33b5cc-f45a-48f9-8c3f-d2a32db8243d" containerID="8a92af631272a636349c2c4a127a4cdb0a78443a77090d3e12b53da52a351185" exitCode=0
Sep 11 17:54:20.138029 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:20.137821    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-2fac54-7965f98cb9-fqn9k" event={"ID":"2b33b5cc-f45a-48f9-8c3f-d2a32db8243d","Type":"ContainerDied","Data":"8a92af631272a636349c2c4a127a4cdb0a78443a77090d3e12b53da52a351185"}
Sep 11 17:54:20.138029 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:20.137845    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-2fac54-7965f98cb9-fqn9k"
Sep 11 17:54:20.138029 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:20.137907    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-2fac54-7965f98cb9-fqn9k" event={"ID":"2b33b5cc-f45a-48f9-8c3f-d2a32db8243d","Type":"ContainerDied","Data":"36568502b2f6bc4842b417e51dd166cc9d112a7216a1eda74a59d029a8374d50"}
Sep 11 17:54:20.138029 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:20.137939    2552 scope.go:117] "RemoveContainer" containerID="8a92af631272a636349c2c4a127a4cdb0a78443a77090d3e12b53da52a351185"
Sep 11 17:54:20.148238 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:20.148212    2552 scope.go:117] "RemoveContainer" containerID="8a92af631272a636349c2c4a127a4cdb0a78443a77090d3e12b53da52a351185"
Sep 11 17:54:20.148597 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:54:20.148577    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8a92af631272a636349c2c4a127a4cdb0a78443a77090d3e12b53da52a351185\": container with ID starting with 8a92af631272a636349c2c4a127a4cdb0a78443a77090d3e12b53da52a351185 not found: ID does not exist" containerID="8a92af631272a636349c2c4a127a4cdb0a78443a77090d3e12b53da52a351185"
Sep 11 17:54:20.148676 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:20.148606    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8a92af631272a636349c2c4a127a4cdb0a78443a77090d3e12b53da52a351185"} err="failed to get container status \"8a92af631272a636349c2c4a127a4cdb0a78443a77090d3e12b53da52a351185\": rpc error: code = NotFound desc = could not find container \"8a92af631272a636349c2c4a127a4cdb0a78443a77090d3e12b53da52a351185\": container with ID starting with 8a92af631272a636349c2c4a127a4cdb0a78443a77090d3e12b53da52a351185 not found: ID does not exist"
Sep 11 17:54:20.157173 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:20.157141    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-2fac54-7965f98cb9-fqn9k"]
Sep 11 17:54:20.160759 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:20.160733    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:54:20.161312 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:20.161295    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-2fac54-7965f98cb9-fqn9k"]
Sep 11 17:54:20.879746 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:20.879714    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:54:21.949057 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:21.949022    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2b33b5cc-f45a-48f9-8c3f-d2a32db8243d" path="/var/lib/kubelet/pods/2b33b5cc-f45a-48f9-8c3f-d2a32db8243d/volumes"
Sep 11 17:54:25.513935 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:25.513892    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:54:26.044044 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:26.044004    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:54:26.144044 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:26.144006    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:54:26.384090 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:26.384053    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:54:27.639932 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:27.639888    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:54:28.409819 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:28.409778    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:54:37.331010 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:37.330978    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-6fc9e6-68fb4f5b48-9tqw5"]
Sep 11 17:54:37.331398 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:37.331236    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-6fc9e6-68fb4f5b48-9tqw5" podUID="46fa2c22-db29-429f-812f-422ae705d742" containerName="maas-api" containerID="cri-o://fe85f241d9d61781a29630e2d7536694b35a8ef3fb74ce75666b21910a60d8ea" gracePeriod=30
Sep 11 17:54:37.364679 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:37.364643    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-6fc9e6-29819145-szflz"]
Sep 11 17:54:37.365421 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:37.365393    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:54:37.368359 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:37.368333    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-6fc9e6-29819145-szflz"]
Sep 11 17:54:37.467478 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:37.467448    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:54:37.565628 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:37.565593    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:54:37.579666 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:37.579641    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-6fc9e6-68fb4f5b48-9tqw5"
Sep 11 17:54:37.661306 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:37.661256    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:54:37.709399 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:37.709362    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/46fa2c22-db29-429f-812f-422ae705d742-metrics-certs\") pod \"46fa2c22-db29-429f-812f-422ae705d742\" (UID: \"46fa2c22-db29-429f-812f-422ae705d742\") "
Sep 11 17:54:37.709399 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:37.709396    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/46fa2c22-db29-429f-812f-422ae705d742-maas-api-tls\") pod \"46fa2c22-db29-429f-812f-422ae705d742\" (UID: \"46fa2c22-db29-429f-812f-422ae705d742\") "
Sep 11 17:54:37.709662 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:37.709469    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g74xj\" (UniqueName: \"kubernetes.io/projected/46fa2c22-db29-429f-812f-422ae705d742-kube-api-access-g74xj\") pod \"46fa2c22-db29-429f-812f-422ae705d742\" (UID: \"46fa2c22-db29-429f-812f-422ae705d742\") "
Sep 11 17:54:37.709662 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:37.709523    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/46fa2c22-db29-429f-812f-422ae705d742-openshift-service-ca\") pod \"46fa2c22-db29-429f-812f-422ae705d742\" (UID: \"46fa2c22-db29-429f-812f-422ae705d742\") "
Sep 11 17:54:37.710053 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:37.709989    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/46fa2c22-db29-429f-812f-422ae705d742-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "46fa2c22-db29-429f-812f-422ae705d742" (UID: "46fa2c22-db29-429f-812f-422ae705d742"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 17:54:37.712248 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:37.712217    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/46fa2c22-db29-429f-812f-422ae705d742-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "46fa2c22-db29-429f-812f-422ae705d742" (UID: "46fa2c22-db29-429f-812f-422ae705d742"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 17:54:37.712374 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:37.712227    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/46fa2c22-db29-429f-812f-422ae705d742-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "46fa2c22-db29-429f-812f-422ae705d742" (UID: "46fa2c22-db29-429f-812f-422ae705d742"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 17:54:37.712374 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:37.712287    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/46fa2c22-db29-429f-812f-422ae705d742-kube-api-access-g74xj" (OuterVolumeSpecName: "kube-api-access-g74xj") pod "46fa2c22-db29-429f-812f-422ae705d742" (UID: "46fa2c22-db29-429f-812f-422ae705d742"). InnerVolumeSpecName "kube-api-access-g74xj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:54:37.810587 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:37.810555    2552 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/46fa2c22-db29-429f-812f-422ae705d742-openshift-service-ca\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:54:37.810587 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:37.810585    2552 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/46fa2c22-db29-429f-812f-422ae705d742-metrics-certs\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:54:37.810587 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:37.810595    2552 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/46fa2c22-db29-429f-812f-422ae705d742-maas-api-tls\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:54:37.810854 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:37.810602    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g74xj\" (UniqueName: \"kubernetes.io/projected/46fa2c22-db29-429f-812f-422ae705d742-kube-api-access-g74xj\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:54:37.949758 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:37.949677    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dcd7f9a8-1a7a-4eb8-abf0-0945972796a9" path="/var/lib/kubelet/pods/dcd7f9a8-1a7a-4eb8-abf0-0945972796a9/volumes"
Sep 11 17:54:38.033653 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:38.033600    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:54:38.168570 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:38.168518    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:54:38.242468 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:38.242378    2552 generic.go:356] "Generic (PLEG): container finished" podID="46fa2c22-db29-429f-812f-422ae705d742" containerID="fe85f241d9d61781a29630e2d7536694b35a8ef3fb74ce75666b21910a60d8ea" exitCode=0
Sep 11 17:54:38.242468 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:38.242425    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-6fc9e6-68fb4f5b48-9tqw5"
Sep 11 17:54:38.242468 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:38.242449    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-6fc9e6-68fb4f5b48-9tqw5" event={"ID":"46fa2c22-db29-429f-812f-422ae705d742","Type":"ContainerDied","Data":"fe85f241d9d61781a29630e2d7536694b35a8ef3fb74ce75666b21910a60d8ea"}
Sep 11 17:54:38.242719 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:38.242502    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-6fc9e6-68fb4f5b48-9tqw5" event={"ID":"46fa2c22-db29-429f-812f-422ae705d742","Type":"ContainerDied","Data":"69b2cfcb977b2b4f8b10ad86ac9b09b2943eff3755fc702900eb07becbe2d103"}
Sep 11 17:54:38.242719 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:38.242524    2552 scope.go:117] "RemoveContainer" containerID="fe85f241d9d61781a29630e2d7536694b35a8ef3fb74ce75666b21910a60d8ea"
Sep 11 17:54:38.252736 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:38.252713    2552 scope.go:117] "RemoveContainer" containerID="fe85f241d9d61781a29630e2d7536694b35a8ef3fb74ce75666b21910a60d8ea"
Sep 11 17:54:38.253057 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:54:38.253036    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fe85f241d9d61781a29630e2d7536694b35a8ef3fb74ce75666b21910a60d8ea\": container with ID starting with fe85f241d9d61781a29630e2d7536694b35a8ef3fb74ce75666b21910a60d8ea not found: ID does not exist" containerID="fe85f241d9d61781a29630e2d7536694b35a8ef3fb74ce75666b21910a60d8ea"
Sep 11 17:54:38.253139 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:38.253068    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fe85f241d9d61781a29630e2d7536694b35a8ef3fb74ce75666b21910a60d8ea"} err="failed to get container status \"fe85f241d9d61781a29630e2d7536694b35a8ef3fb74ce75666b21910a60d8ea\": rpc error: code = NotFound desc = could not find container \"fe85f241d9d61781a29630e2d7536694b35a8ef3fb74ce75666b21910a60d8ea\": container with ID starting with fe85f241d9d61781a29630e2d7536694b35a8ef3fb74ce75666b21910a60d8ea not found: ID does not exist"
Sep 11 17:54:38.277810 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:38.277772    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-6fc9e6-68fb4f5b48-9tqw5"]
Sep 11 17:54:38.284047 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:38.284016    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-6fc9e6-68fb4f5b48-9tqw5"]
Sep 11 17:54:39.950132 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:39.950090    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="46fa2c22-db29-429f-812f-422ae705d742" path="/var/lib/kubelet/pods/46fa2c22-db29-429f-812f-422ae705d742/volumes"
Sep 11 17:54:46.554715 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:46.554678    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:54:46.776670 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:46.776632    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:54:46.946952 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:46.946915    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:54:57.389000 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:54:57.388961    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:55:06.443315 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:55:06.443279    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:55:06.544047 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:55:06.544010    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:55:26.442959 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:55:26.442927    2552 scope.go:117] "RemoveContainer" containerID="6f93d3c8e8be11597da774ab24e38506d8b4cea5eb0d2ddc769fe3d18c2454fb"
Sep 11 17:55:34.918316 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:55:34.918282    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:56:13.100320 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:56:13.100284    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:56:14.618602 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:56:14.618567    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:57:16.160398 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:57:16.160360    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-fzs45"]
Sep 11 17:57:16.160903 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:57:16.160594    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-fzs45" podUID="3ebd01e5-f712-42b8-9e89-deff14ec577a" containerName="manager" containerID="cri-o://abed4f1ed03e5825bf40ce82670fbe15fcf82e5905d1419576c22e170d5104bb" gracePeriod=10
Sep 11 17:57:17.909987 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:57:17.909962    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-fzs45"
Sep 11 17:57:18.021963 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:57:18.021932    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3ebd01e5-f712-42b8-9e89-deff14ec577a-extensions-socket-volume\") pod \"3ebd01e5-f712-42b8-9e89-deff14ec577a\" (UID: \"3ebd01e5-f712-42b8-9e89-deff14ec577a\") "
Sep 11 17:57:18.022171 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:57:18.022018    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hbqnb\" (UniqueName: \"kubernetes.io/projected/3ebd01e5-f712-42b8-9e89-deff14ec577a-kube-api-access-hbqnb\") pod \"3ebd01e5-f712-42b8-9e89-deff14ec577a\" (UID: \"3ebd01e5-f712-42b8-9e89-deff14ec577a\") "
Sep 11 17:57:18.022406 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:57:18.022381    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ebd01e5-f712-42b8-9e89-deff14ec577a-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "3ebd01e5-f712-42b8-9e89-deff14ec577a" (UID: "3ebd01e5-f712-42b8-9e89-deff14ec577a"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:57:18.024091 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:57:18.024028    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3ebd01e5-f712-42b8-9e89-deff14ec577a-kube-api-access-hbqnb" (OuterVolumeSpecName: "kube-api-access-hbqnb") pod "3ebd01e5-f712-42b8-9e89-deff14ec577a" (UID: "3ebd01e5-f712-42b8-9e89-deff14ec577a"). InnerVolumeSpecName "kube-api-access-hbqnb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:57:18.123021 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:57:18.122982    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hbqnb\" (UniqueName: \"kubernetes.io/projected/3ebd01e5-f712-42b8-9e89-deff14ec577a-kube-api-access-hbqnb\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:57:18.123021 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:57:18.123014    2552 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3ebd01e5-f712-42b8-9e89-deff14ec577a-extensions-socket-volume\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 17:57:18.160379 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:57:18.160345    2552 generic.go:356] "Generic (PLEG): container finished" podID="3ebd01e5-f712-42b8-9e89-deff14ec577a" containerID="abed4f1ed03e5825bf40ce82670fbe15fcf82e5905d1419576c22e170d5104bb" exitCode=0
Sep 11 17:57:18.160523 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:57:18.160393    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-fzs45"
Sep 11 17:57:18.160523 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:57:18.160433    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-fzs45" event={"ID":"3ebd01e5-f712-42b8-9e89-deff14ec577a","Type":"ContainerDied","Data":"abed4f1ed03e5825bf40ce82670fbe15fcf82e5905d1419576c22e170d5104bb"}
Sep 11 17:57:18.160523 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:57:18.160484    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-fzs45" event={"ID":"3ebd01e5-f712-42b8-9e89-deff14ec577a","Type":"ContainerDied","Data":"ead3d834ae9e8ba03452fe1208eedc5267773df521d5626d63d0997c6307151c"}
Sep 11 17:57:18.160523 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:57:18.160506    2552 scope.go:117] "RemoveContainer" containerID="abed4f1ed03e5825bf40ce82670fbe15fcf82e5905d1419576c22e170d5104bb"
Sep 11 17:57:18.170407 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:57:18.170390    2552 scope.go:117] "RemoveContainer" containerID="18dcd4d7b55049a0bcd1aea91d80e7b4a1f7409a1631ba55d4369637dab0bbdb"
Sep 11 17:57:18.178350 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:57:18.178333    2552 scope.go:117] "RemoveContainer" containerID="abed4f1ed03e5825bf40ce82670fbe15fcf82e5905d1419576c22e170d5104bb"
Sep 11 17:57:18.178594 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:57:18.178575    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"abed4f1ed03e5825bf40ce82670fbe15fcf82e5905d1419576c22e170d5104bb\": container with ID starting with abed4f1ed03e5825bf40ce82670fbe15fcf82e5905d1419576c22e170d5104bb not found: ID does not exist" containerID="abed4f1ed03e5825bf40ce82670fbe15fcf82e5905d1419576c22e170d5104bb"
Sep 11 17:57:18.178656 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:57:18.178603    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"abed4f1ed03e5825bf40ce82670fbe15fcf82e5905d1419576c22e170d5104bb"} err="failed to get container status \"abed4f1ed03e5825bf40ce82670fbe15fcf82e5905d1419576c22e170d5104bb\": rpc error: code = NotFound desc = could not find container \"abed4f1ed03e5825bf40ce82670fbe15fcf82e5905d1419576c22e170d5104bb\": container with ID starting with abed4f1ed03e5825bf40ce82670fbe15fcf82e5905d1419576c22e170d5104bb not found: ID does not exist"
Sep 11 17:57:18.178656 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:57:18.178620    2552 scope.go:117] "RemoveContainer" containerID="18dcd4d7b55049a0bcd1aea91d80e7b4a1f7409a1631ba55d4369637dab0bbdb"
Sep 11 17:57:18.178876 ip-10-0-140-51 kubenswrapper[2552]: E0911 17:57:18.178863    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"18dcd4d7b55049a0bcd1aea91d80e7b4a1f7409a1631ba55d4369637dab0bbdb\": container with ID starting with 18dcd4d7b55049a0bcd1aea91d80e7b4a1f7409a1631ba55d4369637dab0bbdb not found: ID does not exist" containerID="18dcd4d7b55049a0bcd1aea91d80e7b4a1f7409a1631ba55d4369637dab0bbdb"
Sep 11 17:57:18.178918 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:57:18.178878    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"18dcd4d7b55049a0bcd1aea91d80e7b4a1f7409a1631ba55d4369637dab0bbdb"} err="failed to get container status \"18dcd4d7b55049a0bcd1aea91d80e7b4a1f7409a1631ba55d4369637dab0bbdb\": rpc error: code = NotFound desc = could not find container \"18dcd4d7b55049a0bcd1aea91d80e7b4a1f7409a1631ba55d4369637dab0bbdb\": container with ID starting with 18dcd4d7b55049a0bcd1aea91d80e7b4a1f7409a1631ba55d4369637dab0bbdb not found: ID does not exist"
Sep 11 17:57:18.185289 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:57:18.185268    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-fzs45"]
Sep 11 17:57:18.189063 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:57:18.189043    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-fzs45"]
Sep 11 17:57:19.948773 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:57:19.948739    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3ebd01e5-f712-42b8-9e89-deff14ec577a" path="/var/lib/kubelet/pods/3ebd01e5-f712-42b8-9e89-deff14ec577a/volumes"
Sep 11 17:58:16.881569 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:58:16.881534    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-rfgh8"]
Sep 11 17:58:16.882018 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:58:16.881878    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="69a4dccc-5986-4985-9494-9c2acf232e41" containerName="istio-proxy"
Sep 11 17:58:16.882018 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:58:16.881889    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="69a4dccc-5986-4985-9494-9c2acf232e41" containerName="istio-proxy"
Sep 11 17:58:16.882018 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:58:16.881896    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="287b139c-ae79-4112-8188-031eb9bb66a1" containerName="maas-api"
Sep 11 17:58:16.882018 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:58:16.881901    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="287b139c-ae79-4112-8188-031eb9bb66a1" containerName="maas-api"
Sep 11 17:58:16.882018 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:58:16.881912    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="96210882-8a70-4795-910a-01c409c73c78" containerName="istio-proxy"
Sep 11 17:58:16.882018 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:58:16.881918    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="96210882-8a70-4795-910a-01c409c73c78" containerName="istio-proxy"
Sep 11 17:58:16.882018 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:58:16.881928    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="69a4dccc-5986-4985-9494-9c2acf232e41" containerName="istio-proxy"
Sep 11 17:58:16.882018 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:58:16.881934    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="69a4dccc-5986-4985-9494-9c2acf232e41" containerName="istio-proxy"
Sep 11 17:58:16.882018 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:58:16.881941    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2b33b5cc-f45a-48f9-8c3f-d2a32db8243d" containerName="maas-api"
Sep 11 17:58:16.882018 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:58:16.881946    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="2b33b5cc-f45a-48f9-8c3f-d2a32db8243d" containerName="maas-api"
Sep 11 17:58:16.882018 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:58:16.881955    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ebd01e5-f712-42b8-9e89-deff14ec577a" containerName="manager"
Sep 11 17:58:16.882018 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:58:16.881959    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ebd01e5-f712-42b8-9e89-deff14ec577a" containerName="manager"
Sep 11 17:58:16.882018 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:58:16.881967    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="46fa2c22-db29-429f-812f-422ae705d742" containerName="maas-api"
Sep 11 17:58:16.882018 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:58:16.881972    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="46fa2c22-db29-429f-812f-422ae705d742" containerName="maas-api"
Sep 11 17:58:16.882018 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:58:16.881979    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="250eabc9-048b-4de1-a4f1-a07773b7c393" containerName="maas-api"
Sep 11 17:58:16.882018 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:58:16.881984    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="250eabc9-048b-4de1-a4f1-a07773b7c393" containerName="maas-api"
Sep 11 17:58:16.882018 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:58:16.881993    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="222814af-b6d3-474b-bf27-f6ba4d77dc84" containerName="maas-api"
Sep 11 17:58:16.882018 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:58:16.881997    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="222814af-b6d3-474b-bf27-f6ba4d77dc84" containerName="maas-api"
Sep 11 17:58:16.882602 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:58:16.882053    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="69a4dccc-5986-4985-9494-9c2acf232e41" containerName="istio-proxy"
Sep 11 17:58:16.882602 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:58:16.882060    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="96210882-8a70-4795-910a-01c409c73c78" containerName="istio-proxy"
Sep 11 17:58:16.882602 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:58:16.882065    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="3ebd01e5-f712-42b8-9e89-deff14ec577a" containerName="manager"
Sep 11 17:58:16.882602 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:58:16.882072    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="222814af-b6d3-474b-bf27-f6ba4d77dc84" containerName="maas-api"
Sep 11 17:58:16.882602 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:58:16.882077    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="46fa2c22-db29-429f-812f-422ae705d742" containerName="maas-api"
Sep 11 17:58:16.882602 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:58:16.882083    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="2b33b5cc-f45a-48f9-8c3f-d2a32db8243d" containerName="maas-api"
Sep 11 17:58:16.882602 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:58:16.882088    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="96210882-8a70-4795-910a-01c409c73c78" containerName="istio-proxy"
Sep 11 17:58:16.882602 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:58:16.882094    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="69a4dccc-5986-4985-9494-9c2acf232e41" containerName="istio-proxy"
Sep 11 17:58:16.882602 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:58:16.882100    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="250eabc9-048b-4de1-a4f1-a07773b7c393" containerName="maas-api"
Sep 11 17:58:16.882602 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:58:16.882105    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="287b139c-ae79-4112-8188-031eb9bb66a1" containerName="maas-api"
Sep 11 17:58:16.885166 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:58:16.885151    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-rfgh8"
Sep 11 17:58:16.887494 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:58:16.887467    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-dztsz\""
Sep 11 17:58:16.897082 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:58:16.897058    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-rfgh8"]
Sep 11 17:58:17.029034 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:58:17.028999    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tz7kw\" (UniqueName: \"kubernetes.io/projected/1813eee6-0985-4f90-99ca-672d76738bcc-kube-api-access-tz7kw\") pod \"kuadrant-operator-controller-manager-5769f99d9f-rfgh8\" (UID: \"1813eee6-0985-4f90-99ca-672d76738bcc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-rfgh8"
Sep 11 17:58:17.029230 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:58:17.029067    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1813eee6-0985-4f90-99ca-672d76738bcc-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-rfgh8\" (UID: \"1813eee6-0985-4f90-99ca-672d76738bcc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-rfgh8"
Sep 11 17:58:17.130238 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:58:17.130203    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tz7kw\" (UniqueName: \"kubernetes.io/projected/1813eee6-0985-4f90-99ca-672d76738bcc-kube-api-access-tz7kw\") pod \"kuadrant-operator-controller-manager-5769f99d9f-rfgh8\" (UID: \"1813eee6-0985-4f90-99ca-672d76738bcc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-rfgh8"
Sep 11 17:58:17.130392 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:58:17.130285    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1813eee6-0985-4f90-99ca-672d76738bcc-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-rfgh8\" (UID: \"1813eee6-0985-4f90-99ca-672d76738bcc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-rfgh8"
Sep 11 17:58:17.130680 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:58:17.130660    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1813eee6-0985-4f90-99ca-672d76738bcc-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-rfgh8\" (UID: \"1813eee6-0985-4f90-99ca-672d76738bcc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-rfgh8"
Sep 11 17:58:17.145621 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:58:17.145571    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tz7kw\" (UniqueName: \"kubernetes.io/projected/1813eee6-0985-4f90-99ca-672d76738bcc-kube-api-access-tz7kw\") pod \"kuadrant-operator-controller-manager-5769f99d9f-rfgh8\" (UID: \"1813eee6-0985-4f90-99ca-672d76738bcc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-rfgh8"
Sep 11 17:58:17.195596 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:58:17.195567    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-rfgh8"
Sep 11 17:58:17.475593 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:58:17.475567    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-rfgh8"]
Sep 11 17:58:17.478238 ip-10-0-140-51 kubenswrapper[2552]: W0911 17:58:17.478213    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1813eee6_0985_4f90_99ca_672d76738bcc.slice/crio-85ba190aa8361559a92e5e8bda2aef6418bfbf79ea38127334976351f10fe1ed WatchSource:0}: Error finding container 85ba190aa8361559a92e5e8bda2aef6418bfbf79ea38127334976351f10fe1ed: Status 404 returned error can't find the container with id 85ba190aa8361559a92e5e8bda2aef6418bfbf79ea38127334976351f10fe1ed
Sep 11 17:58:18.483100 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:58:18.483062    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-rfgh8" event={"ID":"1813eee6-0985-4f90-99ca-672d76738bcc","Type":"ContainerStarted","Data":"216c55212fd203f050f3c2a5f39ed02eceee270c127c49e8c9ae4382061f036e"}
Sep 11 17:58:18.483501 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:58:18.483137    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-rfgh8"
Sep 11 17:58:18.483501 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:58:18.483156    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-rfgh8" event={"ID":"1813eee6-0985-4f90-99ca-672d76738bcc","Type":"ContainerStarted","Data":"85ba190aa8361559a92e5e8bda2aef6418bfbf79ea38127334976351f10fe1ed"}
Sep 11 17:58:18.506977 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:58:18.506926    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-rfgh8" podStartSLOduration=2.506910533 podStartE2EDuration="2.506910533s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:58:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:58:18.504224598 +0000 UTC m=+2393.025516119" watchObservedRunningTime="2026-09-11 17:58:18.506910533 +0000 UTC m=+2393.028202056"
Sep 11 17:58:26.184481 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:58:26.184453    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-hg7p8_eb5f3be1-2e36-40ba-b3a0-3bec413056e8/console-operator/1.log"
Sep 11 17:58:26.188218 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:58:26.188198    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6k9tr_0620c0a0-8070-4307-86d9-c69a76ea14a9/ovn-acl-logging/0.log"
Sep 11 17:58:26.206291 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:58:26.206267    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-hg7p8_eb5f3be1-2e36-40ba-b3a0-3bec413056e8/console-operator/1.log"
Sep 11 17:58:26.209941 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:58:26.209921    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6k9tr_0620c0a0-8070-4307-86d9-c69a76ea14a9/ovn-acl-logging/0.log"
Sep 11 17:58:29.490707 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:58:29.490672    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-rfgh8"
Sep 11 17:58:55.196691 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:58:55.196653    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:59:11.630098 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:59:11.630057    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:59:27.498909 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:59:27.498868    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:59:37.172694 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:59:37.172653    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:59:46.392556 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:59:46.392524    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 17:59:48.240241 ip-10-0-140-51 kubenswrapper[2552]: I0911 17:59:48.240202    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 18:00:00.157944 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:00:00.157905    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29819160-gwlzj"]
Sep 11 18:00:00.158430 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:00:00.158284    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ebd01e5-f712-42b8-9e89-deff14ec577a" containerName="manager"
Sep 11 18:00:00.158430 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:00:00.158296    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ebd01e5-f712-42b8-9e89-deff14ec577a" containerName="manager"
Sep 11 18:00:00.158430 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:00:00.158318    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="96210882-8a70-4795-910a-01c409c73c78" containerName="istio-proxy"
Sep 11 18:00:00.158430 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:00:00.158324    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="96210882-8a70-4795-910a-01c409c73c78" containerName="istio-proxy"
Sep 11 18:00:00.158430 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:00:00.158381    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="3ebd01e5-f712-42b8-9e89-deff14ec577a" containerName="manager"
Sep 11 18:00:00.161606 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:00:00.161588    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29819160-gwlzj"
Sep 11 18:00:00.168578 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:00:00.168221    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9n8gf\""
Sep 11 18:00:00.171296 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:00:00.171277    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29819160-gwlzj"
Sep 11 18:00:00.196022 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:00:00.195977    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29819160-gwlzj"]
Sep 11 18:00:00.443604 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:00:00.443578    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29819160-gwlzj"]
Sep 11 18:00:00.445957 ip-10-0-140-51 kubenswrapper[2552]: W0911 18:00:00.445929    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb63bd3d9_b88e_40eb_a004_ee3424728cdf.slice/crio-b88f9416a2ad35edaad373fe2c8de3939a33271b11d3ed9b2ea8b69ece0ee384 WatchSource:0}: Error finding container b88f9416a2ad35edaad373fe2c8de3939a33271b11d3ed9b2ea8b69ece0ee384: Status 404 returned error can't find the container with id b88f9416a2ad35edaad373fe2c8de3939a33271b11d3ed9b2ea8b69ece0ee384
Sep 11 18:00:01.076237 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:00:01.076145    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29819160-gwlzj" event={"ID":"b63bd3d9-b88e-40eb-a004-ee3424728cdf","Type":"ContainerStarted","Data":"dfb404d3f3bfad3ef7bf547c3a49e8980d7a29e257e16b1ecb97f087a8a19df0"}
Sep 11 18:00:01.076237 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:00:01.076193    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29819160-gwlzj" event={"ID":"b63bd3d9-b88e-40eb-a004-ee3424728cdf","Type":"ContainerStarted","Data":"b88f9416a2ad35edaad373fe2c8de3939a33271b11d3ed9b2ea8b69ece0ee384"}
Sep 11 18:00:01.099629 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:00:01.099576    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29819160-gwlzj" podStartSLOduration=1.099561676 podStartE2EDuration="1.099561676s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:00:01.097709989 +0000 UTC m=+2495.619001509" watchObservedRunningTime="2026-09-11 18:00:01.099561676 +0000 UTC m=+2495.620853197"
Sep 11 18:00:02.083296 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:00:02.083263    2552 generic.go:356] "Generic (PLEG): container finished" podID="b63bd3d9-b88e-40eb-a004-ee3424728cdf" containerID="dfb404d3f3bfad3ef7bf547c3a49e8980d7a29e257e16b1ecb97f087a8a19df0" exitCode=0
Sep 11 18:00:02.083698 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:00:02.083315    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29819160-gwlzj" event={"ID":"b63bd3d9-b88e-40eb-a004-ee3424728cdf","Type":"ContainerDied","Data":"dfb404d3f3bfad3ef7bf547c3a49e8980d7a29e257e16b1ecb97f087a8a19df0"}
Sep 11 18:00:03.226579 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:00:03.226551    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29819160-gwlzj"
Sep 11 18:00:04.097010 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:00:04.096983    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29819160-gwlzj"
Sep 11 18:00:04.097184 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:00:04.096981    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29819160-gwlzj" event={"ID":"b63bd3d9-b88e-40eb-a004-ee3424728cdf","Type":"ContainerDied","Data":"b88f9416a2ad35edaad373fe2c8de3939a33271b11d3ed9b2ea8b69ece0ee384"}
Sep 11 18:00:04.097184 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:00:04.097097    2552 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b88f9416a2ad35edaad373fe2c8de3939a33271b11d3ed9b2ea8b69ece0ee384"
Sep 11 18:00:11.330121 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:00:11.330074    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-5ddbcf49fc-j7zbq"]
Sep 11 18:00:11.330500 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:00:11.330440    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b63bd3d9-b88e-40eb-a004-ee3424728cdf" containerName="cleanup"
Sep 11 18:00:11.330500 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:00:11.330451    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="b63bd3d9-b88e-40eb-a004-ee3424728cdf" containerName="cleanup"
Sep 11 18:00:11.330588 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:00:11.330529    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="b63bd3d9-b88e-40eb-a004-ee3424728cdf" containerName="cleanup"
Sep 11 18:00:11.333584 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:00:11.333560    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5ddbcf49fc-j7zbq"
Sep 11 18:00:11.337412 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:00:11.337374    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 18:00:11.337839 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:00:11.337819    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Sep 11 18:00:11.337941 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:00:11.337836    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 11 18:00:11.338272 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:00:11.338254    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-zp7dq\""
Sep 11 18:00:11.338384 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:00:11.338251    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 11 18:00:11.348889 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:00:11.348868    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5ddbcf49fc-j7zbq"]
Sep 11 18:00:11.358798 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:00:11.358769    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-7dd7c69b97-lxf98"]
Sep 11 18:00:11.362659 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:00:11.362638    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7dd7c69b97-lxf98"
Sep 11 18:00:11.374951 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:00:11.374924    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7dd7c69b97-lxf98"]
Sep 11 18:00:11.445152 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:00:11.445093    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fd4bf6d7-5e3d-4ca4-936e-52dfcf9f65ef-maas-api-tls\") pod \"maas-api-5ddbcf49fc-j7zbq\" (UID: \"fd4bf6d7-5e3d-4ca4-936e-52dfcf9f65ef\") " pod="odh-ai-gateway-infra/maas-api-5ddbcf49fc-j7zbq"
Sep 11 18:00:11.445347 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:00:11.445245    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qp9hg\" (UniqueName: \"kubernetes.io/projected/fd4bf6d7-5e3d-4ca4-936e-52dfcf9f65ef-kube-api-access-qp9hg\") pod \"maas-api-5ddbcf49fc-j7zbq\" (UID: \"fd4bf6d7-5e3d-4ca4-936e-52dfcf9f65ef\") " pod="odh-ai-gateway-infra/maas-api-5ddbcf49fc-j7zbq"
Sep 11 18:00:11.445347 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:00:11.445296    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fd4bf6d7-5e3d-4ca4-936e-52dfcf9f65ef-openshift-service-ca\") pod \"maas-api-5ddbcf49fc-j7zbq\" (UID: \"fd4bf6d7-5e3d-4ca4-936e-52dfcf9f65ef\") " pod="odh-ai-gateway-infra/maas-api-5ddbcf49fc-j7zbq"
Sep 11 18:00:11.445347 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:00:11.445340    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fd4bf6d7-5e3d-4ca4-936e-52dfcf9f65ef-metrics-certs\") pod \"maas-api-5ddbcf49fc-j7zbq\" (UID: \"fd4bf6d7-5e3d-4ca4-936e-52dfcf9f65ef\") " pod="odh-ai-gateway-infra/maas-api-5ddbcf49fc-j7zbq"
Sep 11 18:00:11.546538 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:00:11.546502    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fd4bf6d7-5e3d-4ca4-936e-52dfcf9f65ef-openshift-service-ca\") pod \"maas-api-5ddbcf49fc-j7zbq\" (UID: \"fd4bf6d7-5e3d-4ca4-936e-52dfcf9f65ef\") " pod="odh-ai-gateway-infra/maas-api-5ddbcf49fc-j7zbq"
Sep 11 18:00:11.546538 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:00:11.546544    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3113bc48-6bcf-425a-a295-7e5109562e54-openshift-service-ca\") pod \"maas-api-7dd7c69b97-lxf98\" (UID: \"3113bc48-6bcf-425a-a295-7e5109562e54\") " pod="odh-ai-gateway-infra/maas-api-7dd7c69b97-lxf98"
Sep 11 18:00:11.546788 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:00:11.546569    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3113bc48-6bcf-425a-a295-7e5109562e54-maas-api-tls\") pod \"maas-api-7dd7c69b97-lxf98\" (UID: \"3113bc48-6bcf-425a-a295-7e5109562e54\") " pod="odh-ai-gateway-infra/maas-api-7dd7c69b97-lxf98"
Sep 11 18:00:11.546788 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:00:11.546586    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fd4bf6d7-5e3d-4ca4-936e-52dfcf9f65ef-metrics-certs\") pod \"maas-api-5ddbcf49fc-j7zbq\" (UID: \"fd4bf6d7-5e3d-4ca4-936e-52dfcf9f65ef\") " pod="odh-ai-gateway-infra/maas-api-5ddbcf49fc-j7zbq"
Sep 11 18:00:11.546788 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:00:11.546669    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3113bc48-6bcf-425a-a295-7e5109562e54-metrics-certs\") pod \"maas-api-7dd7c69b97-lxf98\" (UID: \"3113bc48-6bcf-425a-a295-7e5109562e54\") " pod="odh-ai-gateway-infra/maas-api-7dd7c69b97-lxf98"
Sep 11 18:00:11.546788 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:00:11.546734    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fd4bf6d7-5e3d-4ca4-936e-52dfcf9f65ef-maas-api-tls\") pod \"maas-api-5ddbcf49fc-j7zbq\" (UID: \"fd4bf6d7-5e3d-4ca4-936e-52dfcf9f65ef\") " pod="odh-ai-gateway-infra/maas-api-5ddbcf49fc-j7zbq"
Sep 11 18:00:11.546788 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:00:11.546783    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-twkww\" (UniqueName: \"kubernetes.io/projected/3113bc48-6bcf-425a-a295-7e5109562e54-kube-api-access-twkww\") pod \"maas-api-7dd7c69b97-lxf98\" (UID: \"3113bc48-6bcf-425a-a295-7e5109562e54\") " pod="odh-ai-gateway-infra/maas-api-7dd7c69b97-lxf98"
Sep 11 18:00:11.546993 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:00:11.546888    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qp9hg\" (UniqueName: \"kubernetes.io/projected/fd4bf6d7-5e3d-4ca4-936e-52dfcf9f65ef-kube-api-access-qp9hg\") pod \"maas-api-5ddbcf49fc-j7zbq\" (UID: \"fd4bf6d7-5e3d-4ca4-936e-52dfcf9f65ef\") " pod="odh-ai-gateway-infra/maas-api-5ddbcf49fc-j7zbq"
Sep 11 18:00:11.547369 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:00:11.547348    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fd4bf6d7-5e3d-4ca4-936e-52dfcf9f65ef-openshift-service-ca\") pod \"maas-api-5ddbcf49fc-j7zbq\" (UID: \"fd4bf6d7-5e3d-4ca4-936e-52dfcf9f65ef\") " pod="odh-ai-gateway-infra/maas-api-5ddbcf49fc-j7zbq"
Sep 11 18:00:11.549059 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:00:11.549033    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fd4bf6d7-5e3d-4ca4-936e-52dfcf9f65ef-metrics-certs\") pod \"maas-api-5ddbcf49fc-j7zbq\" (UID: \"fd4bf6d7-5e3d-4ca4-936e-52dfcf9f65ef\") " pod="odh-ai-gateway-infra/maas-api-5ddbcf49fc-j7zbq"
Sep 11 18:00:11.549240 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:00:11.549219    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fd4bf6d7-5e3d-4ca4-936e-52dfcf9f65ef-maas-api-tls\") pod \"maas-api-5ddbcf49fc-j7zbq\" (UID: \"fd4bf6d7-5e3d-4ca4-936e-52dfcf9f65ef\") " pod="odh-ai-gateway-infra/maas-api-5ddbcf49fc-j7zbq"
Sep 11 18:00:11.556638 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:00:11.556615    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qp9hg\" (UniqueName: \"kubernetes.io/projected/fd4bf6d7-5e3d-4ca4-936e-52dfcf9f65ef-kube-api-access-qp9hg\") pod \"maas-api-5ddbcf49fc-j7zbq\" (UID: \"fd4bf6d7-5e3d-4ca4-936e-52dfcf9f65ef\") " pod="odh-ai-gateway-infra/maas-api-5ddbcf49fc-j7zbq"
Sep 11 18:00:11.644576 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:00:11.644542    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5ddbcf49fc-j7zbq"
Sep 11 18:00:11.647909 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:00:11.647882    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3113bc48-6bcf-425a-a295-7e5109562e54-openshift-service-ca\") pod \"maas-api-7dd7c69b97-lxf98\" (UID: \"3113bc48-6bcf-425a-a295-7e5109562e54\") " pod="odh-ai-gateway-infra/maas-api-7dd7c69b97-lxf98"
Sep 11 18:00:11.648061 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:00:11.647928    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3113bc48-6bcf-425a-a295-7e5109562e54-maas-api-tls\") pod \"maas-api-7dd7c69b97-lxf98\" (UID: \"3113bc48-6bcf-425a-a295-7e5109562e54\") " pod="odh-ai-gateway-infra/maas-api-7dd7c69b97-lxf98"
Sep 11 18:00:11.648061 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:00:11.647970    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3113bc48-6bcf-425a-a295-7e5109562e54-metrics-certs\") pod \"maas-api-7dd7c69b97-lxf98\" (UID: \"3113bc48-6bcf-425a-a295-7e5109562e54\") " pod="odh-ai-gateway-infra/maas-api-7dd7c69b97-lxf98"
Sep 11 18:00:11.648061 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:00:11.648005    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-twkww\" (UniqueName: \"kubernetes.io/projected/3113bc48-6bcf-425a-a295-7e5109562e54-kube-api-access-twkww\") pod \"maas-api-7dd7c69b97-lxf98\" (UID: \"3113bc48-6bcf-425a-a295-7e5109562e54\") " pod="odh-ai-gateway-infra/maas-api-7dd7c69b97-lxf98"
Sep 11 18:00:11.648676 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:00:11.648617    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3113bc48-6bcf-425a-a295-7e5109562e54-openshift-service-ca\") pod \"maas-api-7dd7c69b97-lxf98\" (UID: \"3113bc48-6bcf-425a-a295-7e5109562e54\") " pod="odh-ai-gateway-infra/maas-api-7dd7c69b97-lxf98"
Sep 11 18:00:11.650521 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:00:11.650494    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3113bc48-6bcf-425a-a295-7e5109562e54-maas-api-tls\") pod \"maas-api-7dd7c69b97-lxf98\" (UID: \"3113bc48-6bcf-425a-a295-7e5109562e54\") " pod="odh-ai-gateway-infra/maas-api-7dd7c69b97-lxf98"
Sep 11 18:00:11.650652 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:00:11.650633    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3113bc48-6bcf-425a-a295-7e5109562e54-metrics-certs\") pod \"maas-api-7dd7c69b97-lxf98\" (UID: \"3113bc48-6bcf-425a-a295-7e5109562e54\") " pod="odh-ai-gateway-infra/maas-api-7dd7c69b97-lxf98"
Sep 11 18:00:11.658935 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:00:11.658912    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-twkww\" (UniqueName: \"kubernetes.io/projected/3113bc48-6bcf-425a-a295-7e5109562e54-kube-api-access-twkww\") pod \"maas-api-7dd7c69b97-lxf98\" (UID: \"3113bc48-6bcf-425a-a295-7e5109562e54\") " pod="odh-ai-gateway-infra/maas-api-7dd7c69b97-lxf98"
Sep 11 18:00:11.673715 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:00:11.673687    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7dd7c69b97-lxf98"
Sep 11 18:00:11.925466 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:00:11.925442    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5ddbcf49fc-j7zbq"]
Sep 11 18:00:11.927337 ip-10-0-140-51 kubenswrapper[2552]: W0911 18:00:11.927302    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd4bf6d7_5e3d_4ca4_936e_52dfcf9f65ef.slice/crio-ce6697aff1d688334a2ba85bd85de83c1ece414a1c8f0096c3fa025bf495fca9 WatchSource:0}: Error finding container ce6697aff1d688334a2ba85bd85de83c1ece414a1c8f0096c3fa025bf495fca9: Status 404 returned error can't find the container with id ce6697aff1d688334a2ba85bd85de83c1ece414a1c8f0096c3fa025bf495fca9
Sep 11 18:00:11.928684 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:00:11.928666    2552 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 18:00:11.960845 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:00:11.960818    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7dd7c69b97-lxf98"]
Sep 11 18:00:11.960977 ip-10-0-140-51 kubenswrapper[2552]: W0911 18:00:11.960852    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3113bc48_6bcf_425a_a295_7e5109562e54.slice/crio-24f280a97fbf569922a38d76eb36be33bdc9e99238049d60b3a507ede8406b65 WatchSource:0}: Error finding container 24f280a97fbf569922a38d76eb36be33bdc9e99238049d60b3a507ede8406b65: Status 404 returned error can't find the container with id 24f280a97fbf569922a38d76eb36be33bdc9e99238049d60b3a507ede8406b65
Sep 11 18:00:12.142368 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:00:12.142332    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5ddbcf49fc-j7zbq" event={"ID":"fd4bf6d7-5e3d-4ca4-936e-52dfcf9f65ef","Type":"ContainerStarted","Data":"ce6697aff1d688334a2ba85bd85de83c1ece414a1c8f0096c3fa025bf495fca9"}
Sep 11 18:00:12.143538 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:00:12.143510    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7dd7c69b97-lxf98" event={"ID":"3113bc48-6bcf-425a-a295-7e5109562e54","Type":"ContainerStarted","Data":"24f280a97fbf569922a38d76eb36be33bdc9e99238049d60b3a507ede8406b65"}
Sep 11 18:00:13.151684 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:00:13.151648    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5ddbcf49fc-j7zbq" event={"ID":"fd4bf6d7-5e3d-4ca4-936e-52dfcf9f65ef","Type":"ContainerStarted","Data":"a46b3a0c76eda1df6a34002ef4e4cf4adb4c6317caefce8cf0890379664034e0"}
Sep 11 18:00:13.152202 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:00:13.151825    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-5ddbcf49fc-j7zbq"
Sep 11 18:00:13.153377 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:00:13.153351    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7dd7c69b97-lxf98" event={"ID":"3113bc48-6bcf-425a-a295-7e5109562e54","Type":"ContainerStarted","Data":"c50859ee4bee7d0ace78d1c9d7fe5d5eabefdb03dfcb131dcec1743129f23787"}
Sep 11 18:00:13.153505 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:00:13.153390    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-7dd7c69b97-lxf98"
Sep 11 18:00:13.171270 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:00:13.171219    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-5ddbcf49fc-j7zbq" podStartSLOduration=1.7243910059999998 podStartE2EDuration="2.171198183s" totalImagesPullingTime="446.807177ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:00:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:00:13.169090029 +0000 UTC m=+2507.690381552" watchObservedRunningTime="2026-09-11 18:00:13.171198183 +0000 UTC m=+2507.692489705"
Sep 11 18:00:13.189048 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:00:13.189001    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-7dd7c69b97-lxf98" podStartSLOduration=1.172973074 podStartE2EDuration="2.188985117s" totalImagesPullingTime="1.016012043s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:00:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:00:13.18693014 +0000 UTC m=+2507.708221661" watchObservedRunningTime="2026-09-11 18:00:13.188985117 +0000 UTC m=+2507.710276638"
Sep 11 18:00:19.165476 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:00:19.165381    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-7dd7c69b97-lxf98"
Sep 11 18:00:19.165924 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:00:19.165763    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-5ddbcf49fc-j7zbq"
Sep 11 18:00:19.241920 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:00:19.241882    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-5ddbcf49fc-j7zbq"]
Sep 11 18:00:19.242238 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:00:19.242197    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-5ddbcf49fc-j7zbq" podUID="fd4bf6d7-5e3d-4ca4-936e-52dfcf9f65ef" containerName="maas-api" containerID="cri-o://a46b3a0c76eda1df6a34002ef4e4cf4adb4c6317caefce8cf0890379664034e0" gracePeriod=30
Sep 11 18:00:19.496658 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:00:19.496632    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5ddbcf49fc-j7zbq"
Sep 11 18:00:19.520438 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:00:19.520410    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fd4bf6d7-5e3d-4ca4-936e-52dfcf9f65ef-metrics-certs\") pod \"fd4bf6d7-5e3d-4ca4-936e-52dfcf9f65ef\" (UID: \"fd4bf6d7-5e3d-4ca4-936e-52dfcf9f65ef\") "
Sep 11 18:00:19.520614 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:00:19.520452    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fd4bf6d7-5e3d-4ca4-936e-52dfcf9f65ef-openshift-service-ca\") pod \"fd4bf6d7-5e3d-4ca4-936e-52dfcf9f65ef\" (UID: \"fd4bf6d7-5e3d-4ca4-936e-52dfcf9f65ef\") "
Sep 11 18:00:19.520614 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:00:19.520492    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qp9hg\" (UniqueName: \"kubernetes.io/projected/fd4bf6d7-5e3d-4ca4-936e-52dfcf9f65ef-kube-api-access-qp9hg\") pod \"fd4bf6d7-5e3d-4ca4-936e-52dfcf9f65ef\" (UID: \"fd4bf6d7-5e3d-4ca4-936e-52dfcf9f65ef\") "
Sep 11 18:00:19.520614 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:00:19.520580    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fd4bf6d7-5e3d-4ca4-936e-52dfcf9f65ef-maas-api-tls\") pod \"fd4bf6d7-5e3d-4ca4-936e-52dfcf9f65ef\" (UID: \"fd4bf6d7-5e3d-4ca4-936e-52dfcf9f65ef\") "
Sep 11 18:00:19.520915 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:00:19.520886    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fd4bf6d7-5e3d-4ca4-936e-52dfcf9f65ef-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "fd4bf6d7-5e3d-4ca4-936e-52dfcf9f65ef" (UID: "fd4bf6d7-5e3d-4ca4-936e-52dfcf9f65ef"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 18:00:19.522719 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:00:19.522681    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fd4bf6d7-5e3d-4ca4-936e-52dfcf9f65ef-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "fd4bf6d7-5e3d-4ca4-936e-52dfcf9f65ef" (UID: "fd4bf6d7-5e3d-4ca4-936e-52dfcf9f65ef"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 18:00:19.522846 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:00:19.522806    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd4bf6d7-5e3d-4ca4-936e-52dfcf9f65ef-kube-api-access-qp9hg" (OuterVolumeSpecName: "kube-api-access-qp9hg") pod "fd4bf6d7-5e3d-4ca4-936e-52dfcf9f65ef" (UID: "fd4bf6d7-5e3d-4ca4-936e-52dfcf9f65ef"). InnerVolumeSpecName "kube-api-access-qp9hg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:00:19.523091 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:00:19.523068    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fd4bf6d7-5e3d-4ca4-936e-52dfcf9f65ef-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "fd4bf6d7-5e3d-4ca4-936e-52dfcf9f65ef" (UID: "fd4bf6d7-5e3d-4ca4-936e-52dfcf9f65ef"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 18:00:19.621540 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:00:19.621497    2552 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fd4bf6d7-5e3d-4ca4-936e-52dfcf9f65ef-maas-api-tls\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 18:00:19.621540 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:00:19.621532    2552 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fd4bf6d7-5e3d-4ca4-936e-52dfcf9f65ef-metrics-certs\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 18:00:19.621760 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:00:19.621551    2552 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fd4bf6d7-5e3d-4ca4-936e-52dfcf9f65ef-openshift-service-ca\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 18:00:19.621760 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:00:19.621561    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qp9hg\" (UniqueName: \"kubernetes.io/projected/fd4bf6d7-5e3d-4ca4-936e-52dfcf9f65ef-kube-api-access-qp9hg\") on node \"ip-10-0-140-51.ec2.internal\" DevicePath \"\""
Sep 11 18:00:20.198086 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:00:20.198046    2552 generic.go:356] "Generic (PLEG): container finished" podID="fd4bf6d7-5e3d-4ca4-936e-52dfcf9f65ef" containerID="a46b3a0c76eda1df6a34002ef4e4cf4adb4c6317caefce8cf0890379664034e0" exitCode=0
Sep 11 18:00:20.198611 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:00:20.198098    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5ddbcf49fc-j7zbq"
Sep 11 18:00:20.198611 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:00:20.198145    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5ddbcf49fc-j7zbq" event={"ID":"fd4bf6d7-5e3d-4ca4-936e-52dfcf9f65ef","Type":"ContainerDied","Data":"a46b3a0c76eda1df6a34002ef4e4cf4adb4c6317caefce8cf0890379664034e0"}
Sep 11 18:00:20.198611 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:00:20.198215    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5ddbcf49fc-j7zbq" event={"ID":"fd4bf6d7-5e3d-4ca4-936e-52dfcf9f65ef","Type":"ContainerDied","Data":"ce6697aff1d688334a2ba85bd85de83c1ece414a1c8f0096c3fa025bf495fca9"}
Sep 11 18:00:20.198611 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:00:20.198249    2552 scope.go:117] "RemoveContainer" containerID="a46b3a0c76eda1df6a34002ef4e4cf4adb4c6317caefce8cf0890379664034e0"
Sep 11 18:00:20.207706 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:00:20.207685    2552 scope.go:117] "RemoveContainer" containerID="a46b3a0c76eda1df6a34002ef4e4cf4adb4c6317caefce8cf0890379664034e0"
Sep 11 18:00:20.208022 ip-10-0-140-51 kubenswrapper[2552]: E0911 18:00:20.207999    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a46b3a0c76eda1df6a34002ef4e4cf4adb4c6317caefce8cf0890379664034e0\": container with ID starting with a46b3a0c76eda1df6a34002ef4e4cf4adb4c6317caefce8cf0890379664034e0 not found: ID does not exist" containerID="a46b3a0c76eda1df6a34002ef4e4cf4adb4c6317caefce8cf0890379664034e0"
Sep 11 18:00:20.208151 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:00:20.208034    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a46b3a0c76eda1df6a34002ef4e4cf4adb4c6317caefce8cf0890379664034e0"} err="failed to get container status \"a46b3a0c76eda1df6a34002ef4e4cf4adb4c6317caefce8cf0890379664034e0\": rpc error: code = NotFound desc = could not find container \"a46b3a0c76eda1df6a34002ef4e4cf4adb4c6317caefce8cf0890379664034e0\": container with ID starting with a46b3a0c76eda1df6a34002ef4e4cf4adb4c6317caefce8cf0890379664034e0 not found: ID does not exist"
Sep 11 18:00:20.218442 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:00:20.218381    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-5ddbcf49fc-j7zbq"]
Sep 11 18:00:20.222868 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:00:20.222846    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-5ddbcf49fc-j7zbq"]
Sep 11 18:00:21.949786 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:00:21.949746    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fd4bf6d7-5e3d-4ca4-936e-52dfcf9f65ef" path="/var/lib/kubelet/pods/fd4bf6d7-5e3d-4ca4-936e-52dfcf9f65ef/volumes"
Sep 11 18:00:42.383808 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:00:42.383774    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-5cdcddb749-szqt9_48df3a88-833b-480f-9e21-def162c86d75/authorino/0.log"
Sep 11 18:00:48.352276 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:00:48.352236    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-glcc5_6e0966ed-9453-40cc-9237-5fede02f1d73/manager/0.log"
Sep 11 18:00:48.643452 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:00:48.643362    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-7c92n_d550ef84-b465-4c3d-81fb-a0a457d635cb/server/0.log"
Sep 11 18:00:48.767533 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:00:48.767492    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-vq8q8_a25b4854-2484-4787-8656-9cba06d6d7c4/manager/0.log"
Sep 11 18:00:48.898344 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:00:48.898266    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-b887fdd9f-5v6pg_b7b66d08-cda4-4fae-85a2-742fb95922c0/copy-manifests/0.log"
Sep 11 18:00:48.905147 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:00:48.905105    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-b887fdd9f-5v6pg_b7b66d08-cda4-4fae-85a2-742fb95922c0/manager/0.log"
Sep 11 18:00:49.573729 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:00:49.573700    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-7dd7c69b97-lxf98_3113bc48-6bcf-425a-a295-7e5109562e54/maas-api/0.log"
Sep 11 18:00:49.818735 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:00:49.818707    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29819130-txfjr_bee1e8bc-89ab-492d-a578-be2125b7d47a/cleanup/0.log"
Sep 11 18:00:50.059445 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:00:50.059408    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29819160-gwlzj_b63bd3d9-b88e-40eb-a004-ee3424728cdf/cleanup/0.log"
Sep 11 18:00:51.086634 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:00:51.086596    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-5cdcddb749-szqt9_48df3a88-833b-480f-9e21-def162c86d75/authorino/0.log"
Sep 11 18:00:51.610725 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:00:51.610695    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5769f99d9f-rfgh8_1813eee6-0985-4f90-99ca-672d76738bcc/manager/0.log"
Sep 11 18:00:51.888265 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:00:51.888183    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-74495c69f8-xb8h4_69263d34-2a29-4704-b91d-f292eb09e9f0/manager/0.log"
Sep 11 18:00:52.290456 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:00:52.290366    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-dd9vm_280dfef1-c4c9-487b-bb3e-a1ff2b63d769/istio-proxy/0.log"
Sep 11 18:00:53.159738 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:00:53.159707    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6f8f7958c8-2p4xx_2a34a4aa-8cb7-42f3-9f87-da690cfd8031/router/0.log"
Sep 11 18:00:53.760828 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:00:53.760793    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-embedding-simulated-kserve-bcc9ff8d8-lv2n9_c6d2302e-3deb-44cb-a6e0-ada382faac64/main/0.log"
Sep 11 18:01:00.475250 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:01:00.475210    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-tsftd/must-gather-nzshb"]
Sep 11 18:01:00.475661 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:01:00.475643    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fd4bf6d7-5e3d-4ca4-936e-52dfcf9f65ef" containerName="maas-api"
Sep 11 18:01:00.475721 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:01:00.475664    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="fd4bf6d7-5e3d-4ca4-936e-52dfcf9f65ef" containerName="maas-api"
Sep 11 18:01:00.475758 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:01:00.475742    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="fd4bf6d7-5e3d-4ca4-936e-52dfcf9f65ef" containerName="maas-api"
Sep 11 18:01:00.478616 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:01:00.478595    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-tsftd/must-gather-nzshb"
Sep 11 18:01:00.481311 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:01:00.481288    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-tsftd\"/\"openshift-service-ca.crt\""
Sep 11 18:01:00.481432 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:01:00.481332    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-tsftd\"/\"kube-root-ca.crt\""
Sep 11 18:01:00.488412 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:01:00.488384    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-tsftd/must-gather-nzshb"]
Sep 11 18:01:00.502060 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:01:00.502034    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/1345393d-8035-4739-ab43-2af6a5f465a9-must-gather-output\") pod \"must-gather-nzshb\" (UID: \"1345393d-8035-4739-ab43-2af6a5f465a9\") " pod="openshift-must-gather-tsftd/must-gather-nzshb"
Sep 11 18:01:00.502241 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:01:00.502100    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t9gfc\" (UniqueName: \"kubernetes.io/projected/1345393d-8035-4739-ab43-2af6a5f465a9-kube-api-access-t9gfc\") pod \"must-gather-nzshb\" (UID: \"1345393d-8035-4739-ab43-2af6a5f465a9\") " pod="openshift-must-gather-tsftd/must-gather-nzshb"
Sep 11 18:01:00.507301 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:01:00.507266    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 18:01:00.587089 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:01:00.587051    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 18:01:00.602702 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:01:00.602672    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t9gfc\" (UniqueName: \"kubernetes.io/projected/1345393d-8035-4739-ab43-2af6a5f465a9-kube-api-access-t9gfc\") pod \"must-gather-nzshb\" (UID: \"1345393d-8035-4739-ab43-2af6a5f465a9\") " pod="openshift-must-gather-tsftd/must-gather-nzshb"
Sep 11 18:01:00.602818 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:01:00.602751    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/1345393d-8035-4739-ab43-2af6a5f465a9-must-gather-output\") pod \"must-gather-nzshb\" (UID: \"1345393d-8035-4739-ab43-2af6a5f465a9\") " pod="openshift-must-gather-tsftd/must-gather-nzshb"
Sep 11 18:01:00.603156 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:01:00.603101    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/1345393d-8035-4739-ab43-2af6a5f465a9-must-gather-output\") pod \"must-gather-nzshb\" (UID: \"1345393d-8035-4739-ab43-2af6a5f465a9\") " pod="openshift-must-gather-tsftd/must-gather-nzshb"
Sep 11 18:01:00.613548 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:01:00.613524    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t9gfc\" (UniqueName: \"kubernetes.io/projected/1345393d-8035-4739-ab43-2af6a5f465a9-kube-api-access-t9gfc\") pod \"must-gather-nzshb\" (UID: \"1345393d-8035-4739-ab43-2af6a5f465a9\") " pod="openshift-must-gather-tsftd/must-gather-nzshb"
Sep 11 18:01:00.788684 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:01:00.788563    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-tsftd/must-gather-nzshb"
Sep 11 18:01:01.106483 ip-10-0-140-51 kubenswrapper[2552]: W0911 18:01:01.106457    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1345393d_8035_4739_ab43_2af6a5f465a9.slice/crio-60b1db2af3b84237c2032c2e6e8a051409e9c08bf984ec2590485c2fb2a1a45a WatchSource:0}: Error finding container 60b1db2af3b84237c2032c2e6e8a051409e9c08bf984ec2590485c2fb2a1a45a: Status 404 returned error can't find the container with id 60b1db2af3b84237c2032c2e6e8a051409e9c08bf984ec2590485c2fb2a1a45a
Sep 11 18:01:01.107092 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:01:01.107073    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-tsftd/must-gather-nzshb"]
Sep 11 18:01:01.434982 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:01:01.434885    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-tsftd/must-gather-nzshb" event={"ID":"1345393d-8035-4739-ab43-2af6a5f465a9","Type":"ContainerStarted","Data":"60b1db2af3b84237c2032c2e6e8a051409e9c08bf984ec2590485c2fb2a1a45a"}
Sep 11 18:01:02.444617 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:01:02.444583    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-tsftd/must-gather-nzshb" event={"ID":"1345393d-8035-4739-ab43-2af6a5f465a9","Type":"ContainerStarted","Data":"d647774c920ce59db3490b13dec13828f8bfdc08b3cff1a4179ba363144f1f8e"}
Sep 11 18:01:02.445140 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:01:02.444636    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-tsftd/must-gather-nzshb" event={"ID":"1345393d-8035-4739-ab43-2af6a5f465a9","Type":"ContainerStarted","Data":"c85a0c2f08850e0a78bde395ad0598c39f41b0803542eb9cff92bf0e076881b6"}
Sep 11 18:01:02.459250 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:01:02.459185    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-tsftd/must-gather-nzshb" podStartSLOduration=1.504930589 podStartE2EDuration="2.459163813s" totalImagesPullingTime="954.233224ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:01:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:01:02.458254538 +0000 UTC m=+2556.979546071" watchObservedRunningTime="2026-09-11 18:01:02.459163813 +0000 UTC m=+2556.980455335"
Sep 11 18:01:03.740319 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:01:03.740282    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-lx6ht_72eae49b-cb97-4819-aaf5-5c6eb23ce9c8/global-pull-secret-syncer/0.log"
Sep 11 18:01:03.801476 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:01:03.801439    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log"
Sep 11 18:01:03.933811 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:01:03.933777    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-140-51.ec2.internal_144be191d43f74dcac73bb27936b49ab/haproxy/0.log"
Sep 11 18:01:09.097656 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:01:09.097163    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-5cdcddb749-szqt9_48df3a88-833b-480f-9e21-def162c86d75/authorino/0.log"
Sep 11 18:01:09.256564 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:01:09.256518    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5769f99d9f-rfgh8_1813eee6-0985-4f90-99ca-672d76738bcc/manager/0.log"
Sep 11 18:01:09.369978 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:01:09.369868    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-74495c69f8-xb8h4_69263d34-2a29-4704-b91d-f292eb09e9f0/manager/0.log"
Sep 11 18:01:10.936478 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:01:10.936441    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-6cd954b897-vtd7t_24021eec-97e8-4bd2-91b0-c31be7bd9c88/cluster-monitoring-operator/0.log"
Sep 11 18:01:11.277647 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:01:11.277569    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-78gb7_781705c1-5c64-4a55-b898-1943a2c11a1a/node-exporter/0.log"
Sep 11 18:01:11.300246 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:01:11.300218    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-78gb7_781705c1-5c64-4a55-b898-1943a2c11a1a/kube-rbac-proxy/0.log"
Sep 11 18:01:11.326628 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:01:11.326599    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-78gb7_781705c1-5c64-4a55-b898-1943a2c11a1a/init-textfile/0.log"
Sep 11 18:01:11.774021 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:01:11.773988    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-5864cd99d7-7bhdv_3277d6b0-b7ff-4851-917c-a46c45c3a25d/prometheus-operator-admission-webhook/0.log"
Sep 11 18:01:13.373469 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:01:13.373435    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-5dcbc7df6b-7kvsj_3e8c629b-c41f-4eef-9c3e-7eed516e00a7/networking-console-plugin/0.log"
Sep 11 18:01:13.993754 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:01:13.993717    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-tsftd/perf-node-gather-daemonset-bp5gv"]
Sep 11 18:01:14.001905 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:01:14.001877    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-tsftd/perf-node-gather-daemonset-bp5gv"
Sep 11 18:01:14.003367 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:01:14.003341    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-tsftd/perf-node-gather-daemonset-bp5gv"]
Sep 11 18:01:14.004050 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:01:14.004028    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-tsftd\"/\"default-dockercfg-cw6vd\""
Sep 11 18:01:14.039141 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:01:14.039091    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e36ffd75-a8b6-4569-8d49-f85e4bb481a6-proc\") pod \"perf-node-gather-daemonset-bp5gv\" (UID: \"e36ffd75-a8b6-4569-8d49-f85e4bb481a6\") " pod="openshift-must-gather-tsftd/perf-node-gather-daemonset-bp5gv"
Sep 11 18:01:14.039544 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:01:14.039495    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ggrpd\" (UniqueName: \"kubernetes.io/projected/e36ffd75-a8b6-4569-8d49-f85e4bb481a6-kube-api-access-ggrpd\") pod \"perf-node-gather-daemonset-bp5gv\" (UID: \"e36ffd75-a8b6-4569-8d49-f85e4bb481a6\") " pod="openshift-must-gather-tsftd/perf-node-gather-daemonset-bp5gv"
Sep 11 18:01:14.039699 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:01:14.039677    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e36ffd75-a8b6-4569-8d49-f85e4bb481a6-sys\") pod \"perf-node-gather-daemonset-bp5gv\" (UID: \"e36ffd75-a8b6-4569-8d49-f85e4bb481a6\") " pod="openshift-must-gather-tsftd/perf-node-gather-daemonset-bp5gv"
Sep 11 18:01:14.039773 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:01:14.039736    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e36ffd75-a8b6-4569-8d49-f85e4bb481a6-lib-modules\") pod \"perf-node-gather-daemonset-bp5gv\" (UID: \"e36ffd75-a8b6-4569-8d49-f85e4bb481a6\") " pod="openshift-must-gather-tsftd/perf-node-gather-daemonset-bp5gv"
Sep 11 18:01:14.039773 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:01:14.039764    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e36ffd75-a8b6-4569-8d49-f85e4bb481a6-podres\") pod \"perf-node-gather-daemonset-bp5gv\" (UID: \"e36ffd75-a8b6-4569-8d49-f85e4bb481a6\") " pod="openshift-must-gather-tsftd/perf-node-gather-daemonset-bp5gv"
Sep 11 18:01:14.067104 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:01:14.067062    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-hg7p8_eb5f3be1-2e36-40ba-b3a0-3bec413056e8/console-operator/1.log"
Sep 11 18:01:14.073486 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:01:14.073453    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-hg7p8_eb5f3be1-2e36-40ba-b3a0-3bec413056e8/console-operator/2.log"
Sep 11 18:01:14.140365 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:01:14.140332    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e36ffd75-a8b6-4569-8d49-f85e4bb481a6-proc\") pod \"perf-node-gather-daemonset-bp5gv\" (UID: \"e36ffd75-a8b6-4569-8d49-f85e4bb481a6\") " pod="openshift-must-gather-tsftd/perf-node-gather-daemonset-bp5gv"
Sep 11 18:01:14.140679 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:01:14.140657    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ggrpd\" (UniqueName: \"kubernetes.io/projected/e36ffd75-a8b6-4569-8d49-f85e4bb481a6-kube-api-access-ggrpd\") pod \"perf-node-gather-daemonset-bp5gv\" (UID: \"e36ffd75-a8b6-4569-8d49-f85e4bb481a6\") " pod="openshift-must-gather-tsftd/perf-node-gather-daemonset-bp5gv"
Sep 11 18:01:14.140868 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:01:14.140849    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e36ffd75-a8b6-4569-8d49-f85e4bb481a6-sys\") pod \"perf-node-gather-daemonset-bp5gv\" (UID: \"e36ffd75-a8b6-4569-8d49-f85e4bb481a6\") " pod="openshift-must-gather-tsftd/perf-node-gather-daemonset-bp5gv"
Sep 11 18:01:14.141075 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:01:14.141059    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e36ffd75-a8b6-4569-8d49-f85e4bb481a6-lib-modules\") pod \"perf-node-gather-daemonset-bp5gv\" (UID: \"e36ffd75-a8b6-4569-8d49-f85e4bb481a6\") " pod="openshift-must-gather-tsftd/perf-node-gather-daemonset-bp5gv"
Sep 11 18:01:14.141394 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:01:14.141375    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e36ffd75-a8b6-4569-8d49-f85e4bb481a6-podres\") pod \"perf-node-gather-daemonset-bp5gv\" (UID: \"e36ffd75-a8b6-4569-8d49-f85e4bb481a6\") " pod="openshift-must-gather-tsftd/perf-node-gather-daemonset-bp5gv"
Sep 11 18:01:14.141718 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:01:14.141021    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e36ffd75-a8b6-4569-8d49-f85e4bb481a6-sys\") pod \"perf-node-gather-daemonset-bp5gv\" (UID: \"e36ffd75-a8b6-4569-8d49-f85e4bb481a6\") " pod="openshift-must-gather-tsftd/perf-node-gather-daemonset-bp5gv"
Sep 11 18:01:14.141854 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:01:14.141282    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e36ffd75-a8b6-4569-8d49-f85e4bb481a6-lib-modules\") pod \"perf-node-gather-daemonset-bp5gv\" (UID: \"e36ffd75-a8b6-4569-8d49-f85e4bb481a6\") " pod="openshift-must-gather-tsftd/perf-node-gather-daemonset-bp5gv"
Sep 11 18:01:14.141951 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:01:14.140539    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e36ffd75-a8b6-4569-8d49-f85e4bb481a6-proc\") pod \"perf-node-gather-daemonset-bp5gv\" (UID: \"e36ffd75-a8b6-4569-8d49-f85e4bb481a6\") " pod="openshift-must-gather-tsftd/perf-node-gather-daemonset-bp5gv"
Sep 11 18:01:14.142042 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:01:14.141591    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e36ffd75-a8b6-4569-8d49-f85e4bb481a6-podres\") pod \"perf-node-gather-daemonset-bp5gv\" (UID: \"e36ffd75-a8b6-4569-8d49-f85e4bb481a6\") " pod="openshift-must-gather-tsftd/perf-node-gather-daemonset-bp5gv"
Sep 11 18:01:14.152105 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:01:14.152075    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ggrpd\" (UniqueName: \"kubernetes.io/projected/e36ffd75-a8b6-4569-8d49-f85e4bb481a6-kube-api-access-ggrpd\") pod \"perf-node-gather-daemonset-bp5gv\" (UID: \"e36ffd75-a8b6-4569-8d49-f85e4bb481a6\") " pod="openshift-must-gather-tsftd/perf-node-gather-daemonset-bp5gv"
Sep 11 18:01:14.315554 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:01:14.315455    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-tsftd/perf-node-gather-daemonset-bp5gv"
Sep 11 18:01:14.612571 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:01:14.612532    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-tsftd/perf-node-gather-daemonset-bp5gv"]
Sep 11 18:01:14.613162 ip-10-0-140-51 kubenswrapper[2552]: W0911 18:01:14.613133    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pode36ffd75_a8b6_4569_8d49_f85e4bb481a6.slice/crio-be41c18f95c370f09e4545879e4ef952ad371e594d149f911905b4bb530cd3fd WatchSource:0}: Error finding container be41c18f95c370f09e4545879e4ef952ad371e594d149f911905b4bb530cd3fd: Status 404 returned error can't find the container with id be41c18f95c370f09e4545879e4ef952ad371e594d149f911905b4bb530cd3fd
Sep 11 18:01:14.622545 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:01:14.622510    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-5b55b4bcf8-kmnkh_280cbc95-d762-4661-bf63-0f5606364d06/console/0.log"
Sep 11 18:01:14.666827 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:01:14.666795    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-76f99584f4-s92rh_4e82ca32-df13-4755-9e88-08a339c69abe/download-server/0.log"
Sep 11 18:01:15.402813 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:01:15.402770    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-86695b7cc7-5sknr_3d5f2687-9edd-426d-bb9d-3c7221a5aeb1/volume-data-source-validator/0.log"
Sep 11 18:01:15.562399 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:01:15.562353    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-tsftd/perf-node-gather-daemonset-bp5gv" event={"ID":"e36ffd75-a8b6-4569-8d49-f85e4bb481a6","Type":"ContainerStarted","Data":"b26c622d20be3b60ce75adb59cbe323f45969af1d2c1858f4253f4e6239ad174"}
Sep 11 18:01:15.562585 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:01:15.562444    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-tsftd/perf-node-gather-daemonset-bp5gv" event={"ID":"e36ffd75-a8b6-4569-8d49-f85e4bb481a6","Type":"ContainerStarted","Data":"be41c18f95c370f09e4545879e4ef952ad371e594d149f911905b4bb530cd3fd"}
Sep 11 18:01:15.562585 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:01:15.562473    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-tsftd/perf-node-gather-daemonset-bp5gv"
Sep 11 18:01:15.583243 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:01:15.583182    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-tsftd/perf-node-gather-daemonset-bp5gv" podStartSLOduration=2.583159783 podStartE2EDuration="2.583159783s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 18:01:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:01:15.580324673 +0000 UTC m=+2570.101616205" watchObservedRunningTime="2026-09-11 18:01:15.583159783 +0000 UTC m=+2570.104451305"
Sep 11 18:01:16.557213 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:01:16.557182    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-6h5ct_b7d42fd1-2cb9-429b-a57a-656a5d7ecb91/dns/0.log"
Sep 11 18:01:16.577043 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:01:16.577014    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-6h5ct_b7d42fd1-2cb9-429b-a57a-656a5d7ecb91/kube-rbac-proxy/0.log"
Sep 11 18:01:16.723708 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:01:16.723678    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-h6lht_8b4c1f98-046f-47c4-889f-c2bd807dddbe/dns-node-resolver/0.log"
Sep 11 18:01:17.328124 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:01:17.328087    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-gbf8j_d0ec6f78-bdc6-420a-a6aa-953d61643e27/node-ca/0.log"
Sep 11 18:01:18.285525 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:01:18.285487    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-dd9vm_280dfef1-c4c9-487b-bb3e-a1ff2b63d769/istio-proxy/0.log"
Sep 11 18:01:19.324815 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:01:19.324780    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6f8f7958c8-2p4xx_2a34a4aa-8cb7-42f3-9f87-da690cfd8031/router/0.log"
Sep 11 18:01:20.290450 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:01:20.290410    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-nzjpd_62af7c53-a9e7-40d3-ac2d-595a8dcfc54f/serve-healthcheck-canary/0.log"
Sep 11 18:01:20.786980 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:01:20.786948    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5844b856db-88bn9_7583d484-6dd5-48e3-9485-05e57c62443e/insights-operator/0.log"
Sep 11 18:01:20.791148 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:01:20.791106    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5844b856db-88bn9_7583d484-6dd5-48e3-9485-05e57c62443e/insights-operator/1.log"
Sep 11 18:01:20.813001 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:01:20.812955    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-99f6z_3f805648-71b2-435c-9942-1701f83fe7a3/kube-rbac-proxy/0.log"
Sep 11 18:01:20.833876 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:01:20.833847    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-99f6z_3f805648-71b2-435c-9942-1701f83fe7a3/exporter/0.log"
Sep 11 18:01:20.857475 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:01:20.857443    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-99f6z_3f805648-71b2-435c-9942-1701f83fe7a3/extractor/0.log"
Sep 11 18:01:21.582581 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:01:21.582552    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-tsftd/perf-node-gather-daemonset-bp5gv"
Sep 11 18:01:24.028511 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:01:24.028468    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-glcc5_6e0966ed-9453-40cc-9237-5fede02f1d73/manager/0.log"
Sep 11 18:01:24.122670 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:01:24.122632    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-7c92n_d550ef84-b465-4c3d-81fb-a0a457d635cb/server/0.log"
Sep 11 18:01:24.287651 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:01:24.287616    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-vq8q8_a25b4854-2484-4787-8656-9cba06d6d7c4/manager/0.log"
Sep 11 18:01:24.316619 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:01:24.316585    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-b887fdd9f-5v6pg_b7b66d08-cda4-4fae-85a2-742fb95922c0/manager/0.log"
Sep 11 18:01:24.338286 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:01:24.338255    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-b887fdd9f-5v6pg_b7b66d08-cda4-4fae-85a2-742fb95922c0/copy-manifests/0.log"
Sep 11 18:01:24.720599 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:01:24.720570    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-dd9vm_280dfef1-c4c9-487b-bb3e-a1ff2b63d769/istio-proxy/0.log"
Sep 11 18:01:25.696242 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:01:25.696193    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6f8f7958c8-2p4xx_2a34a4aa-8cb7-42f3-9f87-da690cfd8031/router/0.log"
Sep 11 18:01:25.727382 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:01:25.727347    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-845d467d5c-4zthg_91e20f54-4a44-4f22-b9e6-015c4be287a7/manager/0.log"
Sep 11 18:01:25.834315 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:01:25.834224    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-mfsgs_da644d9f-2b46-44dc-8757-60e68db81d23/openshift-lws-operator/0.log"
Sep 11 18:01:27.217605 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:01:27.217571    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-dd9vm_280dfef1-c4c9-487b-bb3e-a1ff2b63d769/istio-proxy/0.log"
Sep 11 18:01:27.782944 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:01:27.782909    2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8j6jf_5b082ce5-2ea9-46dc-8307-08a722392886/konnectivity-agent/0.log" limit=1024
Sep 11 18:01:28.265708 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:01:28.265666    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6f8f7958c8-2p4xx_2a34a4aa-8cb7-42f3-9f87-da690cfd8031/router/0.log"
Sep 11 18:01:31.316578 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:01:31.316547    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-8b4d697c5-f5bc2_c41df48e-7ab3-41d3-a549-ddd96b123d99/migrator/0.log"
Sep 11 18:01:31.340930 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:01:31.340887    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-8b4d697c5-f5bc2_c41df48e-7ab3-41d3-a549-ddd96b123d99/graceful-termination/0.log"
Sep 11 18:01:31.823064 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:01:31.823034    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6c8f84f897-pmvkp_676c8e42-836e-4f84-b5b0-c1a49eb8c5f3/kube-storage-version-migrator-operator/1.log"
Sep 11 18:01:31.825226 ip-10-0-140-51 kubenswrapper[2552]: I0911 18:01:31.825194    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6c8f84f897-pmvkp_676c8e42-836e-4f84-b5b0-c1a49eb8c5f3/kube-storage-version-migrator-operator/0.log"