Sep 16 18:42:32.601266 ip-10-0-139-182 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Sep 16 18:42:32.601282 ip-10-0-139-182 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Sep 16 18:42:32.601291 ip-10-0-139-182 systemd[1]: kubelet.service: Failed with result 'resources'.
Sep 16 18:42:32.601521 ip-10-0-139-182 systemd[1]: Failed to start Kubernetes Kubelet.
Sep 16 18:42:42.821774 ip-10-0-139-182 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Sep 16 18:42:42.821792 ip-10-0-139-182 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 9f1b9e67dbde41b88d6aa4f43447b3cf --
Sep 16 18:44:29.562141 ip-10-0-139-182 systemd[1]: Starting Kubernetes Kubelet...
Sep 16 18:44:29.946791 ip-10-0-139-182 kubenswrapper[2584]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 16 18:44:29.946791 ip-10-0-139-182 kubenswrapper[2584]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 16 18:44:29.946791 ip-10-0-139-182 kubenswrapper[2584]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 16 18:44:29.946791 ip-10-0-139-182 kubenswrapper[2584]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 16 18:44:29.946791 ip-10-0-139-182 kubenswrapper[2584]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 16 18:44:29.947478 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.946891    2584 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 16 18:44:29.950225 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.950209    2584 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 16 18:44:29.950225 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.950224    2584 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 16 18:44:29.950290 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.950228    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 16 18:44:29.950290 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.950231    2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 16 18:44:29.950290 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.950234    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 16 18:44:29.950290 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.950239    2584 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 16 18:44:29.950290 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.950243    2584 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 16 18:44:29.950290 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.950253    2584 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 16 18:44:29.950290 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.950256    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 16 18:44:29.950290 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.950259    2584 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 16 18:44:29.950290 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.950263    2584 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 16 18:44:29.950290 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.950266    2584 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 16 18:44:29.950290 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.950268    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 16 18:44:29.950290 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.950271    2584 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 16 18:44:29.950290 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.950274    2584 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 16 18:44:29.950290 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.950277    2584 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 16 18:44:29.950290 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.950279    2584 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 16 18:44:29.950290 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.950282    2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 16 18:44:29.950290 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.950284    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 16 18:44:29.950290 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.950287    2584 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 16 18:44:29.950290 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.950290    2584 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 16 18:44:29.950724 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.950292    2584 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 16 18:44:29.950724 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.950295    2584 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 16 18:44:29.950724 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.950298    2584 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 16 18:44:29.950724 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.950301    2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 16 18:44:29.950724 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.950304    2584 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 16 18:44:29.950724 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.950306    2584 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 16 18:44:29.950724 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.950309    2584 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 16 18:44:29.950724 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.950311    2584 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 16 18:44:29.950724 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.950315    2584 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 16 18:44:29.950724 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.950319    2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 16 18:44:29.950724 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.950322    2584 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 16 18:44:29.950724 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.950324    2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 16 18:44:29.950724 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.950327    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 16 18:44:29.950724 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.950329    2584 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 16 18:44:29.950724 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.950332    2584 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 16 18:44:29.950724 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.950334    2584 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 16 18:44:29.950724 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.950337    2584 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 16 18:44:29.950724 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.950339    2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 16 18:44:29.950724 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.950343    2584 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 16 18:44:29.951197 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.950345    2584 feature_gate.go:328] unrecognized feature gate: Example2
Sep 16 18:44:29.951197 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.950348    2584 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 16 18:44:29.951197 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.950351    2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 16 18:44:29.951197 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.950354    2584 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 16 18:44:29.951197 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.950356    2584 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 16 18:44:29.951197 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.950358    2584 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 16 18:44:29.951197 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.950361    2584 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 16 18:44:29.951197 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.950363    2584 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 16 18:44:29.951197 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.950366    2584 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 16 18:44:29.951197 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.950369    2584 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 16 18:44:29.951197 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.950372    2584 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 16 18:44:29.951197 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.950375    2584 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 16 18:44:29.951197 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.950377    2584 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 16 18:44:29.951197 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.950380    2584 feature_gate.go:328] unrecognized feature gate: Example
Sep 16 18:44:29.951197 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.950383    2584 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 16 18:44:29.951197 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.950386    2584 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 16 18:44:29.951197 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.950389    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 16 18:44:29.951197 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.950391    2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 16 18:44:29.951197 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.950394    2584 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 16 18:44:29.951197 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.950396    2584 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 16 18:44:29.951661 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.950399    2584 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 16 18:44:29.951661 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.950401    2584 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 16 18:44:29.951661 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.950404    2584 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 16 18:44:29.951661 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.950406    2584 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 16 18:44:29.951661 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.950409    2584 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 16 18:44:29.951661 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.950411    2584 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 16 18:44:29.951661 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.950414    2584 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 16 18:44:29.951661 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.950416    2584 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 16 18:44:29.951661 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.950419    2584 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 16 18:44:29.951661 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.950421    2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 16 18:44:29.951661 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.950423    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 16 18:44:29.951661 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.950426    2584 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 16 18:44:29.951661 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.950429    2584 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 16 18:44:29.951661 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.950431    2584 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 16 18:44:29.951661 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.950435    2584 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 16 18:44:29.951661 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.950437    2584 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 16 18:44:29.951661 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.950439    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 16 18:44:29.951661 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.950442    2584 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 16 18:44:29.951661 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.950444    2584 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 16 18:44:29.951661 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.950446    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 16 18:44:29.952173 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.950449    2584 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 16 18:44:29.952173 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.950451    2584 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 16 18:44:29.952173 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.950453    2584 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 16 18:44:29.952173 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.950456    2584 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 16 18:44:29.952173 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.950458    2584 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 16 18:44:29.952173 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.950460    2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 16 18:44:29.952173 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.950466    2584 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 16 18:44:29.952173 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.950468    2584 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 16 18:44:29.952173 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.950471    2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 16 18:44:29.952173 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.950474    2584 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 16 18:44:29.952173 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.950476    2584 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 16 18:44:29.952173 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.950478    2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 16 18:44:29.952173 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.950481    2584 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 16 18:44:29.952173 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.950483    2584 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 16 18:44:29.952173 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.950486    2584 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 16 18:44:29.952173 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.950488    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 16 18:44:29.952173 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.950490    2584 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 16 18:44:29.952173 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.950493    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 16 18:44:29.952173 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.950495    2584 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 16 18:44:29.952628 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951160    2584 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 16 18:44:29.952628 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951166    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 16 18:44:29.952628 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951169    2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 16 18:44:29.952628 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951172    2584 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 16 18:44:29.952628 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951174    2584 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 16 18:44:29.952628 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951177    2584 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 16 18:44:29.952628 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951180    2584 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 16 18:44:29.952628 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951183    2584 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 16 18:44:29.952628 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951186    2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 16 18:44:29.952628 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951189    2584 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 16 18:44:29.952628 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951191    2584 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 16 18:44:29.952628 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951194    2584 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 16 18:44:29.952628 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951196    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 16 18:44:29.952628 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951199    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 16 18:44:29.952628 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951202    2584 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 16 18:44:29.952628 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951204    2584 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 16 18:44:29.952628 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951207    2584 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 16 18:44:29.952628 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951210    2584 feature_gate.go:328] unrecognized feature gate: Example2
Sep 16 18:44:29.952628 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951212    2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 16 18:44:29.952628 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951216    2584 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 16 18:44:29.953146 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951219    2584 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 16 18:44:29.953146 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951222    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 16 18:44:29.953146 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951224    2584 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 16 18:44:29.953146 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951227    2584 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 16 18:44:29.953146 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951230    2584 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 16 18:44:29.953146 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951232    2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 16 18:44:29.953146 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951235    2584 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 16 18:44:29.953146 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951238    2584 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 16 18:44:29.953146 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951240    2584 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 16 18:44:29.953146 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951243    2584 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 16 18:44:29.953146 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951246    2584 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 16 18:44:29.953146 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951249    2584 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 16 18:44:29.953146 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951251    2584 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 16 18:44:29.953146 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951254    2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 16 18:44:29.953146 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951257    2584 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 16 18:44:29.953146 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951259    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 16 18:44:29.953146 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951262    2584 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 16 18:44:29.953146 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951264    2584 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 16 18:44:29.953146 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951267    2584 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 16 18:44:29.953593 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951270    2584 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 16 18:44:29.953593 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951272    2584 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 16 18:44:29.953593 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951274    2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 16 18:44:29.953593 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951277    2584 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 16 18:44:29.953593 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951279    2584 feature_gate.go:328] unrecognized feature gate: Example
Sep 16 18:44:29.953593 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951282    2584 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 16 18:44:29.953593 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951284    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 16 18:44:29.953593 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951287    2584 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 16 18:44:29.953593 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951289    2584 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 16 18:44:29.953593 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951291    2584 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 16 18:44:29.953593 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951294    2584 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 16 18:44:29.953593 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951296    2584 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 16 18:44:29.953593 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951299    2584 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 16 18:44:29.953593 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951302    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 16 18:44:29.953593 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951304    2584 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 16 18:44:29.953593 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951307    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 16 18:44:29.953593 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951309    2584 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 16 18:44:29.953593 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951311    2584 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 16 18:44:29.953593 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951314    2584 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 16 18:44:29.953593 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951317    2584 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 16 18:44:29.954079 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951319    2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 16 18:44:29.954079 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951321    2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 16 18:44:29.954079 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951324    2584 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 16 18:44:29.954079 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951328    2584 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 16 18:44:29.954079 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951331    2584 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 16 18:44:29.954079 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951334    2584 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 16 18:44:29.954079 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951336    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 16 18:44:29.954079 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951339    2584 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 16 18:44:29.954079 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951341    2584 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 16 18:44:29.954079 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951345    2584 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 16 18:44:29.954079 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951349    2584 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 16 18:44:29.954079 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951351    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 16 18:44:29.954079 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951354    2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 16 18:44:29.954079 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951359    2584 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 16 18:44:29.954079 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951362    2584 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 16 18:44:29.954079 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951365    2584 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 16 18:44:29.954079 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951368    2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 16 18:44:29.954079 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951370    2584 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 16 18:44:29.954509 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951373    2584 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 16 18:44:29.954509 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951376    2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 16 18:44:29.954509 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951378    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 16 18:44:29.954509 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951380    2584 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 16 18:44:29.954509 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951383    2584 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 16 18:44:29.954509 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951385    2584 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 16 18:44:29.954509 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951389    2584 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 16 18:44:29.954509 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951391    2584 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 16 18:44:29.954509 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951394    2584 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 16 18:44:29.954509 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951396    2584 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 16 18:44:29.954509 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951399    2584 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 16 18:44:29.954509 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951401    2584 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 16 18:44:29.954509 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951404    2584 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 16 18:44:29.954509 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951407    2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 16 18:44:29.954509 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951409    2584 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 16 18:44:29.954509 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951412    2584 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 16 18:44:29.954509 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951414    2584 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 16 18:44:29.954509 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951416    2584 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 16 18:44:29.954509 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951419    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 16 18:44:29.954509 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951421    2584 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 16 18:44:29.954995 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951424    2584 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 16 18:44:29.954995 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951426    2584 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 16 18:44:29.954995 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951786    2584 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 16 18:44:29.954995 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951790    2584 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 16 18:44:29.954995 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951793    2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 16 18:44:29.954995 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951796    2584 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 16 18:44:29.954995 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951798    2584 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 16 18:44:29.954995 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951802    2584 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 16 18:44:29.954995 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951805    2584 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 16 18:44:29.954995 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951808    2584 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 16 18:44:29.954995 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951811    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 16 18:44:29.954995 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951813    2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 16 18:44:29.954995 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951816    2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 16 18:44:29.954995 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951818    2584 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 16 18:44:29.954995 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951821    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 16 18:44:29.954995 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951823    2584 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 16 18:44:29.954995 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951842    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 16 18:44:29.954995 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951846    2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 16 18:44:29.954995 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951849    2584 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 16 18:44:29.954995 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951852    2584 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 16 18:44:29.955481 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951855    2584 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 16 18:44:29.955481 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951859    2584 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 16 18:44:29.955481 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951862    2584 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 16 18:44:29.955481 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951866    2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 16 18:44:29.955481 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951868    2584 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 16 18:44:29.955481 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951871    2584 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 16 18:44:29.955481 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951873    2584 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 16 18:44:29.955481 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951875    2584 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 16 18:44:29.955481 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951878    2584 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 16 18:44:29.955481 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951881    2584 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 16 18:44:29.955481 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951883    2584 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 16 18:44:29.955481 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951886    2584 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 16 18:44:29.955481 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951888    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 16 18:44:29.955481 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951891    2584 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 16 18:44:29.955481 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951893    2584 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 16 18:44:29.955481 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951896    2584 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 16 18:44:29.955481 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951898    2584 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 16 18:44:29.955481 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951900    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 16 18:44:29.955481 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951903    2584 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 16 18:44:29.955481 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951906    2584 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 16 18:44:29.955987 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951909    2584 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 16 18:44:29.955987 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951912    2584 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 16 18:44:29.955987 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951914    2584 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 16 18:44:29.955987 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951916    2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 16 18:44:29.955987 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951919    2584 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 16 18:44:29.955987 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951921    2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 16 18:44:29.955987 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951924    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 16 18:44:29.955987 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951926    2584 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 16 18:44:29.955987 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951929    2584 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 16 18:44:29.955987 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951931    2584 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 16 18:44:29.955987 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951934    2584 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 16 18:44:29.955987 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951937    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 16 18:44:29.955987 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951940    2584 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 16 18:44:29.955987 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951942    2584 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 16 18:44:29.955987 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951945    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 16 18:44:29.955987 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951948    2584 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 16 18:44:29.955987 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951950    2584 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 16 18:44:29.955987 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951953    2584 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 16 18:44:29.955987 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951955    2584 feature_gate.go:328] unrecognized feature gate: Example2
Sep 16 18:44:29.955987 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951957    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 16 18:44:29.956489 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951960    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 16 18:44:29.956489 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951962    2584 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 16 18:44:29.956489 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951965    2584 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 16 18:44:29.956489 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951967    2584 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 16 18:44:29.956489 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951969    2584 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 16 18:44:29.956489 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951972    2584 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 16 18:44:29.956489 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951974    2584 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 16 18:44:29.956489 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951977    2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 16 18:44:29.956489 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951979    2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 16 18:44:29.956489 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951981    2584 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 16 18:44:29.956489 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951984    2584 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 16 18:44:29.956489 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951986    2584 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 16 18:44:29.956489 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951991    2584 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 16 18:44:29.956489 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951995    2584 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 16 18:44:29.956489 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.951998    2584 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 16 18:44:29.956489 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.952001    2584 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 16 18:44:29.956489 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.952003    2584 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 16 18:44:29.956489 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.952006    2584 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 16 18:44:29.956489 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.952008    2584 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 16 18:44:29.956982 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.952011    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 16 18:44:29.956982 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.952013    2584 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 16 18:44:29.956982 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.952016    2584 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 16 18:44:29.956982 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.952019    2584 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 16 18:44:29.956982 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.952022    2584 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 16 18:44:29.956982 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.952026    2584 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 16 18:44:29.956982 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.952029    2584 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 16 18:44:29.956982 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.952032    2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 16 18:44:29.956982 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.952035    2584 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 16 18:44:29.956982 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.952037    2584 feature_gate.go:328] unrecognized feature gate: Example
Sep 16 18:44:29.956982 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.952040    2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 16 18:44:29.956982 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.952043    2584 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 16 18:44:29.956982 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.952045    2584 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 16 18:44:29.956982 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.952048    2584 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 16 18:44:29.956982 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.952050    2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 16 18:44:29.956982 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.952052    2584 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 16 18:44:29.956982 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.952055    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 16 18:44:29.956982 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.952057    2584 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 16 18:44:29.956982 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.952060    2584 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 16 18:44:29.957437 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.952062    2584 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 16 18:44:29.957437 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.952064    2584 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 16 18:44:29.957437 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.952067    2584 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 16 18:44:29.957437 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952136    2584 flags.go:64] FLAG: --address="0.0.0.0"
Sep 16 18:44:29.957437 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952176    2584 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 16 18:44:29.957437 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952183    2584 flags.go:64] FLAG: --anonymous-auth="true"
Sep 16 18:44:29.957437 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952188    2584 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 16 18:44:29.957437 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952191    2584 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 16 18:44:29.957437 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952195    2584 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 16 18:44:29.957437 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952199    2584 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 16 18:44:29.957437 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952204    2584 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 16 18:44:29.957437 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952207    2584 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 16 18:44:29.957437 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952210    2584 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 16 18:44:29.957437 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952213    2584 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 16 18:44:29.957437 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952217    2584 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 16 18:44:29.957437 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952220    2584 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 16 18:44:29.957437 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952223    2584 flags.go:64] FLAG: --cgroup-root=""
Sep 16 18:44:29.957437 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952226    2584 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 16 18:44:29.957437 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952229    2584 flags.go:64] FLAG: --client-ca-file=""
Sep 16 18:44:29.957437 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952232    2584 flags.go:64] FLAG: --cloud-provider="external"
Sep 16 18:44:29.957437 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952235    2584 flags.go:64] FLAG: --cluster-dns="[]"
Sep 16 18:44:29.957437 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952240    2584 flags.go:64] FLAG: --cluster-domain=""
Sep 16 18:44:29.957437 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952243    2584 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 16 18:44:29.957995 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952246    2584 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 16 18:44:29.957995 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952249    2584 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 16 18:44:29.957995 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952252    2584 flags.go:64] FLAG: --container-log-max-files="5"
Sep 16 18:44:29.957995 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952256    2584 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 16 18:44:29.957995 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952259    2584 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 16 18:44:29.957995 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952262    2584 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 16 18:44:29.957995 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952265    2584 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 16 18:44:29.957995 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952268    2584 flags.go:64] FLAG: --contention-profiling="false"
Sep 16 18:44:29.957995 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952271    2584 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 16 18:44:29.957995 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952274    2584 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 16 18:44:29.957995 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952277    2584 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 16 18:44:29.957995 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952280    2584 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 16 18:44:29.957995 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952283    2584 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 16 18:44:29.957995 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952286    2584 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 16 18:44:29.957995 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952289    2584 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 16 18:44:29.957995 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952292    2584 flags.go:64] FLAG: --enable-load-reader="false"
Sep 16 18:44:29.957995 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952295    2584 flags.go:64] FLAG: --enable-server="true"
Sep 16 18:44:29.957995 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952298    2584 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 16 18:44:29.957995 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952302    2584 flags.go:64] FLAG: --event-burst="100"
Sep 16 18:44:29.957995 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952305    2584 flags.go:64] FLAG: --event-qps="50"
Sep 16 18:44:29.957995 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952308    2584 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 16 18:44:29.957995 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952311    2584 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 16 18:44:29.957995 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952314    2584 flags.go:64] FLAG: --eviction-hard=""
Sep 16 18:44:29.957995 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952320    2584 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 16 18:44:29.958552 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952323    2584 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 16 18:44:29.958552 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952326    2584 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 16 18:44:29.958552 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952329    2584 flags.go:64] FLAG: --eviction-soft=""
Sep 16 18:44:29.958552 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952332    2584 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 16 18:44:29.958552 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952334    2584 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 16 18:44:29.958552 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952337    2584 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 16 18:44:29.958552 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952341    2584 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 16 18:44:29.958552 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952343    2584 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 16 18:44:29.958552 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952346    2584 flags.go:64] FLAG: --fail-swap-on="true"
Sep 16 18:44:29.958552 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952349    2584 flags.go:64] FLAG: --feature-gates=""
Sep 16 18:44:29.958552 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952353    2584 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 16 18:44:29.958552 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952356    2584 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 16 18:44:29.958552 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952359    2584 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 16 18:44:29.958552 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952362    2584 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 16 18:44:29.958552 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952365    2584 flags.go:64] FLAG: --healthz-port="10248"
Sep 16 18:44:29.958552 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952368    2584 flags.go:64] FLAG: --help="false"
Sep 16 18:44:29.958552 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952371    2584 flags.go:64] FLAG: --hostname-override="ip-10-0-139-182.ec2.internal"
Sep 16 18:44:29.958552 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952374    2584 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 16 18:44:29.958552 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952376    2584 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 16 18:44:29.958552 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952380    2584 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 16 18:44:29.958552 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952383    2584 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 16 18:44:29.958552 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952386    2584 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 16 18:44:29.958552 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952389    2584 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 16 18:44:29.958552 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952392    2584 flags.go:64] FLAG: --image-service-endpoint=""
Sep 16 18:44:29.959197 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952395    2584 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 16 18:44:29.959197 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952397    2584 flags.go:64] FLAG: --kube-api-burst="100"
Sep 16 18:44:29.959197 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952400    2584 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 16 18:44:29.959197 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952405    2584 flags.go:64] FLAG: --kube-api-qps="50"
Sep 16 18:44:29.959197 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952408    2584 flags.go:64] FLAG: --kube-reserved=""
Sep 16 18:44:29.959197 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952411    2584 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 16 18:44:29.959197 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952415    2584 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 16 18:44:29.959197 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952419    2584 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 16 18:44:29.959197 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952422    2584 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 16 18:44:29.959197 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952424    2584 flags.go:64] FLAG: --lock-file=""
Sep 16 18:44:29.959197 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952428    2584 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 16 18:44:29.959197 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952430    2584 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 16 18:44:29.959197 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952433    2584 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 16 18:44:29.959197 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952438    2584 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 16 18:44:29.959197 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952441    2584 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 16 18:44:29.959197 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952444    2584 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 16 18:44:29.959197 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952447    2584 flags.go:64] FLAG: --logging-format="text"
Sep 16 18:44:29.959197 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952449    2584 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 16 18:44:29.959197 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952453    2584 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 16 18:44:29.959197 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952455    2584 flags.go:64] FLAG: --manifest-url=""
Sep 16 18:44:29.959197 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952458    2584 flags.go:64] FLAG: --manifest-url-header=""
Sep 16 18:44:29.959197 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952462    2584 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 16 18:44:29.959197 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952466    2584 flags.go:64] FLAG: --max-open-files="1000000"
Sep 16 18:44:29.959197 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952470    2584 flags.go:64] FLAG: --max-pods="110"
Sep 16 18:44:29.959197 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952473    2584 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 16 18:44:29.960154 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952476    2584 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 16 18:44:29.960154 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952478    2584 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 16 18:44:29.960154 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952482    2584 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 16 18:44:29.960154 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952484    2584 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 16 18:44:29.960154 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952487    2584 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 16 18:44:29.960154 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952490    2584 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 16 18:44:29.960154 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952497    2584 flags.go:64] FLAG: --node-status-max-images="50"
Sep 16 18:44:29.960154 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952500    2584 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 16 18:44:29.960154 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952503    2584 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 16 18:44:29.960154 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952506    2584 flags.go:64] FLAG: --pod-cidr=""
Sep 16 18:44:29.960154 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952509    2584 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:eee392dbe3ac2ed4a38bca29ae3bd486f5b7780406c62c600154cd1906871b10"
Sep 16 18:44:29.960154 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952515    2584 flags.go:64] FLAG: --pod-manifest-path=""
Sep 16 18:44:29.960154 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952518    2584 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 16 18:44:29.960154 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952522    2584 flags.go:64] FLAG: --pods-per-core="0"
Sep 16 18:44:29.960154 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952526    2584 flags.go:64] FLAG: --port="10250"
Sep 16 18:44:29.960154 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952529    2584 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 16 18:44:29.960154 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952532    2584 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-06f648e66f0494365"
Sep 16 18:44:29.960154 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952535    2584 flags.go:64] FLAG: --qos-reserved=""
Sep 16 18:44:29.960154 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952538    2584 flags.go:64] FLAG: --read-only-port="10255"
Sep 16 18:44:29.960154 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952541    2584 flags.go:64] FLAG: --register-node="true"
Sep 16 18:44:29.960154 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952544    2584 flags.go:64] FLAG: --register-with-taints=""
Sep 16 18:44:29.960154 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952547    2584 flags.go:64] FLAG: --registry-burst="10"
Sep 16 18:44:29.960154 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952550    2584 flags.go:64] FLAG: --registry-qps="5"
Sep 16 18:44:29.960154 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952553    2584 flags.go:64] FLAG: --reserved-cpus=""
Sep 16 18:44:29.960847 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952555    2584 flags.go:64] FLAG: --reserved-memory=""
Sep 16 18:44:29.960847 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952559    2584 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 16 18:44:29.960847 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952562    2584 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 16 18:44:29.960847 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952565    2584 flags.go:64] FLAG: --rotate-certificates="false"
Sep 16 18:44:29.960847 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952568    2584 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 16 18:44:29.960847 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952571    2584 flags.go:64] FLAG: --runonce="false"
Sep 16 18:44:29.960847 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952574    2584 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 16 18:44:29.960847 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952577    2584 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 16 18:44:29.960847 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952580    2584 flags.go:64] FLAG: --seccomp-default="false"
Sep 16 18:44:29.960847 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952582    2584 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 16 18:44:29.960847 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952585    2584 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 16 18:44:29.960847 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952588    2584 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 16 18:44:29.960847 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952591    2584 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 16 18:44:29.960847 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952594    2584 flags.go:64] FLAG: --storage-driver-password="root"
Sep 16 18:44:29.960847 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952597    2584 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 16 18:44:29.960847 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952600    2584 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 16 18:44:29.960847 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952602    2584 flags.go:64] FLAG: --storage-driver-user="root"
Sep 16 18:44:29.960847 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952605    2584 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 16 18:44:29.960847 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952608    2584 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 16 18:44:29.960847 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952611    2584 flags.go:64] FLAG: --system-cgroups=""
Sep 16 18:44:29.960847 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952615    2584 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 16 18:44:29.960847 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952620    2584 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 16 18:44:29.960847 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952625    2584 flags.go:64] FLAG: --tls-cert-file=""
Sep 16 18:44:29.960847 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952628    2584 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 16 18:44:29.960847 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952632    2584 flags.go:64] FLAG: --tls-min-version=""
Sep 16 18:44:29.961441 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952635    2584 flags.go:64] FLAG: --tls-private-key-file=""
Sep 16 18:44:29.961441 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952638    2584 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 16 18:44:29.961441 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952640    2584 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 16 18:44:29.961441 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952643    2584 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 16 18:44:29.961441 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952646    2584 flags.go:64] FLAG: --v="2"
Sep 16 18:44:29.961441 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952650    2584 flags.go:64] FLAG: --version="false"
Sep 16 18:44:29.961441 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952654    2584 flags.go:64] FLAG: --vmodule=""
Sep 16 18:44:29.961441 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952658    2584 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 16 18:44:29.961441 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.952661    2584 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 16 18:44:29.961441 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.952781    2584 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 16 18:44:29.961441 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.952786    2584 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 16 18:44:29.961441 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.952789    2584 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 16 18:44:29.961441 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.952792    2584 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 16 18:44:29.961441 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.952795    2584 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 16 18:44:29.961441 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.952798    2584 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 16 18:44:29.961441 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.952800    2584 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 16 18:44:29.961441 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.952803    2584 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 16 18:44:29.961441 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.952805    2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 16 18:44:29.961441 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.952808    2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 16 18:44:29.961441 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.952811    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 16 18:44:29.961441 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.952813    2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 16 18:44:29.961441 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.952816    2584 feature_gate.go:328] unrecognized feature gate: Example2
Sep 16 18:44:29.961989 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.952819    2584 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 16 18:44:29.961989 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.952823    2584 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 16 18:44:29.961989 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.952840    2584 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 16 18:44:29.961989 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.952844    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 16 18:44:29.961989 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.952846    2584 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 16 18:44:29.961989 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.952849    2584 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 16 18:44:29.961989 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.952852    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 16 18:44:29.961989 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.952858    2584 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 16 18:44:29.961989 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.952861    2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 16 18:44:29.961989 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.952864    2584 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 16 18:44:29.961989 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.952866    2584 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 16 18:44:29.961989 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.952869    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 16 18:44:29.961989 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.952871    2584 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 16 18:44:29.961989 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.952874    2584 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 16 18:44:29.961989 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.952877    2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 16 18:44:29.961989 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.952879    2584 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 16 18:44:29.961989 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.952882    2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 16 18:44:29.961989 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.952884    2584 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 16 18:44:29.961989 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.952886    2584 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 16 18:44:29.962453 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.952889    2584 feature_gate.go:328] unrecognized feature gate: Example
Sep 16 18:44:29.962453 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.952892    2584 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 16 18:44:29.962453 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.952894    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 16 18:44:29.962453 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.952897    2584 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 16 18:44:29.962453 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.952899    2584 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 16 18:44:29.962453 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.952902    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 16 18:44:29.962453 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.952904    2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 16 18:44:29.962453 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.952907    2584 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 16 18:44:29.962453 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.952910    2584 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 16 18:44:29.962453 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.952913    2584 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 16 18:44:29.962453 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.952916    2584 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 16 18:44:29.962453 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.952918    2584 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 16 18:44:29.962453 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.952921    2584 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 16 18:44:29.962453 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.952923    2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 16 18:44:29.962453 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.952926    2584 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 16 18:44:29.962453 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.952929    2584 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 16 18:44:29.962453 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.952931    2584 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 16 18:44:29.962453 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.952934    2584 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 16 18:44:29.962453 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.952937    2584 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 16 18:44:29.962453 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.952939    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 16 18:44:29.962948 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.952945    2584 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 16 18:44:29.962948 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.952947    2584 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 16 18:44:29.962948 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.952950    2584 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 16 18:44:29.962948 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.952953    2584 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 16 18:44:29.962948 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.952955    2584 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 16 18:44:29.962948 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.952958    2584 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 16 18:44:29.962948 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.952960    2584 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 16 18:44:29.962948 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.952963    2584 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 16 18:44:29.962948 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.952965    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 16 18:44:29.962948 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.952968    2584 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 16 18:44:29.962948 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.952970    2584 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 16 18:44:29.962948 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.952973    2584 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 16 18:44:29.962948 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.952975    2584 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 16 18:44:29.962948 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.952978    2584 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 16 18:44:29.962948 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.952980    2584 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 16 18:44:29.962948 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.952983    2584 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 16 18:44:29.962948 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.952985    2584 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 16 18:44:29.962948 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.952988    2584 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 16 18:44:29.962948 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.952990    2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 16 18:44:29.962948 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.952993    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 16 18:44:29.963418 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.952996    2584 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 16 18:44:29.963418 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.952998    2584 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 16 18:44:29.963418 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.953001    2584 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 16 18:44:29.963418 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.953003    2584 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 16 18:44:29.963418 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.953006    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 16 18:44:29.963418 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.953010    2584 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 16 18:44:29.963418 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.953013    2584 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 16 18:44:29.963418 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.953015    2584 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 16 18:44:29.963418 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.953018    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 16 18:44:29.963418 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.953020    2584 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 16 18:44:29.963418 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.953023    2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 16 18:44:29.963418 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.953026    2584 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 16 18:44:29.963418 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.953031    2584 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 16 18:44:29.963418 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.953034    2584 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 16 18:44:29.963418 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.953036    2584 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 16 18:44:29.963418 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.953039    2584 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 16 18:44:29.963418 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.953042    2584 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 16 18:44:29.963418 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.953044    2584 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 16 18:44:29.963418 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.953047    2584 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 16 18:44:29.963418 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.953049    2584 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 16 18:44:29.963943 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.953052    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 16 18:44:29.963943 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.953054    2584 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 16 18:44:29.963943 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.953057    2584 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 16 18:44:29.963943 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.953060    2584 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 16 18:44:29.963943 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.953064    2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 16 18:44:29.963943 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.953066    2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 16 18:44:29.963943 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.953069    2584 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 16 18:44:29.963943 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.953582    2584 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 16 18:44:29.963943 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.961512    2584 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 16 18:44:29.963943 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.961528    2584 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 16 18:44:29.963943 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.962396    2584 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 16 18:44:29.963943 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.962404    2584 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 16 18:44:29.963943 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.962459    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 16 18:44:29.963943 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.962464    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 16 18:44:29.964366 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.962468    2584 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 16 18:44:29.964366 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.962472    2584 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 16 18:44:29.964366 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.962477    2584 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 16 18:44:29.964366 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.962480    2584 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 16 18:44:29.964366 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.962483    2584 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 16 18:44:29.964366 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.962487    2584 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 16 18:44:29.964366 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.962489    2584 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 16 18:44:29.964366 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.962492    2584 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 16 18:44:29.964366 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.962495    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 16 18:44:29.964366 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.962498    2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 16 18:44:29.964366 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.962501    2584 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 16 18:44:29.964366 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.962504    2584 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 16 18:44:29.964366 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.962507    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 16 18:44:29.964366 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.962509    2584 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 16 18:44:29.964366 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.962511    2584 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 16 18:44:29.964366 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.962514    2584 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 16 18:44:29.964366 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.962517    2584 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 16 18:44:29.964366 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.962520    2584 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 16 18:44:29.964366 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.962522    2584 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 16 18:44:29.964826 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.962525    2584 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 16 18:44:29.964826 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.962527    2584 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 16 18:44:29.964826 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.962530    2584 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 16 18:44:29.964826 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.962533    2584 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 16 18:44:29.964826 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.962535    2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 16 18:44:29.964826 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.962537    2584 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 16 18:44:29.964826 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.962540    2584 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 16 18:44:29.964826 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.962543    2584 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 16 18:44:29.964826 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.962547    2584 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 16 18:44:29.964826 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.962550    2584 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 16 18:44:29.964826 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.962553    2584 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 16 18:44:29.964826 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.962555    2584 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 16 18:44:29.964826 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.962558    2584 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 16 18:44:29.964826 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.962560    2584 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 16 18:44:29.964826 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.962563    2584 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 16 18:44:29.964826 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.962565    2584 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 16 18:44:29.964826 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.962568    2584 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 16 18:44:29.964826 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.962571    2584 feature_gate.go:328] unrecognized feature gate: Example
Sep 16 18:44:29.964826 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.962574    2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 16 18:44:29.964826 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.962576    2584 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 16 18:44:29.965320 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.962579    2584 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 16 18:44:29.965320 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.962581    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 16 18:44:29.965320 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.962584    2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 16 18:44:29.965320 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.962586    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 16 18:44:29.965320 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.962589    2584 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 16 18:44:29.965320 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.962592    2584 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 16 18:44:29.965320 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.962596    2584 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 16 18:44:29.965320 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.962599    2584 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 16 18:44:29.965320 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.962602    2584 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 16 18:44:29.965320 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.962605    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 16 18:44:29.965320 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.962608    2584 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 16 18:44:29.965320 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.962611    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 16 18:44:29.965320 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.962614    2584 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 16 18:44:29.965320 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.962617    2584 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 16 18:44:29.965320 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.962619    2584 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 16 18:44:29.965320 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.962622    2584 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 16 18:44:29.965320 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.962625    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 16 18:44:29.965320 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.962627    2584 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 16 18:44:29.965320 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.962630    2584 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 16 18:44:29.965771 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.962632    2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 16 18:44:29.965771 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.962636    2584 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 16 18:44:29.965771 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.962639    2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 16 18:44:29.965771 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.962641    2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 16 18:44:29.965771 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.962644    2584 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 16 18:44:29.965771 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.962646    2584 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 16 18:44:29.965771 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.962648    2584 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 16 18:44:29.965771 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.962651    2584 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 16 18:44:29.965771 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.962654    2584 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 16 18:44:29.965771 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.962656    2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 16 18:44:29.965771 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.962658    2584 feature_gate.go:328] unrecognized feature gate: Example2
Sep 16 18:44:29.965771 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.962662    2584 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 16 18:44:29.965771 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.962664    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 16 18:44:29.965771 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.962666    2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 16 18:44:29.965771 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.962669    2584 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 16 18:44:29.965771 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.962671    2584 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 16 18:44:29.965771 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.962674    2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 16 18:44:29.965771 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.962676    2584 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 16 18:44:29.965771 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.962679    2584 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 16 18:44:29.965771 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.962682    2584 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 16 18:44:29.966264 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.962685    2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 16 18:44:29.966264 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.962687    2584 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 16 18:44:29.966264 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.962691    2584 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 16 18:44:29.966264 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.962694    2584 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 16 18:44:29.966264 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.962697    2584 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 16 18:44:29.966264 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.962700    2584 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 16 18:44:29.966264 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.962703    2584 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 16 18:44:29.966264 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.962705    2584 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 16 18:44:29.966264 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.962708    2584 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 16 18:44:29.966264 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.962710    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 16 18:44:29.966264 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.962713    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 16 18:44:29.966264 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.962715    2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 16 18:44:29.966264 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.962717    2584 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 16 18:44:29.966264 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.962720    2584 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 16 18:44:29.966264 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.962723    2584 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 16 18:44:29.966264 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.962725    2584 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 16 18:44:29.966264 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.962728    2584 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 16 18:44:29.966264 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.962730    2584 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 16 18:44:29.966264 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.962733    2584 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 16 18:44:29.966712 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.962738    2584 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 16 18:44:29.966712 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.962879    2584 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 16 18:44:29.966712 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.962885    2584 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 16 18:44:29.966712 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.962889    2584 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 16 18:44:29.966712 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.962892    2584 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 16 18:44:29.966712 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.962895    2584 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 16 18:44:29.966712 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.962897    2584 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 16 18:44:29.966712 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.962900    2584 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 16 18:44:29.966712 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.962902    2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 16 18:44:29.966712 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.962905    2584 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 16 18:44:29.966712 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.962907    2584 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 16 18:44:29.966712 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.962910    2584 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 16 18:44:29.966712 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.962912    2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 16 18:44:29.966712 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.962916    2584 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 16 18:44:29.966712 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.962919    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 16 18:44:29.967098 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.962922    2584 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 16 18:44:29.967098 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.962924    2584 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 16 18:44:29.967098 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.962927    2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 16 18:44:29.967098 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.962929    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 16 18:44:29.967098 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.962932    2584 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 16 18:44:29.967098 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.962934    2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 16 18:44:29.967098 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.962936    2584 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 16 18:44:29.967098 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.962939    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 16 18:44:29.967098 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.962941    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 16 18:44:29.967098 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.962944    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 16 18:44:29.967098 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.962946    2584 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 16 18:44:29.967098 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.962949    2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 16 18:44:29.967098 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.962952    2584 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 16 18:44:29.967098 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.962954    2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 16 18:44:29.967098 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.962957    2584 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 16 18:44:29.967098 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.962960    2584 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 16 18:44:29.967098 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.962962    2584 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 16 18:44:29.967098 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.962965    2584 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 16 18:44:29.967098 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.962968    2584 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 16 18:44:29.967098 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.962970    2584 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 16 18:44:29.967601 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.962972    2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 16 18:44:29.967601 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.962975    2584 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 16 18:44:29.967601 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.962977    2584 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 16 18:44:29.967601 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.962980    2584 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 16 18:44:29.967601 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.962982    2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 16 18:44:29.967601 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.962984    2584 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 16 18:44:29.967601 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.962988    2584 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 16 18:44:29.967601 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.962992    2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 16 18:44:29.967601 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.962994    2584 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 16 18:44:29.967601 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.962997    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 16 18:44:29.967601 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.962999    2584 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 16 18:44:29.967601 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.963003    2584 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 16 18:44:29.967601 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.963005    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 16 18:44:29.967601 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.963008    2584 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 16 18:44:29.967601 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.963010    2584 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 16 18:44:29.967601 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.963012    2584 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 16 18:44:29.967601 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.963015    2584 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 16 18:44:29.967601 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.963017    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 16 18:44:29.967601 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.963020    2584 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 16 18:44:29.968248 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.963023    2584 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 16 18:44:29.968248 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.963025    2584 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 16 18:44:29.968248 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.963028    2584 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 16 18:44:29.968248 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.963030    2584 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 16 18:44:29.968248 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.963033    2584 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 16 18:44:29.968248 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.963036    2584 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 16 18:44:29.968248 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.963038    2584 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 16 18:44:29.968248 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.963041    2584 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 16 18:44:29.968248 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.963043    2584 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 16 18:44:29.968248 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.963046    2584 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 16 18:44:29.968248 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.963048    2584 feature_gate.go:328] unrecognized feature gate: Example2
Sep 16 18:44:29.968248 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.963052    2584 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 16 18:44:29.968248 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.963055    2584 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 16 18:44:29.968248 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.963058    2584 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 16 18:44:29.968248 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.963060    2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 16 18:44:29.968248 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.963063    2584 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 16 18:44:29.968248 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.963065    2584 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 16 18:44:29.968248 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.963069    2584 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 16 18:44:29.968248 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.963072    2584 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 16 18:44:29.969034 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.963075    2584 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 16 18:44:29.969034 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.963078    2584 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 16 18:44:29.969034 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.963081    2584 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 16 18:44:29.969034 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.963083    2584 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 16 18:44:29.969034 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.963086    2584 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 16 18:44:29.969034 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.963088    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 16 18:44:29.969034 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.963091    2584 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 16 18:44:29.969034 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.963094    2584 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 16 18:44:29.969034 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.963096    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 16 18:44:29.969034 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.963099    2584 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 16 18:44:29.969034 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.963101    2584 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 16 18:44:29.969034 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.963104    2584 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 16 18:44:29.969034 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.963106    2584 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 16 18:44:29.969034 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.963108    2584 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 16 18:44:29.969034 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.963111    2584 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 16 18:44:29.969034 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.963113    2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 16 18:44:29.969034 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.963116    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 16 18:44:29.969034 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.963119    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 16 18:44:29.969034 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.963122    2584 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 16 18:44:29.969034 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.963124    2584 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 16 18:44:29.969536 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.963126    2584 feature_gate.go:328] unrecognized feature gate: Example
Sep 16 18:44:29.969536 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.963129    2584 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 16 18:44:29.969536 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.963131    2584 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 16 18:44:29.969536 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.963134    2584 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 16 18:44:29.969536 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.963136    2584 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 16 18:44:29.969536 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.963139    2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 16 18:44:29.969536 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:29.963141    2584 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 16 18:44:29.969536 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.963145    2584 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 16 18:44:29.969536 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.963256    2584 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 16 18:44:29.969536 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.966920    2584 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 16 18:44:29.969536 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.967867    2584 server.go:1019] "Starting client certificate rotation"
Sep 16 18:44:29.969536 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.967982    2584 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 16 18:44:29.969536 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.968030    2584 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 16 18:44:29.970300 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.970285    2584 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 16 18:44:29.971423 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.971404    2584 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 16 18:44:29.980800 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.980781    2584 log.go:25] "Validated CRI v1 runtime API"
Sep 16 18:44:29.983132 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.983119    2584 log.go:25] "Validated CRI v1 image API"
Sep 16 18:44:29.983954 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.983934    2584 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 16 18:44:29.987304 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.987286    2584 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 a3deb4fa-1e30-4312-8824-f851dc6d6817:/dev/nvme0n1p4 af29878f-20c8-491d-bc9a-44bc685fa622:/dev/nvme0n1p3]
Sep 16 18:44:29.987376 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.987303    2584 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 16 18:44:29.992799 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.992673    2584 manager.go:216] Machine: {Timestamp:2026-09-16 18:44:29.990679847 +0000 UTC m=+0.320182249 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3210758 MemoryCapacity:33164476416 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2b36ec921c25459a85f2f20f254aff SystemUUID:ec2b36ec-921c-2545-9a85-f2f20f254aff BootID:9f1b9e67-dbde-41b8-8d6a-a4f43447b3cf Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582238208 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:f2:e1:cc:37 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:f2:e1:cc:37 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:62:ff:0c:9e:bb:77 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164476416 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 16 18:44:29.992799 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.992788    2584 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 16 18:44:29.992941 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.992891    2584 manager.go:232] Version: {KernelVersion:5.14.0-570.140.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260908-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 16 18:44:29.997149 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.996951    2584 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 16 18:44:29.997305 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.997152    2584 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-139-182.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 16 18:44:29.997708 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.997686    2584 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 16 18:44:29.997889 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.997876    2584 topology_manager.go:138] "Creating topology manager with none policy"
Sep 16 18:44:29.997937 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.997893    2584 container_manager_linux.go:309] "Creating device plugin manager"
Sep 16 18:44:29.997937 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.997901    2584 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 16 18:44:29.998459 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.998449    2584 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 16 18:44:29.999346 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:29.999337    2584 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 16 18:44:30.003111 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:30.003099    2584 state_mem.go:36] "Initialized new in-memory state store"
Sep 16 18:44:30.003229 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:30.003220    2584 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 16 18:44:30.003258 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:30.003252    2584 kubelet.go:486] "Attempting to sync node with API server"
Sep 16 18:44:30.003291 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:30.003264    2584 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 16 18:44:30.003291 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:30.003278    2584 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 16 18:44:30.003291 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:30.003287    2584 kubelet.go:398] "Adding apiserver pod source"
Sep 16 18:44:30.003384 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:30.003295    2584 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 16 18:44:30.003808 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:30.003796    2584 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 16 18:44:30.003859 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:30.003822    2584 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 16 18:44:30.006199 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:30.006186    2584 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 16 18:44:30.008457 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:30.008443    2584 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 16 18:44:30.008503 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:30.008472    2584 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 16 18:44:30.009660 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:30.009646    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 16 18:44:30.009725 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:30.009673    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 16 18:44:30.009725 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:30.009685    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 16 18:44:30.009725 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:30.009694    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 16 18:44:30.009725 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:30.009710    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 16 18:44:30.009725 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:30.009720    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 16 18:44:30.009962 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:30.009742    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 16 18:44:30.009962 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:30.009748    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 16 18:44:30.009962 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:30.009754    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 16 18:44:30.009962 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:30.009759    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 16 18:44:30.009962 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:30.009767    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 16 18:44:30.009962 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:30.009773    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 16 18:44:30.010419 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:30.010408    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 16 18:44:30.010419 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:30.010418    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 16 18:44:30.014240 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:30.014226    2584 server.go:1268] "Started kubelet"
Sep 16 18:44:30.014332 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:30.014305    2584 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 16 18:44:30.014564 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:30.014450    2584 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 16 18:44:30.014671 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:30.014634    2584 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 16 18:44:30.015383 ip-10-0-139-182 systemd[1]: Started Kubernetes Kubelet.
Sep 16 18:44:30.015600 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:30.015460    2584 server.go:310] "Adding debug handlers to kubelet server"
Sep 16 18:44:30.015818 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:30.015802    2584 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 16 18:44:30.016118 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:44:30.016095    2584 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-139-182.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 16 18:44:30.016118 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:30.016104    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-139-182.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 16 18:44:30.016233 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:44:30.016165    2584 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 16 18:44:30.019894 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:30.019876    2584 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 16 18:44:30.019984 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:30.019889    2584 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 16 18:44:30.020040 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:44:30.020016    2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-182.ec2.internal\" not found"
Sep 16 18:44:30.020562 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:30.020538    2584 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 16 18:44:30.020562 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:30.020558    2584 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 16 18:44:30.020744 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:30.020537    2584 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 16 18:44:30.020744 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:30.020623    2584 reconstruct.go:97] "Volume reconstruction finished"
Sep 16 18:44:30.020744 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:30.020633    2584 reconciler.go:29] "Reconciler: start to sync state"
Sep 16 18:44:30.021757 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:30.021354    2584 factory.go:55] Registering systemd factory
Sep 16 18:44:30.021757 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:30.021368    2584 factory.go:223] Registration of the systemd container factory successfully
Sep 16 18:44:30.021757 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:30.021638    2584 factory.go:153] Registering CRI-O factory
Sep 16 18:44:30.021757 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:30.021651    2584 factory.go:223] Registration of the crio container factory successfully
Sep 16 18:44:30.021757 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:30.021710    2584 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 16 18:44:30.021757 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:30.021729    2584 factory.go:103] Registering Raw factory
Sep 16 18:44:30.021757 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:44:30.021739    2584 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 16 18:44:30.021757 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:30.021747    2584 manager.go:1196] Started watching for new ooms in manager
Sep 16 18:44:30.022151 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:30.022137    2584 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-k2gkh"
Sep 16 18:44:30.022201 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:30.022187    2584 manager.go:349] Starting recovery of all containers
Sep 16 18:44:30.023753 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:44:30.022810    2584 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-182.ec2.internal.18d5e16975d2b73b  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-182.ec2.internal,UID:ip-10-0-139-182.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-139-182.ec2.internal,},FirstTimestamp:2026-09-16 18:44:30.014199611 +0000 UTC m=+0.343702013,LastTimestamp:2026-09-16 18:44:30.014199611 +0000 UTC m=+0.343702013,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-182.ec2.internal,}"
Sep 16 18:44:30.027377 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:44:30.027347    2584 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 16 18:44:30.027476 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:44:30.027443    2584 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-139-182.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 16 18:44:30.029844 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:30.029798    2584 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 16 18:44:30.030732 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:30.030709    2584 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-k2gkh"
Sep 16 18:44:30.033624 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:30.033448    2584 manager.go:354] Recovery completed
Sep 16 18:44:30.038528 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:30.038517    2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 16 18:44:30.040194 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:30.040177    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-182.ec2.internal" event="NodeHasSufficientMemory"
Sep 16 18:44:30.040271 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:30.040207    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-182.ec2.internal" event="NodeHasNoDiskPressure"
Sep 16 18:44:30.040271 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:30.040217    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-182.ec2.internal" event="NodeHasSufficientPID"
Sep 16 18:44:30.041602 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:30.041590    2584 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 16 18:44:30.041602 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:30.041600    2584 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 16 18:44:30.041698 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:30.041614    2584 state_mem.go:36] "Initialized new in-memory state store"
Sep 16 18:44:30.041698 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:30.041631    2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 16 18:44:30.042383 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:30.042367    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-182.ec2.internal" event="NodeHasSufficientMemory"
Sep 16 18:44:30.042457 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:44:30.042324    2584 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-182.ec2.internal.18d5e169775f5bbf  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-182.ec2.internal,UID:ip-10-0-139-182.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-139-182.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-139-182.ec2.internal,},FirstTimestamp:2026-09-16 18:44:30.040193983 +0000 UTC m=+0.369696387,LastTimestamp:2026-09-16 18:44:30.040193983 +0000 UTC m=+0.369696387,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-182.ec2.internal,}"
Sep 16 18:44:30.042457 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:30.042400    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-182.ec2.internal" event="NodeHasNoDiskPressure"
Sep 16 18:44:30.042457 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:30.042412    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-182.ec2.internal" event="NodeHasSufficientPID"
Sep 16 18:44:30.043743 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:30.043727    2584 policy_none.go:49] "None policy: Start"
Sep 16 18:44:30.043743 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:30.043744    2584 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 16 18:44:30.043873 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:30.043754    2584 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 16 18:44:30.045752 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:30.045739    2584 policy_none.go:47] "Start"
Sep 16 18:44:30.069434 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:30.069418    2584 manager.go:337] "Starting Device Plugin manager"
Sep 16 18:44:30.096114 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:44:30.069458    2584 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 16 18:44:30.096114 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:30.069468    2584 server.go:87] "Starting device plugin registration server"
Sep 16 18:44:30.096114 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:30.069822    2584 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 16 18:44:30.096114 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:30.069852    2584 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 16 18:44:30.096114 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:30.069948    2584 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 16 18:44:30.096114 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:30.070025    2584 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 16 18:44:30.096114 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:30.070034    2584 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 16 18:44:30.096114 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:44:30.070552    2584 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 16 18:44:30.096114 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:44:30.070594    2584 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-139-182.ec2.internal\" not found"
Sep 16 18:44:30.110154 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:30.110130    2584 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 16 18:44:30.110239 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:30.110181    2584 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 16 18:44:30.110239 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:30.110206    2584 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 16 18:44:30.110326 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:44:30.110298    2584 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 16 18:44:30.112114 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:30.112094    2584 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 16 18:44:30.169981 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:30.169957    2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 16 18:44:30.170907 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:30.170891    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-182.ec2.internal" event="NodeHasSufficientMemory"
Sep 16 18:44:30.170991 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:30.170922    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-182.ec2.internal" event="NodeHasNoDiskPressure"
Sep 16 18:44:30.170991 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:30.170936    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-182.ec2.internal" event="NodeHasSufficientPID"
Sep 16 18:44:30.170991 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:30.170958    2584 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-139-182.ec2.internal"
Sep 16 18:44:30.180971 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:30.180953    2584 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-139-182.ec2.internal"
Sep 16 18:44:30.181025 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:44:30.180977    2584 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-139-182.ec2.internal\": node \"ip-10-0-139-182.ec2.internal\" not found"
Sep 16 18:44:30.195126 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:44:30.195100    2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-182.ec2.internal\" not found"
Sep 16 18:44:30.211273 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:30.211187    2584 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-139-182.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-182.ec2.internal"]
Sep 16 18:44:30.211273 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:30.211269    2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 16 18:44:30.212118 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:30.212100    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-182.ec2.internal" event="NodeHasSufficientMemory"
Sep 16 18:44:30.212208 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:30.212135    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-182.ec2.internal" event="NodeHasNoDiskPressure"
Sep 16 18:44:30.212208 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:30.212150    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-182.ec2.internal" event="NodeHasSufficientPID"
Sep 16 18:44:30.214348 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:30.214334    2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 16 18:44:30.214447 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:30.214433    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-182.ec2.internal"
Sep 16 18:44:30.214490 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:30.214464    2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 16 18:44:30.215028 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:30.215012    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-182.ec2.internal" event="NodeHasSufficientMemory"
Sep 16 18:44:30.215097 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:30.215044    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-182.ec2.internal" event="NodeHasNoDiskPressure"
Sep 16 18:44:30.215097 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:30.215054    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-182.ec2.internal" event="NodeHasSufficientPID"
Sep 16 18:44:30.215097 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:30.215016    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-182.ec2.internal" event="NodeHasSufficientMemory"
Sep 16 18:44:30.215200 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:30.215113    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-182.ec2.internal" event="NodeHasNoDiskPressure"
Sep 16 18:44:30.215200 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:30.215126    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-182.ec2.internal" event="NodeHasSufficientPID"
Sep 16 18:44:30.217249 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:30.217234    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-182.ec2.internal"
Sep 16 18:44:30.217337 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:30.217265    2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 16 18:44:30.217901 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:30.217885    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-182.ec2.internal" event="NodeHasSufficientMemory"
Sep 16 18:44:30.217978 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:30.217913    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-182.ec2.internal" event="NodeHasNoDiskPressure"
Sep 16 18:44:30.217978 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:30.217925    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-182.ec2.internal" event="NodeHasSufficientPID"
Sep 16 18:44:30.239437 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:44:30.239421    2584 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-182.ec2.internal\" not found" node="ip-10-0-139-182.ec2.internal"
Sep 16 18:44:30.243714 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:44:30.243699    2584 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-182.ec2.internal\" not found" node="ip-10-0-139-182.ec2.internal"
Sep 16 18:44:30.296072 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:44:30.296048    2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-182.ec2.internal\" not found"
Sep 16 18:44:30.321001 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:30.320973    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/228e51ceefc0d99659afe6cee7fbb736-config\") pod \"kube-apiserver-proxy-ip-10-0-139-182.ec2.internal\" (UID: \"228e51ceefc0d99659afe6cee7fbb736\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-182.ec2.internal"
Sep 16 18:44:30.321121 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:30.321010    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/4fa23c77cdd18b7265820cf09e7b3e9d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-182.ec2.internal\" (UID: \"4fa23c77cdd18b7265820cf09e7b3e9d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-182.ec2.internal"
Sep 16 18:44:30.321121 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:30.321033    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4fa23c77cdd18b7265820cf09e7b3e9d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-182.ec2.internal\" (UID: \"4fa23c77cdd18b7265820cf09e7b3e9d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-182.ec2.internal"
Sep 16 18:44:30.396423 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:44:30.396399    2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-182.ec2.internal\" not found"
Sep 16 18:44:30.421691 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:30.421670    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4fa23c77cdd18b7265820cf09e7b3e9d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-182.ec2.internal\" (UID: \"4fa23c77cdd18b7265820cf09e7b3e9d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-182.ec2.internal"
Sep 16 18:44:30.421745 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:30.421697    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/228e51ceefc0d99659afe6cee7fbb736-config\") pod \"kube-apiserver-proxy-ip-10-0-139-182.ec2.internal\" (UID: \"228e51ceefc0d99659afe6cee7fbb736\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-182.ec2.internal"
Sep 16 18:44:30.421745 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:30.421713    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/4fa23c77cdd18b7265820cf09e7b3e9d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-182.ec2.internal\" (UID: \"4fa23c77cdd18b7265820cf09e7b3e9d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-182.ec2.internal"
Sep 16 18:44:30.421811 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:30.421760    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4fa23c77cdd18b7265820cf09e7b3e9d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-182.ec2.internal\" (UID: \"4fa23c77cdd18b7265820cf09e7b3e9d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-182.ec2.internal"
Sep 16 18:44:30.421811 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:30.421765    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/228e51ceefc0d99659afe6cee7fbb736-config\") pod \"kube-apiserver-proxy-ip-10-0-139-182.ec2.internal\" (UID: \"228e51ceefc0d99659afe6cee7fbb736\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-182.ec2.internal"
Sep 16 18:44:30.421896 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:30.421821    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/4fa23c77cdd18b7265820cf09e7b3e9d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-182.ec2.internal\" (UID: \"4fa23c77cdd18b7265820cf09e7b3e9d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-182.ec2.internal"
Sep 16 18:44:30.496991 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:44:30.496926    2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-182.ec2.internal\" not found"
Sep 16 18:44:30.543546 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:30.543520    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-182.ec2.internal"
Sep 16 18:44:30.546517 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:30.546501    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-182.ec2.internal"
Sep 16 18:44:30.597060 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:44:30.597031    2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-182.ec2.internal\" not found"
Sep 16 18:44:30.697562 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:44:30.697540    2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-182.ec2.internal\" not found"
Sep 16 18:44:30.798088 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:44:30.798021    2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-182.ec2.internal\" not found"
Sep 16 18:44:30.898818 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:44:30.898791    2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-182.ec2.internal\" not found"
Sep 16 18:44:30.968277 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:30.968243    2584 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 16 18:44:30.968706 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:30.968469    2584 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 16 18:44:30.999758 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:44:30.999729    2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-182.ec2.internal\" not found"
Sep 16 18:44:31.035789 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:31.035747    2584 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-15 18:39:30 +0000 UTC" deadline="2028-03-05 12:52:20.080574047 +0000 UTC"
Sep 16 18:44:31.035789 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:31.035780    2584 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12858h7m49.044797627s"
Sep 16 18:44:31.046349 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:31.046324    2584 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 16 18:44:31.047242 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:31.047212    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod228e51ceefc0d99659afe6cee7fbb736.slice/crio-194ef84196b866b437a1e1f33810c17bbbc89b23a8808b8166f210e9bc1c9ed1 WatchSource:0}: Error finding container 194ef84196b866b437a1e1f33810c17bbbc89b23a8808b8166f210e9bc1c9ed1: Status 404 returned error can't find the container with id 194ef84196b866b437a1e1f33810c17bbbc89b23a8808b8166f210e9bc1c9ed1
Sep 16 18:44:31.059285 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:31.059266    2584 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 16 18:44:31.067875 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:31.067845    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4fa23c77cdd18b7265820cf09e7b3e9d.slice/crio-2b07fd850978235a8183a6f5199d7d385aa3990baadf9f115d7ee3436e5aea76 WatchSource:0}: Error finding container 2b07fd850978235a8183a6f5199d7d385aa3990baadf9f115d7ee3436e5aea76: Status 404 returned error can't find the container with id 2b07fd850978235a8183a6f5199d7d385aa3990baadf9f115d7ee3436e5aea76
Sep 16 18:44:31.113378 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:31.113320    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-182.ec2.internal" event={"ID":"4fa23c77cdd18b7265820cf09e7b3e9d","Type":"ContainerStarted","Data":"2b07fd850978235a8183a6f5199d7d385aa3990baadf9f115d7ee3436e5aea76"}
Sep 16 18:44:31.114526 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:31.114501    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-182.ec2.internal" event={"ID":"228e51ceefc0d99659afe6cee7fbb736","Type":"ContainerStarted","Data":"194ef84196b866b437a1e1f33810c17bbbc89b23a8808b8166f210e9bc1c9ed1"}
Sep 16 18:44:31.121192 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:31.121173    2584 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-182.ec2.internal"
Sep 16 18:44:31.133181 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:31.133162    2584 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 16 18:44:31.134916 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:31.134894    2584 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-182.ec2.internal"
Sep 16 18:44:31.144228 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:31.144209    2584 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 16 18:44:31.157209 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:31.157190    2584 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 16 18:44:31.200976 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:31.200951    2584 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 16 18:44:32.004964 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.004922    2584 apiserver.go:52] "Watching apiserver"
Sep 16 18:44:32.013703 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.013676    2584 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 16 18:44:32.014114 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.014089    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-6klb4","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gpwqt","openshift-cluster-node-tuning-operator/tuned-9z8sh","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-182.ec2.internal","openshift-multus/multus-additional-cni-plugins-fbw2l","openshift-multus/multus-d9bgl","openshift-network-diagnostics/network-check-target-f4md6","openshift-network-operator/iptables-alerter-qqjgk","kube-system/global-pull-secret-syncer-t69xl","kube-system/kube-apiserver-proxy-ip-10-0-139-182.ec2.internal","openshift-dns/node-resolver-bd22w","openshift-image-registry/node-ca-f8jpf","openshift-multus/network-metrics-daemon-bp5mv","openshift-ovn-kubernetes/ovnkube-node-blv9h"]
Sep 16 18:44:32.017150 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.017124    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bp5mv"
Sep 16 18:44:32.017265 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:44:32.017207    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bp5mv" podUID="9b96048a-0005-410b-a710-909d21fbfde8"
Sep 16 18:44:32.019400 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.019376    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gpwqt"
Sep 16 18:44:32.021013 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.020992    2584 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 16 18:44:32.021823 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.021786    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-9z8sh"
Sep 16 18:44:32.021999 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.021932    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 16 18:44:32.022410 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.022389    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 16 18:44:32.022497 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.022391    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-ppz9c\""
Sep 16 18:44:32.022497 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.022449    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 16 18:44:32.024052 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.024015    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-6klb4"
Sep 16 18:44:32.024514 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.024494    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-zfq85\""
Sep 16 18:44:32.024514 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.024502    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 16 18:44:32.024938 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.024917    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 16 18:44:32.026795 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.026772    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 16 18:44:32.026908 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.026779    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 16 18:44:32.026908 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.026821    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-tdrlg\""
Sep 16 18:44:32.028279 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.028244    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-fbw2l"
Sep 16 18:44:32.030522 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.030470    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s7b79\" (UniqueName: \"kubernetes.io/projected/7892f55c-beff-42d9-989e-52eeb778bdc5-kube-api-access-s7b79\") pod \"tuned-9z8sh\" (UID: \"7892f55c-beff-42d9-989e-52eeb778bdc5\") " pod="openshift-cluster-node-tuning-operator/tuned-9z8sh"
Sep 16 18:44:32.030522 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.030501    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ac357968-eba1-4c7d-97a7-b1a8e691e22e-device-dir\") pod \"aws-ebs-csi-driver-node-gpwqt\" (UID: \"ac357968-eba1-4c7d-97a7-b1a8e691e22e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gpwqt"
Sep 16 18:44:32.030687 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.030527    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ac357968-eba1-4c7d-97a7-b1a8e691e22e-etc-selinux\") pod \"aws-ebs-csi-driver-node-gpwqt\" (UID: \"ac357968-eba1-4c7d-97a7-b1a8e691e22e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gpwqt"
Sep 16 18:44:32.030687 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.030551    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s2sqd\" (UniqueName: \"kubernetes.io/projected/ac357968-eba1-4c7d-97a7-b1a8e691e22e-kube-api-access-s2sqd\") pod \"aws-ebs-csi-driver-node-gpwqt\" (UID: \"ac357968-eba1-4c7d-97a7-b1a8e691e22e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gpwqt"
Sep 16 18:44:32.030687 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.030577    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/7892f55c-beff-42d9-989e-52eeb778bdc5-etc-modprobe-d\") pod \"tuned-9z8sh\" (UID: \"7892f55c-beff-42d9-989e-52eeb778bdc5\") " pod="openshift-cluster-node-tuning-operator/tuned-9z8sh"
Sep 16 18:44:32.030687 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.030584    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-d9bgl"
Sep 16 18:44:32.030687 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.030599    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/7892f55c-beff-42d9-989e-52eeb778bdc5-etc-sysctl-conf\") pod \"tuned-9z8sh\" (UID: \"7892f55c-beff-42d9-989e-52eeb778bdc5\") " pod="openshift-cluster-node-tuning-operator/tuned-9z8sh"
Sep 16 18:44:32.030687 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.030623    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7892f55c-beff-42d9-989e-52eeb778bdc5-var-lib-kubelet\") pod \"tuned-9z8sh\" (UID: \"7892f55c-beff-42d9-989e-52eeb778bdc5\") " pod="openshift-cluster-node-tuning-operator/tuned-9z8sh"
Sep 16 18:44:32.030687 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.030646    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/7892f55c-beff-42d9-989e-52eeb778bdc5-etc-tuned\") pod \"tuned-9z8sh\" (UID: \"7892f55c-beff-42d9-989e-52eeb778bdc5\") " pod="openshift-cluster-node-tuning-operator/tuned-9z8sh"
Sep 16 18:44:32.030687 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.030671    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9b96048a-0005-410b-a710-909d21fbfde8-metrics-certs\") pod \"network-metrics-daemon-bp5mv\" (UID: \"9b96048a-0005-410b-a710-909d21fbfde8\") " pod="openshift-multus/network-metrics-daemon-bp5mv"
Sep 16 18:44:32.031121 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.030695    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/7892f55c-beff-42d9-989e-52eeb778bdc5-run\") pod \"tuned-9z8sh\" (UID: \"7892f55c-beff-42d9-989e-52eeb778bdc5\") " pod="openshift-cluster-node-tuning-operator/tuned-9z8sh"
Sep 16 18:44:32.031121 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.030721    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7892f55c-beff-42d9-989e-52eeb778bdc5-lib-modules\") pod \"tuned-9z8sh\" (UID: \"7892f55c-beff-42d9-989e-52eeb778bdc5\") " pod="openshift-cluster-node-tuning-operator/tuned-9z8sh"
Sep 16 18:44:32.031121 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.030743    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7892f55c-beff-42d9-989e-52eeb778bdc5-tmp\") pod \"tuned-9z8sh\" (UID: \"7892f55c-beff-42d9-989e-52eeb778bdc5\") " pod="openshift-cluster-node-tuning-operator/tuned-9z8sh"
Sep 16 18:44:32.031121 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.030766    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s782z\" (UniqueName: \"kubernetes.io/projected/9b96048a-0005-410b-a710-909d21fbfde8-kube-api-access-s782z\") pod \"network-metrics-daemon-bp5mv\" (UID: \"9b96048a-0005-410b-a710-909d21fbfde8\") " pod="openshift-multus/network-metrics-daemon-bp5mv"
Sep 16 18:44:32.031121 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.030790    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ac357968-eba1-4c7d-97a7-b1a8e691e22e-socket-dir\") pod \"aws-ebs-csi-driver-node-gpwqt\" (UID: \"ac357968-eba1-4c7d-97a7-b1a8e691e22e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gpwqt"
Sep 16 18:44:32.031121 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.030815    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ac357968-eba1-4c7d-97a7-b1a8e691e22e-registration-dir\") pod \"aws-ebs-csi-driver-node-gpwqt\" (UID: \"ac357968-eba1-4c7d-97a7-b1a8e691e22e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gpwqt"
Sep 16 18:44:32.031121 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.030837    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/7892f55c-beff-42d9-989e-52eeb778bdc5-etc-sysconfig\") pod \"tuned-9z8sh\" (UID: \"7892f55c-beff-42d9-989e-52eeb778bdc5\") " pod="openshift-cluster-node-tuning-operator/tuned-9z8sh"
Sep 16 18:44:32.031121 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.030844    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 16 18:44:32.031121 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.030859    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7892f55c-beff-42d9-989e-52eeb778bdc5-etc-kubernetes\") pod \"tuned-9z8sh\" (UID: \"7892f55c-beff-42d9-989e-52eeb778bdc5\") " pod="openshift-cluster-node-tuning-operator/tuned-9z8sh"
Sep 16 18:44:32.031121 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.031059    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-mjbtd\""
Sep 16 18:44:32.031121 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.031093    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/7892f55c-beff-42d9-989e-52eeb778bdc5-etc-sysctl-d\") pod \"tuned-9z8sh\" (UID: \"7892f55c-beff-42d9-989e-52eeb778bdc5\") " pod="openshift-cluster-node-tuning-operator/tuned-9z8sh"
Sep 16 18:44:32.031121 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.031122    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/7892f55c-beff-42d9-989e-52eeb778bdc5-etc-systemd\") pod \"tuned-9z8sh\" (UID: \"7892f55c-beff-42d9-989e-52eeb778bdc5\") " pod="openshift-cluster-node-tuning-operator/tuned-9z8sh"
Sep 16 18:44:32.031658 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.031150    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7892f55c-beff-42d9-989e-52eeb778bdc5-host\") pod \"tuned-9z8sh\" (UID: \"7892f55c-beff-42d9-989e-52eeb778bdc5\") " pod="openshift-cluster-node-tuning-operator/tuned-9z8sh"
Sep 16 18:44:32.031658 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.031177    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ac357968-eba1-4c7d-97a7-b1a8e691e22e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-gpwqt\" (UID: \"ac357968-eba1-4c7d-97a7-b1a8e691e22e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gpwqt"
Sep 16 18:44:32.031658 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.031200    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ac357968-eba1-4c7d-97a7-b1a8e691e22e-sys-fs\") pod \"aws-ebs-csi-driver-node-gpwqt\" (UID: \"ac357968-eba1-4c7d-97a7-b1a8e691e22e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gpwqt"
Sep 16 18:44:32.031658 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.031219    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 16 18:44:32.031658 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.031223    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7892f55c-beff-42d9-989e-52eeb778bdc5-sys\") pod \"tuned-9z8sh\" (UID: \"7892f55c-beff-42d9-989e-52eeb778bdc5\") " pod="openshift-cluster-node-tuning-operator/tuned-9z8sh"
Sep 16 18:44:32.031658 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.031430    2584 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 16 18:44:32.031658 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.031452    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 16 18:44:32.032063 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.032039    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 16 18:44:32.032169 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.032086    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 16 18:44:32.034393 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.033365    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 16 18:44:32.034393 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.033426    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-qt468\""
Sep 16 18:44:32.035017 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.034997    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f4md6"
Sep 16 18:44:32.035116 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:44:32.035071    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f4md6" podUID="df3c2b31-63a6-43e4-a4ae-0199c34a6a86"
Sep 16 18:44:32.037443 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.037423    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-qqjgk"
Sep 16 18:44:32.040455 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.040428    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-p6fsc\""
Sep 16 18:44:32.040571 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.040451    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 16 18:44:32.040571 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.040496    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 16 18:44:32.040688 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.040621    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 16 18:44:32.040747 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.040704    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t69xl"
Sep 16 18:44:32.040796 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:44:32.040761    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t69xl" podUID="b9f0f3b9-87ee-44a5-a66c-6542791f3506"
Sep 16 18:44:32.043299 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.043274    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-blv9h"
Sep 16 18:44:32.046095 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.045826    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-bd22w"
Sep 16 18:44:32.046324 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.046306    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 16 18:44:32.047457 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.047433    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 16 18:44:32.047553 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.047476    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-b52d8\""
Sep 16 18:44:32.047553 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.047476    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 16 18:44:32.047553 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.047542    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 16 18:44:32.047713 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.047553    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 16 18:44:32.047713 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.047500    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 16 18:44:32.048440 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.048419    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-f8jpf"
Sep 16 18:44:32.048796 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.048778    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 16 18:44:32.048796 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.048789    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 16 18:44:32.049014 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.048781    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-gqvgp\""
Sep 16 18:44:32.051096 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.051076    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-8bzs6\""
Sep 16 18:44:32.051202 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.051159    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 16 18:44:32.051202 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.051181    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 16 18:44:32.051306 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.051208    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 16 18:44:32.059744 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.059711    2584 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-z72g6"
Sep 16 18:44:32.068195 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.068171    2584 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-z72g6"
Sep 16 18:44:32.121550 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.121522    2584 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 16 18:44:32.132089 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.132063    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1498d81a-223d-470c-9812-9b2910e3ea35-serviceca\") pod \"node-ca-f8jpf\" (UID: \"1498d81a-223d-470c-9812-9b2910e3ea35\") " pod="openshift-image-registry/node-ca-f8jpf"
Sep 16 18:44:32.132225 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.132102    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9b96048a-0005-410b-a710-909d21fbfde8-metrics-certs\") pod \"network-metrics-daemon-bp5mv\" (UID: \"9b96048a-0005-410b-a710-909d21fbfde8\") " pod="openshift-multus/network-metrics-daemon-bp5mv"
Sep 16 18:44:32.132225 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.132125    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4a80b1e8-39b4-44a2-80cb-aa52d96e80fb-hosts-file\") pod \"node-resolver-bd22w\" (UID: \"4a80b1e8-39b4-44a2-80cb-aa52d96e80fb\") " pod="openshift-dns/node-resolver-bd22w"
Sep 16 18:44:32.132225 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.132149    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c42d9b1b-2d14-469c-b986-ae99600b602c-multus-cni-dir\") pod \"multus-d9bgl\" (UID: \"c42d9b1b-2d14-469c-b986-ae99600b602c\") " pod="openshift-multus/multus-d9bgl"
Sep 16 18:44:32.132225 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.132170    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/08febeb1-4769-4fd6-911f-a0a02c7c4223-etc-openvswitch\") pod \"ovnkube-node-blv9h\" (UID: \"08febeb1-4769-4fd6-911f-a0a02c7c4223\") " pod="openshift-ovn-kubernetes/ovnkube-node-blv9h"
Sep 16 18:44:32.132411 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:44:32.132271    2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 18:44:32.132411 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.132284    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/08febeb1-4769-4fd6-911f-a0a02c7c4223-run-ovn\") pod \"ovnkube-node-blv9h\" (UID: \"08febeb1-4769-4fd6-911f-a0a02c7c4223\") " pod="openshift-ovn-kubernetes/ovnkube-node-blv9h"
Sep 16 18:44:32.132411 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.132335    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/08febeb1-4769-4fd6-911f-a0a02c7c4223-ovnkube-config\") pod \"ovnkube-node-blv9h\" (UID: \"08febeb1-4769-4fd6-911f-a0a02c7c4223\") " pod="openshift-ovn-kubernetes/ovnkube-node-blv9h"
Sep 16 18:44:32.132411 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:44:32.132351    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9b96048a-0005-410b-a710-909d21fbfde8-metrics-certs podName:9b96048a-0005-410b-a710-909d21fbfde8 nodeName:}" failed. No retries permitted until 2026-09-16 18:44:32.632330185 +0000 UTC m=+2.961832600 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9b96048a-0005-410b-a710-909d21fbfde8-metrics-certs") pod "network-metrics-daemon-bp5mv" (UID: "9b96048a-0005-410b-a710-909d21fbfde8") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 18:44:32.132607 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.132421    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7892f55c-beff-42d9-989e-52eeb778bdc5-etc-kubernetes\") pod \"tuned-9z8sh\" (UID: \"7892f55c-beff-42d9-989e-52eeb778bdc5\") " pod="openshift-cluster-node-tuning-operator/tuned-9z8sh"
Sep 16 18:44:32.132607 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.132454    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l4z5w\" (UniqueName: \"kubernetes.io/projected/df3c2b31-63a6-43e4-a4ae-0199c34a6a86-kube-api-access-l4z5w\") pod \"network-check-target-f4md6\" (UID: \"df3c2b31-63a6-43e4-a4ae-0199c34a6a86\") " pod="openshift-network-diagnostics/network-check-target-f4md6"
Sep 16 18:44:32.132607 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.132483    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/08febeb1-4769-4fd6-911f-a0a02c7c4223-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-blv9h\" (UID: \"08febeb1-4769-4fd6-911f-a0a02c7c4223\") " pod="openshift-ovn-kubernetes/ovnkube-node-blv9h"
Sep 16 18:44:32.132607 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.132510    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ac357968-eba1-4c7d-97a7-b1a8e691e22e-sys-fs\") pod \"aws-ebs-csi-driver-node-gpwqt\" (UID: \"ac357968-eba1-4c7d-97a7-b1a8e691e22e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gpwqt"
Sep 16 18:44:32.132607 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.132536    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c42d9b1b-2d14-469c-b986-ae99600b602c-cnibin\") pod \"multus-d9bgl\" (UID: \"c42d9b1b-2d14-469c-b986-ae99600b602c\") " pod="openshift-multus/multus-d9bgl"
Sep 16 18:44:32.132607 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.132560    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c42d9b1b-2d14-469c-b986-ae99600b602c-host-run-k8s-cni-cncf-io\") pod \"multus-d9bgl\" (UID: \"c42d9b1b-2d14-469c-b986-ae99600b602c\") " pod="openshift-multus/multus-d9bgl"
Sep 16 18:44:32.132607 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.132587    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c42d9b1b-2d14-469c-b986-ae99600b602c-host-run-multus-certs\") pod \"multus-d9bgl\" (UID: \"c42d9b1b-2d14-469c-b986-ae99600b602c\") " pod="openshift-multus/multus-d9bgl"
Sep 16 18:44:32.132607 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.132598    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7892f55c-beff-42d9-989e-52eeb778bdc5-etc-kubernetes\") pod \"tuned-9z8sh\" (UID: \"7892f55c-beff-42d9-989e-52eeb778bdc5\") " pod="openshift-cluster-node-tuning-operator/tuned-9z8sh"
Sep 16 18:44:32.132967 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.132613    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b9f0f3b9-87ee-44a5-a66c-6542791f3506-original-pull-secret\") pod \"global-pull-secret-syncer-t69xl\" (UID: \"b9f0f3b9-87ee-44a5-a66c-6542791f3506\") " pod="kube-system/global-pull-secret-syncer-t69xl"
Sep 16 18:44:32.132967 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.132638    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/08febeb1-4769-4fd6-911f-a0a02c7c4223-systemd-units\") pod \"ovnkube-node-blv9h\" (UID: \"08febeb1-4769-4fd6-911f-a0a02c7c4223\") " pod="openshift-ovn-kubernetes/ovnkube-node-blv9h"
Sep 16 18:44:32.132967 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.132646    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ac357968-eba1-4c7d-97a7-b1a8e691e22e-sys-fs\") pod \"aws-ebs-csi-driver-node-gpwqt\" (UID: \"ac357968-eba1-4c7d-97a7-b1a8e691e22e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gpwqt"
Sep 16 18:44:32.132967 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.132692    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/08febeb1-4769-4fd6-911f-a0a02c7c4223-ovn-node-metrics-cert\") pod \"ovnkube-node-blv9h\" (UID: \"08febeb1-4769-4fd6-911f-a0a02c7c4223\") " pod="openshift-ovn-kubernetes/ovnkube-node-blv9h"
Sep 16 18:44:32.132967 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.132747    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s2sqd\" (UniqueName: \"kubernetes.io/projected/ac357968-eba1-4c7d-97a7-b1a8e691e22e-kube-api-access-s2sqd\") pod \"aws-ebs-csi-driver-node-gpwqt\" (UID: \"ac357968-eba1-4c7d-97a7-b1a8e691e22e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gpwqt"
Sep 16 18:44:32.132967 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.132776    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7892f55c-beff-42d9-989e-52eeb778bdc5-var-lib-kubelet\") pod \"tuned-9z8sh\" (UID: \"7892f55c-beff-42d9-989e-52eeb778bdc5\") " pod="openshift-cluster-node-tuning-operator/tuned-9z8sh"
Sep 16 18:44:32.132967 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.132845    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/71637102-fba0-4c38-8055-6dceacddcde9-system-cni-dir\") pod \"multus-additional-cni-plugins-fbw2l\" (UID: \"71637102-fba0-4c38-8055-6dceacddcde9\") " pod="openshift-multus/multus-additional-cni-plugins-fbw2l"
Sep 16 18:44:32.132967 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.132878    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c42d9b1b-2d14-469c-b986-ae99600b602c-cni-binary-copy\") pod \"multus-d9bgl\" (UID: \"c42d9b1b-2d14-469c-b986-ae99600b602c\") " pod="openshift-multus/multus-d9bgl"
Sep 16 18:44:32.132967 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.132877    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7892f55c-beff-42d9-989e-52eeb778bdc5-var-lib-kubelet\") pod \"tuned-9z8sh\" (UID: \"7892f55c-beff-42d9-989e-52eeb778bdc5\") " pod="openshift-cluster-node-tuning-operator/tuned-9z8sh"
Sep 16 18:44:32.132967 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.132910    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7892f55c-beff-42d9-989e-52eeb778bdc5-lib-modules\") pod \"tuned-9z8sh\" (UID: \"7892f55c-beff-42d9-989e-52eeb778bdc5\") " pod="openshift-cluster-node-tuning-operator/tuned-9z8sh"
Sep 16 18:44:32.132967 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.132943    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c42d9b1b-2d14-469c-b986-ae99600b602c-hostroot\") pod \"multus-d9bgl\" (UID: \"c42d9b1b-2d14-469c-b986-ae99600b602c\") " pod="openshift-multus/multus-d9bgl"
Sep 16 18:44:32.133383 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.133004    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a29102aa-22ea-4623-84fc-0be1e5506274-host-slash\") pod \"iptables-alerter-qqjgk\" (UID: \"a29102aa-22ea-4623-84fc-0be1e5506274\") " pod="openshift-network-operator/iptables-alerter-qqjgk"
Sep 16 18:44:32.133383 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.133026    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7km7r\" (UniqueName: \"kubernetes.io/projected/a29102aa-22ea-4623-84fc-0be1e5506274-kube-api-access-7km7r\") pod \"iptables-alerter-qqjgk\" (UID: \"a29102aa-22ea-4623-84fc-0be1e5506274\") " pod="openshift-network-operator/iptables-alerter-qqjgk"
Sep 16 18:44:32.133383 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.133046    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/08febeb1-4769-4fd6-911f-a0a02c7c4223-node-log\") pod \"ovnkube-node-blv9h\" (UID: \"08febeb1-4769-4fd6-911f-a0a02c7c4223\") " pod="openshift-ovn-kubernetes/ovnkube-node-blv9h"
Sep 16 18:44:32.133383 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.133062    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a29102aa-22ea-4623-84fc-0be1e5506274-iptables-alerter-script\") pod \"iptables-alerter-qqjgk\" (UID: \"a29102aa-22ea-4623-84fc-0be1e5506274\") " pod="openshift-network-operator/iptables-alerter-qqjgk"
Sep 16 18:44:32.133383 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.133089    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s782z\" (UniqueName: \"kubernetes.io/projected/9b96048a-0005-410b-a710-909d21fbfde8-kube-api-access-s782z\") pod \"network-metrics-daemon-bp5mv\" (UID: \"9b96048a-0005-410b-a710-909d21fbfde8\") " pod="openshift-multus/network-metrics-daemon-bp5mv"
Sep 16 18:44:32.133383 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.133122    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/7892f55c-beff-42d9-989e-52eeb778bdc5-etc-sysconfig\") pod \"tuned-9z8sh\" (UID: \"7892f55c-beff-42d9-989e-52eeb778bdc5\") " pod="openshift-cluster-node-tuning-operator/tuned-9z8sh"
Sep 16 18:44:32.133383 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.133128    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7892f55c-beff-42d9-989e-52eeb778bdc5-lib-modules\") pod \"tuned-9z8sh\" (UID: \"7892f55c-beff-42d9-989e-52eeb778bdc5\") " pod="openshift-cluster-node-tuning-operator/tuned-9z8sh"
Sep 16 18:44:32.133383 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.133227    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/7892f55c-beff-42d9-989e-52eeb778bdc5-etc-systemd\") pod \"tuned-9z8sh\" (UID: \"7892f55c-beff-42d9-989e-52eeb778bdc5\") " pod="openshift-cluster-node-tuning-operator/tuned-9z8sh"
Sep 16 18:44:32.133383 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.133258    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/828d6bf1-3d76-4c9b-af83-07de04f8f20f-konnectivity-ca\") pod \"konnectivity-agent-6klb4\" (UID: \"828d6bf1-3d76-4c9b-af83-07de04f8f20f\") " pod="kube-system/konnectivity-agent-6klb4"
Sep 16 18:44:32.133383 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.133319    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/7892f55c-beff-42d9-989e-52eeb778bdc5-etc-sysconfig\") pod \"tuned-9z8sh\" (UID: \"7892f55c-beff-42d9-989e-52eeb778bdc5\") " pod="openshift-cluster-node-tuning-operator/tuned-9z8sh"
Sep 16 18:44:32.133383 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.133328    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/7892f55c-beff-42d9-989e-52eeb778bdc5-etc-systemd\") pod \"tuned-9z8sh\" (UID: \"7892f55c-beff-42d9-989e-52eeb778bdc5\") " pod="openshift-cluster-node-tuning-operator/tuned-9z8sh"
Sep 16 18:44:32.133383 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.133325    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/71637102-fba0-4c38-8055-6dceacddcde9-cni-binary-copy\") pod \"multus-additional-cni-plugins-fbw2l\" (UID: \"71637102-fba0-4c38-8055-6dceacddcde9\") " pod="openshift-multus/multus-additional-cni-plugins-fbw2l"
Sep 16 18:44:32.133383 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.133359    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sc9zn\" (UniqueName: \"kubernetes.io/projected/4a80b1e8-39b4-44a2-80cb-aa52d96e80fb-kube-api-access-sc9zn\") pod \"node-resolver-bd22w\" (UID: \"4a80b1e8-39b4-44a2-80cb-aa52d96e80fb\") " pod="openshift-dns/node-resolver-bd22w"
Sep 16 18:44:32.133383 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.133379    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/08febeb1-4769-4fd6-911f-a0a02c7c4223-env-overrides\") pod \"ovnkube-node-blv9h\" (UID: \"08febeb1-4769-4fd6-911f-a0a02c7c4223\") " pod="openshift-ovn-kubernetes/ovnkube-node-blv9h"
Sep 16 18:44:32.134045 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.133408    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/828d6bf1-3d76-4c9b-af83-07de04f8f20f-agent-certs\") pod \"konnectivity-agent-6klb4\" (UID: \"828d6bf1-3d76-4c9b-af83-07de04f8f20f\") " pod="kube-system/konnectivity-agent-6klb4"
Sep 16 18:44:32.134045 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.133429    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c42d9b1b-2d14-469c-b986-ae99600b602c-system-cni-dir\") pod \"multus-d9bgl\" (UID: \"c42d9b1b-2d14-469c-b986-ae99600b602c\") " pod="openshift-multus/multus-d9bgl"
Sep 16 18:44:32.134045 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.133448    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c42d9b1b-2d14-469c-b986-ae99600b602c-host-var-lib-cni-bin\") pod \"multus-d9bgl\" (UID: \"c42d9b1b-2d14-469c-b986-ae99600b602c\") " pod="openshift-multus/multus-d9bgl"
Sep 16 18:44:32.134045 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.133463    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c42d9b1b-2d14-469c-b986-ae99600b602c-multus-daemon-config\") pod \"multus-d9bgl\" (UID: \"c42d9b1b-2d14-469c-b986-ae99600b602c\") " pod="openshift-multus/multus-d9bgl"
Sep 16 18:44:32.134045 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.133482    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b9f0f3b9-87ee-44a5-a66c-6542791f3506-dbus\") pod \"global-pull-secret-syncer-t69xl\" (UID: \"b9f0f3b9-87ee-44a5-a66c-6542791f3506\") " pod="kube-system/global-pull-secret-syncer-t69xl"
Sep 16 18:44:32.134045 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.133521    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/08febeb1-4769-4fd6-911f-a0a02c7c4223-log-socket\") pod \"ovnkube-node-blv9h\" (UID: \"08febeb1-4769-4fd6-911f-a0a02c7c4223\") " pod="openshift-ovn-kubernetes/ovnkube-node-blv9h"
Sep 16 18:44:32.134045 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.133544    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/08febeb1-4769-4fd6-911f-a0a02c7c4223-host-cni-bin\") pod \"ovnkube-node-blv9h\" (UID: \"08febeb1-4769-4fd6-911f-a0a02c7c4223\") " pod="openshift-ovn-kubernetes/ovnkube-node-blv9h"
Sep 16 18:44:32.134045 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.133564    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dd7rl\" (UniqueName: \"kubernetes.io/projected/1498d81a-223d-470c-9812-9b2910e3ea35-kube-api-access-dd7rl\") pod \"node-ca-f8jpf\" (UID: \"1498d81a-223d-470c-9812-9b2910e3ea35\") " pod="openshift-image-registry/node-ca-f8jpf"
Sep 16 18:44:32.134045 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.133600    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ac357968-eba1-4c7d-97a7-b1a8e691e22e-etc-selinux\") pod \"aws-ebs-csi-driver-node-gpwqt\" (UID: \"ac357968-eba1-4c7d-97a7-b1a8e691e22e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gpwqt"
Sep 16 18:44:32.134045 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.133621    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/7892f55c-beff-42d9-989e-52eeb778bdc5-etc-modprobe-d\") pod \"tuned-9z8sh\" (UID: \"7892f55c-beff-42d9-989e-52eeb778bdc5\") " pod="openshift-cluster-node-tuning-operator/tuned-9z8sh"
Sep 16 18:44:32.134045 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.133636    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/7892f55c-beff-42d9-989e-52eeb778bdc5-etc-sysctl-conf\") pod \"tuned-9z8sh\" (UID: \"7892f55c-beff-42d9-989e-52eeb778bdc5\") " pod="openshift-cluster-node-tuning-operator/tuned-9z8sh"
Sep 16 18:44:32.134045 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.133659    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/7892f55c-beff-42d9-989e-52eeb778bdc5-etc-tuned\") pod \"tuned-9z8sh\" (UID: \"7892f55c-beff-42d9-989e-52eeb778bdc5\") " pod="openshift-cluster-node-tuning-operator/tuned-9z8sh"
Sep 16 18:44:32.134045 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.133674    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7892f55c-beff-42d9-989e-52eeb778bdc5-tmp\") pod \"tuned-9z8sh\" (UID: \"7892f55c-beff-42d9-989e-52eeb778bdc5\") " pod="openshift-cluster-node-tuning-operator/tuned-9z8sh"
Sep 16 18:44:32.134045 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.133695    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/71637102-fba0-4c38-8055-6dceacddcde9-tuning-conf-dir\") pod \"multus-additional-cni-plugins-fbw2l\" (UID: \"71637102-fba0-4c38-8055-6dceacddcde9\") " pod="openshift-multus/multus-additional-cni-plugins-fbw2l"
Sep 16 18:44:32.134045 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.133709    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ac357968-eba1-4c7d-97a7-b1a8e691e22e-etc-selinux\") pod \"aws-ebs-csi-driver-node-gpwqt\" (UID: \"ac357968-eba1-4c7d-97a7-b1a8e691e22e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gpwqt"
Sep 16 18:44:32.134045 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.133718    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/08febeb1-4769-4fd6-911f-a0a02c7c4223-run-openvswitch\") pod \"ovnkube-node-blv9h\" (UID: \"08febeb1-4769-4fd6-911f-a0a02c7c4223\") " pod="openshift-ovn-kubernetes/ovnkube-node-blv9h"
Sep 16 18:44:32.134045 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.133765    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ac357968-eba1-4c7d-97a7-b1a8e691e22e-socket-dir\") pod \"aws-ebs-csi-driver-node-gpwqt\" (UID: \"ac357968-eba1-4c7d-97a7-b1a8e691e22e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gpwqt"
Sep 16 18:44:32.134794 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.133790    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ac357968-eba1-4c7d-97a7-b1a8e691e22e-registration-dir\") pod \"aws-ebs-csi-driver-node-gpwqt\" (UID: \"ac357968-eba1-4c7d-97a7-b1a8e691e22e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gpwqt"
Sep 16 18:44:32.134794 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.133814    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7892f55c-beff-42d9-989e-52eeb778bdc5-host\") pod \"tuned-9z8sh\" (UID: \"7892f55c-beff-42d9-989e-52eeb778bdc5\") " pod="openshift-cluster-node-tuning-operator/tuned-9z8sh"
Sep 16 18:44:32.134794 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.133855    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bbdbf\" (UniqueName: \"kubernetes.io/projected/c42d9b1b-2d14-469c-b986-ae99600b602c-kube-api-access-bbdbf\") pod \"multus-d9bgl\" (UID: \"c42d9b1b-2d14-469c-b986-ae99600b602c\") " pod="openshift-multus/multus-d9bgl"
Sep 16 18:44:32.134794 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.133886    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/08febeb1-4769-4fd6-911f-a0a02c7c4223-ovnkube-script-lib\") pod \"ovnkube-node-blv9h\" (UID: \"08febeb1-4769-4fd6-911f-a0a02c7c4223\") " pod="openshift-ovn-kubernetes/ovnkube-node-blv9h"
Sep 16 18:44:32.134794 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.133912    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ac357968-eba1-4c7d-97a7-b1a8e691e22e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-gpwqt\" (UID: \"ac357968-eba1-4c7d-97a7-b1a8e691e22e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gpwqt"
Sep 16 18:44:32.134794 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.133918    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/7892f55c-beff-42d9-989e-52eeb778bdc5-etc-modprobe-d\") pod \"tuned-9z8sh\" (UID: \"7892f55c-beff-42d9-989e-52eeb778bdc5\") " pod="openshift-cluster-node-tuning-operator/tuned-9z8sh"
Sep 16 18:44:32.134794 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.133937    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4a80b1e8-39b4-44a2-80cb-aa52d96e80fb-tmp-dir\") pod \"node-resolver-bd22w\" (UID: \"4a80b1e8-39b4-44a2-80cb-aa52d96e80fb\") " pod="openshift-dns/node-resolver-bd22w"
Sep 16 18:44:32.134794 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.133806    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/7892f55c-beff-42d9-989e-52eeb778bdc5-etc-sysctl-conf\") pod \"tuned-9z8sh\" (UID: \"7892f55c-beff-42d9-989e-52eeb778bdc5\") " pod="openshift-cluster-node-tuning-operator/tuned-9z8sh"
Sep 16 18:44:32.134794 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.133961    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c42d9b1b-2d14-469c-b986-ae99600b602c-host-run-netns\") pod \"multus-d9bgl\" (UID: \"c42d9b1b-2d14-469c-b986-ae99600b602c\") " pod="openshift-multus/multus-d9bgl"
Sep 16 18:44:32.134794 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.133984    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b9f0f3b9-87ee-44a5-a66c-6542791f3506-kubelet-config\") pod \"global-pull-secret-syncer-t69xl\" (UID: \"b9f0f3b9-87ee-44a5-a66c-6542791f3506\") " pod="kube-system/global-pull-secret-syncer-t69xl"
Sep 16 18:44:32.134794 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.134009    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ac357968-eba1-4c7d-97a7-b1a8e691e22e-device-dir\") pod \"aws-ebs-csi-driver-node-gpwqt\" (UID: \"ac357968-eba1-4c7d-97a7-b1a8e691e22e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gpwqt"
Sep 16 18:44:32.134794 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.134032    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c42d9b1b-2d14-469c-b986-ae99600b602c-host-var-lib-cni-multus\") pod \"multus-d9bgl\" (UID: \"c42d9b1b-2d14-469c-b986-ae99600b602c\") " pod="openshift-multus/multus-d9bgl"
Sep 16 18:44:32.134794 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.134059    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c42d9b1b-2d14-469c-b986-ae99600b602c-host-var-lib-kubelet\") pod \"multus-d9bgl\" (UID: \"c42d9b1b-2d14-469c-b986-ae99600b602c\") " pod="openshift-multus/multus-d9bgl"
Sep 16 18:44:32.134794 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.134064    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ac357968-eba1-4c7d-97a7-b1a8e691e22e-socket-dir\") pod \"aws-ebs-csi-driver-node-gpwqt\" (UID: \"ac357968-eba1-4c7d-97a7-b1a8e691e22e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gpwqt"
Sep 16 18:44:32.134794 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.134057    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ac357968-eba1-4c7d-97a7-b1a8e691e22e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-gpwqt\" (UID: \"ac357968-eba1-4c7d-97a7-b1a8e691e22e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gpwqt"
Sep 16 18:44:32.134794 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.134085    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/08febeb1-4769-4fd6-911f-a0a02c7c4223-var-lib-openvswitch\") pod \"ovnkube-node-blv9h\" (UID: \"08febeb1-4769-4fd6-911f-a0a02c7c4223\") " pod="openshift-ovn-kubernetes/ovnkube-node-blv9h"
Sep 16 18:44:32.134794 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.134093    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ac357968-eba1-4c7d-97a7-b1a8e691e22e-registration-dir\") pod \"aws-ebs-csi-driver-node-gpwqt\" (UID: \"ac357968-eba1-4c7d-97a7-b1a8e691e22e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gpwqt"
Sep 16 18:44:32.135526 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.134112    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/7892f55c-beff-42d9-989e-52eeb778bdc5-run\") pod \"tuned-9z8sh\" (UID: \"7892f55c-beff-42d9-989e-52eeb778bdc5\") " pod="openshift-cluster-node-tuning-operator/tuned-9z8sh"
Sep 16 18:44:32.135526 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.134130    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7892f55c-beff-42d9-989e-52eeb778bdc5-host\") pod \"tuned-9z8sh\" (UID: \"7892f55c-beff-42d9-989e-52eeb778bdc5\") " pod="openshift-cluster-node-tuning-operator/tuned-9z8sh"
Sep 16 18:44:32.135526 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.134138    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/71637102-fba0-4c38-8055-6dceacddcde9-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-fbw2l\" (UID: \"71637102-fba0-4c38-8055-6dceacddcde9\") " pod="openshift-multus/multus-additional-cni-plugins-fbw2l"
Sep 16 18:44:32.135526 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.134188    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/7892f55c-beff-42d9-989e-52eeb778bdc5-run\") pod \"tuned-9z8sh\" (UID: \"7892f55c-beff-42d9-989e-52eeb778bdc5\") " pod="openshift-cluster-node-tuning-operator/tuned-9z8sh"
Sep 16 18:44:32.135526 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.134211    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/08febeb1-4769-4fd6-911f-a0a02c7c4223-host-kubelet\") pod \"ovnkube-node-blv9h\" (UID: \"08febeb1-4769-4fd6-911f-a0a02c7c4223\") " pod="openshift-ovn-kubernetes/ovnkube-node-blv9h"
Sep 16 18:44:32.135526 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.134225    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ac357968-eba1-4c7d-97a7-b1a8e691e22e-device-dir\") pod \"aws-ebs-csi-driver-node-gpwqt\" (UID: \"ac357968-eba1-4c7d-97a7-b1a8e691e22e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gpwqt"
Sep 16 18:44:32.135526 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.134235    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c42d9b1b-2d14-469c-b986-ae99600b602c-etc-kubernetes\") pod \"multus-d9bgl\" (UID: \"c42d9b1b-2d14-469c-b986-ae99600b602c\") " pod="openshift-multus/multus-d9bgl"
Sep 16 18:44:32.135526 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.134258    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/7892f55c-beff-42d9-989e-52eeb778bdc5-etc-sysctl-d\") pod \"tuned-9z8sh\" (UID: \"7892f55c-beff-42d9-989e-52eeb778bdc5\") " pod="openshift-cluster-node-tuning-operator/tuned-9z8sh"
Sep 16 18:44:32.135526 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.134277    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s7b79\" (UniqueName: \"kubernetes.io/projected/7892f55c-beff-42d9-989e-52eeb778bdc5-kube-api-access-s7b79\") pod \"tuned-9z8sh\" (UID: \"7892f55c-beff-42d9-989e-52eeb778bdc5\") " pod="openshift-cluster-node-tuning-operator/tuned-9z8sh"
Sep 16 18:44:32.135526 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.134308    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c42d9b1b-2d14-469c-b986-ae99600b602c-multus-socket-dir-parent\") pod \"multus-d9bgl\" (UID: \"c42d9b1b-2d14-469c-b986-ae99600b602c\") " pod="openshift-multus/multus-d9bgl"
Sep 16 18:44:32.135526 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.134332    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/08febeb1-4769-4fd6-911f-a0a02c7c4223-host-run-netns\") pod \"ovnkube-node-blv9h\" (UID: \"08febeb1-4769-4fd6-911f-a0a02c7c4223\") " pod="openshift-ovn-kubernetes/ovnkube-node-blv9h"
Sep 16 18:44:32.135526 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.134354    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/08febeb1-4769-4fd6-911f-a0a02c7c4223-host-run-ovn-kubernetes\") pod \"ovnkube-node-blv9h\" (UID: \"08febeb1-4769-4fd6-911f-a0a02c7c4223\") " pod="openshift-ovn-kubernetes/ovnkube-node-blv9h"
Sep 16 18:44:32.135526 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.134382    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7892f55c-beff-42d9-989e-52eeb778bdc5-sys\") pod \"tuned-9z8sh\" (UID: \"7892f55c-beff-42d9-989e-52eeb778bdc5\") " pod="openshift-cluster-node-tuning-operator/tuned-9z8sh"
Sep 16 18:44:32.135526 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.134383    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/7892f55c-beff-42d9-989e-52eeb778bdc5-etc-sysctl-d\") pod \"tuned-9z8sh\" (UID: \"7892f55c-beff-42d9-989e-52eeb778bdc5\") " pod="openshift-cluster-node-tuning-operator/tuned-9z8sh"
Sep 16 18:44:32.135526 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.134409    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/71637102-fba0-4c38-8055-6dceacddcde9-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-fbw2l\" (UID: \"71637102-fba0-4c38-8055-6dceacddcde9\") " pod="openshift-multus/multus-additional-cni-plugins-fbw2l"
Sep 16 18:44:32.135526 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.134447    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bh6kf\" (UniqueName: \"kubernetes.io/projected/71637102-fba0-4c38-8055-6dceacddcde9-kube-api-access-bh6kf\") pod \"multus-additional-cni-plugins-fbw2l\" (UID: \"71637102-fba0-4c38-8055-6dceacddcde9\") " pod="openshift-multus/multus-additional-cni-plugins-fbw2l"
Sep 16 18:44:32.135526 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.134464    2584 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 16 18:44:32.136376 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.134471    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/08febeb1-4769-4fd6-911f-a0a02c7c4223-host-slash\") pod \"ovnkube-node-blv9h\" (UID: \"08febeb1-4769-4fd6-911f-a0a02c7c4223\") " pod="openshift-ovn-kubernetes/ovnkube-node-blv9h"
Sep 16 18:44:32.136376 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.134502    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7892f55c-beff-42d9-989e-52eeb778bdc5-sys\") pod \"tuned-9z8sh\" (UID: \"7892f55c-beff-42d9-989e-52eeb778bdc5\") " pod="openshift-cluster-node-tuning-operator/tuned-9z8sh"
Sep 16 18:44:32.136376 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.134511    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/08febeb1-4769-4fd6-911f-a0a02c7c4223-host-cni-netd\") pod \"ovnkube-node-blv9h\" (UID: \"08febeb1-4769-4fd6-911f-a0a02c7c4223\") " pod="openshift-ovn-kubernetes/ovnkube-node-blv9h"
Sep 16 18:44:32.136376 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.134535    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-js86k\" (UniqueName: \"kubernetes.io/projected/08febeb1-4769-4fd6-911f-a0a02c7c4223-kube-api-access-js86k\") pod \"ovnkube-node-blv9h\" (UID: \"08febeb1-4769-4fd6-911f-a0a02c7c4223\") " pod="openshift-ovn-kubernetes/ovnkube-node-blv9h"
Sep 16 18:44:32.136376 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.134560    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/71637102-fba0-4c38-8055-6dceacddcde9-cnibin\") pod \"multus-additional-cni-plugins-fbw2l\" (UID: \"71637102-fba0-4c38-8055-6dceacddcde9\") " pod="openshift-multus/multus-additional-cni-plugins-fbw2l"
Sep 16 18:44:32.136376 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.134589    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c42d9b1b-2d14-469c-b986-ae99600b602c-multus-conf-dir\") pod \"multus-d9bgl\" (UID: \"c42d9b1b-2d14-469c-b986-ae99600b602c\") " pod="openshift-multus/multus-d9bgl"
Sep 16 18:44:32.136376 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.134611    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/08febeb1-4769-4fd6-911f-a0a02c7c4223-run-systemd\") pod \"ovnkube-node-blv9h\" (UID: \"08febeb1-4769-4fd6-911f-a0a02c7c4223\") " pod="openshift-ovn-kubernetes/ovnkube-node-blv9h"
Sep 16 18:44:32.136376 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.134634    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1498d81a-223d-470c-9812-9b2910e3ea35-host\") pod \"node-ca-f8jpf\" (UID: \"1498d81a-223d-470c-9812-9b2910e3ea35\") " pod="openshift-image-registry/node-ca-f8jpf"
Sep 16 18:44:32.138095 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.138070    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/7892f55c-beff-42d9-989e-52eeb778bdc5-etc-tuned\") pod \"tuned-9z8sh\" (UID: \"7892f55c-beff-42d9-989e-52eeb778bdc5\") " pod="openshift-cluster-node-tuning-operator/tuned-9z8sh"
Sep 16 18:44:32.138223 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.138105    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7892f55c-beff-42d9-989e-52eeb778bdc5-tmp\") pod \"tuned-9z8sh\" (UID: \"7892f55c-beff-42d9-989e-52eeb778bdc5\") " pod="openshift-cluster-node-tuning-operator/tuned-9z8sh"
Sep 16 18:44:32.147312 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.147289    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s2sqd\" (UniqueName: \"kubernetes.io/projected/ac357968-eba1-4c7d-97a7-b1a8e691e22e-kube-api-access-s2sqd\") pod \"aws-ebs-csi-driver-node-gpwqt\" (UID: \"ac357968-eba1-4c7d-97a7-b1a8e691e22e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gpwqt"
Sep 16 18:44:32.148312 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.148283    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s782z\" (UniqueName: \"kubernetes.io/projected/9b96048a-0005-410b-a710-909d21fbfde8-kube-api-access-s782z\") pod \"network-metrics-daemon-bp5mv\" (UID: \"9b96048a-0005-410b-a710-909d21fbfde8\") " pod="openshift-multus/network-metrics-daemon-bp5mv"
Sep 16 18:44:32.148652 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.148620    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s7b79\" (UniqueName: \"kubernetes.io/projected/7892f55c-beff-42d9-989e-52eeb778bdc5-kube-api-access-s7b79\") pod \"tuned-9z8sh\" (UID: \"7892f55c-beff-42d9-989e-52eeb778bdc5\") " pod="openshift-cluster-node-tuning-operator/tuned-9z8sh"
Sep 16 18:44:32.234951 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.234918    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/71637102-fba0-4c38-8055-6dceacddcde9-tuning-conf-dir\") pod \"multus-additional-cni-plugins-fbw2l\" (UID: \"71637102-fba0-4c38-8055-6dceacddcde9\") " pod="openshift-multus/multus-additional-cni-plugins-fbw2l"
Sep 16 18:44:32.234951 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.234957    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/08febeb1-4769-4fd6-911f-a0a02c7c4223-run-openvswitch\") pod \"ovnkube-node-blv9h\" (UID: \"08febeb1-4769-4fd6-911f-a0a02c7c4223\") " pod="openshift-ovn-kubernetes/ovnkube-node-blv9h"
Sep 16 18:44:32.235209 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.234975    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bbdbf\" (UniqueName: \"kubernetes.io/projected/c42d9b1b-2d14-469c-b986-ae99600b602c-kube-api-access-bbdbf\") pod \"multus-d9bgl\" (UID: \"c42d9b1b-2d14-469c-b986-ae99600b602c\") " pod="openshift-multus/multus-d9bgl"
Sep 16 18:44:32.235209 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.234996    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/08febeb1-4769-4fd6-911f-a0a02c7c4223-ovnkube-script-lib\") pod \"ovnkube-node-blv9h\" (UID: \"08febeb1-4769-4fd6-911f-a0a02c7c4223\") " pod="openshift-ovn-kubernetes/ovnkube-node-blv9h"
Sep 16 18:44:32.235209 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.235062    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/08febeb1-4769-4fd6-911f-a0a02c7c4223-run-openvswitch\") pod \"ovnkube-node-blv9h\" (UID: \"08febeb1-4769-4fd6-911f-a0a02c7c4223\") " pod="openshift-ovn-kubernetes/ovnkube-node-blv9h"
Sep 16 18:44:32.235209 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.235112    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4a80b1e8-39b4-44a2-80cb-aa52d96e80fb-tmp-dir\") pod \"node-resolver-bd22w\" (UID: \"4a80b1e8-39b4-44a2-80cb-aa52d96e80fb\") " pod="openshift-dns/node-resolver-bd22w"
Sep 16 18:44:32.235209 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.235148    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c42d9b1b-2d14-469c-b986-ae99600b602c-host-run-netns\") pod \"multus-d9bgl\" (UID: \"c42d9b1b-2d14-469c-b986-ae99600b602c\") " pod="openshift-multus/multus-d9bgl"
Sep 16 18:44:32.235209 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.235174    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b9f0f3b9-87ee-44a5-a66c-6542791f3506-kubelet-config\") pod \"global-pull-secret-syncer-t69xl\" (UID: \"b9f0f3b9-87ee-44a5-a66c-6542791f3506\") " pod="kube-system/global-pull-secret-syncer-t69xl"
Sep 16 18:44:32.235209 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.235196    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/71637102-fba0-4c38-8055-6dceacddcde9-tuning-conf-dir\") pod \"multus-additional-cni-plugins-fbw2l\" (UID: \"71637102-fba0-4c38-8055-6dceacddcde9\") " pod="openshift-multus/multus-additional-cni-plugins-fbw2l"
Sep 16 18:44:32.235487 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.235248    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c42d9b1b-2d14-469c-b986-ae99600b602c-host-run-netns\") pod \"multus-d9bgl\" (UID: \"c42d9b1b-2d14-469c-b986-ae99600b602c\") " pod="openshift-multus/multus-d9bgl"
Sep 16 18:44:32.235487 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.235250    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c42d9b1b-2d14-469c-b986-ae99600b602c-host-var-lib-cni-multus\") pod \"multus-d9bgl\" (UID: \"c42d9b1b-2d14-469c-b986-ae99600b602c\") " pod="openshift-multus/multus-d9bgl"
Sep 16 18:44:32.235487 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.235202    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c42d9b1b-2d14-469c-b986-ae99600b602c-host-var-lib-cni-multus\") pod \"multus-d9bgl\" (UID: \"c42d9b1b-2d14-469c-b986-ae99600b602c\") " pod="openshift-multus/multus-d9bgl"
Sep 16 18:44:32.235487 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.235294    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c42d9b1b-2d14-469c-b986-ae99600b602c-host-var-lib-kubelet\") pod \"multus-d9bgl\" (UID: \"c42d9b1b-2d14-469c-b986-ae99600b602c\") " pod="openshift-multus/multus-d9bgl"
Sep 16 18:44:32.235487 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.235312    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/08febeb1-4769-4fd6-911f-a0a02c7c4223-var-lib-openvswitch\") pod \"ovnkube-node-blv9h\" (UID: \"08febeb1-4769-4fd6-911f-a0a02c7c4223\") " pod="openshift-ovn-kubernetes/ovnkube-node-blv9h"
Sep 16 18:44:32.235487 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.235294    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b9f0f3b9-87ee-44a5-a66c-6542791f3506-kubelet-config\") pod \"global-pull-secret-syncer-t69xl\" (UID: \"b9f0f3b9-87ee-44a5-a66c-6542791f3506\") " pod="kube-system/global-pull-secret-syncer-t69xl"
Sep 16 18:44:32.235487 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.235335    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/71637102-fba0-4c38-8055-6dceacddcde9-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-fbw2l\" (UID: \"71637102-fba0-4c38-8055-6dceacddcde9\") " pod="openshift-multus/multus-additional-cni-plugins-fbw2l"
Sep 16 18:44:32.235487 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.235380    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c42d9b1b-2d14-469c-b986-ae99600b602c-host-var-lib-kubelet\") pod \"multus-d9bgl\" (UID: \"c42d9b1b-2d14-469c-b986-ae99600b602c\") " pod="openshift-multus/multus-d9bgl"
Sep 16 18:44:32.235487 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.235402    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/08febeb1-4769-4fd6-911f-a0a02c7c4223-var-lib-openvswitch\") pod \"ovnkube-node-blv9h\" (UID: \"08febeb1-4769-4fd6-911f-a0a02c7c4223\") " pod="openshift-ovn-kubernetes/ovnkube-node-blv9h"
Sep 16 18:44:32.235487 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.235408    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/08febeb1-4769-4fd6-911f-a0a02c7c4223-host-kubelet\") pod \"ovnkube-node-blv9h\" (UID: \"08febeb1-4769-4fd6-911f-a0a02c7c4223\") " pod="openshift-ovn-kubernetes/ovnkube-node-blv9h"
Sep 16 18:44:32.235487 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.235437    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c42d9b1b-2d14-469c-b986-ae99600b602c-etc-kubernetes\") pod \"multus-d9bgl\" (UID: \"c42d9b1b-2d14-469c-b986-ae99600b602c\") " pod="openshift-multus/multus-d9bgl"
Sep 16 18:44:32.235487 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.235475    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c42d9b1b-2d14-469c-b986-ae99600b602c-etc-kubernetes\") pod \"multus-d9bgl\" (UID: \"c42d9b1b-2d14-469c-b986-ae99600b602c\") " pod="openshift-multus/multus-d9bgl"
Sep 16 18:44:32.236155 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.235506    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/08febeb1-4769-4fd6-911f-a0a02c7c4223-host-kubelet\") pod \"ovnkube-node-blv9h\" (UID: \"08febeb1-4769-4fd6-911f-a0a02c7c4223\") " pod="openshift-ovn-kubernetes/ovnkube-node-blv9h"
Sep 16 18:44:32.236155 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.235518    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4a80b1e8-39b4-44a2-80cb-aa52d96e80fb-tmp-dir\") pod \"node-resolver-bd22w\" (UID: \"4a80b1e8-39b4-44a2-80cb-aa52d96e80fb\") " pod="openshift-dns/node-resolver-bd22w"
Sep 16 18:44:32.236155 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.235539    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c42d9b1b-2d14-469c-b986-ae99600b602c-multus-socket-dir-parent\") pod \"multus-d9bgl\" (UID: \"c42d9b1b-2d14-469c-b986-ae99600b602c\") " pod="openshift-multus/multus-d9bgl"
Sep 16 18:44:32.236155 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.235570    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/08febeb1-4769-4fd6-911f-a0a02c7c4223-host-run-netns\") pod \"ovnkube-node-blv9h\" (UID: \"08febeb1-4769-4fd6-911f-a0a02c7c4223\") " pod="openshift-ovn-kubernetes/ovnkube-node-blv9h"
Sep 16 18:44:32.236155 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.235599    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/08febeb1-4769-4fd6-911f-a0a02c7c4223-host-run-ovn-kubernetes\") pod \"ovnkube-node-blv9h\" (UID: \"08febeb1-4769-4fd6-911f-a0a02c7c4223\") " pod="openshift-ovn-kubernetes/ovnkube-node-blv9h"
Sep 16 18:44:32.236155 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.235630    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/71637102-fba0-4c38-8055-6dceacddcde9-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-fbw2l\" (UID: \"71637102-fba0-4c38-8055-6dceacddcde9\") " pod="openshift-multus/multus-additional-cni-plugins-fbw2l"
Sep 16 18:44:32.236155 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.235598    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c42d9b1b-2d14-469c-b986-ae99600b602c-multus-socket-dir-parent\") pod \"multus-d9bgl\" (UID: \"c42d9b1b-2d14-469c-b986-ae99600b602c\") " pod="openshift-multus/multus-d9bgl"
Sep 16 18:44:32.236155 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.235637    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/08febeb1-4769-4fd6-911f-a0a02c7c4223-host-run-ovn-kubernetes\") pod \"ovnkube-node-blv9h\" (UID: \"08febeb1-4769-4fd6-911f-a0a02c7c4223\") " pod="openshift-ovn-kubernetes/ovnkube-node-blv9h"
Sep 16 18:44:32.236155 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.235631    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/08febeb1-4769-4fd6-911f-a0a02c7c4223-host-run-netns\") pod \"ovnkube-node-blv9h\" (UID: \"08febeb1-4769-4fd6-911f-a0a02c7c4223\") " pod="openshift-ovn-kubernetes/ovnkube-node-blv9h"
Sep 16 18:44:32.236155 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.235659    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bh6kf\" (UniqueName: \"kubernetes.io/projected/71637102-fba0-4c38-8055-6dceacddcde9-kube-api-access-bh6kf\") pod \"multus-additional-cni-plugins-fbw2l\" (UID: \"71637102-fba0-4c38-8055-6dceacddcde9\") " pod="openshift-multus/multus-additional-cni-plugins-fbw2l"
Sep 16 18:44:32.236155 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.235696    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/08febeb1-4769-4fd6-911f-a0a02c7c4223-host-slash\") pod \"ovnkube-node-blv9h\" (UID: \"08febeb1-4769-4fd6-911f-a0a02c7c4223\") " pod="openshift-ovn-kubernetes/ovnkube-node-blv9h"
Sep 16 18:44:32.236155 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.235723    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/08febeb1-4769-4fd6-911f-a0a02c7c4223-host-cni-netd\") pod \"ovnkube-node-blv9h\" (UID: \"08febeb1-4769-4fd6-911f-a0a02c7c4223\") " pod="openshift-ovn-kubernetes/ovnkube-node-blv9h"
Sep 16 18:44:32.236155 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.235737    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/08febeb1-4769-4fd6-911f-a0a02c7c4223-ovnkube-script-lib\") pod \"ovnkube-node-blv9h\" (UID: \"08febeb1-4769-4fd6-911f-a0a02c7c4223\") " pod="openshift-ovn-kubernetes/ovnkube-node-blv9h"
Sep 16 18:44:32.236155 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.235749    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-js86k\" (UniqueName: \"kubernetes.io/projected/08febeb1-4769-4fd6-911f-a0a02c7c4223-kube-api-access-js86k\") pod \"ovnkube-node-blv9h\" (UID: \"08febeb1-4769-4fd6-911f-a0a02c7c4223\") " pod="openshift-ovn-kubernetes/ovnkube-node-blv9h"
Sep 16 18:44:32.236155 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.235791    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/71637102-fba0-4c38-8055-6dceacddcde9-cnibin\") pod \"multus-additional-cni-plugins-fbw2l\" (UID: \"71637102-fba0-4c38-8055-6dceacddcde9\") " pod="openshift-multus/multus-additional-cni-plugins-fbw2l"
Sep 16 18:44:32.236155 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.235811    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/08febeb1-4769-4fd6-911f-a0a02c7c4223-host-slash\") pod \"ovnkube-node-blv9h\" (UID: \"08febeb1-4769-4fd6-911f-a0a02c7c4223\") " pod="openshift-ovn-kubernetes/ovnkube-node-blv9h"
Sep 16 18:44:32.236155 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.235829    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c42d9b1b-2d14-469c-b986-ae99600b602c-multus-conf-dir\") pod \"multus-d9bgl\" (UID: \"c42d9b1b-2d14-469c-b986-ae99600b602c\") " pod="openshift-multus/multus-d9bgl"
Sep 16 18:44:32.237003 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.235878    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/08febeb1-4769-4fd6-911f-a0a02c7c4223-run-systemd\") pod \"ovnkube-node-blv9h\" (UID: \"08febeb1-4769-4fd6-911f-a0a02c7c4223\") " pod="openshift-ovn-kubernetes/ovnkube-node-blv9h"
Sep 16 18:44:32.237003 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.235911    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1498d81a-223d-470c-9812-9b2910e3ea35-host\") pod \"node-ca-f8jpf\" (UID: \"1498d81a-223d-470c-9812-9b2910e3ea35\") " pod="openshift-image-registry/node-ca-f8jpf"
Sep 16 18:44:32.237003 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.235925    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c42d9b1b-2d14-469c-b986-ae99600b602c-multus-conf-dir\") pod \"multus-d9bgl\" (UID: \"c42d9b1b-2d14-469c-b986-ae99600b602c\") " pod="openshift-multus/multus-d9bgl"
Sep 16 18:44:32.237003 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.235931    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/08febeb1-4769-4fd6-911f-a0a02c7c4223-run-systemd\") pod \"ovnkube-node-blv9h\" (UID: \"08febeb1-4769-4fd6-911f-a0a02c7c4223\") " pod="openshift-ovn-kubernetes/ovnkube-node-blv9h"
Sep 16 18:44:32.237003 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.235938    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1498d81a-223d-470c-9812-9b2910e3ea35-serviceca\") pod \"node-ca-f8jpf\" (UID: \"1498d81a-223d-470c-9812-9b2910e3ea35\") " pod="openshift-image-registry/node-ca-f8jpf"
Sep 16 18:44:32.237003 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.235984    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4a80b1e8-39b4-44a2-80cb-aa52d96e80fb-hosts-file\") pod \"node-resolver-bd22w\" (UID: \"4a80b1e8-39b4-44a2-80cb-aa52d96e80fb\") " pod="openshift-dns/node-resolver-bd22w"
Sep 16 18:44:32.237003 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.235887    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/71637102-fba0-4c38-8055-6dceacddcde9-cnibin\") pod \"multus-additional-cni-plugins-fbw2l\" (UID: \"71637102-fba0-4c38-8055-6dceacddcde9\") " pod="openshift-multus/multus-additional-cni-plugins-fbw2l"
Sep 16 18:44:32.237003 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.235967    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1498d81a-223d-470c-9812-9b2910e3ea35-host\") pod \"node-ca-f8jpf\" (UID: \"1498d81a-223d-470c-9812-9b2910e3ea35\") " pod="openshift-image-registry/node-ca-f8jpf"
Sep 16 18:44:32.237003 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.236013    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c42d9b1b-2d14-469c-b986-ae99600b602c-multus-cni-dir\") pod \"multus-d9bgl\" (UID: \"c42d9b1b-2d14-469c-b986-ae99600b602c\") " pod="openshift-multus/multus-d9bgl"
Sep 16 18:44:32.237003 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.236041    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/08febeb1-4769-4fd6-911f-a0a02c7c4223-host-cni-netd\") pod \"ovnkube-node-blv9h\" (UID: \"08febeb1-4769-4fd6-911f-a0a02c7c4223\") " pod="openshift-ovn-kubernetes/ovnkube-node-blv9h"
Sep 16 18:44:32.237003 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.236062    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/08febeb1-4769-4fd6-911f-a0a02c7c4223-etc-openvswitch\") pod \"ovnkube-node-blv9h\" (UID: \"08febeb1-4769-4fd6-911f-a0a02c7c4223\") " pod="openshift-ovn-kubernetes/ovnkube-node-blv9h"
Sep 16 18:44:32.237003 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.236065    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/71637102-fba0-4c38-8055-6dceacddcde9-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-fbw2l\" (UID: \"71637102-fba0-4c38-8055-6dceacddcde9\") " pod="openshift-multus/multus-additional-cni-plugins-fbw2l"
Sep 16 18:44:32.237003 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.236083    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c42d9b1b-2d14-469c-b986-ae99600b602c-multus-cni-dir\") pod \"multus-d9bgl\" (UID: \"c42d9b1b-2d14-469c-b986-ae99600b602c\") " pod="openshift-multus/multus-d9bgl"
Sep 16 18:44:32.237003 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.235938    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/71637102-fba0-4c38-8055-6dceacddcde9-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-fbw2l\" (UID: \"71637102-fba0-4c38-8055-6dceacddcde9\") " pod="openshift-multus/multus-additional-cni-plugins-fbw2l"
Sep 16 18:44:32.237003 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.236091    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/08febeb1-4769-4fd6-911f-a0a02c7c4223-run-ovn\") pod \"ovnkube-node-blv9h\" (UID: \"08febeb1-4769-4fd6-911f-a0a02c7c4223\") " pod="openshift-ovn-kubernetes/ovnkube-node-blv9h"
Sep 16 18:44:32.237003 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.236066    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4a80b1e8-39b4-44a2-80cb-aa52d96e80fb-hosts-file\") pod \"node-resolver-bd22w\" (UID: \"4a80b1e8-39b4-44a2-80cb-aa52d96e80fb\") " pod="openshift-dns/node-resolver-bd22w"
Sep 16 18:44:32.237003 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.236120    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/08febeb1-4769-4fd6-911f-a0a02c7c4223-etc-openvswitch\") pod \"ovnkube-node-blv9h\" (UID: \"08febeb1-4769-4fd6-911f-a0a02c7c4223\") " pod="openshift-ovn-kubernetes/ovnkube-node-blv9h"
Sep 16 18:44:32.237003 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.236135    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/08febeb1-4769-4fd6-911f-a0a02c7c4223-ovnkube-config\") pod \"ovnkube-node-blv9h\" (UID: \"08febeb1-4769-4fd6-911f-a0a02c7c4223\") " pod="openshift-ovn-kubernetes/ovnkube-node-blv9h"
Sep 16 18:44:32.237896 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.236162    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/08febeb1-4769-4fd6-911f-a0a02c7c4223-run-ovn\") pod \"ovnkube-node-blv9h\" (UID: \"08febeb1-4769-4fd6-911f-a0a02c7c4223\") " pod="openshift-ovn-kubernetes/ovnkube-node-blv9h"
Sep 16 18:44:32.237896 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.236163    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l4z5w\" (UniqueName: \"kubernetes.io/projected/df3c2b31-63a6-43e4-a4ae-0199c34a6a86-kube-api-access-l4z5w\") pod \"network-check-target-f4md6\" (UID: \"df3c2b31-63a6-43e4-a4ae-0199c34a6a86\") " pod="openshift-network-diagnostics/network-check-target-f4md6"
Sep 16 18:44:32.237896 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.236227    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/08febeb1-4769-4fd6-911f-a0a02c7c4223-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-blv9h\" (UID: \"08febeb1-4769-4fd6-911f-a0a02c7c4223\") " pod="openshift-ovn-kubernetes/ovnkube-node-blv9h"
Sep 16 18:44:32.237896 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.236259    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c42d9b1b-2d14-469c-b986-ae99600b602c-cnibin\") pod \"multus-d9bgl\" (UID: \"c42d9b1b-2d14-469c-b986-ae99600b602c\") " pod="openshift-multus/multus-d9bgl"
Sep 16 18:44:32.237896 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.236285    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c42d9b1b-2d14-469c-b986-ae99600b602c-host-run-k8s-cni-cncf-io\") pod \"multus-d9bgl\" (UID: \"c42d9b1b-2d14-469c-b986-ae99600b602c\") " pod="openshift-multus/multus-d9bgl"
Sep 16 18:44:32.237896 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.236316    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c42d9b1b-2d14-469c-b986-ae99600b602c-host-run-multus-certs\") pod \"multus-d9bgl\" (UID: \"c42d9b1b-2d14-469c-b986-ae99600b602c\") " pod="openshift-multus/multus-d9bgl"
Sep 16 18:44:32.237896 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.236337    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b9f0f3b9-87ee-44a5-a66c-6542791f3506-original-pull-secret\") pod \"global-pull-secret-syncer-t69xl\" (UID: \"b9f0f3b9-87ee-44a5-a66c-6542791f3506\") " pod="kube-system/global-pull-secret-syncer-t69xl"
Sep 16 18:44:32.237896 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.236353    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/08febeb1-4769-4fd6-911f-a0a02c7c4223-systemd-units\") pod \"ovnkube-node-blv9h\" (UID: \"08febeb1-4769-4fd6-911f-a0a02c7c4223\") " pod="openshift-ovn-kubernetes/ovnkube-node-blv9h"
Sep 16 18:44:32.237896 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.236362    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1498d81a-223d-470c-9812-9b2910e3ea35-serviceca\") pod \"node-ca-f8jpf\" (UID: \"1498d81a-223d-470c-9812-9b2910e3ea35\") " pod="openshift-image-registry/node-ca-f8jpf"
Sep 16 18:44:32.237896 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.236368    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/08febeb1-4769-4fd6-911f-a0a02c7c4223-ovn-node-metrics-cert\") pod \"ovnkube-node-blv9h\" (UID: \"08febeb1-4769-4fd6-911f-a0a02c7c4223\") " pod="openshift-ovn-kubernetes/ovnkube-node-blv9h"
Sep 16 18:44:32.237896 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.236383    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c42d9b1b-2d14-469c-b986-ae99600b602c-cnibin\") pod \"multus-d9bgl\" (UID: \"c42d9b1b-2d14-469c-b986-ae99600b602c\") " pod="openshift-multus/multus-d9bgl"
Sep 16 18:44:32.237896 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.236418    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/08febeb1-4769-4fd6-911f-a0a02c7c4223-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-blv9h\" (UID: \"08febeb1-4769-4fd6-911f-a0a02c7c4223\") " pod="openshift-ovn-kubernetes/ovnkube-node-blv9h"
Sep 16 18:44:32.237896 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.236427    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c42d9b1b-2d14-469c-b986-ae99600b602c-host-run-k8s-cni-cncf-io\") pod \"multus-d9bgl\" (UID: \"c42d9b1b-2d14-469c-b986-ae99600b602c\") " pod="openshift-multus/multus-d9bgl"
Sep 16 18:44:32.237896 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:44:32.236439    2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 16 18:44:32.237896 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.236463    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/08febeb1-4769-4fd6-911f-a0a02c7c4223-systemd-units\") pod \"ovnkube-node-blv9h\" (UID: \"08febeb1-4769-4fd6-911f-a0a02c7c4223\") " pod="openshift-ovn-kubernetes/ovnkube-node-blv9h"
Sep 16 18:44:32.237896 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:44:32.236492    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b9f0f3b9-87ee-44a5-a66c-6542791f3506-original-pull-secret podName:b9f0f3b9-87ee-44a5-a66c-6542791f3506 nodeName:}" failed. No retries permitted until 2026-09-16 18:44:32.736478081 +0000 UTC m=+3.065980474 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b9f0f3b9-87ee-44a5-a66c-6542791f3506-original-pull-secret") pod "global-pull-secret-syncer-t69xl" (UID: "b9f0f3b9-87ee-44a5-a66c-6542791f3506") : object "kube-system"/"original-pull-secret" not registered
Sep 16 18:44:32.237896 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.236615    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/71637102-fba0-4c38-8055-6dceacddcde9-system-cni-dir\") pod \"multus-additional-cni-plugins-fbw2l\" (UID: \"71637102-fba0-4c38-8055-6dceacddcde9\") " pod="openshift-multus/multus-additional-cni-plugins-fbw2l"
Sep 16 18:44:32.238696 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.236663    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/71637102-fba0-4c38-8055-6dceacddcde9-system-cni-dir\") pod \"multus-additional-cni-plugins-fbw2l\" (UID: \"71637102-fba0-4c38-8055-6dceacddcde9\") " pod="openshift-multus/multus-additional-cni-plugins-fbw2l"
Sep 16 18:44:32.238696 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.236664    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c42d9b1b-2d14-469c-b986-ae99600b602c-cni-binary-copy\") pod \"multus-d9bgl\" (UID: \"c42d9b1b-2d14-469c-b986-ae99600b602c\") " pod="openshift-multus/multus-d9bgl"
Sep 16 18:44:32.238696 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.236673    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/08febeb1-4769-4fd6-911f-a0a02c7c4223-ovnkube-config\") pod \"ovnkube-node-blv9h\" (UID: \"08febeb1-4769-4fd6-911f-a0a02c7c4223\") " pod="openshift-ovn-kubernetes/ovnkube-node-blv9h"
Sep 16 18:44:32.238696 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.236742    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c42d9b1b-2d14-469c-b986-ae99600b602c-hostroot\") pod \"multus-d9bgl\" (UID: \"c42d9b1b-2d14-469c-b986-ae99600b602c\") " pod="openshift-multus/multus-d9bgl"
Sep 16 18:44:32.238696 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.236789    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a29102aa-22ea-4623-84fc-0be1e5506274-host-slash\") pod \"iptables-alerter-qqjgk\" (UID: \"a29102aa-22ea-4623-84fc-0be1e5506274\") " pod="openshift-network-operator/iptables-alerter-qqjgk"
Sep 16 18:44:32.238696 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.236794    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c42d9b1b-2d14-469c-b986-ae99600b602c-hostroot\") pod \"multus-d9bgl\" (UID: \"c42d9b1b-2d14-469c-b986-ae99600b602c\") " pod="openshift-multus/multus-d9bgl"
Sep 16 18:44:32.238696 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.236825    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c42d9b1b-2d14-469c-b986-ae99600b602c-host-run-multus-certs\") pod \"multus-d9bgl\" (UID: \"c42d9b1b-2d14-469c-b986-ae99600b602c\") " pod="openshift-multus/multus-d9bgl"
Sep 16 18:44:32.238696 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.236862    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7km7r\" (UniqueName: \"kubernetes.io/projected/a29102aa-22ea-4623-84fc-0be1e5506274-kube-api-access-7km7r\") pod \"iptables-alerter-qqjgk\" (UID: \"a29102aa-22ea-4623-84fc-0be1e5506274\") " pod="openshift-network-operator/iptables-alerter-qqjgk"
Sep 16 18:44:32.238696 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.236891    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/08febeb1-4769-4fd6-911f-a0a02c7c4223-node-log\") pod \"ovnkube-node-blv9h\" (UID: \"08febeb1-4769-4fd6-911f-a0a02c7c4223\") " pod="openshift-ovn-kubernetes/ovnkube-node-blv9h"
Sep 16 18:44:32.238696 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.236973    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a29102aa-22ea-4623-84fc-0be1e5506274-iptables-alerter-script\") pod \"iptables-alerter-qqjgk\" (UID: \"a29102aa-22ea-4623-84fc-0be1e5506274\") " pod="openshift-network-operator/iptables-alerter-qqjgk"
Sep 16 18:44:32.238696 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.237030    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/828d6bf1-3d76-4c9b-af83-07de04f8f20f-konnectivity-ca\") pod \"konnectivity-agent-6klb4\" (UID: \"828d6bf1-3d76-4c9b-af83-07de04f8f20f\") " pod="kube-system/konnectivity-agent-6klb4"
Sep 16 18:44:32.238696 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.237058    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/71637102-fba0-4c38-8055-6dceacddcde9-cni-binary-copy\") pod \"multus-additional-cni-plugins-fbw2l\" (UID: \"71637102-fba0-4c38-8055-6dceacddcde9\") " pod="openshift-multus/multus-additional-cni-plugins-fbw2l"
Sep 16 18:44:32.238696 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.237085    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sc9zn\" (UniqueName: \"kubernetes.io/projected/4a80b1e8-39b4-44a2-80cb-aa52d96e80fb-kube-api-access-sc9zn\") pod \"node-resolver-bd22w\" (UID: \"4a80b1e8-39b4-44a2-80cb-aa52d96e80fb\") " pod="openshift-dns/node-resolver-bd22w"
Sep 16 18:44:32.238696 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.237117    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/08febeb1-4769-4fd6-911f-a0a02c7c4223-env-overrides\") pod \"ovnkube-node-blv9h\" (UID: \"08febeb1-4769-4fd6-911f-a0a02c7c4223\") " pod="openshift-ovn-kubernetes/ovnkube-node-blv9h"
Sep 16 18:44:32.238696 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.237144    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/828d6bf1-3d76-4c9b-af83-07de04f8f20f-agent-certs\") pod \"konnectivity-agent-6klb4\" (UID: \"828d6bf1-3d76-4c9b-af83-07de04f8f20f\") " pod="kube-system/konnectivity-agent-6klb4"
Sep 16 18:44:32.238696 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.237163    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c42d9b1b-2d14-469c-b986-ae99600b602c-cni-binary-copy\") pod \"multus-d9bgl\" (UID: \"c42d9b1b-2d14-469c-b986-ae99600b602c\") " pod="openshift-multus/multus-d9bgl"
Sep 16 18:44:32.238696 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.237171    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c42d9b1b-2d14-469c-b986-ae99600b602c-system-cni-dir\") pod \"multus-d9bgl\" (UID: \"c42d9b1b-2d14-469c-b986-ae99600b602c\") " pod="openshift-multus/multus-d9bgl"
Sep 16 18:44:32.238696 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.237188    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/08febeb1-4769-4fd6-911f-a0a02c7c4223-node-log\") pod \"ovnkube-node-blv9h\" (UID: \"08febeb1-4769-4fd6-911f-a0a02c7c4223\") " pod="openshift-ovn-kubernetes/ovnkube-node-blv9h"
Sep 16 18:44:32.239428 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.237204    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c42d9b1b-2d14-469c-b986-ae99600b602c-host-var-lib-cni-bin\") pod \"multus-d9bgl\" (UID: \"c42d9b1b-2d14-469c-b986-ae99600b602c\") " pod="openshift-multus/multus-d9bgl"
Sep 16 18:44:32.239428 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.237282    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c42d9b1b-2d14-469c-b986-ae99600b602c-system-cni-dir\") pod \"multus-d9bgl\" (UID: \"c42d9b1b-2d14-469c-b986-ae99600b602c\") " pod="openshift-multus/multus-d9bgl"
Sep 16 18:44:32.239428 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.237321    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c42d9b1b-2d14-469c-b986-ae99600b602c-multus-daemon-config\") pod \"multus-d9bgl\" (UID: \"c42d9b1b-2d14-469c-b986-ae99600b602c\") " pod="openshift-multus/multus-d9bgl"
Sep 16 18:44:32.239428 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.237348    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b9f0f3b9-87ee-44a5-a66c-6542791f3506-dbus\") pod \"global-pull-secret-syncer-t69xl\" (UID: \"b9f0f3b9-87ee-44a5-a66c-6542791f3506\") " pod="kube-system/global-pull-secret-syncer-t69xl"
Sep 16 18:44:32.239428 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.237381    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/08febeb1-4769-4fd6-911f-a0a02c7c4223-log-socket\") pod \"ovnkube-node-blv9h\" (UID: \"08febeb1-4769-4fd6-911f-a0a02c7c4223\") " pod="openshift-ovn-kubernetes/ovnkube-node-blv9h"
Sep 16 18:44:32.239428 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.237408    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/08febeb1-4769-4fd6-911f-a0a02c7c4223-host-cni-bin\") pod \"ovnkube-node-blv9h\" (UID: \"08febeb1-4769-4fd6-911f-a0a02c7c4223\") " pod="openshift-ovn-kubernetes/ovnkube-node-blv9h"
Sep 16 18:44:32.239428 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.237380    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a29102aa-22ea-4623-84fc-0be1e5506274-host-slash\") pod \"iptables-alerter-qqjgk\" (UID: \"a29102aa-22ea-4623-84fc-0be1e5506274\") " pod="openshift-network-operator/iptables-alerter-qqjgk"
Sep 16 18:44:32.239428 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.237473    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dd7rl\" (UniqueName: \"kubernetes.io/projected/1498d81a-223d-470c-9812-9b2910e3ea35-kube-api-access-dd7rl\") pod \"node-ca-f8jpf\" (UID: \"1498d81a-223d-470c-9812-9b2910e3ea35\") " pod="openshift-image-registry/node-ca-f8jpf"
Sep 16 18:44:32.239428 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.237481    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c42d9b1b-2d14-469c-b986-ae99600b602c-host-var-lib-cni-bin\") pod \"multus-d9bgl\" (UID: \"c42d9b1b-2d14-469c-b986-ae99600b602c\") " pod="openshift-multus/multus-d9bgl"
Sep 16 18:44:32.239428 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.237481    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b9f0f3b9-87ee-44a5-a66c-6542791f3506-dbus\") pod \"global-pull-secret-syncer-t69xl\" (UID: \"b9f0f3b9-87ee-44a5-a66c-6542791f3506\") " pod="kube-system/global-pull-secret-syncer-t69xl"
Sep 16 18:44:32.239428 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.237498    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/08febeb1-4769-4fd6-911f-a0a02c7c4223-host-cni-bin\") pod \"ovnkube-node-blv9h\" (UID: \"08febeb1-4769-4fd6-911f-a0a02c7c4223\") " pod="openshift-ovn-kubernetes/ovnkube-node-blv9h"
Sep 16 18:44:32.239428 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.237512    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/08febeb1-4769-4fd6-911f-a0a02c7c4223-log-socket\") pod \"ovnkube-node-blv9h\" (UID: \"08febeb1-4769-4fd6-911f-a0a02c7c4223\") " pod="openshift-ovn-kubernetes/ovnkube-node-blv9h"
Sep 16 18:44:32.239428 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.237697    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/71637102-fba0-4c38-8055-6dceacddcde9-cni-binary-copy\") pod \"multus-additional-cni-plugins-fbw2l\" (UID: \"71637102-fba0-4c38-8055-6dceacddcde9\") " pod="openshift-multus/multus-additional-cni-plugins-fbw2l"
Sep 16 18:44:32.239428 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.237962    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a29102aa-22ea-4623-84fc-0be1e5506274-iptables-alerter-script\") pod \"iptables-alerter-qqjgk\" (UID: \"a29102aa-22ea-4623-84fc-0be1e5506274\") " pod="openshift-network-operator/iptables-alerter-qqjgk"
Sep 16 18:44:32.239428 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.237987    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c42d9b1b-2d14-469c-b986-ae99600b602c-multus-daemon-config\") pod \"multus-d9bgl\" (UID: \"c42d9b1b-2d14-469c-b986-ae99600b602c\") " pod="openshift-multus/multus-d9bgl"
Sep 16 18:44:32.239428 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.237995    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/08febeb1-4769-4fd6-911f-a0a02c7c4223-env-overrides\") pod \"ovnkube-node-blv9h\" (UID: \"08febeb1-4769-4fd6-911f-a0a02c7c4223\") " pod="openshift-ovn-kubernetes/ovnkube-node-blv9h"
Sep 16 18:44:32.239428 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.238397    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/828d6bf1-3d76-4c9b-af83-07de04f8f20f-konnectivity-ca\") pod \"konnectivity-agent-6klb4\" (UID: \"828d6bf1-3d76-4c9b-af83-07de04f8f20f\") " pod="kube-system/konnectivity-agent-6klb4"
Sep 16 18:44:32.239428 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.239230    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/08febeb1-4769-4fd6-911f-a0a02c7c4223-ovn-node-metrics-cert\") pod \"ovnkube-node-blv9h\" (UID: \"08febeb1-4769-4fd6-911f-a0a02c7c4223\") " pod="openshift-ovn-kubernetes/ovnkube-node-blv9h"
Sep 16 18:44:32.240244 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.240153    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/828d6bf1-3d76-4c9b-af83-07de04f8f20f-agent-certs\") pod \"konnectivity-agent-6klb4\" (UID: \"828d6bf1-3d76-4c9b-af83-07de04f8f20f\") " pod="kube-system/konnectivity-agent-6klb4"
Sep 16 18:44:32.245916 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:44:32.245893    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 16 18:44:32.245916 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:44:32.245919    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 16 18:44:32.246096 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:44:32.245933    2584 projected.go:196] Error preparing data for projected volume kube-api-access-l4z5w for pod openshift-network-diagnostics/network-check-target-f4md6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 18:44:32.246096 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:44:32.246011    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/df3c2b31-63a6-43e4-a4ae-0199c34a6a86-kube-api-access-l4z5w podName:df3c2b31-63a6-43e4-a4ae-0199c34a6a86 nodeName:}" failed. No retries permitted until 2026-09-16 18:44:32.745988826 +0000 UTC m=+3.075491219 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-l4z5w" (UniqueName: "kubernetes.io/projected/df3c2b31-63a6-43e4-a4ae-0199c34a6a86-kube-api-access-l4z5w") pod "network-check-target-f4md6" (UID: "df3c2b31-63a6-43e4-a4ae-0199c34a6a86") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 18:44:32.247057 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.247038    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bbdbf\" (UniqueName: \"kubernetes.io/projected/c42d9b1b-2d14-469c-b986-ae99600b602c-kube-api-access-bbdbf\") pod \"multus-d9bgl\" (UID: \"c42d9b1b-2d14-469c-b986-ae99600b602c\") " pod="openshift-multus/multus-d9bgl"
Sep 16 18:44:32.248618 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.248290    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bh6kf\" (UniqueName: \"kubernetes.io/projected/71637102-fba0-4c38-8055-6dceacddcde9-kube-api-access-bh6kf\") pod \"multus-additional-cni-plugins-fbw2l\" (UID: \"71637102-fba0-4c38-8055-6dceacddcde9\") " pod="openshift-multus/multus-additional-cni-plugins-fbw2l"
Sep 16 18:44:32.248618 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.248571    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-js86k\" (UniqueName: \"kubernetes.io/projected/08febeb1-4769-4fd6-911f-a0a02c7c4223-kube-api-access-js86k\") pod \"ovnkube-node-blv9h\" (UID: \"08febeb1-4769-4fd6-911f-a0a02c7c4223\") " pod="openshift-ovn-kubernetes/ovnkube-node-blv9h"
Sep 16 18:44:32.248920 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.248884    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dd7rl\" (UniqueName: \"kubernetes.io/projected/1498d81a-223d-470c-9812-9b2910e3ea35-kube-api-access-dd7rl\") pod \"node-ca-f8jpf\" (UID: \"1498d81a-223d-470c-9812-9b2910e3ea35\") " pod="openshift-image-registry/node-ca-f8jpf"
Sep 16 18:44:32.249225 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.249209    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sc9zn\" (UniqueName: \"kubernetes.io/projected/4a80b1e8-39b4-44a2-80cb-aa52d96e80fb-kube-api-access-sc9zn\") pod \"node-resolver-bd22w\" (UID: \"4a80b1e8-39b4-44a2-80cb-aa52d96e80fb\") " pod="openshift-dns/node-resolver-bd22w"
Sep 16 18:44:32.250133 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.250117    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7km7r\" (UniqueName: \"kubernetes.io/projected/a29102aa-22ea-4623-84fc-0be1e5506274-kube-api-access-7km7r\") pod \"iptables-alerter-qqjgk\" (UID: \"a29102aa-22ea-4623-84fc-0be1e5506274\") " pod="openshift-network-operator/iptables-alerter-qqjgk"
Sep 16 18:44:32.331629 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.331541    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gpwqt"
Sep 16 18:44:32.342169 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.342144    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-9z8sh"
Sep 16 18:44:32.350608 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.350587    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-6klb4"
Sep 16 18:44:32.356807 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.356786    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-fbw2l"
Sep 16 18:44:32.364002 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.363975    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-f8jpf"
Sep 16 18:44:32.371885 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.371862    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-d9bgl"
Sep 16 18:44:32.379718 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.379696    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-qqjgk"
Sep 16 18:44:32.386576 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.386550    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-blv9h"
Sep 16 18:44:32.390431 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.390412    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-bd22w"
Sep 16 18:44:32.542848 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.542815    2584 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 16 18:44:32.640640 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.640554    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9b96048a-0005-410b-a710-909d21fbfde8-metrics-certs\") pod \"network-metrics-daemon-bp5mv\" (UID: \"9b96048a-0005-410b-a710-909d21fbfde8\") " pod="openshift-multus/network-metrics-daemon-bp5mv"
Sep 16 18:44:32.640794 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:44:32.640698    2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 18:44:32.640794 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:44:32.640773    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9b96048a-0005-410b-a710-909d21fbfde8-metrics-certs podName:9b96048a-0005-410b-a710-909d21fbfde8 nodeName:}" failed. No retries permitted until 2026-09-16 18:44:33.640753642 +0000 UTC m=+3.970256038 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9b96048a-0005-410b-a710-909d21fbfde8-metrics-certs") pod "network-metrics-daemon-bp5mv" (UID: "9b96048a-0005-410b-a710-909d21fbfde8") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 18:44:32.741380 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.741340    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b9f0f3b9-87ee-44a5-a66c-6542791f3506-original-pull-secret\") pod \"global-pull-secret-syncer-t69xl\" (UID: \"b9f0f3b9-87ee-44a5-a66c-6542791f3506\") " pod="kube-system/global-pull-secret-syncer-t69xl"
Sep 16 18:44:32.741577 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:44:32.741482    2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 16 18:44:32.741577 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:44:32.741558    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b9f0f3b9-87ee-44a5-a66c-6542791f3506-original-pull-secret podName:b9f0f3b9-87ee-44a5-a66c-6542791f3506 nodeName:}" failed. No retries permitted until 2026-09-16 18:44:33.741539145 +0000 UTC m=+4.071041550 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b9f0f3b9-87ee-44a5-a66c-6542791f3506-original-pull-secret") pod "global-pull-secret-syncer-t69xl" (UID: "b9f0f3b9-87ee-44a5-a66c-6542791f3506") : object "kube-system"/"original-pull-secret" not registered
Sep 16 18:44:32.842060 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:32.842026    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l4z5w\" (UniqueName: \"kubernetes.io/projected/df3c2b31-63a6-43e4-a4ae-0199c34a6a86-kube-api-access-l4z5w\") pod \"network-check-target-f4md6\" (UID: \"df3c2b31-63a6-43e4-a4ae-0199c34a6a86\") " pod="openshift-network-diagnostics/network-check-target-f4md6"
Sep 16 18:44:32.842243 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:44:32.842223    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 16 18:44:32.842297 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:44:32.842254    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 16 18:44:32.842297 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:44:32.842268    2584 projected.go:196] Error preparing data for projected volume kube-api-access-l4z5w for pod openshift-network-diagnostics/network-check-target-f4md6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 18:44:32.842371 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:44:32.842330    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/df3c2b31-63a6-43e4-a4ae-0199c34a6a86-kube-api-access-l4z5w podName:df3c2b31-63a6-43e4-a4ae-0199c34a6a86 nodeName:}" failed. No retries permitted until 2026-09-16 18:44:33.842310636 +0000 UTC m=+4.171813036 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-l4z5w" (UniqueName: "kubernetes.io/projected/df3c2b31-63a6-43e4-a4ae-0199c34a6a86-kube-api-access-l4z5w") pod "network-check-target-f4md6" (UID: "df3c2b31-63a6-43e4-a4ae-0199c34a6a86") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 18:44:32.973494 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:32.973333    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08febeb1_4769_4fd6_911f_a0a02c7c4223.slice/crio-c7668329f64090b4ddaef1b14f2d40f547886cb20f79b290ed97b78f5d3a5032 WatchSource:0}: Error finding container c7668329f64090b4ddaef1b14f2d40f547886cb20f79b290ed97b78f5d3a5032: Status 404 returned error can't find the container with id c7668329f64090b4ddaef1b14f2d40f547886cb20f79b290ed97b78f5d3a5032
Sep 16 18:44:32.974638 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:32.974608    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac357968_eba1_4c7d_97a7_b1a8e691e22e.slice/crio-549710cd475b3bdcc8d034ec48afd61ce88fcdad603b93a1b63c55b2b1ec5c53 WatchSource:0}: Error finding container 549710cd475b3bdcc8d034ec48afd61ce88fcdad603b93a1b63c55b2b1ec5c53: Status 404 returned error can't find the container with id 549710cd475b3bdcc8d034ec48afd61ce88fcdad603b93a1b63c55b2b1ec5c53
Sep 16 18:44:32.975887 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:32.975842    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7892f55c_beff_42d9_989e_52eeb778bdc5.slice/crio-dd4ef5c092095e74ebf0e475688d4cffab6ccaa953c76936fe8f93903c9904bb WatchSource:0}: Error finding container dd4ef5c092095e74ebf0e475688d4cffab6ccaa953c76936fe8f93903c9904bb: Status 404 returned error can't find the container with id dd4ef5c092095e74ebf0e475688d4cffab6ccaa953c76936fe8f93903c9904bb
Sep 16 18:44:32.977196 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:32.977143    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc42d9b1b_2d14_469c_b986_ae99600b602c.slice/crio-b538fc53a41f809e37a9d21a71390b4a6148d5bd5d908f4961800213fd868780 WatchSource:0}: Error finding container b538fc53a41f809e37a9d21a71390b4a6148d5bd5d908f4961800213fd868780: Status 404 returned error can't find the container with id b538fc53a41f809e37a9d21a71390b4a6148d5bd5d908f4961800213fd868780
Sep 16 18:44:32.979739 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:32.979708    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71637102_fba0_4c38_8055_6dceacddcde9.slice/crio-1e884d941ff0795de1228c9d1aedcd1612a385b77b4d858e71bbe8a0e3d415ae WatchSource:0}: Error finding container 1e884d941ff0795de1228c9d1aedcd1612a385b77b4d858e71bbe8a0e3d415ae: Status 404 returned error can't find the container with id 1e884d941ff0795de1228c9d1aedcd1612a385b77b4d858e71bbe8a0e3d415ae
Sep 16 18:44:32.980658 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:32.980617    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda29102aa_22ea_4623_84fc_0be1e5506274.slice/crio-a5ada3d3ba4702fed8414dc0e36e7c89a66cda4d1196c75316a4058bef66d1b5 WatchSource:0}: Error finding container a5ada3d3ba4702fed8414dc0e36e7c89a66cda4d1196c75316a4058bef66d1b5: Status 404 returned error can't find the container with id a5ada3d3ba4702fed8414dc0e36e7c89a66cda4d1196c75316a4058bef66d1b5
Sep 16 18:44:32.981284 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:32.981262    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a80b1e8_39b4_44a2_80cb_aa52d96e80fb.slice/crio-12145d8f9a0bbbf63759d53d7b259129a64b620b77a86f614a3e95fe255d71ef WatchSource:0}: Error finding container 12145d8f9a0bbbf63759d53d7b259129a64b620b77a86f614a3e95fe255d71ef: Status 404 returned error can't find the container with id 12145d8f9a0bbbf63759d53d7b259129a64b620b77a86f614a3e95fe255d71ef
Sep 16 18:44:32.985540 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:44:32.985029    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1498d81a_223d_470c_9812_9b2910e3ea35.slice/crio-8484ffd078cdd5d243ab18a375aefc6a954390863dd1048f7e437a5a3f21be63 WatchSource:0}: Error finding container 8484ffd078cdd5d243ab18a375aefc6a954390863dd1048f7e437a5a3f21be63: Status 404 returned error can't find the container with id 8484ffd078cdd5d243ab18a375aefc6a954390863dd1048f7e437a5a3f21be63
Sep 16 18:44:33.071095 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:33.071038    2584 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-15 18:39:32 +0000 UTC" deadline="2028-06-05 07:06:16.737733542 +0000 UTC"
Sep 16 18:44:33.071095 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:33.071081    2584 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15060h21m43.666655114s"
Sep 16 18:44:33.111381 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:33.111350    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t69xl"
Sep 16 18:44:33.111502 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:33.111385    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f4md6"
Sep 16 18:44:33.111502 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:44:33.111471    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t69xl" podUID="b9f0f3b9-87ee-44a5-a66c-6542791f3506"
Sep 16 18:44:33.111604 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:44:33.111585    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f4md6" podUID="df3c2b31-63a6-43e4-a4ae-0199c34a6a86"
Sep 16 18:44:33.120211 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:33.120182    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-f8jpf" event={"ID":"1498d81a-223d-470c-9812-9b2910e3ea35","Type":"ContainerStarted","Data":"8484ffd078cdd5d243ab18a375aefc6a954390863dd1048f7e437a5a3f21be63"}
Sep 16 18:44:33.122743 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:33.122718    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-bd22w" event={"ID":"4a80b1e8-39b4-44a2-80cb-aa52d96e80fb","Type":"ContainerStarted","Data":"12145d8f9a0bbbf63759d53d7b259129a64b620b77a86f614a3e95fe255d71ef"}
Sep 16 18:44:33.123953 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:33.123934    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-qqjgk" event={"ID":"a29102aa-22ea-4623-84fc-0be1e5506274","Type":"ContainerStarted","Data":"a5ada3d3ba4702fed8414dc0e36e7c89a66cda4d1196c75316a4058bef66d1b5"}
Sep 16 18:44:33.125406 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:33.125383    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-d9bgl" event={"ID":"c42d9b1b-2d14-469c-b986-ae99600b602c","Type":"ContainerStarted","Data":"b538fc53a41f809e37a9d21a71390b4a6148d5bd5d908f4961800213fd868780"}
Sep 16 18:44:33.127829 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:33.127805    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-9z8sh" event={"ID":"7892f55c-beff-42d9-989e-52eeb778bdc5","Type":"ContainerStarted","Data":"dd4ef5c092095e74ebf0e475688d4cffab6ccaa953c76936fe8f93903c9904bb"}
Sep 16 18:44:33.129473 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:33.129447    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gpwqt" event={"ID":"ac357968-eba1-4c7d-97a7-b1a8e691e22e","Type":"ContainerStarted","Data":"549710cd475b3bdcc8d034ec48afd61ce88fcdad603b93a1b63c55b2b1ec5c53"}
Sep 16 18:44:33.130681 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:33.130661    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-blv9h" event={"ID":"08febeb1-4769-4fd6-911f-a0a02c7c4223","Type":"ContainerStarted","Data":"c7668329f64090b4ddaef1b14f2d40f547886cb20f79b290ed97b78f5d3a5032"}
Sep 16 18:44:33.134457 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:33.134431    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-182.ec2.internal" event={"ID":"228e51ceefc0d99659afe6cee7fbb736","Type":"ContainerStarted","Data":"1542055a387fe531a47bf13fa15899e417e0931b712c029ed4c7bc3c5cc9a21c"}
Sep 16 18:44:33.136207 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:33.136182    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-6klb4" event={"ID":"828d6bf1-3d76-4c9b-af83-07de04f8f20f","Type":"ContainerStarted","Data":"108b1cd569b800f75abc9938f64f3f157678341c1aaa5769572e8918f759e1a5"}
Sep 16 18:44:33.137370 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:33.137351    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fbw2l" event={"ID":"71637102-fba0-4c38-8055-6dceacddcde9","Type":"ContainerStarted","Data":"1e884d941ff0795de1228c9d1aedcd1612a385b77b4d858e71bbe8a0e3d415ae"}
Sep 16 18:44:33.648974 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:33.648875    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9b96048a-0005-410b-a710-909d21fbfde8-metrics-certs\") pod \"network-metrics-daemon-bp5mv\" (UID: \"9b96048a-0005-410b-a710-909d21fbfde8\") " pod="openshift-multus/network-metrics-daemon-bp5mv"
Sep 16 18:44:33.649149 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:44:33.649093    2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 18:44:33.649208 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:44:33.649162    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9b96048a-0005-410b-a710-909d21fbfde8-metrics-certs podName:9b96048a-0005-410b-a710-909d21fbfde8 nodeName:}" failed. No retries permitted until 2026-09-16 18:44:35.649140739 +0000 UTC m=+5.978643146 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9b96048a-0005-410b-a710-909d21fbfde8-metrics-certs") pod "network-metrics-daemon-bp5mv" (UID: "9b96048a-0005-410b-a710-909d21fbfde8") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 18:44:33.749795 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:33.749755    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b9f0f3b9-87ee-44a5-a66c-6542791f3506-original-pull-secret\") pod \"global-pull-secret-syncer-t69xl\" (UID: \"b9f0f3b9-87ee-44a5-a66c-6542791f3506\") " pod="kube-system/global-pull-secret-syncer-t69xl"
Sep 16 18:44:33.750009 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:44:33.749936    2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 16 18:44:33.750009 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:44:33.750010    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b9f0f3b9-87ee-44a5-a66c-6542791f3506-original-pull-secret podName:b9f0f3b9-87ee-44a5-a66c-6542791f3506 nodeName:}" failed. No retries permitted until 2026-09-16 18:44:35.749990483 +0000 UTC m=+6.079492882 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b9f0f3b9-87ee-44a5-a66c-6542791f3506-original-pull-secret") pod "global-pull-secret-syncer-t69xl" (UID: "b9f0f3b9-87ee-44a5-a66c-6542791f3506") : object "kube-system"/"original-pull-secret" not registered
Sep 16 18:44:33.850416 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:33.850375    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l4z5w\" (UniqueName: \"kubernetes.io/projected/df3c2b31-63a6-43e4-a4ae-0199c34a6a86-kube-api-access-l4z5w\") pod \"network-check-target-f4md6\" (UID: \"df3c2b31-63a6-43e4-a4ae-0199c34a6a86\") " pod="openshift-network-diagnostics/network-check-target-f4md6"
Sep 16 18:44:33.850623 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:44:33.850605    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 16 18:44:33.850692 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:44:33.850636    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 16 18:44:33.850692 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:44:33.850649    2584 projected.go:196] Error preparing data for projected volume kube-api-access-l4z5w for pod openshift-network-diagnostics/network-check-target-f4md6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 18:44:33.850804 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:44:33.850710    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/df3c2b31-63a6-43e4-a4ae-0199c34a6a86-kube-api-access-l4z5w podName:df3c2b31-63a6-43e4-a4ae-0199c34a6a86 nodeName:}" failed. No retries permitted until 2026-09-16 18:44:35.850691237 +0000 UTC m=+6.180193633 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-l4z5w" (UniqueName: "kubernetes.io/projected/df3c2b31-63a6-43e4-a4ae-0199c34a6a86-kube-api-access-l4z5w") pod "network-check-target-f4md6" (UID: "df3c2b31-63a6-43e4-a4ae-0199c34a6a86") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 18:44:34.071708 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:34.071589    2584 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-15 18:39:32 +0000 UTC" deadline="2028-04-04 14:58:23.504120103 +0000 UTC"
Sep 16 18:44:34.071708 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:34.071631    2584 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13580h13m49.432494077s"
Sep 16 18:44:34.114263 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:34.114233    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bp5mv"
Sep 16 18:44:34.114438 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:44:34.114375    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bp5mv" podUID="9b96048a-0005-410b-a710-909d21fbfde8"
Sep 16 18:44:34.155367 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:34.155325    2584 generic.go:356] "Generic (PLEG): container finished" podID="4fa23c77cdd18b7265820cf09e7b3e9d" containerID="6cf4af1371c8030733c659ff8c21fc521dd3bd50b9303831c1438af1abd78cf2" exitCode=0
Sep 16 18:44:34.156590 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:34.156540    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-182.ec2.internal" event={"ID":"4fa23c77cdd18b7265820cf09e7b3e9d","Type":"ContainerDied","Data":"6cf4af1371c8030733c659ff8c21fc521dd3bd50b9303831c1438af1abd78cf2"}
Sep 16 18:44:34.174989 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:34.174219    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-182.ec2.internal" podStartSLOduration=3.17420077 podStartE2EDuration="3.17420077s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 18:44:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:44:33.150853407 +0000 UTC m=+3.480355816" watchObservedRunningTime="2026-09-16 18:44:34.17420077 +0000 UTC m=+4.503703188"
Sep 16 18:44:35.111335 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:35.111294    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t69xl"
Sep 16 18:44:35.112192 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:44:35.111435    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t69xl" podUID="b9f0f3b9-87ee-44a5-a66c-6542791f3506"
Sep 16 18:44:35.112192 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:35.111882    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f4md6"
Sep 16 18:44:35.112192 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:44:35.111985    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f4md6" podUID="df3c2b31-63a6-43e4-a4ae-0199c34a6a86"
Sep 16 18:44:35.192645 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:35.192527    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-182.ec2.internal" event={"ID":"4fa23c77cdd18b7265820cf09e7b3e9d","Type":"ContainerStarted","Data":"b192704be065f376674830c40dde65fdae4506a204f23005cf436928e095a39c"}
Sep 16 18:44:35.210213 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:35.210134    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-182.ec2.internal" podStartSLOduration=4.210112843 podStartE2EDuration="4.210112843s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 18:44:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:44:35.209754268 +0000 UTC m=+5.539256685" watchObservedRunningTime="2026-09-16 18:44:35.210112843 +0000 UTC m=+5.539615260"
Sep 16 18:44:35.670209 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:35.670158    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9b96048a-0005-410b-a710-909d21fbfde8-metrics-certs\") pod \"network-metrics-daemon-bp5mv\" (UID: \"9b96048a-0005-410b-a710-909d21fbfde8\") " pod="openshift-multus/network-metrics-daemon-bp5mv"
Sep 16 18:44:35.670477 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:44:35.670323    2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 18:44:35.670477 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:44:35.670392    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9b96048a-0005-410b-a710-909d21fbfde8-metrics-certs podName:9b96048a-0005-410b-a710-909d21fbfde8 nodeName:}" failed. No retries permitted until 2026-09-16 18:44:39.670373324 +0000 UTC m=+9.999875720 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9b96048a-0005-410b-a710-909d21fbfde8-metrics-certs") pod "network-metrics-daemon-bp5mv" (UID: "9b96048a-0005-410b-a710-909d21fbfde8") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 18:44:35.771363 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:35.771307    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b9f0f3b9-87ee-44a5-a66c-6542791f3506-original-pull-secret\") pod \"global-pull-secret-syncer-t69xl\" (UID: \"b9f0f3b9-87ee-44a5-a66c-6542791f3506\") " pod="kube-system/global-pull-secret-syncer-t69xl"
Sep 16 18:44:35.771565 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:44:35.771451    2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 16 18:44:35.771565 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:44:35.771525    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b9f0f3b9-87ee-44a5-a66c-6542791f3506-original-pull-secret podName:b9f0f3b9-87ee-44a5-a66c-6542791f3506 nodeName:}" failed. No retries permitted until 2026-09-16 18:44:39.771506839 +0000 UTC m=+10.101009245 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b9f0f3b9-87ee-44a5-a66c-6542791f3506-original-pull-secret") pod "global-pull-secret-syncer-t69xl" (UID: "b9f0f3b9-87ee-44a5-a66c-6542791f3506") : object "kube-system"/"original-pull-secret" not registered
Sep 16 18:44:35.872605 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:35.872565    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l4z5w\" (UniqueName: \"kubernetes.io/projected/df3c2b31-63a6-43e4-a4ae-0199c34a6a86-kube-api-access-l4z5w\") pod \"network-check-target-f4md6\" (UID: \"df3c2b31-63a6-43e4-a4ae-0199c34a6a86\") " pod="openshift-network-diagnostics/network-check-target-f4md6"
Sep 16 18:44:35.872883 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:44:35.872779    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 16 18:44:35.872883 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:44:35.872801    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 16 18:44:35.872883 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:44:35.872814    2584 projected.go:196] Error preparing data for projected volume kube-api-access-l4z5w for pod openshift-network-diagnostics/network-check-target-f4md6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 18:44:35.873075 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:44:35.872872    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/df3c2b31-63a6-43e4-a4ae-0199c34a6a86-kube-api-access-l4z5w podName:df3c2b31-63a6-43e4-a4ae-0199c34a6a86 nodeName:}" failed. No retries permitted until 2026-09-16 18:44:39.872854424 +0000 UTC m=+10.202356832 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-l4z5w" (UniqueName: "kubernetes.io/projected/df3c2b31-63a6-43e4-a4ae-0199c34a6a86-kube-api-access-l4z5w") pod "network-check-target-f4md6" (UID: "df3c2b31-63a6-43e4-a4ae-0199c34a6a86") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 18:44:36.111496 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:36.110940    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bp5mv"
Sep 16 18:44:36.111496 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:44:36.111083    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bp5mv" podUID="9b96048a-0005-410b-a710-909d21fbfde8"
Sep 16 18:44:37.110651 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:37.110573    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t69xl"
Sep 16 18:44:37.110821 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:44:37.110720    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t69xl" podUID="b9f0f3b9-87ee-44a5-a66c-6542791f3506"
Sep 16 18:44:37.111128 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:37.111102    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f4md6"
Sep 16 18:44:37.111256 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:44:37.111212    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f4md6" podUID="df3c2b31-63a6-43e4-a4ae-0199c34a6a86"
Sep 16 18:44:38.111277 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:38.111123    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bp5mv"
Sep 16 18:44:38.111277 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:44:38.111265    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bp5mv" podUID="9b96048a-0005-410b-a710-909d21fbfde8"
Sep 16 18:44:39.110573 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:39.110497    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t69xl"
Sep 16 18:44:39.110731 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:39.110627    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f4md6"
Sep 16 18:44:39.110731 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:44:39.110629    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t69xl" podUID="b9f0f3b9-87ee-44a5-a66c-6542791f3506"
Sep 16 18:44:39.110731 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:44:39.110713    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f4md6" podUID="df3c2b31-63a6-43e4-a4ae-0199c34a6a86"
Sep 16 18:44:39.707063 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:39.706810    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9b96048a-0005-410b-a710-909d21fbfde8-metrics-certs\") pod \"network-metrics-daemon-bp5mv\" (UID: \"9b96048a-0005-410b-a710-909d21fbfde8\") " pod="openshift-multus/network-metrics-daemon-bp5mv"
Sep 16 18:44:39.707063 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:44:39.707032    2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 18:44:39.707603 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:44:39.707102    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9b96048a-0005-410b-a710-909d21fbfde8-metrics-certs podName:9b96048a-0005-410b-a710-909d21fbfde8 nodeName:}" failed. No retries permitted until 2026-09-16 18:44:47.707082994 +0000 UTC m=+18.036585387 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9b96048a-0005-410b-a710-909d21fbfde8-metrics-certs") pod "network-metrics-daemon-bp5mv" (UID: "9b96048a-0005-410b-a710-909d21fbfde8") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 18:44:39.808089 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:39.808056    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b9f0f3b9-87ee-44a5-a66c-6542791f3506-original-pull-secret\") pod \"global-pull-secret-syncer-t69xl\" (UID: \"b9f0f3b9-87ee-44a5-a66c-6542791f3506\") " pod="kube-system/global-pull-secret-syncer-t69xl"
Sep 16 18:44:39.808279 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:44:39.808195    2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 16 18:44:39.808279 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:44:39.808257    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b9f0f3b9-87ee-44a5-a66c-6542791f3506-original-pull-secret podName:b9f0f3b9-87ee-44a5-a66c-6542791f3506 nodeName:}" failed. No retries permitted until 2026-09-16 18:44:47.808239959 +0000 UTC m=+18.137742362 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b9f0f3b9-87ee-44a5-a66c-6542791f3506-original-pull-secret") pod "global-pull-secret-syncer-t69xl" (UID: "b9f0f3b9-87ee-44a5-a66c-6542791f3506") : object "kube-system"/"original-pull-secret" not registered
Sep 16 18:44:39.909349 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:39.909311    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l4z5w\" (UniqueName: \"kubernetes.io/projected/df3c2b31-63a6-43e4-a4ae-0199c34a6a86-kube-api-access-l4z5w\") pod \"network-check-target-f4md6\" (UID: \"df3c2b31-63a6-43e4-a4ae-0199c34a6a86\") " pod="openshift-network-diagnostics/network-check-target-f4md6"
Sep 16 18:44:39.909542 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:44:39.909505    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 16 18:44:39.909542 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:44:39.909536    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 16 18:44:39.909542 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:44:39.909551    2584 projected.go:196] Error preparing data for projected volume kube-api-access-l4z5w for pod openshift-network-diagnostics/network-check-target-f4md6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 18:44:39.909714 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:44:39.909612    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/df3c2b31-63a6-43e4-a4ae-0199c34a6a86-kube-api-access-l4z5w podName:df3c2b31-63a6-43e4-a4ae-0199c34a6a86 nodeName:}" failed. No retries permitted until 2026-09-16 18:44:47.909595127 +0000 UTC m=+18.239097532 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-l4z5w" (UniqueName: "kubernetes.io/projected/df3c2b31-63a6-43e4-a4ae-0199c34a6a86-kube-api-access-l4z5w") pod "network-check-target-f4md6" (UID: "df3c2b31-63a6-43e4-a4ae-0199c34a6a86") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 18:44:40.112513 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:40.111950    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bp5mv"
Sep 16 18:44:40.112513 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:44:40.112088    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bp5mv" podUID="9b96048a-0005-410b-a710-909d21fbfde8"
Sep 16 18:44:41.111132 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:41.111100    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t69xl"
Sep 16 18:44:41.111550 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:44:41.111251    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t69xl" podUID="b9f0f3b9-87ee-44a5-a66c-6542791f3506"
Sep 16 18:44:41.111550 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:41.111107    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f4md6"
Sep 16 18:44:41.111550 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:44:41.111365    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f4md6" podUID="df3c2b31-63a6-43e4-a4ae-0199c34a6a86"
Sep 16 18:44:42.110808 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:42.110725    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bp5mv"
Sep 16 18:44:42.111007 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:44:42.110872    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bp5mv" podUID="9b96048a-0005-410b-a710-909d21fbfde8"
Sep 16 18:44:43.110719 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:43.110689    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f4md6"
Sep 16 18:44:43.111094 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:44:43.110789    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f4md6" podUID="df3c2b31-63a6-43e4-a4ae-0199c34a6a86"
Sep 16 18:44:43.111094 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:43.110689    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t69xl"
Sep 16 18:44:43.111094 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:44:43.110881    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t69xl" podUID="b9f0f3b9-87ee-44a5-a66c-6542791f3506"
Sep 16 18:44:44.111371 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:44.111331    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bp5mv"
Sep 16 18:44:44.111877 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:44:44.111474    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bp5mv" podUID="9b96048a-0005-410b-a710-909d21fbfde8"
Sep 16 18:44:45.110929 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:45.110897    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t69xl"
Sep 16 18:44:45.111102 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:45.110903    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f4md6"
Sep 16 18:44:45.111102 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:44:45.110997    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t69xl" podUID="b9f0f3b9-87ee-44a5-a66c-6542791f3506"
Sep 16 18:44:45.111191 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:44:45.111111    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f4md6" podUID="df3c2b31-63a6-43e4-a4ae-0199c34a6a86"
Sep 16 18:44:46.110998 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:46.110958    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bp5mv"
Sep 16 18:44:46.111419 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:44:46.111094    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bp5mv" podUID="9b96048a-0005-410b-a710-909d21fbfde8"
Sep 16 18:44:47.111220 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:47.111183    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f4md6"
Sep 16 18:44:47.111603 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:47.111182    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t69xl"
Sep 16 18:44:47.111603 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:44:47.111290    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f4md6" podUID="df3c2b31-63a6-43e4-a4ae-0199c34a6a86"
Sep 16 18:44:47.111603 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:44:47.111360    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t69xl" podUID="b9f0f3b9-87ee-44a5-a66c-6542791f3506"
Sep 16 18:44:47.768659 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:47.768617    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9b96048a-0005-410b-a710-909d21fbfde8-metrics-certs\") pod \"network-metrics-daemon-bp5mv\" (UID: \"9b96048a-0005-410b-a710-909d21fbfde8\") " pod="openshift-multus/network-metrics-daemon-bp5mv"
Sep 16 18:44:47.768855 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:44:47.768750    2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 18:44:47.768855 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:44:47.768843    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9b96048a-0005-410b-a710-909d21fbfde8-metrics-certs podName:9b96048a-0005-410b-a710-909d21fbfde8 nodeName:}" failed. No retries permitted until 2026-09-16 18:45:03.768809518 +0000 UTC m=+34.098311914 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9b96048a-0005-410b-a710-909d21fbfde8-metrics-certs") pod "network-metrics-daemon-bp5mv" (UID: "9b96048a-0005-410b-a710-909d21fbfde8") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 18:44:47.869729 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:47.869689    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b9f0f3b9-87ee-44a5-a66c-6542791f3506-original-pull-secret\") pod \"global-pull-secret-syncer-t69xl\" (UID: \"b9f0f3b9-87ee-44a5-a66c-6542791f3506\") " pod="kube-system/global-pull-secret-syncer-t69xl"
Sep 16 18:44:47.869943 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:44:47.869822    2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 16 18:44:47.869943 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:44:47.869907    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b9f0f3b9-87ee-44a5-a66c-6542791f3506-original-pull-secret podName:b9f0f3b9-87ee-44a5-a66c-6542791f3506 nodeName:}" failed. No retries permitted until 2026-09-16 18:45:03.869887986 +0000 UTC m=+34.199390397 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b9f0f3b9-87ee-44a5-a66c-6542791f3506-original-pull-secret") pod "global-pull-secret-syncer-t69xl" (UID: "b9f0f3b9-87ee-44a5-a66c-6542791f3506") : object "kube-system"/"original-pull-secret" not registered
Sep 16 18:44:47.970105 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:47.970056    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l4z5w\" (UniqueName: \"kubernetes.io/projected/df3c2b31-63a6-43e4-a4ae-0199c34a6a86-kube-api-access-l4z5w\") pod \"network-check-target-f4md6\" (UID: \"df3c2b31-63a6-43e4-a4ae-0199c34a6a86\") " pod="openshift-network-diagnostics/network-check-target-f4md6"
Sep 16 18:44:47.970295 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:44:47.970222    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 16 18:44:47.970295 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:44:47.970250    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 16 18:44:47.970295 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:44:47.970262    2584 projected.go:196] Error preparing data for projected volume kube-api-access-l4z5w for pod openshift-network-diagnostics/network-check-target-f4md6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 18:44:47.970448 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:44:47.970332    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/df3c2b31-63a6-43e4-a4ae-0199c34a6a86-kube-api-access-l4z5w podName:df3c2b31-63a6-43e4-a4ae-0199c34a6a86 nodeName:}" failed. No retries permitted until 2026-09-16 18:45:03.97031316 +0000 UTC m=+34.299815564 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-l4z5w" (UniqueName: "kubernetes.io/projected/df3c2b31-63a6-43e4-a4ae-0199c34a6a86-kube-api-access-l4z5w") pod "network-check-target-f4md6" (UID: "df3c2b31-63a6-43e4-a4ae-0199c34a6a86") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 18:44:48.111358 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:48.111270    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bp5mv"
Sep 16 18:44:48.111764 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:44:48.111419    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bp5mv" podUID="9b96048a-0005-410b-a710-909d21fbfde8"
Sep 16 18:44:49.110902 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:49.110870    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f4md6"
Sep 16 18:44:49.111075 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:49.110873    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t69xl"
Sep 16 18:44:49.111075 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:44:49.111003    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f4md6" podUID="df3c2b31-63a6-43e4-a4ae-0199c34a6a86"
Sep 16 18:44:49.111179 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:44:49.111079    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t69xl" podUID="b9f0f3b9-87ee-44a5-a66c-6542791f3506"
Sep 16 18:44:50.111647 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:50.111610    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bp5mv"
Sep 16 18:44:50.112117 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:44:50.111716    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bp5mv" podUID="9b96048a-0005-410b-a710-909d21fbfde8"
Sep 16 18:44:51.110982 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:51.110953    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f4md6"
Sep 16 18:44:51.111118 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:44:51.111049    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f4md6" podUID="df3c2b31-63a6-43e4-a4ae-0199c34a6a86"
Sep 16 18:44:51.111285 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:51.110952    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t69xl"
Sep 16 18:44:51.111285 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:44:51.111261    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t69xl" podUID="b9f0f3b9-87ee-44a5-a66c-6542791f3506"
Sep 16 18:44:51.230115 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:51.230082    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-6klb4" event={"ID":"828d6bf1-3d76-4c9b-af83-07de04f8f20f","Type":"ContainerStarted","Data":"156bbb95adf08285090233a72cfd2fd309515ddbb57b6dc02b9b2825079afa63"}
Sep 16 18:44:51.232195 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:51.232079    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fbw2l" event={"ID":"71637102-fba0-4c38-8055-6dceacddcde9","Type":"ContainerStarted","Data":"c895dc5a4ab4602b9baada83f20b7e6155d3621a9ed99d8865aa008eb700c83d"}
Sep 16 18:44:51.234090 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:51.234059    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-f8jpf" event={"ID":"1498d81a-223d-470c-9812-9b2910e3ea35","Type":"ContainerStarted","Data":"46cc22347e831ff7316ce9a11524b0e0ca42d4e3b23a3a992fb2ad5bba4d8039"}
Sep 16 18:44:51.245859 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:51.245802    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-6klb4" podStartSLOduration=6.992457279 podStartE2EDuration="21.245789454s" totalImagesPullingTime="14.253332175s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:44:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:44:51.245632993 +0000 UTC m=+21.575135416" watchObservedRunningTime="2026-09-16 18:44:51.245789454 +0000 UTC m=+21.575291866"
Sep 16 18:44:51.287258 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:51.286758    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-f8jpf" podStartSLOduration=3.312984601 podStartE2EDuration="21.286739032s" totalImagesPullingTime="17.973754431s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 18:44:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:44:51.285986614 +0000 UTC m=+21.615489025" watchObservedRunningTime="2026-09-16 18:44:51.286739032 +0000 UTC m=+21.616241446"
Sep 16 18:44:51.312544 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:51.312513    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:51.394892 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:51.394727    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:52.110934 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:52.110896    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bp5mv"
Sep 16 18:44:52.111176 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:44:52.111057    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bp5mv" podUID="9b96048a-0005-410b-a710-909d21fbfde8"
Sep 16 18:44:52.237848 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:52.237810    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-bd22w" event={"ID":"4a80b1e8-39b4-44a2-80cb-aa52d96e80fb","Type":"ContainerStarted","Data":"cb1d6bda93eccfd4494b259cdfe10d63afdfacd46fa232d15e1b79c0d81f1dde"}
Sep 16 18:44:52.239343 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:52.239276    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-d9bgl" event={"ID":"c42d9b1b-2d14-469c-b986-ae99600b602c","Type":"ContainerStarted","Data":"77262050b7f2bb47a0e44f427af7b790ce09490f8b266b13ee02163aec23f15f"}
Sep 16 18:44:52.240774 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:52.240747    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-9z8sh" event={"ID":"7892f55c-beff-42d9-989e-52eeb778bdc5","Type":"ContainerStarted","Data":"063cf1d807ce26ca59ec88381a40d742aa655cf42ed8f5f31ea649be1c1f0931"}
Sep 16 18:44:52.242259 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:52.242235    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gpwqt" event={"ID":"ac357968-eba1-4c7d-97a7-b1a8e691e22e","Type":"ContainerStarted","Data":"138d13a0b78b8793e3678b03c6cbd1217da51f5ff7322414d8ab91e9de90bc34"}
Sep 16 18:44:52.245433 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:52.245409    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-blv9h" event={"ID":"08febeb1-4769-4fd6-911f-a0a02c7c4223","Type":"ContainerStarted","Data":"286dd6c4eb39e9d5a92ee530f1b1c920166fb7353f918d9fca9f0715ed458687"}
Sep 16 18:44:52.245544 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:52.245486    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-blv9h" event={"ID":"08febeb1-4769-4fd6-911f-a0a02c7c4223","Type":"ContainerStarted","Data":"58b294297cabd26a8dc442ffe75f015b3e6cf05d1c5b4c5fefe15de49dcb3bc0"}
Sep 16 18:44:52.245544 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:52.245539    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-blv9h" event={"ID":"08febeb1-4769-4fd6-911f-a0a02c7c4223","Type":"ContainerStarted","Data":"49a2ee70f56a889f9082b62085e084602e71d10b068e39ed56e81e882fb8f17e"}
Sep 16 18:44:52.245617 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:52.245589    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-blv9h" event={"ID":"08febeb1-4769-4fd6-911f-a0a02c7c4223","Type":"ContainerStarted","Data":"8ba689113336ce7342c1c6b327764b7f9230c021938ccf27e5c5f8a848fe8eb0"}
Sep 16 18:44:52.245661 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:52.245637    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-blv9h" event={"ID":"08febeb1-4769-4fd6-911f-a0a02c7c4223","Type":"ContainerStarted","Data":"bb3cfaede07173f096e1b8d2ad771b8b8ed9d059d05528fbe2b3b0bf68ee53d1"}
Sep 16 18:44:52.245715 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:52.245668    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-blv9h" event={"ID":"08febeb1-4769-4fd6-911f-a0a02c7c4223","Type":"ContainerStarted","Data":"677710be4cfe5c162e56c499f511c1861caee498108773540f4411816cc0deef"}
Sep 16 18:44:52.246992 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:52.246967    2584 generic.go:356] "Generic (PLEG): container finished" podID="71637102-fba0-4c38-8055-6dceacddcde9" containerID="c895dc5a4ab4602b9baada83f20b7e6155d3621a9ed99d8865aa008eb700c83d" exitCode=0
Sep 16 18:44:52.247104 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:52.247045    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fbw2l" event={"ID":"71637102-fba0-4c38-8055-6dceacddcde9","Type":"ContainerDied","Data":"c895dc5a4ab4602b9baada83f20b7e6155d3621a9ed99d8865aa008eb700c83d"}
Sep 16 18:44:52.253483 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:52.253446    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-bd22w" podStartSLOduration=4.270707952 podStartE2EDuration="22.253433174s" totalImagesPullingTime="17.982725222s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 18:44:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:44:52.252864852 +0000 UTC m=+22.582367266" watchObservedRunningTime="2026-09-16 18:44:52.253433174 +0000 UTC m=+22.582935584"
Sep 16 18:44:52.292099 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:52.292050    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-d9bgl" podStartSLOduration=3.963179435 podStartE2EDuration="22.292035933s" totalImagesPullingTime="18.328856498s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 18:44:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:44:52.291703429 +0000 UTC m=+22.621205841" watchObservedRunningTime="2026-09-16 18:44:52.292035933 +0000 UTC m=+22.621538346"
Sep 16 18:44:52.448002 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:52.447962    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:52.546745 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:52.546711    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:52.634049 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:52.634023    2584 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 16 18:44:53.085445 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:53.085326    2584 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-16T18:44:52.634042318Z","UUID":"75af6b5e-085c-4e54-a114-43048e8e8cba","Handler":null,"Name":"","Endpoint":""}
Sep 16 18:44:53.087189 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:53.087163    2584 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 16 18:44:53.087328 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:53.087200    2584 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 16 18:44:53.110657 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:53.110577    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t69xl"
Sep 16 18:44:53.110869 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:44:53.110700    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t69xl" podUID="b9f0f3b9-87ee-44a5-a66c-6542791f3506"
Sep 16 18:44:53.110869 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:53.110826    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f4md6"
Sep 16 18:44:53.111014 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:44:53.110991    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f4md6" podUID="df3c2b31-63a6-43e4-a4ae-0199c34a6a86"
Sep 16 18:44:53.252426 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:53.252346    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gpwqt" event={"ID":"ac357968-eba1-4c7d-97a7-b1a8e691e22e","Type":"ContainerStarted","Data":"4b9999e6d15715d768750d400fef63ad86e9186ec8ee41bd5cc72ec5e1872348"}
Sep 16 18:44:53.254619 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:53.254469    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-qqjgk" event={"ID":"a29102aa-22ea-4623-84fc-0be1e5506274","Type":"ContainerStarted","Data":"23c4c738a0078da670653894031febc15884f93383bf99d43ad1277e46ea9dda"}
Sep 16 18:44:53.270312 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:53.270260    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-9z8sh" podStartSLOduration=5.286781842 podStartE2EDuration="23.270239844s" totalImagesPullingTime="17.983458002s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 18:44:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:44:52.318006736 +0000 UTC m=+22.647509149" watchObservedRunningTime="2026-09-16 18:44:53.270239844 +0000 UTC m=+23.599742259"
Sep 16 18:44:53.270615 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:53.270586    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-qqjgk" podStartSLOduration=5.293252074 podStartE2EDuration="23.270574794s" totalImagesPullingTime="17.97732272s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 18:44:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:44:53.269913155 +0000 UTC m=+23.599415572" watchObservedRunningTime="2026-09-16 18:44:53.270574794 +0000 UTC m=+23.600077206"
Sep 16 18:44:53.388294 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:53.388212    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:53.487911 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:53.487866    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:54.111488 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:54.111452    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bp5mv"
Sep 16 18:44:54.111774 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:44:54.111600    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bp5mv" podUID="9b96048a-0005-410b-a710-909d21fbfde8"
Sep 16 18:44:54.260089 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:54.260043    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gpwqt" event={"ID":"ac357968-eba1-4c7d-97a7-b1a8e691e22e","Type":"ContainerStarted","Data":"0ed7baf6b01a4798a6d296649bbdef03946c2135664a22caf2bcc1fe6c153ffd"}
Sep 16 18:44:54.264329 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:54.264281    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-blv9h" event={"ID":"08febeb1-4769-4fd6-911f-a0a02c7c4223","Type":"ContainerStarted","Data":"a71c02f5ac3554129be44747c228fa05856543726eadd8a8dd5df7191947789a"}
Sep 16 18:44:55.111040 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:55.111001    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t69xl"
Sep 16 18:44:55.111040 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:55.111037    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f4md6"
Sep 16 18:44:55.111288 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:44:55.111112    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t69xl" podUID="b9f0f3b9-87ee-44a5-a66c-6542791f3506"
Sep 16 18:44:55.111288 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:44:55.111249    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f4md6" podUID="df3c2b31-63a6-43e4-a4ae-0199c34a6a86"
Sep 16 18:44:55.907287 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:55.907234    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-6klb4"
Sep 16 18:44:55.907894 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:55.907873    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-6klb4"
Sep 16 18:44:55.923506 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:55.923448    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gpwqt" podStartSLOduration=5.818865032 podStartE2EDuration="25.923426052s" totalImagesPullingTime="20.10456102s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 18:44:30 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:44:54.277026631 +0000 UTC m=+24.606529044" watchObservedRunningTime="2026-09-16 18:44:55.923426052 +0000 UTC m=+26.252928466"
Sep 16 18:44:56.110955 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:56.110876    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bp5mv"
Sep 16 18:44:56.111120 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:44:56.111008    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bp5mv" podUID="9b96048a-0005-410b-a710-909d21fbfde8"
Sep 16 18:44:57.110899 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:57.110712    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t69xl"
Sep 16 18:44:57.111256 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:57.110712    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f4md6"
Sep 16 18:44:57.111256 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:44:57.110968    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t69xl" podUID="b9f0f3b9-87ee-44a5-a66c-6542791f3506"
Sep 16 18:44:57.111256 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:44:57.111052    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f4md6" podUID="df3c2b31-63a6-43e4-a4ae-0199c34a6a86"
Sep 16 18:44:57.275385 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:57.275351    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-blv9h" event={"ID":"08febeb1-4769-4fd6-911f-a0a02c7c4223","Type":"ContainerStarted","Data":"bd0390203c3adca7b144d9e486088ee55bbda3737ca40f2c3d363752920b6aa1"}
Sep 16 18:44:57.275622 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:57.275593    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-blv9h"
Sep 16 18:44:57.277260 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:57.277235    2584 generic.go:356] "Generic (PLEG): container finished" podID="71637102-fba0-4c38-8055-6dceacddcde9" containerID="3b1f403eab67ac3eaed22b01c60ce823c3d6032c9f8f76b9126f915c847dbb27" exitCode=0
Sep 16 18:44:57.277365 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:57.277268    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fbw2l" event={"ID":"71637102-fba0-4c38-8055-6dceacddcde9","Type":"ContainerDied","Data":"3b1f403eab67ac3eaed22b01c60ce823c3d6032c9f8f76b9126f915c847dbb27"}
Sep 16 18:44:57.293222 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:57.293198    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-blv9h"
Sep 16 18:44:57.301794 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:57.301750    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-blv9h" podStartSLOduration=9.295144666 podStartE2EDuration="27.301735361s" totalImagesPullingTime="18.006590695s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 18:44:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:44:57.301100309 +0000 UTC m=+27.630602723" watchObservedRunningTime="2026-09-16 18:44:57.301735361 +0000 UTC m=+27.631237792"
Sep 16 18:44:58.005156 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:58.005093    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-6klb4"
Sep 16 18:44:58.005437 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:58.005261    2584 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 16 18:44:58.005930 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:58.005909    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-6klb4"
Sep 16 18:44:58.063700 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:58.063646    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:58.111160 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:58.111127    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bp5mv"
Sep 16 18:44:58.111825 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:44:58.111250    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bp5mv" podUID="9b96048a-0005-410b-a710-909d21fbfde8"
Sep 16 18:44:58.243172 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:58.243141    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-f4md6"]
Sep 16 18:44:58.243340 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:58.243261    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f4md6"
Sep 16 18:44:58.243395 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:44:58.243341    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f4md6" podUID="df3c2b31-63a6-43e4-a4ae-0199c34a6a86"
Sep 16 18:44:58.246244 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:58.246210    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-bp5mv"]
Sep 16 18:44:58.246815 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:58.246791    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-t69xl"]
Sep 16 18:44:58.246967 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:58.246897    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t69xl"
Sep 16 18:44:58.247032 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:44:58.246992    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t69xl" podUID="b9f0f3b9-87ee-44a5-a66c-6542791f3506"
Sep 16 18:44:58.279821 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:58.279780    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bp5mv"
Sep 16 18:44:58.280003 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:44:58.279977    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bp5mv" podUID="9b96048a-0005-410b-a710-909d21fbfde8"
Sep 16 18:44:58.280485 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:58.280469    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-blv9h"
Sep 16 18:44:58.280549 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:58.280532    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-blv9h"
Sep 16 18:44:58.295102 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:58.295079    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-blv9h"
Sep 16 18:44:59.284235 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:59.284203    2584 generic.go:356] "Generic (PLEG): container finished" podID="71637102-fba0-4c38-8055-6dceacddcde9" containerID="b87b39400c0df737aa1895a823a05bc5d25b0ceb163398b4e83794520b917cca" exitCode=0
Sep 16 18:44:59.284635 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:44:59.284297    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fbw2l" event={"ID":"71637102-fba0-4c38-8055-6dceacddcde9","Type":"ContainerDied","Data":"b87b39400c0df737aa1895a823a05bc5d25b0ceb163398b4e83794520b917cca"}
Sep 16 18:45:00.114229 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:45:00.113972    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f4md6"
Sep 16 18:45:00.114229 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:45:00.113977    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bp5mv"
Sep 16 18:45:00.114229 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:45:00.114226    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f4md6" podUID="df3c2b31-63a6-43e4-a4ae-0199c34a6a86"
Sep 16 18:45:00.114541 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:45:00.113987    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t69xl"
Sep 16 18:45:00.114541 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:45:00.114377    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bp5mv" podUID="9b96048a-0005-410b-a710-909d21fbfde8"
Sep 16 18:45:00.114541 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:45:00.114447    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t69xl" podUID="b9f0f3b9-87ee-44a5-a66c-6542791f3506"
Sep 16 18:45:00.461183 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:45:00.461155    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:45:00.543079 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:45:00.543048    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:45:00.643334 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:45:00.643302    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:45:01.290534 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:45:01.290506    2584 generic.go:356] "Generic (PLEG): container finished" podID="71637102-fba0-4c38-8055-6dceacddcde9" containerID="30dda966a0833f84615b856a07c8c0e7dbe6a84ab77aa3b6aba1f7d56f5e31f0" exitCode=0
Sep 16 18:45:01.290701 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:45:01.290547    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fbw2l" event={"ID":"71637102-fba0-4c38-8055-6dceacddcde9","Type":"ContainerDied","Data":"30dda966a0833f84615b856a07c8c0e7dbe6a84ab77aa3b6aba1f7d56f5e31f0"}
Sep 16 18:45:01.507405 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:45:01.507374    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:45:01.706976 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:45:01.706893    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:45:02.110548 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:45:02.110515    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f4md6"
Sep 16 18:45:02.110719 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:45:02.110515    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bp5mv"
Sep 16 18:45:02.110719 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:45:02.110643    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f4md6" podUID="df3c2b31-63a6-43e4-a4ae-0199c34a6a86"
Sep 16 18:45:02.110719 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:45:02.110520    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t69xl"
Sep 16 18:45:02.111500 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:45:02.111366    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bp5mv" podUID="9b96048a-0005-410b-a710-909d21fbfde8"
Sep 16 18:45:02.111615 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:45:02.111510    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t69xl" podUID="b9f0f3b9-87ee-44a5-a66c-6542791f3506"
Sep 16 18:45:03.783327 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:45:03.783292    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9b96048a-0005-410b-a710-909d21fbfde8-metrics-certs\") pod \"network-metrics-daemon-bp5mv\" (UID: \"9b96048a-0005-410b-a710-909d21fbfde8\") " pod="openshift-multus/network-metrics-daemon-bp5mv"
Sep 16 18:45:03.783722 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:45:03.783459    2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 18:45:03.783722 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:45:03.783522    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9b96048a-0005-410b-a710-909d21fbfde8-metrics-certs podName:9b96048a-0005-410b-a710-909d21fbfde8 nodeName:}" failed. No retries permitted until 2026-09-16 18:45:35.78350763 +0000 UTC m=+66.113010037 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9b96048a-0005-410b-a710-909d21fbfde8-metrics-certs") pod "network-metrics-daemon-bp5mv" (UID: "9b96048a-0005-410b-a710-909d21fbfde8") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 18:45:03.884527 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:45:03.884496    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b9f0f3b9-87ee-44a5-a66c-6542791f3506-original-pull-secret\") pod \"global-pull-secret-syncer-t69xl\" (UID: \"b9f0f3b9-87ee-44a5-a66c-6542791f3506\") " pod="kube-system/global-pull-secret-syncer-t69xl"
Sep 16 18:45:03.884672 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:45:03.884635    2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 16 18:45:03.884731 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:45:03.884704    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b9f0f3b9-87ee-44a5-a66c-6542791f3506-original-pull-secret podName:b9f0f3b9-87ee-44a5-a66c-6542791f3506 nodeName:}" failed. No retries permitted until 2026-09-16 18:45:35.884685456 +0000 UTC m=+66.214187861 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b9f0f3b9-87ee-44a5-a66c-6542791f3506-original-pull-secret") pod "global-pull-secret-syncer-t69xl" (UID: "b9f0f3b9-87ee-44a5-a66c-6542791f3506") : object "kube-system"/"original-pull-secret" not registered
Sep 16 18:45:03.985083 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:45:03.984989    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l4z5w\" (UniqueName: \"kubernetes.io/projected/df3c2b31-63a6-43e4-a4ae-0199c34a6a86-kube-api-access-l4z5w\") pod \"network-check-target-f4md6\" (UID: \"df3c2b31-63a6-43e4-a4ae-0199c34a6a86\") " pod="openshift-network-diagnostics/network-check-target-f4md6"
Sep 16 18:45:03.985227 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:45:03.985182    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 16 18:45:03.985227 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:45:03.985208    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 16 18:45:03.985227 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:45:03.985218    2584 projected.go:196] Error preparing data for projected volume kube-api-access-l4z5w for pod openshift-network-diagnostics/network-check-target-f4md6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 18:45:03.985339 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:45:03.985279    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/df3c2b31-63a6-43e4-a4ae-0199c34a6a86-kube-api-access-l4z5w podName:df3c2b31-63a6-43e4-a4ae-0199c34a6a86 nodeName:}" failed. No retries permitted until 2026-09-16 18:45:35.985257062 +0000 UTC m=+66.314759476 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-l4z5w" (UniqueName: "kubernetes.io/projected/df3c2b31-63a6-43e4-a4ae-0199c34a6a86-kube-api-access-l4z5w") pod "network-check-target-f4md6" (UID: "df3c2b31-63a6-43e4-a4ae-0199c34a6a86") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 18:45:04.010763 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:45:04.010732    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-182.ec2.internal" event="NodeReady"
Sep 16 18:45:04.010918 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:45:04.010902    2584 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 16 18:45:04.074012 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:45:04.073980    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-tv99q"]
Sep 16 18:45:04.076331 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:45:04.076311    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-tv99q"
Sep 16 18:45:04.079621 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:45:04.079595    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-l7s4t\""
Sep 16 18:45:04.079747 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:45:04.079675    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 16 18:45:04.079804 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:45:04.079753    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 16 18:45:04.082414 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:45:04.082388    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-s4gvd"]
Sep 16 18:45:04.084088 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:45:04.084066    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-s4gvd"
Sep 16 18:45:04.087019 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:45:04.086998    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 16 18:45:04.087132 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:45:04.087032    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-p2cz7\""
Sep 16 18:45:04.087195 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:45:04.087140    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 16 18:45:04.087301 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:45:04.087280    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 16 18:45:04.095313 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:45:04.095291    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-tv99q"]
Sep 16 18:45:04.096055 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:45:04.096031    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-s4gvd"]
Sep 16 18:45:04.110535 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:45:04.110457    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t69xl"
Sep 16 18:45:04.110685 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:45:04.110458    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f4md6"
Sep 16 18:45:04.110685 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:45:04.110591    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bp5mv"
Sep 16 18:45:04.113275 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:45:04.113245    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 16 18:45:04.113421 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:45:04.113330    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 16 18:45:04.113421 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:45:04.113383    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 16 18:45:04.113612 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:45:04.113576    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-5l9rg\""
Sep 16 18:45:04.114281 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:45:04.114264    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-7prs7\""
Sep 16 18:45:04.114383 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:45:04.114281    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 16 18:45:04.176660 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:45:04.176636    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:45:04.186576 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:45:04.186538    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7a4aa47d-be1a-4b2c-bd88-3edc40ef5d6a-tmp-dir\") pod \"dns-default-tv99q\" (UID: \"7a4aa47d-be1a-4b2c-bd88-3edc40ef5d6a\") " pod="openshift-dns/dns-default-tv99q"
Sep 16 18:45:04.186740 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:45:04.186590    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/80b7ba2b-64e7-4bf3-9f78-d8a2b2b0e66a-cert\") pod \"ingress-canary-s4gvd\" (UID: \"80b7ba2b-64e7-4bf3-9f78-d8a2b2b0e66a\") " pod="openshift-ingress-canary/ingress-canary-s4gvd"
Sep 16 18:45:04.186740 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:45:04.186637    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2hnpn\" (UniqueName: \"kubernetes.io/projected/80b7ba2b-64e7-4bf3-9f78-d8a2b2b0e66a-kube-api-access-2hnpn\") pod \"ingress-canary-s4gvd\" (UID: \"80b7ba2b-64e7-4bf3-9f78-d8a2b2b0e66a\") " pod="openshift-ingress-canary/ingress-canary-s4gvd"
Sep 16 18:45:04.186740 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:45:04.186688    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wdml6\" (UniqueName: \"kubernetes.io/projected/7a4aa47d-be1a-4b2c-bd88-3edc40ef5d6a-kube-api-access-wdml6\") pod \"dns-default-tv99q\" (UID: \"7a4aa47d-be1a-4b2c-bd88-3edc40ef5d6a\") " pod="openshift-dns/dns-default-tv99q"
Sep 16 18:45:04.186740 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:45:04.186730    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/7a4aa47d-be1a-4b2c-bd88-3edc40ef5d6a-config-volume\") pod \"dns-default-tv99q\" (UID: \"7a4aa47d-be1a-4b2c-bd88-3edc40ef5d6a\") " pod="openshift-dns/dns-default-tv99q"
Sep 16 18:45:04.186924 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:45:04.186761    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7a4aa47d-be1a-4b2c-bd88-3edc40ef5d6a-metrics-tls\") pod \"dns-default-tv99q\" (UID: \"7a4aa47d-be1a-4b2c-bd88-3edc40ef5d6a\") " pod="openshift-dns/dns-default-tv99q"
Sep 16 18:45:04.234152 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:45:04.234121    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:45:04.287296 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:45:04.287269    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/7a4aa47d-be1a-4b2c-bd88-3edc40ef5d6a-config-volume\") pod \"dns-default-tv99q\" (UID: \"7a4aa47d-be1a-4b2c-bd88-3edc40ef5d6a\") " pod="openshift-dns/dns-default-tv99q"
Sep 16 18:45:04.287478 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:45:04.287321    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7a4aa47d-be1a-4b2c-bd88-3edc40ef5d6a-metrics-tls\") pod \"dns-default-tv99q\" (UID: \"7a4aa47d-be1a-4b2c-bd88-3edc40ef5d6a\") " pod="openshift-dns/dns-default-tv99q"
Sep 16 18:45:04.287478 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:45:04.287430    2584 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 16 18:45:04.287478 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:45:04.287447    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7a4aa47d-be1a-4b2c-bd88-3edc40ef5d6a-tmp-dir\") pod \"dns-default-tv99q\" (UID: \"7a4aa47d-be1a-4b2c-bd88-3edc40ef5d6a\") " pod="openshift-dns/dns-default-tv99q"
Sep 16 18:45:04.287635 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:45:04.287489    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7a4aa47d-be1a-4b2c-bd88-3edc40ef5d6a-metrics-tls podName:7a4aa47d-be1a-4b2c-bd88-3edc40ef5d6a nodeName:}" failed. No retries permitted until 2026-09-16 18:45:04.78746854 +0000 UTC m=+35.116970996 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7a4aa47d-be1a-4b2c-bd88-3edc40ef5d6a-metrics-tls") pod "dns-default-tv99q" (UID: "7a4aa47d-be1a-4b2c-bd88-3edc40ef5d6a") : secret "dns-default-metrics-tls" not found
Sep 16 18:45:04.288207 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:45:04.287883    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/80b7ba2b-64e7-4bf3-9f78-d8a2b2b0e66a-cert\") pod \"ingress-canary-s4gvd\" (UID: \"80b7ba2b-64e7-4bf3-9f78-d8a2b2b0e66a\") " pod="openshift-ingress-canary/ingress-canary-s4gvd"
Sep 16 18:45:04.288207 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:45:04.287970    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2hnpn\" (UniqueName: \"kubernetes.io/projected/80b7ba2b-64e7-4bf3-9f78-d8a2b2b0e66a-kube-api-access-2hnpn\") pod \"ingress-canary-s4gvd\" (UID: \"80b7ba2b-64e7-4bf3-9f78-d8a2b2b0e66a\") " pod="openshift-ingress-canary/ingress-canary-s4gvd"
Sep 16 18:45:04.288207 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:45:04.288015    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wdml6\" (UniqueName: \"kubernetes.io/projected/7a4aa47d-be1a-4b2c-bd88-3edc40ef5d6a-kube-api-access-wdml6\") pod \"dns-default-tv99q\" (UID: \"7a4aa47d-be1a-4b2c-bd88-3edc40ef5d6a\") " pod="openshift-dns/dns-default-tv99q"
Sep 16 18:45:04.288207 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:45:04.288087    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7a4aa47d-be1a-4b2c-bd88-3edc40ef5d6a-tmp-dir\") pod \"dns-default-tv99q\" (UID: \"7a4aa47d-be1a-4b2c-bd88-3edc40ef5d6a\") " pod="openshift-dns/dns-default-tv99q"
Sep 16 18:45:04.288746 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:45:04.288721    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/7a4aa47d-be1a-4b2c-bd88-3edc40ef5d6a-config-volume\") pod \"dns-default-tv99q\" (UID: \"7a4aa47d-be1a-4b2c-bd88-3edc40ef5d6a\") " pod="openshift-dns/dns-default-tv99q"
Sep 16 18:45:04.293539 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:45:04.293516    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/80b7ba2b-64e7-4bf3-9f78-d8a2b2b0e66a-cert\") pod \"ingress-canary-s4gvd\" (UID: \"80b7ba2b-64e7-4bf3-9f78-d8a2b2b0e66a\") " pod="openshift-ingress-canary/ingress-canary-s4gvd"
Sep 16 18:45:04.297549 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:45:04.297523    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2hnpn\" (UniqueName: \"kubernetes.io/projected/80b7ba2b-64e7-4bf3-9f78-d8a2b2b0e66a-kube-api-access-2hnpn\") pod \"ingress-canary-s4gvd\" (UID: \"80b7ba2b-64e7-4bf3-9f78-d8a2b2b0e66a\") " pod="openshift-ingress-canary/ingress-canary-s4gvd"
Sep 16 18:45:04.297879 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:45:04.297851    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wdml6\" (UniqueName: \"kubernetes.io/projected/7a4aa47d-be1a-4b2c-bd88-3edc40ef5d6a-kube-api-access-wdml6\") pod \"dns-default-tv99q\" (UID: \"7a4aa47d-be1a-4b2c-bd88-3edc40ef5d6a\") " pod="openshift-dns/dns-default-tv99q"
Sep 16 18:45:04.394978 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:45:04.394942    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-s4gvd"
Sep 16 18:45:04.792663 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:45:04.792629    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7a4aa47d-be1a-4b2c-bd88-3edc40ef5d6a-metrics-tls\") pod \"dns-default-tv99q\" (UID: \"7a4aa47d-be1a-4b2c-bd88-3edc40ef5d6a\") " pod="openshift-dns/dns-default-tv99q"
Sep 16 18:45:04.793133 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:45:04.792783    2584 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 16 18:45:04.793133 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:45:04.792853    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7a4aa47d-be1a-4b2c-bd88-3edc40ef5d6a-metrics-tls podName:7a4aa47d-be1a-4b2c-bd88-3edc40ef5d6a nodeName:}" failed. No retries permitted until 2026-09-16 18:45:05.792823562 +0000 UTC m=+36.122325971 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7a4aa47d-be1a-4b2c-bd88-3edc40ef5d6a-metrics-tls") pod "dns-default-tv99q" (UID: "7a4aa47d-be1a-4b2c-bd88-3edc40ef5d6a") : secret "dns-default-metrics-tls" not found
Sep 16 18:45:05.509930 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:45:05.509742    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-s4gvd"]
Sep 16 18:45:05.517713 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:45:05.517680    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80b7ba2b_64e7_4bf3_9f78_d8a2b2b0e66a.slice/crio-194603d766fc867813fe7053f1cebe5271f1ce5165eb8feaaf16b67b0a853512 WatchSource:0}: Error finding container 194603d766fc867813fe7053f1cebe5271f1ce5165eb8feaaf16b67b0a853512: Status 404 returned error can't find the container with id 194603d766fc867813fe7053f1cebe5271f1ce5165eb8feaaf16b67b0a853512
Sep 16 18:45:05.800207 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:45:05.800125    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7a4aa47d-be1a-4b2c-bd88-3edc40ef5d6a-metrics-tls\") pod \"dns-default-tv99q\" (UID: \"7a4aa47d-be1a-4b2c-bd88-3edc40ef5d6a\") " pod="openshift-dns/dns-default-tv99q"
Sep 16 18:45:05.800699 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:45:05.800249    2584 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 16 18:45:05.800699 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:45:05.800302    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7a4aa47d-be1a-4b2c-bd88-3edc40ef5d6a-metrics-tls podName:7a4aa47d-be1a-4b2c-bd88-3edc40ef5d6a nodeName:}" failed. No retries permitted until 2026-09-16 18:45:07.800289149 +0000 UTC m=+38.129791541 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7a4aa47d-be1a-4b2c-bd88-3edc40ef5d6a-metrics-tls") pod "dns-default-tv99q" (UID: "7a4aa47d-be1a-4b2c-bd88-3edc40ef5d6a") : secret "dns-default-metrics-tls" not found
Sep 16 18:45:06.304251 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:45:06.304184    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-s4gvd" event={"ID":"80b7ba2b-64e7-4bf3-9f78-d8a2b2b0e66a","Type":"ContainerStarted","Data":"194603d766fc867813fe7053f1cebe5271f1ce5165eb8feaaf16b67b0a853512"}
Sep 16 18:45:06.307924 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:45:06.307892    2584 generic.go:356] "Generic (PLEG): container finished" podID="71637102-fba0-4c38-8055-6dceacddcde9" containerID="5241ffb822976cceff3916661975cf5509623d5bc2d31099e7327927d08e5303" exitCode=0
Sep 16 18:45:06.308102 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:45:06.307934    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fbw2l" event={"ID":"71637102-fba0-4c38-8055-6dceacddcde9","Type":"ContainerDied","Data":"5241ffb822976cceff3916661975cf5509623d5bc2d31099e7327927d08e5303"}
Sep 16 18:45:07.314944 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:45:07.314912    2584 generic.go:356] "Generic (PLEG): container finished" podID="71637102-fba0-4c38-8055-6dceacddcde9" containerID="b423fa90d26832da4a07918e961c0e64869b222ba8424a4b616c72a4af974ab1" exitCode=0
Sep 16 18:45:07.315574 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:45:07.314952    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fbw2l" event={"ID":"71637102-fba0-4c38-8055-6dceacddcde9","Type":"ContainerDied","Data":"b423fa90d26832da4a07918e961c0e64869b222ba8424a4b616c72a4af974ab1"}
Sep 16 18:45:07.815746 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:45:07.815710    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7a4aa47d-be1a-4b2c-bd88-3edc40ef5d6a-metrics-tls\") pod \"dns-default-tv99q\" (UID: \"7a4aa47d-be1a-4b2c-bd88-3edc40ef5d6a\") " pod="openshift-dns/dns-default-tv99q"
Sep 16 18:45:07.815907 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:45:07.815849    2584 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 16 18:45:07.815907 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:45:07.815903    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7a4aa47d-be1a-4b2c-bd88-3edc40ef5d6a-metrics-tls podName:7a4aa47d-be1a-4b2c-bd88-3edc40ef5d6a nodeName:}" failed. No retries permitted until 2026-09-16 18:45:11.815889442 +0000 UTC m=+42.145391833 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7a4aa47d-be1a-4b2c-bd88-3edc40ef5d6a-metrics-tls") pod "dns-default-tv99q" (UID: "7a4aa47d-be1a-4b2c-bd88-3edc40ef5d6a") : secret "dns-default-metrics-tls" not found
Sep 16 18:45:08.319357 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:45:08.319319    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-s4gvd" event={"ID":"80b7ba2b-64e7-4bf3-9f78-d8a2b2b0e66a","Type":"ContainerStarted","Data":"9600d72fa44ac49ce21a94b2cea405009c3569bd63c97638dac8bf729095f519"}
Sep 16 18:45:08.323023 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:45:08.322997    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fbw2l" event={"ID":"71637102-fba0-4c38-8055-6dceacddcde9","Type":"ContainerStarted","Data":"ec8905b3b70dda27f9e167b1dfcda5a7b11f117d9cf929bbcc437e0368facf61"}
Sep 16 18:45:08.336825 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:45:08.336780    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-s4gvd" podStartSLOduration=2.184312358 podStartE2EDuration="4.336766464s" totalImagesPullingTime="2.152454106s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:45:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:45:08.335163132 +0000 UTC m=+38.664665546" watchObservedRunningTime="2026-09-16 18:45:08.336766464 +0000 UTC m=+38.666268876"
Sep 16 18:45:08.357903 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:45:08.357819    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-fbw2l" podStartSLOduration=10.158142323 podStartE2EDuration="38.357795867s" totalImagesPullingTime="28.199653544s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 18:44:30 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:45:08.355700121 +0000 UTC m=+38.685202535" watchObservedRunningTime="2026-09-16 18:45:08.357795867 +0000 UTC m=+38.687298281"
Sep 16 18:45:08.384805 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:45:08.384765    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:45:08.446511 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:45:08.446477    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:45:08.506424 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:45:08.506392    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:45:08.565207 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:45:08.565176    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:45:08.826392 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:45:08.826364    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:45:08.926427 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:45:08.926392    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:45:11.842150 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:45:11.842111    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7a4aa47d-be1a-4b2c-bd88-3edc40ef5d6a-metrics-tls\") pod \"dns-default-tv99q\" (UID: \"7a4aa47d-be1a-4b2c-bd88-3edc40ef5d6a\") " pod="openshift-dns/dns-default-tv99q"
Sep 16 18:45:11.842506 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:45:11.842257    2584 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 16 18:45:11.842506 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:45:11.842334    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7a4aa47d-be1a-4b2c-bd88-3edc40ef5d6a-metrics-tls podName:7a4aa47d-be1a-4b2c-bd88-3edc40ef5d6a nodeName:}" failed. No retries permitted until 2026-09-16 18:45:19.842319448 +0000 UTC m=+50.171821839 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7a4aa47d-be1a-4b2c-bd88-3edc40ef5d6a-metrics-tls") pod "dns-default-tv99q" (UID: "7a4aa47d-be1a-4b2c-bd88-3edc40ef5d6a") : secret "dns-default-metrics-tls" not found
Sep 16 18:45:13.522165 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:45:13.522132    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:45:15.585450 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:45:15.585417    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:45:15.680723 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:45:15.680691    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:45:17.968269 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:45:17.968231    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:45:18.027718 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:45:18.027686    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:45:19.900414 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:45:19.900366    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7a4aa47d-be1a-4b2c-bd88-3edc40ef5d6a-metrics-tls\") pod \"dns-default-tv99q\" (UID: \"7a4aa47d-be1a-4b2c-bd88-3edc40ef5d6a\") " pod="openshift-dns/dns-default-tv99q"
Sep 16 18:45:19.902399 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:45:19.900506    2584 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 16 18:45:19.902399 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:45:19.900572    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7a4aa47d-be1a-4b2c-bd88-3edc40ef5d6a-metrics-tls podName:7a4aa47d-be1a-4b2c-bd88-3edc40ef5d6a nodeName:}" failed. No retries permitted until 2026-09-16 18:45:35.900557336 +0000 UTC m=+66.230059728 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7a4aa47d-be1a-4b2c-bd88-3edc40ef5d6a-metrics-tls") pod "dns-default-tv99q" (UID: "7a4aa47d-be1a-4b2c-bd88-3edc40ef5d6a") : secret "dns-default-metrics-tls" not found
Sep 16 18:45:30.296484 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:45:30.296453    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-blv9h"
Sep 16 18:45:30.447256 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:45:30.447224    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:45:30.502936 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:45:30.502900    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:45:35.803548 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:45:35.803503    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9b96048a-0005-410b-a710-909d21fbfde8-metrics-certs\") pod \"network-metrics-daemon-bp5mv\" (UID: \"9b96048a-0005-410b-a710-909d21fbfde8\") " pod="openshift-multus/network-metrics-daemon-bp5mv"
Sep 16 18:45:35.806547 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:45:35.806530    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 16 18:45:35.814675 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:45:35.814656    2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 16 18:45:35.814740 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:45:35.814712    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9b96048a-0005-410b-a710-909d21fbfde8-metrics-certs podName:9b96048a-0005-410b-a710-909d21fbfde8 nodeName:}" failed. No retries permitted until 2026-09-16 18:46:39.814697166 +0000 UTC m=+130.144199557 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9b96048a-0005-410b-a710-909d21fbfde8-metrics-certs") pod "network-metrics-daemon-bp5mv" (UID: "9b96048a-0005-410b-a710-909d21fbfde8") : secret "metrics-daemon-secret" not found
Sep 16 18:45:35.904321 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:45:35.904288    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b9f0f3b9-87ee-44a5-a66c-6542791f3506-original-pull-secret\") pod \"global-pull-secret-syncer-t69xl\" (UID: \"b9f0f3b9-87ee-44a5-a66c-6542791f3506\") " pod="kube-system/global-pull-secret-syncer-t69xl"
Sep 16 18:45:35.904321 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:45:35.904321    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7a4aa47d-be1a-4b2c-bd88-3edc40ef5d6a-metrics-tls\") pod \"dns-default-tv99q\" (UID: \"7a4aa47d-be1a-4b2c-bd88-3edc40ef5d6a\") " pod="openshift-dns/dns-default-tv99q"
Sep 16 18:45:35.904520 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:45:35.904503    2584 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 16 18:45:35.904578 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:45:35.904567    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7a4aa47d-be1a-4b2c-bd88-3edc40ef5d6a-metrics-tls podName:7a4aa47d-be1a-4b2c-bd88-3edc40ef5d6a nodeName:}" failed. No retries permitted until 2026-09-16 18:46:07.904550958 +0000 UTC m=+98.234053352 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7a4aa47d-be1a-4b2c-bd88-3edc40ef5d6a-metrics-tls") pod "dns-default-tv99q" (UID: "7a4aa47d-be1a-4b2c-bd88-3edc40ef5d6a") : secret "dns-default-metrics-tls" not found
Sep 16 18:45:35.907586 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:45:35.907569    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 16 18:45:35.917658 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:45:35.917638    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b9f0f3b9-87ee-44a5-a66c-6542791f3506-original-pull-secret\") pod \"global-pull-secret-syncer-t69xl\" (UID: \"b9f0f3b9-87ee-44a5-a66c-6542791f3506\") " pod="kube-system/global-pull-secret-syncer-t69xl"
Sep 16 18:45:35.921600 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:45:35.921580    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t69xl"
Sep 16 18:45:36.005401 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:45:36.005369    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l4z5w\" (UniqueName: \"kubernetes.io/projected/df3c2b31-63a6-43e4-a4ae-0199c34a6a86-kube-api-access-l4z5w\") pod \"network-check-target-f4md6\" (UID: \"df3c2b31-63a6-43e4-a4ae-0199c34a6a86\") " pod="openshift-network-diagnostics/network-check-target-f4md6"
Sep 16 18:45:36.008534 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:45:36.008510    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 16 18:45:36.019446 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:45:36.019425    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 16 18:45:36.029194 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:45:36.029171    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l4z5w\" (UniqueName: \"kubernetes.io/projected/df3c2b31-63a6-43e4-a4ae-0199c34a6a86-kube-api-access-l4z5w\") pod \"network-check-target-f4md6\" (UID: \"df3c2b31-63a6-43e4-a4ae-0199c34a6a86\") " pod="openshift-network-diagnostics/network-check-target-f4md6"
Sep 16 18:45:36.216088 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:45:36.215994    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-t69xl"]
Sep 16 18:45:36.219487 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:45:36.219457    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9f0f3b9_87ee_44a5_a66c_6542791f3506.slice/crio-0fa8f5cda5a4cef96a38c01f0e50a1948d19cdd580463dc7b031214b56dd7d85 WatchSource:0}: Error finding container 0fa8f5cda5a4cef96a38c01f0e50a1948d19cdd580463dc7b031214b56dd7d85: Status 404 returned error can't find the container with id 0fa8f5cda5a4cef96a38c01f0e50a1948d19cdd580463dc7b031214b56dd7d85
Sep 16 18:45:36.231183 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:45:36.231162    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-5l9rg\""
Sep 16 18:45:36.239326 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:45:36.239309    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f4md6"
Sep 16 18:45:36.386146 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:45:36.386114    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-t69xl" event={"ID":"b9f0f3b9-87ee-44a5-a66c-6542791f3506","Type":"ContainerStarted","Data":"0fa8f5cda5a4cef96a38c01f0e50a1948d19cdd580463dc7b031214b56dd7d85"}
Sep 16 18:45:36.530681 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:45:36.530646    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-f4md6"]
Sep 16 18:45:36.533819 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:45:36.533795    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf3c2b31_63a6_43e4_a4ae_0199c34a6a86.slice/crio-b897bf1a7d86d074ada0ad6ec4bb3ebdd2eeb6556622dd840e1ec5afa1cd8f9f WatchSource:0}: Error finding container b897bf1a7d86d074ada0ad6ec4bb3ebdd2eeb6556622dd840e1ec5afa1cd8f9f: Status 404 returned error can't find the container with id b897bf1a7d86d074ada0ad6ec4bb3ebdd2eeb6556622dd840e1ec5afa1cd8f9f
Sep 16 18:45:37.391745 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:45:37.391676    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-f4md6" event={"ID":"df3c2b31-63a6-43e4-a4ae-0199c34a6a86","Type":"ContainerStarted","Data":"b897bf1a7d86d074ada0ad6ec4bb3ebdd2eeb6556622dd840e1ec5afa1cd8f9f"}
Sep 16 18:45:38.532990 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:45:38.532954    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:45:38.600435 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:45:38.600401    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:45:38.664707 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:45:38.664671    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:45:41.403302 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:45:41.403265    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-t69xl" event={"ID":"b9f0f3b9-87ee-44a5-a66c-6542791f3506","Type":"ContainerStarted","Data":"868bd045f643015a051d9ef2d7a9286bdf7b13f82cb6a68bd61895278f134979"}
Sep 16 18:45:41.404901 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:45:41.404877    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-f4md6" event={"ID":"df3c2b31-63a6-43e4-a4ae-0199c34a6a86","Type":"ContainerStarted","Data":"af7f6486759295b52b167f7f4fdbdedc652b2581283737ea9d983fe909e4ce15"}
Sep 16 18:45:41.405007 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:45:41.404925    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-f4md6"
Sep 16 18:45:41.421521 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:45:41.421478    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-t69xl" podStartSLOduration=66.772354802 podStartE2EDuration="1m11.42146501s" totalImagesPullingTime="4.649110208s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 18:44:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:45:41.421173799 +0000 UTC m=+71.750676213" watchObservedRunningTime="2026-09-16 18:45:41.42146501 +0000 UTC m=+71.750967423"
Sep 16 18:45:41.435218 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:45:41.435171    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-f4md6" podStartSLOduration=67.101476727 podStartE2EDuration="1m11.435158941s" totalImagesPullingTime="4.333682214s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:44:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:45:41.434670967 +0000 UTC m=+71.764173392" watchObservedRunningTime="2026-09-16 18:45:41.435158941 +0000 UTC m=+71.764661354"
Sep 16 18:46:07.916790 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:46:07.916742    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7a4aa47d-be1a-4b2c-bd88-3edc40ef5d6a-metrics-tls\") pod \"dns-default-tv99q\" (UID: \"7a4aa47d-be1a-4b2c-bd88-3edc40ef5d6a\") " pod="openshift-dns/dns-default-tv99q"
Sep 16 18:46:07.917313 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:46:07.916931    2584 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 16 18:46:07.917313 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:46:07.917020    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7a4aa47d-be1a-4b2c-bd88-3edc40ef5d6a-metrics-tls podName:7a4aa47d-be1a-4b2c-bd88-3edc40ef5d6a nodeName:}" failed. No retries permitted until 2026-09-16 18:47:11.916998455 +0000 UTC m=+162.246500866 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7a4aa47d-be1a-4b2c-bd88-3edc40ef5d6a-metrics-tls") pod "dns-default-tv99q" (UID: "7a4aa47d-be1a-4b2c-bd88-3edc40ef5d6a") : secret "dns-default-metrics-tls" not found
Sep 16 18:46:12.409430 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:46:12.409399    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-f4md6"
Sep 16 18:46:12.639376 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:46:12.639341    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:46:12.740015 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:46:12.739940    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:46:16.018737 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:46:16.018707    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:46:37.065114 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:46:37.065078    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-84b65c6cc4-nqz45"]
Sep 16 18:46:37.071266 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:46:37.071243    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-84b65c6cc4-nqz45"
Sep 16 18:46:37.073922 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:46:37.073902    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Sep 16 18:46:37.074038 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:46:37.073951    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Sep 16 18:46:37.075192 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:46:37.075177    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Sep 16 18:46:37.075510 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:46:37.075495    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Sep 16 18:46:37.075594 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:46:37.075500    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-k7cqx\""
Sep 16 18:46:37.077942 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:46:37.077924    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-84b65c6cc4-nqz45"]
Sep 16 18:46:37.104286 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:46:37.104257    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1e773a50-def6-4284-987f-eeb6f6cda2e4-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-84b65c6cc4-nqz45\" (UID: \"1e773a50-def6-4284-987f-eeb6f6cda2e4\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-84b65c6cc4-nqz45"
Sep 16 18:46:37.104408 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:46:37.104302    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9v8xx\" (UniqueName: \"kubernetes.io/projected/1e773a50-def6-4284-987f-eeb6f6cda2e4-kube-api-access-9v8xx\") pod \"managed-serviceaccount-addon-agent-84b65c6cc4-nqz45\" (UID: \"1e773a50-def6-4284-987f-eeb6f6cda2e4\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-84b65c6cc4-nqz45"
Sep 16 18:46:37.205102 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:46:37.205057    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1e773a50-def6-4284-987f-eeb6f6cda2e4-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-84b65c6cc4-nqz45\" (UID: \"1e773a50-def6-4284-987f-eeb6f6cda2e4\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-84b65c6cc4-nqz45"
Sep 16 18:46:37.205262 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:46:37.205114    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9v8xx\" (UniqueName: \"kubernetes.io/projected/1e773a50-def6-4284-987f-eeb6f6cda2e4-kube-api-access-9v8xx\") pod \"managed-serviceaccount-addon-agent-84b65c6cc4-nqz45\" (UID: \"1e773a50-def6-4284-987f-eeb6f6cda2e4\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-84b65c6cc4-nqz45"
Sep 16 18:46:37.207566 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:46:37.207533    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1e773a50-def6-4284-987f-eeb6f6cda2e4-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-84b65c6cc4-nqz45\" (UID: \"1e773a50-def6-4284-987f-eeb6f6cda2e4\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-84b65c6cc4-nqz45"
Sep 16 18:46:37.213465 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:46:37.213437    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9v8xx\" (UniqueName: \"kubernetes.io/projected/1e773a50-def6-4284-987f-eeb6f6cda2e4-kube-api-access-9v8xx\") pod \"managed-serviceaccount-addon-agent-84b65c6cc4-nqz45\" (UID: \"1e773a50-def6-4284-987f-eeb6f6cda2e4\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-84b65c6cc4-nqz45"
Sep 16 18:46:37.391659 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:46:37.391573    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-84b65c6cc4-nqz45"
Sep 16 18:46:37.656016 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:46:37.655920    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-84b65c6cc4-nqz45"]
Sep 16 18:46:37.659744 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:46:37.659716    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1e773a50_def6_4284_987f_eeb6f6cda2e4.slice/crio-61c5bd32e2bcadeb4a4c466ba64506bd6db645aac41179215332fe2a5eea07d7 WatchSource:0}: Error finding container 61c5bd32e2bcadeb4a4c466ba64506bd6db645aac41179215332fe2a5eea07d7: Status 404 returned error can't find the container with id 61c5bd32e2bcadeb4a4c466ba64506bd6db645aac41179215332fe2a5eea07d7
Sep 16 18:46:38.538694 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:46:38.538647    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-84b65c6cc4-nqz45" event={"ID":"1e773a50-def6-4284-987f-eeb6f6cda2e4","Type":"ContainerStarted","Data":"61c5bd32e2bcadeb4a4c466ba64506bd6db645aac41179215332fe2a5eea07d7"}
Sep 16 18:46:39.822865 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:46:39.822808    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9b96048a-0005-410b-a710-909d21fbfde8-metrics-certs\") pod \"network-metrics-daemon-bp5mv\" (UID: \"9b96048a-0005-410b-a710-909d21fbfde8\") " pod="openshift-multus/network-metrics-daemon-bp5mv"
Sep 16 18:46:39.823212 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:46:39.822971    2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 16 18:46:39.823212 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:46:39.823046    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9b96048a-0005-410b-a710-909d21fbfde8-metrics-certs podName:9b96048a-0005-410b-a710-909d21fbfde8 nodeName:}" failed. No retries permitted until 2026-09-16 18:48:41.823030484 +0000 UTC m=+252.152532891 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9b96048a-0005-410b-a710-909d21fbfde8-metrics-certs") pod "network-metrics-daemon-bp5mv" (UID: "9b96048a-0005-410b-a710-909d21fbfde8") : secret "metrics-daemon-secret" not found
Sep 16 18:46:40.545712 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:46:40.545673    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-84b65c6cc4-nqz45" event={"ID":"1e773a50-def6-4284-987f-eeb6f6cda2e4","Type":"ContainerStarted","Data":"0285cf82e0a70c2ab856af6c0a94a2d5980d845e773c2e245fce94ce092e163b"}
Sep 16 18:46:40.561406 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:46:40.561332    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-84b65c6cc4-nqz45" podStartSLOduration=1.534200566 podStartE2EDuration="3.561312588s" totalImagesPullingTime="2.027112022s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:46:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:46:40.560962885 +0000 UTC m=+130.890465298" watchObservedRunningTime="2026-09-16 18:46:40.561312588 +0000 UTC m=+130.890815002"
Sep 16 18:47:07.087746 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:47:07.087691    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-tv99q" podUID="7a4aa47d-be1a-4b2c-bd88-3edc40ef5d6a"
Sep 16 18:47:07.133694 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:47:07.133658    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-bp5mv" podUID="9b96048a-0005-410b-a710-909d21fbfde8"
Sep 16 18:47:07.610327 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:47:07.610299    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-tv99q"
Sep 16 18:47:11.950536 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:47:11.950491    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7a4aa47d-be1a-4b2c-bd88-3edc40ef5d6a-metrics-tls\") pod \"dns-default-tv99q\" (UID: \"7a4aa47d-be1a-4b2c-bd88-3edc40ef5d6a\") " pod="openshift-dns/dns-default-tv99q"
Sep 16 18:47:11.951046 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:47:11.950625    2584 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 16 18:47:11.951046 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:47:11.950685    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7a4aa47d-be1a-4b2c-bd88-3edc40ef5d6a-metrics-tls podName:7a4aa47d-be1a-4b2c-bd88-3edc40ef5d6a nodeName:}" failed. No retries permitted until 2026-09-16 18:49:13.950672111 +0000 UTC m=+284.280174503 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7a4aa47d-be1a-4b2c-bd88-3edc40ef5d6a-metrics-tls") pod "dns-default-tv99q" (UID: "7a4aa47d-be1a-4b2c-bd88-3edc40ef5d6a") : secret "dns-default-metrics-tls" not found
Sep 16 18:47:20.111848 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:47:20.111804    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bp5mv"
Sep 16 18:47:40.474498 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:47:40.474462    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:13.991263 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:13.991230    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:39.829156 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:39.829127    2584 generic.go:356] "Generic (PLEG): container finished" podID="1e773a50-def6-4284-987f-eeb6f6cda2e4" containerID="0285cf82e0a70c2ab856af6c0a94a2d5980d845e773c2e245fce94ce092e163b" exitCode=255
Sep 16 18:48:39.829564 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:39.829172    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-84b65c6cc4-nqz45" event={"ID":"1e773a50-def6-4284-987f-eeb6f6cda2e4","Type":"ContainerDied","Data":"0285cf82e0a70c2ab856af6c0a94a2d5980d845e773c2e245fce94ce092e163b"}
Sep 16 18:48:39.829564 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:39.829532    2584 scope.go:117] "RemoveContainer" containerID="0285cf82e0a70c2ab856af6c0a94a2d5980d845e773c2e245fce94ce092e163b"
Sep 16 18:48:40.502393 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:40.502358    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-7ff679ddf8-ctqk5"]
Sep 16 18:48:40.505112 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:40.505095    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-7ff679ddf8-ctqk5"
Sep 16 18:48:40.509442 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:40.508412    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 16 18:48:40.509442 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:40.508679    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 16 18:48:40.509442 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:40.508922    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Sep 16 18:48:40.509442 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:40.509016    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Sep 16 18:48:40.509442 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:40.509142    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-fc7gw\""
Sep 16 18:48:40.509442 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:40.509154    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Sep 16 18:48:40.509442 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:40.509250    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Sep 16 18:48:40.529897 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:40.529863    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-7ff679ddf8-ctqk5"]
Sep 16 18:48:40.562499 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:40.562469    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/ec6fcb0d-227c-492a-923a-db5e74797aa6-stats-auth\") pod \"router-default-7ff679ddf8-ctqk5\" (UID: \"ec6fcb0d-227c-492a-923a-db5e74797aa6\") " pod="openshift-ingress/router-default-7ff679ddf8-ctqk5"
Sep 16 18:48:40.562499 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:40.562500    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ec6fcb0d-227c-492a-923a-db5e74797aa6-metrics-certs\") pod \"router-default-7ff679ddf8-ctqk5\" (UID: \"ec6fcb0d-227c-492a-923a-db5e74797aa6\") " pod="openshift-ingress/router-default-7ff679ddf8-ctqk5"
Sep 16 18:48:40.562697 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:40.562530    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zntnl\" (UniqueName: \"kubernetes.io/projected/ec6fcb0d-227c-492a-923a-db5e74797aa6-kube-api-access-zntnl\") pod \"router-default-7ff679ddf8-ctqk5\" (UID: \"ec6fcb0d-227c-492a-923a-db5e74797aa6\") " pod="openshift-ingress/router-default-7ff679ddf8-ctqk5"
Sep 16 18:48:40.562697 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:40.562585    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/ec6fcb0d-227c-492a-923a-db5e74797aa6-default-certificate\") pod \"router-default-7ff679ddf8-ctqk5\" (UID: \"ec6fcb0d-227c-492a-923a-db5e74797aa6\") " pod="openshift-ingress/router-default-7ff679ddf8-ctqk5"
Sep 16 18:48:40.562697 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:40.562614    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ec6fcb0d-227c-492a-923a-db5e74797aa6-service-ca-bundle\") pod \"router-default-7ff679ddf8-ctqk5\" (UID: \"ec6fcb0d-227c-492a-923a-db5e74797aa6\") " pod="openshift-ingress/router-default-7ff679ddf8-ctqk5"
Sep 16 18:48:40.611806 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:40.611774    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-qkzj5"]
Sep 16 18:48:40.614611 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:40.614590    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-696884fb9c-crjw7"]
Sep 16 18:48:40.614729 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:40.614710    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-qkzj5"
Sep 16 18:48:40.617496 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:40.617470    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-cf7c776b7-6xjxf"]
Sep 16 18:48:40.617608 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:40.617565    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Sep 16 18:48:40.617608 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:40.617585    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-696884fb9c-crjw7"
Sep 16 18:48:40.618144 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:40.618125    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Sep 16 18:48:40.618247 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:40.618159    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-c5pqg\""
Sep 16 18:48:40.618247 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:40.618132    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Sep 16 18:48:40.618428 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:40.618413    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Sep 16 18:48:40.620408 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:40.620391    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-76c66cc6d5-zsjkq"]
Sep 16 18:48:40.620472 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:40.620457    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-6xjxf"
Sep 16 18:48:40.621417 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:40.621393    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-6hm76\""
Sep 16 18:48:40.621536 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:40.621516    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Sep 16 18:48:40.621772 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:40.621752    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Sep 16 18:48:40.623102 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:40.623081    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-76c66cc6d5-zsjkq"
Sep 16 18:48:40.623664 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:40.623648    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Sep 16 18:48:40.623854 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:40.623787    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Sep 16 18:48:40.623958 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:40.623865    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Sep 16 18:48:40.624371 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:40.624349    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Sep 16 18:48:40.624488 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:40.624464    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Sep 16 18:48:40.624553 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:40.624488    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-77mgh\""
Sep 16 18:48:40.624553 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:40.624488    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Sep 16 18:48:40.626070 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:40.626052    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-lpjvx\""
Sep 16 18:48:40.628098 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:40.628082    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 16 18:48:40.629399 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:40.629373    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 16 18:48:40.629490 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:40.629379    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 16 18:48:40.629700 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:40.629679    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Sep 16 18:48:40.631590 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:40.631572    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 16 18:48:40.632299 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:40.632279    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-cf7c776b7-6xjxf"]
Sep 16 18:48:40.633266 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:40.633247    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-qkzj5"]
Sep 16 18:48:40.634353 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:40.634335    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-76c66cc6d5-zsjkq"]
Sep 16 18:48:40.644116 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:40.644094    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-696884fb9c-crjw7"]
Sep 16 18:48:40.663886 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:40.663846    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/93861b72-565d-497a-a7d5-098fc010f83c-ca-trust-extracted\") pod \"image-registry-76c66cc6d5-zsjkq\" (UID: \"93861b72-565d-497a-a7d5-098fc010f83c\") " pod="openshift-image-registry/image-registry-76c66cc6d5-zsjkq"
Sep 16 18:48:40.663886 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:40.663886    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/93861b72-565d-497a-a7d5-098fc010f83c-bound-sa-token\") pod \"image-registry-76c66cc6d5-zsjkq\" (UID: \"93861b72-565d-497a-a7d5-098fc010f83c\") " pod="openshift-image-registry/image-registry-76c66cc6d5-zsjkq"
Sep 16 18:48:40.664101 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:40.663943    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/93861b72-565d-497a-a7d5-098fc010f83c-image-registry-private-configuration\") pod \"image-registry-76c66cc6d5-zsjkq\" (UID: \"93861b72-565d-497a-a7d5-098fc010f83c\") " pod="openshift-image-registry/image-registry-76c66cc6d5-zsjkq"
Sep 16 18:48:40.664101 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:40.663980    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/ec6fcb0d-227c-492a-923a-db5e74797aa6-default-certificate\") pod \"router-default-7ff679ddf8-ctqk5\" (UID: \"ec6fcb0d-227c-492a-923a-db5e74797aa6\") " pod="openshift-ingress/router-default-7ff679ddf8-ctqk5"
Sep 16 18:48:40.664101 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:40.664016    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/12570b86-13fd-4b6d-adbf-e2d515c0492c-config\") pod \"service-ca-operator-cf7c776b7-6xjxf\" (UID: \"12570b86-13fd-4b6d-adbf-e2d515c0492c\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-6xjxf"
Sep 16 18:48:40.664101 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:40.664057    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/93861b72-565d-497a-a7d5-098fc010f83c-registry-certificates\") pod \"image-registry-76c66cc6d5-zsjkq\" (UID: \"93861b72-565d-497a-a7d5-098fc010f83c\") " pod="openshift-image-registry/image-registry-76c66cc6d5-zsjkq"
Sep 16 18:48:40.664101 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:40.664091    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/93861b72-565d-497a-a7d5-098fc010f83c-registry-tls\") pod \"image-registry-76c66cc6d5-zsjkq\" (UID: \"93861b72-565d-497a-a7d5-098fc010f83c\") " pod="openshift-image-registry/image-registry-76c66cc6d5-zsjkq"
Sep 16 18:48:40.664273 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:40.664125    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ecf273b5-b465-4008-b035-f7ac0e740d58-config\") pod \"console-operator-696884fb9c-crjw7\" (UID: \"ecf273b5-b465-4008-b035-f7ac0e740d58\") " pod="openshift-console-operator/console-operator-696884fb9c-crjw7"
Sep 16 18:48:40.664273 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:40.664181    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/93861b72-565d-497a-a7d5-098fc010f83c-installation-pull-secrets\") pod \"image-registry-76c66cc6d5-zsjkq\" (UID: \"93861b72-565d-497a-a7d5-098fc010f83c\") " pod="openshift-image-registry/image-registry-76c66cc6d5-zsjkq"
Sep 16 18:48:40.664273 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:40.664209    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/65e92d96-65ad-4891-b693-1293389b7855-serving-cert\") pod \"kube-storage-version-migrator-operator-6c8f84f897-qkzj5\" (UID: \"65e92d96-65ad-4891-b693-1293389b7855\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-qkzj5"
Sep 16 18:48:40.664273 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:40.664227    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7mgk7\" (UniqueName: \"kubernetes.io/projected/ecf273b5-b465-4008-b035-f7ac0e740d58-kube-api-access-7mgk7\") pod \"console-operator-696884fb9c-crjw7\" (UID: \"ecf273b5-b465-4008-b035-f7ac0e740d58\") " pod="openshift-console-operator/console-operator-696884fb9c-crjw7"
Sep 16 18:48:40.664273 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:40.664252    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zntnl\" (UniqueName: \"kubernetes.io/projected/ec6fcb0d-227c-492a-923a-db5e74797aa6-kube-api-access-zntnl\") pod \"router-default-7ff679ddf8-ctqk5\" (UID: \"ec6fcb0d-227c-492a-923a-db5e74797aa6\") " pod="openshift-ingress/router-default-7ff679ddf8-ctqk5"
Sep 16 18:48:40.664475 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:40.664279    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/93861b72-565d-497a-a7d5-098fc010f83c-trusted-ca\") pod \"image-registry-76c66cc6d5-zsjkq\" (UID: \"93861b72-565d-497a-a7d5-098fc010f83c\") " pod="openshift-image-registry/image-registry-76c66cc6d5-zsjkq"
Sep 16 18:48:40.664475 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:40.664305    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/12570b86-13fd-4b6d-adbf-e2d515c0492c-serving-cert\") pod \"service-ca-operator-cf7c776b7-6xjxf\" (UID: \"12570b86-13fd-4b6d-adbf-e2d515c0492c\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-6xjxf"
Sep 16 18:48:40.664475 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:40.664421    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sghhl\" (UniqueName: \"kubernetes.io/projected/93861b72-565d-497a-a7d5-098fc010f83c-kube-api-access-sghhl\") pod \"image-registry-76c66cc6d5-zsjkq\" (UID: \"93861b72-565d-497a-a7d5-098fc010f83c\") " pod="openshift-image-registry/image-registry-76c66cc6d5-zsjkq"
Sep 16 18:48:40.664475 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:40.664457    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ec6fcb0d-227c-492a-923a-db5e74797aa6-service-ca-bundle\") pod \"router-default-7ff679ddf8-ctqk5\" (UID: \"ec6fcb0d-227c-492a-923a-db5e74797aa6\") " pod="openshift-ingress/router-default-7ff679ddf8-ctqk5"
Sep 16 18:48:40.664630 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:40.664494    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ecf273b5-b465-4008-b035-f7ac0e740d58-serving-cert\") pod \"console-operator-696884fb9c-crjw7\" (UID: \"ecf273b5-b465-4008-b035-f7ac0e740d58\") " pod="openshift-console-operator/console-operator-696884fb9c-crjw7"
Sep 16 18:48:40.664630 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:40.664521    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ecf273b5-b465-4008-b035-f7ac0e740d58-trusted-ca\") pod \"console-operator-696884fb9c-crjw7\" (UID: \"ecf273b5-b465-4008-b035-f7ac0e740d58\") " pod="openshift-console-operator/console-operator-696884fb9c-crjw7"
Sep 16 18:48:40.664630 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:40.664548    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tgqc7\" (UniqueName: \"kubernetes.io/projected/12570b86-13fd-4b6d-adbf-e2d515c0492c-kube-api-access-tgqc7\") pod \"service-ca-operator-cf7c776b7-6xjxf\" (UID: \"12570b86-13fd-4b6d-adbf-e2d515c0492c\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-6xjxf"
Sep 16 18:48:40.664630 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:48:40.664590    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ec6fcb0d-227c-492a-923a-db5e74797aa6-service-ca-bundle podName:ec6fcb0d-227c-492a-923a-db5e74797aa6 nodeName:}" failed. No retries permitted until 2026-09-16 18:48:41.164568787 +0000 UTC m=+251.494071182 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/ec6fcb0d-227c-492a-923a-db5e74797aa6-service-ca-bundle") pod "router-default-7ff679ddf8-ctqk5" (UID: "ec6fcb0d-227c-492a-923a-db5e74797aa6") : configmap references non-existent config key: service-ca.crt
Sep 16 18:48:40.664807 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:40.664630    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/ec6fcb0d-227c-492a-923a-db5e74797aa6-stats-auth\") pod \"router-default-7ff679ddf8-ctqk5\" (UID: \"ec6fcb0d-227c-492a-923a-db5e74797aa6\") " pod="openshift-ingress/router-default-7ff679ddf8-ctqk5"
Sep 16 18:48:40.664807 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:40.664655    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ec6fcb0d-227c-492a-923a-db5e74797aa6-metrics-certs\") pod \"router-default-7ff679ddf8-ctqk5\" (UID: \"ec6fcb0d-227c-492a-923a-db5e74797aa6\") " pod="openshift-ingress/router-default-7ff679ddf8-ctqk5"
Sep 16 18:48:40.664807 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:40.664683    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tzttb\" (UniqueName: \"kubernetes.io/projected/65e92d96-65ad-4891-b693-1293389b7855-kube-api-access-tzttb\") pod \"kube-storage-version-migrator-operator-6c8f84f897-qkzj5\" (UID: \"65e92d96-65ad-4891-b693-1293389b7855\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-qkzj5"
Sep 16 18:48:40.664807 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:40.664721    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/65e92d96-65ad-4891-b693-1293389b7855-config\") pod \"kube-storage-version-migrator-operator-6c8f84f897-qkzj5\" (UID: \"65e92d96-65ad-4891-b693-1293389b7855\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-qkzj5"
Sep 16 18:48:40.664992 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:48:40.664815    2584 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 16 18:48:40.664992 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:48:40.664906    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec6fcb0d-227c-492a-923a-db5e74797aa6-metrics-certs podName:ec6fcb0d-227c-492a-923a-db5e74797aa6 nodeName:}" failed. No retries permitted until 2026-09-16 18:48:41.164887692 +0000 UTC m=+251.494390102 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ec6fcb0d-227c-492a-923a-db5e74797aa6-metrics-certs") pod "router-default-7ff679ddf8-ctqk5" (UID: "ec6fcb0d-227c-492a-923a-db5e74797aa6") : secret "router-metrics-certs-default" not found
Sep 16 18:48:40.666445 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:40.666423    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/ec6fcb0d-227c-492a-923a-db5e74797aa6-default-certificate\") pod \"router-default-7ff679ddf8-ctqk5\" (UID: \"ec6fcb0d-227c-492a-923a-db5e74797aa6\") " pod="openshift-ingress/router-default-7ff679ddf8-ctqk5"
Sep 16 18:48:40.666812 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:40.666795    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/ec6fcb0d-227c-492a-923a-db5e74797aa6-stats-auth\") pod \"router-default-7ff679ddf8-ctqk5\" (UID: \"ec6fcb0d-227c-492a-923a-db5e74797aa6\") " pod="openshift-ingress/router-default-7ff679ddf8-ctqk5"
Sep 16 18:48:40.672778 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:40.672752    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zntnl\" (UniqueName: \"kubernetes.io/projected/ec6fcb0d-227c-492a-923a-db5e74797aa6-kube-api-access-zntnl\") pod \"router-default-7ff679ddf8-ctqk5\" (UID: \"ec6fcb0d-227c-492a-923a-db5e74797aa6\") " pod="openshift-ingress/router-default-7ff679ddf8-ctqk5"
Sep 16 18:48:40.729003 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:40.728968    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:40.765319 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:40.765294    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/12570b86-13fd-4b6d-adbf-e2d515c0492c-config\") pod \"service-ca-operator-cf7c776b7-6xjxf\" (UID: \"12570b86-13fd-4b6d-adbf-e2d515c0492c\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-6xjxf"
Sep 16 18:48:40.765458 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:40.765333    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/93861b72-565d-497a-a7d5-098fc010f83c-registry-certificates\") pod \"image-registry-76c66cc6d5-zsjkq\" (UID: \"93861b72-565d-497a-a7d5-098fc010f83c\") " pod="openshift-image-registry/image-registry-76c66cc6d5-zsjkq"
Sep 16 18:48:40.765458 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:40.765354    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/93861b72-565d-497a-a7d5-098fc010f83c-registry-tls\") pod \"image-registry-76c66cc6d5-zsjkq\" (UID: \"93861b72-565d-497a-a7d5-098fc010f83c\") " pod="openshift-image-registry/image-registry-76c66cc6d5-zsjkq"
Sep 16 18:48:40.765458 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:40.765370    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ecf273b5-b465-4008-b035-f7ac0e740d58-config\") pod \"console-operator-696884fb9c-crjw7\" (UID: \"ecf273b5-b465-4008-b035-f7ac0e740d58\") " pod="openshift-console-operator/console-operator-696884fb9c-crjw7"
Sep 16 18:48:40.765458 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:40.765391    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/93861b72-565d-497a-a7d5-098fc010f83c-installation-pull-secrets\") pod \"image-registry-76c66cc6d5-zsjkq\" (UID: \"93861b72-565d-497a-a7d5-098fc010f83c\") " pod="openshift-image-registry/image-registry-76c66cc6d5-zsjkq"
Sep 16 18:48:40.765458 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:40.765410    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/65e92d96-65ad-4891-b693-1293389b7855-serving-cert\") pod \"kube-storage-version-migrator-operator-6c8f84f897-qkzj5\" (UID: \"65e92d96-65ad-4891-b693-1293389b7855\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-qkzj5"
Sep 16 18:48:40.765458 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:40.765433    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7mgk7\" (UniqueName: \"kubernetes.io/projected/ecf273b5-b465-4008-b035-f7ac0e740d58-kube-api-access-7mgk7\") pod \"console-operator-696884fb9c-crjw7\" (UID: \"ecf273b5-b465-4008-b035-f7ac0e740d58\") " pod="openshift-console-operator/console-operator-696884fb9c-crjw7"
Sep 16 18:48:40.765766 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:48:40.765479    2584 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 16 18:48:40.765766 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:48:40.765502    2584 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-76c66cc6d5-zsjkq: secret "image-registry-tls" not found
Sep 16 18:48:40.765766 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:48:40.765567    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/93861b72-565d-497a-a7d5-098fc010f83c-registry-tls podName:93861b72-565d-497a-a7d5-098fc010f83c nodeName:}" failed. No retries permitted until 2026-09-16 18:48:41.265547499 +0000 UTC m=+251.595049903 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/93861b72-565d-497a-a7d5-098fc010f83c-registry-tls") pod "image-registry-76c66cc6d5-zsjkq" (UID: "93861b72-565d-497a-a7d5-098fc010f83c") : secret "image-registry-tls" not found
Sep 16 18:48:40.765766 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:40.765590    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/93861b72-565d-497a-a7d5-098fc010f83c-trusted-ca\") pod \"image-registry-76c66cc6d5-zsjkq\" (UID: \"93861b72-565d-497a-a7d5-098fc010f83c\") " pod="openshift-image-registry/image-registry-76c66cc6d5-zsjkq"
Sep 16 18:48:40.765766 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:40.765622    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/12570b86-13fd-4b6d-adbf-e2d515c0492c-serving-cert\") pod \"service-ca-operator-cf7c776b7-6xjxf\" (UID: \"12570b86-13fd-4b6d-adbf-e2d515c0492c\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-6xjxf"
Sep 16 18:48:40.765766 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:40.765655    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sghhl\" (UniqueName: \"kubernetes.io/projected/93861b72-565d-497a-a7d5-098fc010f83c-kube-api-access-sghhl\") pod \"image-registry-76c66cc6d5-zsjkq\" (UID: \"93861b72-565d-497a-a7d5-098fc010f83c\") " pod="openshift-image-registry/image-registry-76c66cc6d5-zsjkq"
Sep 16 18:48:40.765766 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:40.765724    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ecf273b5-b465-4008-b035-f7ac0e740d58-serving-cert\") pod \"console-operator-696884fb9c-crjw7\" (UID: \"ecf273b5-b465-4008-b035-f7ac0e740d58\") " pod="openshift-console-operator/console-operator-696884fb9c-crjw7"
Sep 16 18:48:40.765766 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:40.765749    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ecf273b5-b465-4008-b035-f7ac0e740d58-trusted-ca\") pod \"console-operator-696884fb9c-crjw7\" (UID: \"ecf273b5-b465-4008-b035-f7ac0e740d58\") " pod="openshift-console-operator/console-operator-696884fb9c-crjw7"
Sep 16 18:48:40.766136 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:40.766057    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tgqc7\" (UniqueName: \"kubernetes.io/projected/12570b86-13fd-4b6d-adbf-e2d515c0492c-kube-api-access-tgqc7\") pod \"service-ca-operator-cf7c776b7-6xjxf\" (UID: \"12570b86-13fd-4b6d-adbf-e2d515c0492c\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-6xjxf"
Sep 16 18:48:40.766244 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:40.766225    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tzttb\" (UniqueName: \"kubernetes.io/projected/65e92d96-65ad-4891-b693-1293389b7855-kube-api-access-tzttb\") pod \"kube-storage-version-migrator-operator-6c8f84f897-qkzj5\" (UID: \"65e92d96-65ad-4891-b693-1293389b7855\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-qkzj5"
Sep 16 18:48:40.766298 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:40.766285    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/65e92d96-65ad-4891-b693-1293389b7855-config\") pod \"kube-storage-version-migrator-operator-6c8f84f897-qkzj5\" (UID: \"65e92d96-65ad-4891-b693-1293389b7855\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-qkzj5"
Sep 16 18:48:40.766353 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:40.766313    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/93861b72-565d-497a-a7d5-098fc010f83c-ca-trust-extracted\") pod \"image-registry-76c66cc6d5-zsjkq\" (UID: \"93861b72-565d-497a-a7d5-098fc010f83c\") " pod="openshift-image-registry/image-registry-76c66cc6d5-zsjkq"
Sep 16 18:48:40.766353 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:40.766346    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/93861b72-565d-497a-a7d5-098fc010f83c-bound-sa-token\") pod \"image-registry-76c66cc6d5-zsjkq\" (UID: \"93861b72-565d-497a-a7d5-098fc010f83c\") " pod="openshift-image-registry/image-registry-76c66cc6d5-zsjkq"
Sep 16 18:48:40.766453 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:40.766358    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/12570b86-13fd-4b6d-adbf-e2d515c0492c-config\") pod \"service-ca-operator-cf7c776b7-6xjxf\" (UID: \"12570b86-13fd-4b6d-adbf-e2d515c0492c\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-6xjxf"
Sep 16 18:48:40.766453 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:40.766389    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/93861b72-565d-497a-a7d5-098fc010f83c-image-registry-private-configuration\") pod \"image-registry-76c66cc6d5-zsjkq\" (UID: \"93861b72-565d-497a-a7d5-098fc010f83c\") " pod="openshift-image-registry/image-registry-76c66cc6d5-zsjkq"
Sep 16 18:48:40.766716 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:40.766693    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/93861b72-565d-497a-a7d5-098fc010f83c-ca-trust-extracted\") pod \"image-registry-76c66cc6d5-zsjkq\" (UID: \"93861b72-565d-497a-a7d5-098fc010f83c\") " pod="openshift-image-registry/image-registry-76c66cc6d5-zsjkq"
Sep 16 18:48:40.767144 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:40.766914    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/93861b72-565d-497a-a7d5-098fc010f83c-trusted-ca\") pod \"image-registry-76c66cc6d5-zsjkq\" (UID: \"93861b72-565d-497a-a7d5-098fc010f83c\") " pod="openshift-image-registry/image-registry-76c66cc6d5-zsjkq"
Sep 16 18:48:40.767144 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:40.766915    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/65e92d96-65ad-4891-b693-1293389b7855-config\") pod \"kube-storage-version-migrator-operator-6c8f84f897-qkzj5\" (UID: \"65e92d96-65ad-4891-b693-1293389b7855\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-qkzj5"
Sep 16 18:48:40.767503 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:40.767447    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/93861b72-565d-497a-a7d5-098fc010f83c-registry-certificates\") pod \"image-registry-76c66cc6d5-zsjkq\" (UID: \"93861b72-565d-497a-a7d5-098fc010f83c\") " pod="openshift-image-registry/image-registry-76c66cc6d5-zsjkq"
Sep 16 18:48:40.768221 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:40.768157    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ecf273b5-b465-4008-b035-f7ac0e740d58-config\") pod \"console-operator-696884fb9c-crjw7\" (UID: \"ecf273b5-b465-4008-b035-f7ac0e740d58\") " pod="openshift-console-operator/console-operator-696884fb9c-crjw7"
Sep 16 18:48:40.769184 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:40.768872    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/65e92d96-65ad-4891-b693-1293389b7855-serving-cert\") pod \"kube-storage-version-migrator-operator-6c8f84f897-qkzj5\" (UID: \"65e92d96-65ad-4891-b693-1293389b7855\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-qkzj5"
Sep 16 18:48:40.769670 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:40.769643    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/12570b86-13fd-4b6d-adbf-e2d515c0492c-serving-cert\") pod \"service-ca-operator-cf7c776b7-6xjxf\" (UID: \"12570b86-13fd-4b6d-adbf-e2d515c0492c\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-6xjxf"
Sep 16 18:48:40.769886 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:40.769852    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/93861b72-565d-497a-a7d5-098fc010f83c-image-registry-private-configuration\") pod \"image-registry-76c66cc6d5-zsjkq\" (UID: \"93861b72-565d-497a-a7d5-098fc010f83c\") " pod="openshift-image-registry/image-registry-76c66cc6d5-zsjkq"
Sep 16 18:48:40.770135 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:40.770113    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ecf273b5-b465-4008-b035-f7ac0e740d58-serving-cert\") pod \"console-operator-696884fb9c-crjw7\" (UID: \"ecf273b5-b465-4008-b035-f7ac0e740d58\") " pod="openshift-console-operator/console-operator-696884fb9c-crjw7"
Sep 16 18:48:40.771469 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:40.771437    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/93861b72-565d-497a-a7d5-098fc010f83c-installation-pull-secrets\") pod \"image-registry-76c66cc6d5-zsjkq\" (UID: \"93861b72-565d-497a-a7d5-098fc010f83c\") " pod="openshift-image-registry/image-registry-76c66cc6d5-zsjkq"
Sep 16 18:48:40.772872 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:40.772820    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ecf273b5-b465-4008-b035-f7ac0e740d58-trusted-ca\") pod \"console-operator-696884fb9c-crjw7\" (UID: \"ecf273b5-b465-4008-b035-f7ac0e740d58\") " pod="openshift-console-operator/console-operator-696884fb9c-crjw7"
Sep 16 18:48:40.773244 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:40.773227    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7mgk7\" (UniqueName: \"kubernetes.io/projected/ecf273b5-b465-4008-b035-f7ac0e740d58-kube-api-access-7mgk7\") pod \"console-operator-696884fb9c-crjw7\" (UID: \"ecf273b5-b465-4008-b035-f7ac0e740d58\") " pod="openshift-console-operator/console-operator-696884fb9c-crjw7"
Sep 16 18:48:40.775632 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:40.775609    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tgqc7\" (UniqueName: \"kubernetes.io/projected/12570b86-13fd-4b6d-adbf-e2d515c0492c-kube-api-access-tgqc7\") pod \"service-ca-operator-cf7c776b7-6xjxf\" (UID: \"12570b86-13fd-4b6d-adbf-e2d515c0492c\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-6xjxf"
Sep 16 18:48:40.775821 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:40.775804    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/93861b72-565d-497a-a7d5-098fc010f83c-bound-sa-token\") pod \"image-registry-76c66cc6d5-zsjkq\" (UID: \"93861b72-565d-497a-a7d5-098fc010f83c\") " pod="openshift-image-registry/image-registry-76c66cc6d5-zsjkq"
Sep 16 18:48:40.776186 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:40.776170    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sghhl\" (UniqueName: \"kubernetes.io/projected/93861b72-565d-497a-a7d5-098fc010f83c-kube-api-access-sghhl\") pod \"image-registry-76c66cc6d5-zsjkq\" (UID: \"93861b72-565d-497a-a7d5-098fc010f83c\") " pod="openshift-image-registry/image-registry-76c66cc6d5-zsjkq"
Sep 16 18:48:40.776353 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:40.776337    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tzttb\" (UniqueName: \"kubernetes.io/projected/65e92d96-65ad-4891-b693-1293389b7855-kube-api-access-tzttb\") pod \"kube-storage-version-migrator-operator-6c8f84f897-qkzj5\" (UID: \"65e92d96-65ad-4891-b693-1293389b7855\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-qkzj5"
Sep 16 18:48:40.829789 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:40.829760    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:40.832993 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:40.832969    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-84b65c6cc4-nqz45" event={"ID":"1e773a50-def6-4284-987f-eeb6f6cda2e4","Type":"ContainerStarted","Data":"6017bfc9775be805567661f42964ce5fc459ec54ba2560bb9493cbcb08579cd3"}
Sep 16 18:48:40.925050 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:40.925021    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-qkzj5"
Sep 16 18:48:40.931416 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:40.931398    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-696884fb9c-crjw7"
Sep 16 18:48:40.939668 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:40.939645    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-6xjxf"
Sep 16 18:48:41.170644 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:41.170562    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ec6fcb0d-227c-492a-923a-db5e74797aa6-metrics-certs\") pod \"router-default-7ff679ddf8-ctqk5\" (UID: \"ec6fcb0d-227c-492a-923a-db5e74797aa6\") " pod="openshift-ingress/router-default-7ff679ddf8-ctqk5"
Sep 16 18:48:41.170644 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:41.170637    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ec6fcb0d-227c-492a-923a-db5e74797aa6-service-ca-bundle\") pod \"router-default-7ff679ddf8-ctqk5\" (UID: \"ec6fcb0d-227c-492a-923a-db5e74797aa6\") " pod="openshift-ingress/router-default-7ff679ddf8-ctqk5"
Sep 16 18:48:41.170814 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:48:41.170695    2584 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 16 18:48:41.170814 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:48:41.170752    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec6fcb0d-227c-492a-923a-db5e74797aa6-metrics-certs podName:ec6fcb0d-227c-492a-923a-db5e74797aa6 nodeName:}" failed. No retries permitted until 2026-09-16 18:48:42.170738023 +0000 UTC m=+252.500240413 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ec6fcb0d-227c-492a-923a-db5e74797aa6-metrics-certs") pod "router-default-7ff679ddf8-ctqk5" (UID: "ec6fcb0d-227c-492a-923a-db5e74797aa6") : secret "router-metrics-certs-default" not found
Sep 16 18:48:41.170814 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:48:41.170764    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ec6fcb0d-227c-492a-923a-db5e74797aa6-service-ca-bundle podName:ec6fcb0d-227c-492a-923a-db5e74797aa6 nodeName:}" failed. No retries permitted until 2026-09-16 18:48:42.170758624 +0000 UTC m=+252.500261016 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/ec6fcb0d-227c-492a-923a-db5e74797aa6-service-ca-bundle") pod "router-default-7ff679ddf8-ctqk5" (UID: "ec6fcb0d-227c-492a-923a-db5e74797aa6") : configmap references non-existent config key: service-ca.crt
Sep 16 18:48:41.197663 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:41.197626    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-qkzj5"]
Sep 16 18:48:41.200671 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:48:41.200645    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65e92d96_65ad_4891_b693_1293389b7855.slice/crio-797de694a527ec8f950e9e9b82d70d908e3c89fefd9357cef514b4e3d07f7b56 WatchSource:0}: Error finding container 797de694a527ec8f950e9e9b82d70d908e3c89fefd9357cef514b4e3d07f7b56: Status 404 returned error can't find the container with id 797de694a527ec8f950e9e9b82d70d908e3c89fefd9357cef514b4e3d07f7b56
Sep 16 18:48:41.221263 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:41.221233    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-cf7c776b7-6xjxf"]
Sep 16 18:48:41.224012 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:48:41.223973    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12570b86_13fd_4b6d_adbf_e2d515c0492c.slice/crio-ecb988dadfc31a39422cb3410228f23f9e75c7836774a254ade5b83a9019164c WatchSource:0}: Error finding container ecb988dadfc31a39422cb3410228f23f9e75c7836774a254ade5b83a9019164c: Status 404 returned error can't find the container with id ecb988dadfc31a39422cb3410228f23f9e75c7836774a254ade5b83a9019164c
Sep 16 18:48:41.224153 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:41.224134    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-696884fb9c-crjw7"]
Sep 16 18:48:41.226662 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:48:41.226642    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podecf273b5_b465_4008_b035_f7ac0e740d58.slice/crio-99f1daee94661510d49242912939ad956e6f09dab31e66cc59e4ee73355a8db4 WatchSource:0}: Error finding container 99f1daee94661510d49242912939ad956e6f09dab31e66cc59e4ee73355a8db4: Status 404 returned error can't find the container with id 99f1daee94661510d49242912939ad956e6f09dab31e66cc59e4ee73355a8db4
Sep 16 18:48:41.270978 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:41.270953    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/93861b72-565d-497a-a7d5-098fc010f83c-registry-tls\") pod \"image-registry-76c66cc6d5-zsjkq\" (UID: \"93861b72-565d-497a-a7d5-098fc010f83c\") " pod="openshift-image-registry/image-registry-76c66cc6d5-zsjkq"
Sep 16 18:48:41.271087 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:48:41.271059    2584 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 16 18:48:41.271087 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:48:41.271071    2584 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-76c66cc6d5-zsjkq: secret "image-registry-tls" not found
Sep 16 18:48:41.271169 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:48:41.271114    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/93861b72-565d-497a-a7d5-098fc010f83c-registry-tls podName:93861b72-565d-497a-a7d5-098fc010f83c nodeName:}" failed. No retries permitted until 2026-09-16 18:48:42.271102095 +0000 UTC m=+252.600604486 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/93861b72-565d-497a-a7d5-098fc010f83c-registry-tls") pod "image-registry-76c66cc6d5-zsjkq" (UID: "93861b72-565d-497a-a7d5-098fc010f83c") : secret "image-registry-tls" not found
Sep 16 18:48:41.837278 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:41.837208    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-crjw7" event={"ID":"ecf273b5-b465-4008-b035-f7ac0e740d58","Type":"ContainerStarted","Data":"99f1daee94661510d49242912939ad956e6f09dab31e66cc59e4ee73355a8db4"}
Sep 16 18:48:41.838637 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:41.838603    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-qkzj5" event={"ID":"65e92d96-65ad-4891-b693-1293389b7855","Type":"ContainerStarted","Data":"797de694a527ec8f950e9e9b82d70d908e3c89fefd9357cef514b4e3d07f7b56"}
Sep 16 18:48:41.840038 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:41.840016    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-6xjxf" event={"ID":"12570b86-13fd-4b6d-adbf-e2d515c0492c","Type":"ContainerStarted","Data":"ecb988dadfc31a39422cb3410228f23f9e75c7836774a254ade5b83a9019164c"}
Sep 16 18:48:41.874967 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:41.874931    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9b96048a-0005-410b-a710-909d21fbfde8-metrics-certs\") pod \"network-metrics-daemon-bp5mv\" (UID: \"9b96048a-0005-410b-a710-909d21fbfde8\") " pod="openshift-multus/network-metrics-daemon-bp5mv"
Sep 16 18:48:41.875193 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:48:41.875170    2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 16 18:48:41.875278 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:48:41.875249    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9b96048a-0005-410b-a710-909d21fbfde8-metrics-certs podName:9b96048a-0005-410b-a710-909d21fbfde8 nodeName:}" failed. No retries permitted until 2026-09-16 18:50:43.875229002 +0000 UTC m=+374.204731392 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9b96048a-0005-410b-a710-909d21fbfde8-metrics-certs") pod "network-metrics-daemon-bp5mv" (UID: "9b96048a-0005-410b-a710-909d21fbfde8") : secret "metrics-daemon-secret" not found
Sep 16 18:48:42.177008 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:42.176922    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ec6fcb0d-227c-492a-923a-db5e74797aa6-service-ca-bundle\") pod \"router-default-7ff679ddf8-ctqk5\" (UID: \"ec6fcb0d-227c-492a-923a-db5e74797aa6\") " pod="openshift-ingress/router-default-7ff679ddf8-ctqk5"
Sep 16 18:48:42.177008 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:42.177006    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ec6fcb0d-227c-492a-923a-db5e74797aa6-metrics-certs\") pod \"router-default-7ff679ddf8-ctqk5\" (UID: \"ec6fcb0d-227c-492a-923a-db5e74797aa6\") " pod="openshift-ingress/router-default-7ff679ddf8-ctqk5"
Sep 16 18:48:42.177235 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:48:42.177108    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ec6fcb0d-227c-492a-923a-db5e74797aa6-service-ca-bundle podName:ec6fcb0d-227c-492a-923a-db5e74797aa6 nodeName:}" failed. No retries permitted until 2026-09-16 18:48:44.177086296 +0000 UTC m=+254.506588699 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/ec6fcb0d-227c-492a-923a-db5e74797aa6-service-ca-bundle") pod "router-default-7ff679ddf8-ctqk5" (UID: "ec6fcb0d-227c-492a-923a-db5e74797aa6") : configmap references non-existent config key: service-ca.crt
Sep 16 18:48:42.177235 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:48:42.177146    2584 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 16 18:48:42.177235 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:48:42.177182    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec6fcb0d-227c-492a-923a-db5e74797aa6-metrics-certs podName:ec6fcb0d-227c-492a-923a-db5e74797aa6 nodeName:}" failed. No retries permitted until 2026-09-16 18:48:44.177171241 +0000 UTC m=+254.506673645 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ec6fcb0d-227c-492a-923a-db5e74797aa6-metrics-certs") pod "router-default-7ff679ddf8-ctqk5" (UID: "ec6fcb0d-227c-492a-923a-db5e74797aa6") : secret "router-metrics-certs-default" not found
Sep 16 18:48:42.277750 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:42.277715    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/93861b72-565d-497a-a7d5-098fc010f83c-registry-tls\") pod \"image-registry-76c66cc6d5-zsjkq\" (UID: \"93861b72-565d-497a-a7d5-098fc010f83c\") " pod="openshift-image-registry/image-registry-76c66cc6d5-zsjkq"
Sep 16 18:48:42.277949 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:48:42.277870    2584 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 16 18:48:42.277949 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:48:42.277884    2584 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-76c66cc6d5-zsjkq: secret "image-registry-tls" not found
Sep 16 18:48:42.277949 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:48:42.277942    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/93861b72-565d-497a-a7d5-098fc010f83c-registry-tls podName:93861b72-565d-497a-a7d5-098fc010f83c nodeName:}" failed. No retries permitted until 2026-09-16 18:48:44.27792415 +0000 UTC m=+254.607426548 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/93861b72-565d-497a-a7d5-098fc010f83c-registry-tls") pod "image-registry-76c66cc6d5-zsjkq" (UID: "93861b72-565d-497a-a7d5-098fc010f83c") : secret "image-registry-tls" not found
Sep 16 18:48:43.397324 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:43.397273    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:43.498487 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:43.498447    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:44.193407 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:44.193366    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ec6fcb0d-227c-492a-923a-db5e74797aa6-service-ca-bundle\") pod \"router-default-7ff679ddf8-ctqk5\" (UID: \"ec6fcb0d-227c-492a-923a-db5e74797aa6\") " pod="openshift-ingress/router-default-7ff679ddf8-ctqk5"
Sep 16 18:48:44.193563 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:44.193443    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ec6fcb0d-227c-492a-923a-db5e74797aa6-metrics-certs\") pod \"router-default-7ff679ddf8-ctqk5\" (UID: \"ec6fcb0d-227c-492a-923a-db5e74797aa6\") " pod="openshift-ingress/router-default-7ff679ddf8-ctqk5"
Sep 16 18:48:44.193563 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:48:44.193548    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ec6fcb0d-227c-492a-923a-db5e74797aa6-service-ca-bundle podName:ec6fcb0d-227c-492a-923a-db5e74797aa6 nodeName:}" failed. No retries permitted until 2026-09-16 18:48:48.193529052 +0000 UTC m=+258.523031462 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/ec6fcb0d-227c-492a-923a-db5e74797aa6-service-ca-bundle") pod "router-default-7ff679ddf8-ctqk5" (UID: "ec6fcb0d-227c-492a-923a-db5e74797aa6") : configmap references non-existent config key: service-ca.crt
Sep 16 18:48:44.193563 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:48:44.193557    2584 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 16 18:48:44.193673 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:48:44.193591    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec6fcb0d-227c-492a-923a-db5e74797aa6-metrics-certs podName:ec6fcb0d-227c-492a-923a-db5e74797aa6 nodeName:}" failed. No retries permitted until 2026-09-16 18:48:48.193581303 +0000 UTC m=+258.523083697 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ec6fcb0d-227c-492a-923a-db5e74797aa6-metrics-certs") pod "router-default-7ff679ddf8-ctqk5" (UID: "ec6fcb0d-227c-492a-923a-db5e74797aa6") : secret "router-metrics-certs-default" not found
Sep 16 18:48:44.293805 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:44.293779    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/93861b72-565d-497a-a7d5-098fc010f83c-registry-tls\") pod \"image-registry-76c66cc6d5-zsjkq\" (UID: \"93861b72-565d-497a-a7d5-098fc010f83c\") " pod="openshift-image-registry/image-registry-76c66cc6d5-zsjkq"
Sep 16 18:48:44.293948 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:48:44.293929    2584 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 16 18:48:44.293998 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:48:44.293951    2584 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-76c66cc6d5-zsjkq: secret "image-registry-tls" not found
Sep 16 18:48:44.294037 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:48:44.294004    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/93861b72-565d-497a-a7d5-098fc010f83c-registry-tls podName:93861b72-565d-497a-a7d5-098fc010f83c nodeName:}" failed. No retries permitted until 2026-09-16 18:48:48.293989349 +0000 UTC m=+258.623491744 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/93861b72-565d-497a-a7d5-098fc010f83c-registry-tls") pod "image-registry-76c66cc6d5-zsjkq" (UID: "93861b72-565d-497a-a7d5-098fc010f83c") : secret "image-registry-tls" not found
Sep 16 18:48:44.850791 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:44.850758    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-crjw7_ecf273b5-b465-4008-b035-f7ac0e740d58/console-operator/0.log"
Sep 16 18:48:44.851236 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:44.850806    2584 generic.go:356] "Generic (PLEG): container finished" podID="ecf273b5-b465-4008-b035-f7ac0e740d58" containerID="6690b322440198f3ed2f1380eba7e12998a3c2e9d4467a68055ebbeb46432fba" exitCode=255
Sep 16 18:48:44.851236 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:44.850901    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-crjw7" event={"ID":"ecf273b5-b465-4008-b035-f7ac0e740d58","Type":"ContainerDied","Data":"6690b322440198f3ed2f1380eba7e12998a3c2e9d4467a68055ebbeb46432fba"}
Sep 16 18:48:44.851236 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:44.851179    2584 scope.go:117] "RemoveContainer" containerID="6690b322440198f3ed2f1380eba7e12998a3c2e9d4467a68055ebbeb46432fba"
Sep 16 18:48:44.852784 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:44.852752    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-qkzj5" event={"ID":"65e92d96-65ad-4891-b693-1293389b7855","Type":"ContainerStarted","Data":"3e43e5ab2579e28cb9b49e1c0f33676301580da88e31f58e99a092772c3fac02"}
Sep 16 18:48:44.854352 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:44.854324    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-6xjxf" event={"ID":"12570b86-13fd-4b6d-adbf-e2d515c0492c","Type":"ContainerStarted","Data":"6f9b34eb6f87767efde8c7fa4119f45761c14b1d8392d9317220fa3ab74a811a"}
Sep 16 18:48:44.883417 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:44.883362    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-6xjxf" podStartSLOduration=1.879171771 podStartE2EDuration="4.883342463s" totalImagesPullingTime="3.004170692s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:48:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:48:44.88294261 +0000 UTC m=+255.212445024" watchObservedRunningTime="2026-09-16 18:48:44.883342463 +0000 UTC m=+255.212844877"
Sep 16 18:48:44.901335 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:44.901285    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-qkzj5" podStartSLOduration=1.877106245 podStartE2EDuration="4.901262071s" totalImagesPullingTime="3.024155826s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:48:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:48:44.900089175 +0000 UTC m=+255.229591589" watchObservedRunningTime="2026-09-16 18:48:44.901262071 +0000 UTC m=+255.230764487"
Sep 16 18:48:45.081848 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:45.081799    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-9gqhw"]
Sep 16 18:48:45.085217 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:45.085199    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-9gqhw"
Sep 16 18:48:45.087895 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:45.087866    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 16 18:48:45.088039 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:45.087877    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 16 18:48:45.088039 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:45.087944    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 16 18:48:45.088225 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:45.088207    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 16 18:48:45.088371 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:45.088356    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-lpzdr\""
Sep 16 18:48:45.091149 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:45.090977    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-9gqhw"]
Sep 16 18:48:45.104141 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:45.104083    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:45.160145 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:45.160113    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:45.202180 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:45.202147    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/efe7abc6-7f9b-4141-aaa5-fbe85db5cead-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9gqhw\" (UID: \"efe7abc6-7f9b-4141-aaa5-fbe85db5cead\") " pod="openshift-insights/insights-runtime-extractor-9gqhw"
Sep 16 18:48:45.202300 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:45.202191    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/efe7abc6-7f9b-4141-aaa5-fbe85db5cead-crio-socket\") pod \"insights-runtime-extractor-9gqhw\" (UID: \"efe7abc6-7f9b-4141-aaa5-fbe85db5cead\") " pod="openshift-insights/insights-runtime-extractor-9gqhw"
Sep 16 18:48:45.202300 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:45.202281    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/efe7abc6-7f9b-4141-aaa5-fbe85db5cead-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9gqhw\" (UID: \"efe7abc6-7f9b-4141-aaa5-fbe85db5cead\") " pod="openshift-insights/insights-runtime-extractor-9gqhw"
Sep 16 18:48:45.202390 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:45.202310    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w7d7r\" (UniqueName: \"kubernetes.io/projected/efe7abc6-7f9b-4141-aaa5-fbe85db5cead-kube-api-access-w7d7r\") pod \"insights-runtime-extractor-9gqhw\" (UID: \"efe7abc6-7f9b-4141-aaa5-fbe85db5cead\") " pod="openshift-insights/insights-runtime-extractor-9gqhw"
Sep 16 18:48:45.202390 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:45.202349    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/efe7abc6-7f9b-4141-aaa5-fbe85db5cead-data-volume\") pod \"insights-runtime-extractor-9gqhw\" (UID: \"efe7abc6-7f9b-4141-aaa5-fbe85db5cead\") " pod="openshift-insights/insights-runtime-extractor-9gqhw"
Sep 16 18:48:45.303724 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:45.303685    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/efe7abc6-7f9b-4141-aaa5-fbe85db5cead-data-volume\") pod \"insights-runtime-extractor-9gqhw\" (UID: \"efe7abc6-7f9b-4141-aaa5-fbe85db5cead\") " pod="openshift-insights/insights-runtime-extractor-9gqhw"
Sep 16 18:48:45.303894 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:45.303736    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/efe7abc6-7f9b-4141-aaa5-fbe85db5cead-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9gqhw\" (UID: \"efe7abc6-7f9b-4141-aaa5-fbe85db5cead\") " pod="openshift-insights/insights-runtime-extractor-9gqhw"
Sep 16 18:48:45.303894 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:45.303871    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/efe7abc6-7f9b-4141-aaa5-fbe85db5cead-crio-socket\") pod \"insights-runtime-extractor-9gqhw\" (UID: \"efe7abc6-7f9b-4141-aaa5-fbe85db5cead\") " pod="openshift-insights/insights-runtime-extractor-9gqhw"
Sep 16 18:48:45.303975 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:48:45.303959    2584 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 16 18:48:45.304020 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:45.303980    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/efe7abc6-7f9b-4141-aaa5-fbe85db5cead-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9gqhw\" (UID: \"efe7abc6-7f9b-4141-aaa5-fbe85db5cead\") " pod="openshift-insights/insights-runtime-extractor-9gqhw"
Sep 16 18:48:45.304020 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:48:45.304016    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/efe7abc6-7f9b-4141-aaa5-fbe85db5cead-kube-rbac-proxy-cm podName:efe7abc6-7f9b-4141-aaa5-fbe85db5cead nodeName:}" failed. No retries permitted until 2026-09-16 18:48:45.803996109 +0000 UTC m=+256.133498506 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/efe7abc6-7f9b-4141-aaa5-fbe85db5cead-kube-rbac-proxy-cm") pod "insights-runtime-extractor-9gqhw" (UID: "efe7abc6-7f9b-4141-aaa5-fbe85db5cead") : configmap "kube-rbac-proxy" not found
Sep 16 18:48:45.304130 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:45.304054    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w7d7r\" (UniqueName: \"kubernetes.io/projected/efe7abc6-7f9b-4141-aaa5-fbe85db5cead-kube-api-access-w7d7r\") pod \"insights-runtime-extractor-9gqhw\" (UID: \"efe7abc6-7f9b-4141-aaa5-fbe85db5cead\") " pod="openshift-insights/insights-runtime-extractor-9gqhw"
Sep 16 18:48:45.304130 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:45.304062    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/efe7abc6-7f9b-4141-aaa5-fbe85db5cead-crio-socket\") pod \"insights-runtime-extractor-9gqhw\" (UID: \"efe7abc6-7f9b-4141-aaa5-fbe85db5cead\") " pod="openshift-insights/insights-runtime-extractor-9gqhw"
Sep 16 18:48:45.304130 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:48:45.304089    2584 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 16 18:48:45.304130 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:45.304109    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/efe7abc6-7f9b-4141-aaa5-fbe85db5cead-data-volume\") pod \"insights-runtime-extractor-9gqhw\" (UID: \"efe7abc6-7f9b-4141-aaa5-fbe85db5cead\") " pod="openshift-insights/insights-runtime-extractor-9gqhw"
Sep 16 18:48:45.304263 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:48:45.304152    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/efe7abc6-7f9b-4141-aaa5-fbe85db5cead-insights-runtime-extractor-tls podName:efe7abc6-7f9b-4141-aaa5-fbe85db5cead nodeName:}" failed. No retries permitted until 2026-09-16 18:48:45.804142057 +0000 UTC m=+256.133644455 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/efe7abc6-7f9b-4141-aaa5-fbe85db5cead-insights-runtime-extractor-tls") pod "insights-runtime-extractor-9gqhw" (UID: "efe7abc6-7f9b-4141-aaa5-fbe85db5cead") : secret "insights-runtime-extractor-tls" not found
Sep 16 18:48:45.312580 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:45.312557    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w7d7r\" (UniqueName: \"kubernetes.io/projected/efe7abc6-7f9b-4141-aaa5-fbe85db5cead-kube-api-access-w7d7r\") pod \"insights-runtime-extractor-9gqhw\" (UID: \"efe7abc6-7f9b-4141-aaa5-fbe85db5cead\") " pod="openshift-insights/insights-runtime-extractor-9gqhw"
Sep 16 18:48:45.613270 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:45.613234    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:45.669434 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:45.669394    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:45.808173 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:45.808139    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/efe7abc6-7f9b-4141-aaa5-fbe85db5cead-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9gqhw\" (UID: \"efe7abc6-7f9b-4141-aaa5-fbe85db5cead\") " pod="openshift-insights/insights-runtime-extractor-9gqhw"
Sep 16 18:48:45.808363 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:45.808214    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/efe7abc6-7f9b-4141-aaa5-fbe85db5cead-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9gqhw\" (UID: \"efe7abc6-7f9b-4141-aaa5-fbe85db5cead\") " pod="openshift-insights/insights-runtime-extractor-9gqhw"
Sep 16 18:48:45.808363 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:48:45.808284    2584 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 16 18:48:45.808363 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:48:45.808304    2584 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 16 18:48:45.808493 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:48:45.808376    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/efe7abc6-7f9b-4141-aaa5-fbe85db5cead-kube-rbac-proxy-cm podName:efe7abc6-7f9b-4141-aaa5-fbe85db5cead nodeName:}" failed. No retries permitted until 2026-09-16 18:48:46.808355249 +0000 UTC m=+257.137857657 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/efe7abc6-7f9b-4141-aaa5-fbe85db5cead-kube-rbac-proxy-cm") pod "insights-runtime-extractor-9gqhw" (UID: "efe7abc6-7f9b-4141-aaa5-fbe85db5cead") : configmap "kube-rbac-proxy" not found
Sep 16 18:48:45.808493 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:48:45.808400    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/efe7abc6-7f9b-4141-aaa5-fbe85db5cead-insights-runtime-extractor-tls podName:efe7abc6-7f9b-4141-aaa5-fbe85db5cead nodeName:}" failed. No retries permitted until 2026-09-16 18:48:46.808388846 +0000 UTC m=+257.137891251 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/efe7abc6-7f9b-4141-aaa5-fbe85db5cead-insights-runtime-extractor-tls") pod "insights-runtime-extractor-9gqhw" (UID: "efe7abc6-7f9b-4141-aaa5-fbe85db5cead") : secret "insights-runtime-extractor-tls" not found
Sep 16 18:48:45.854649 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:45.854615    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:45.858159 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:45.858137    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-crjw7_ecf273b5-b465-4008-b035-f7ac0e740d58/console-operator/1.log"
Sep 16 18:48:45.858533 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:45.858519    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-crjw7_ecf273b5-b465-4008-b035-f7ac0e740d58/console-operator/0.log"
Sep 16 18:48:45.858581 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:45.858552    2584 generic.go:356] "Generic (PLEG): container finished" podID="ecf273b5-b465-4008-b035-f7ac0e740d58" containerID="afa27c6465cfad7ad388ace7433d367335073c3373173449c41e3b1816dea9ef" exitCode=255
Sep 16 18:48:45.858644 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:45.858628    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-crjw7" event={"ID":"ecf273b5-b465-4008-b035-f7ac0e740d58","Type":"ContainerDied","Data":"afa27c6465cfad7ad388ace7433d367335073c3373173449c41e3b1816dea9ef"}
Sep 16 18:48:45.858689 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:45.858676    2584 scope.go:117] "RemoveContainer" containerID="6690b322440198f3ed2f1380eba7e12998a3c2e9d4467a68055ebbeb46432fba"
Sep 16 18:48:45.858943 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:45.858893    2584 scope.go:117] "RemoveContainer" containerID="afa27c6465cfad7ad388ace7433d367335073c3373173449c41e3b1816dea9ef"
Sep 16 18:48:45.859114 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:48:45.859089    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-696884fb9c-crjw7_openshift-console-operator(ecf273b5-b465-4008-b035-f7ac0e740d58)\"" pod="openshift-console-operator/console-operator-696884fb9c-crjw7" podUID="ecf273b5-b465-4008-b035-f7ac0e740d58"
Sep 16 18:48:45.948990 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:45.948953    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:46.048912 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:46.048880    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:46.106500 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:46.106460    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:46.332559 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:46.332527    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:46.431931 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:46.431890    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:46.817566 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:46.817511    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/efe7abc6-7f9b-4141-aaa5-fbe85db5cead-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9gqhw\" (UID: \"efe7abc6-7f9b-4141-aaa5-fbe85db5cead\") " pod="openshift-insights/insights-runtime-extractor-9gqhw"
Sep 16 18:48:46.817790 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:46.817606    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/efe7abc6-7f9b-4141-aaa5-fbe85db5cead-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9gqhw\" (UID: \"efe7abc6-7f9b-4141-aaa5-fbe85db5cead\") " pod="openshift-insights/insights-runtime-extractor-9gqhw"
Sep 16 18:48:46.817790 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:48:46.817665    2584 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 16 18:48:46.817790 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:48:46.817693    2584 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 16 18:48:46.817790 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:48:46.817758    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/efe7abc6-7f9b-4141-aaa5-fbe85db5cead-kube-rbac-proxy-cm podName:efe7abc6-7f9b-4141-aaa5-fbe85db5cead nodeName:}" failed. No retries permitted until 2026-09-16 18:48:48.817738254 +0000 UTC m=+259.147240658 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/efe7abc6-7f9b-4141-aaa5-fbe85db5cead-kube-rbac-proxy-cm") pod "insights-runtime-extractor-9gqhw" (UID: "efe7abc6-7f9b-4141-aaa5-fbe85db5cead") : configmap "kube-rbac-proxy" not found
Sep 16 18:48:46.817790 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:48:46.817777    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/efe7abc6-7f9b-4141-aaa5-fbe85db5cead-insights-runtime-extractor-tls podName:efe7abc6-7f9b-4141-aaa5-fbe85db5cead nodeName:}" failed. No retries permitted until 2026-09-16 18:48:48.817769489 +0000 UTC m=+259.147271883 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/efe7abc6-7f9b-4141-aaa5-fbe85db5cead-insights-runtime-extractor-tls") pod "insights-runtime-extractor-9gqhw" (UID: "efe7abc6-7f9b-4141-aaa5-fbe85db5cead") : secret "insights-runtime-extractor-tls" not found
Sep 16 18:48:46.862884 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:46.862862    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-crjw7_ecf273b5-b465-4008-b035-f7ac0e740d58/console-operator/1.log"
Sep 16 18:48:46.863194 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:46.863180    2584 scope.go:117] "RemoveContainer" containerID="afa27c6465cfad7ad388ace7433d367335073c3373173449c41e3b1816dea9ef"
Sep 16 18:48:46.863337 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:48:46.863320    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-696884fb9c-crjw7_openshift-console-operator(ecf273b5-b465-4008-b035-f7ac0e740d58)\"" pod="openshift-console-operator/console-operator-696884fb9c-crjw7" podUID="ecf273b5-b465-4008-b035-f7ac0e740d58"
Sep 16 18:48:47.135643 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:47.135570    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-bd22w_4a80b1e8-39b4-44a2-80cb-aa52d96e80fb/dns-node-resolver/0.log"
Sep 16 18:48:47.988617 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:47.988583    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:48.042169 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:48.042126    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:48.136117 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:48.136089    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-f8jpf_1498d81a-223d-470c-9812-9b2910e3ea35/node-ca/0.log"
Sep 16 18:48:48.229033 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:48.229000    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ec6fcb0d-227c-492a-923a-db5e74797aa6-service-ca-bundle\") pod \"router-default-7ff679ddf8-ctqk5\" (UID: \"ec6fcb0d-227c-492a-923a-db5e74797aa6\") " pod="openshift-ingress/router-default-7ff679ddf8-ctqk5"
Sep 16 18:48:48.229201 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:48.229065    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ec6fcb0d-227c-492a-923a-db5e74797aa6-metrics-certs\") pod \"router-default-7ff679ddf8-ctqk5\" (UID: \"ec6fcb0d-227c-492a-923a-db5e74797aa6\") " pod="openshift-ingress/router-default-7ff679ddf8-ctqk5"
Sep 16 18:48:48.229201 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:48:48.229193    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ec6fcb0d-227c-492a-923a-db5e74797aa6-service-ca-bundle podName:ec6fcb0d-227c-492a-923a-db5e74797aa6 nodeName:}" failed. No retries permitted until 2026-09-16 18:48:56.22917162 +0000 UTC m=+266.558674024 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/ec6fcb0d-227c-492a-923a-db5e74797aa6-service-ca-bundle") pod "router-default-7ff679ddf8-ctqk5" (UID: "ec6fcb0d-227c-492a-923a-db5e74797aa6") : configmap references non-existent config key: service-ca.crt
Sep 16 18:48:48.229306 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:48:48.229199    2584 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 16 18:48:48.229306 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:48:48.229241    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec6fcb0d-227c-492a-923a-db5e74797aa6-metrics-certs podName:ec6fcb0d-227c-492a-923a-db5e74797aa6 nodeName:}" failed. No retries permitted until 2026-09-16 18:48:56.229230631 +0000 UTC m=+266.558733041 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ec6fcb0d-227c-492a-923a-db5e74797aa6-metrics-certs") pod "router-default-7ff679ddf8-ctqk5" (UID: "ec6fcb0d-227c-492a-923a-db5e74797aa6") : secret "router-metrics-certs-default" not found
Sep 16 18:48:48.329571 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:48.329538    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/93861b72-565d-497a-a7d5-098fc010f83c-registry-tls\") pod \"image-registry-76c66cc6d5-zsjkq\" (UID: \"93861b72-565d-497a-a7d5-098fc010f83c\") " pod="openshift-image-registry/image-registry-76c66cc6d5-zsjkq"
Sep 16 18:48:48.329711 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:48:48.329684    2584 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 16 18:48:48.329711 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:48:48.329700    2584 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-76c66cc6d5-zsjkq: secret "image-registry-tls" not found
Sep 16 18:48:48.329795 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:48:48.329758    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/93861b72-565d-497a-a7d5-098fc010f83c-registry-tls podName:93861b72-565d-497a-a7d5-098fc010f83c nodeName:}" failed. No retries permitted until 2026-09-16 18:48:56.329740175 +0000 UTC m=+266.659242568 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/93861b72-565d-497a-a7d5-098fc010f83c-registry-tls") pod "image-registry-76c66cc6d5-zsjkq" (UID: "93861b72-565d-497a-a7d5-098fc010f83c") : secret "image-registry-tls" not found
Sep 16 18:48:48.513497 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:48.513462    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:48.833927 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:48.833885    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/efe7abc6-7f9b-4141-aaa5-fbe85db5cead-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9gqhw\" (UID: \"efe7abc6-7f9b-4141-aaa5-fbe85db5cead\") " pod="openshift-insights/insights-runtime-extractor-9gqhw"
Sep 16 18:48:48.834092 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:48.833962    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/efe7abc6-7f9b-4141-aaa5-fbe85db5cead-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9gqhw\" (UID: \"efe7abc6-7f9b-4141-aaa5-fbe85db5cead\") " pod="openshift-insights/insights-runtime-extractor-9gqhw"
Sep 16 18:48:48.834092 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:48:48.834007    2584 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 16 18:48:48.834092 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:48:48.834063    2584 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 16 18:48:48.834092 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:48:48.834081    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/efe7abc6-7f9b-4141-aaa5-fbe85db5cead-kube-rbac-proxy-cm podName:efe7abc6-7f9b-4141-aaa5-fbe85db5cead nodeName:}" failed. No retries permitted until 2026-09-16 18:48:52.834066361 +0000 UTC m=+263.163568751 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/efe7abc6-7f9b-4141-aaa5-fbe85db5cead-kube-rbac-proxy-cm") pod "insights-runtime-extractor-9gqhw" (UID: "efe7abc6-7f9b-4141-aaa5-fbe85db5cead") : configmap "kube-rbac-proxy" not found
Sep 16 18:48:48.834226 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:48:48.834103    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/efe7abc6-7f9b-4141-aaa5-fbe85db5cead-insights-runtime-extractor-tls podName:efe7abc6-7f9b-4141-aaa5-fbe85db5cead nodeName:}" failed. No retries permitted until 2026-09-16 18:48:52.834092363 +0000 UTC m=+263.163594754 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/efe7abc6-7f9b-4141-aaa5-fbe85db5cead-insights-runtime-extractor-tls") pod "insights-runtime-extractor-9gqhw" (UID: "efe7abc6-7f9b-4141-aaa5-fbe85db5cead") : secret "insights-runtime-extractor-tls" not found
Sep 16 18:48:48.868069 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:48.868023    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:48.936762 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:48.936736    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-s4gvd_80b7ba2b-64e7-4bf3-9f78-d8a2b2b0e66a/serve-healthcheck-canary/0.log"
Sep 16 18:48:50.931803 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:50.931761    2584 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-696884fb9c-crjw7"
Sep 16 18:48:50.932202 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:50.931821    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-696884fb9c-crjw7"
Sep 16 18:48:50.932202 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:50.932171    2584 scope.go:117] "RemoveContainer" containerID="afa27c6465cfad7ad388ace7433d367335073c3373173449c41e3b1816dea9ef"
Sep 16 18:48:50.932325 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:48:50.932311    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-696884fb9c-crjw7_openshift-console-operator(ecf273b5-b465-4008-b035-f7ac0e740d58)\"" pod="openshift-console-operator/console-operator-696884fb9c-crjw7" podUID="ecf273b5-b465-4008-b035-f7ac0e740d58"
Sep 16 18:48:51.689409 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:51.689377    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:51.987800 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:51.987722    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:52.198559 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:52.198518    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:52.316586 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:52.316553    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:52.424203 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:52.424171    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:52.571144 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:52.571045    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:52.676033 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:52.675965    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:52.787506 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:52.787467    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:52.871785 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:52.871695    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/efe7abc6-7f9b-4141-aaa5-fbe85db5cead-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9gqhw\" (UID: \"efe7abc6-7f9b-4141-aaa5-fbe85db5cead\") " pod="openshift-insights/insights-runtime-extractor-9gqhw"
Sep 16 18:48:52.871929 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:52.871785    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/efe7abc6-7f9b-4141-aaa5-fbe85db5cead-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9gqhw\" (UID: \"efe7abc6-7f9b-4141-aaa5-fbe85db5cead\") " pod="openshift-insights/insights-runtime-extractor-9gqhw"
Sep 16 18:48:52.871929 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:48:52.871914    2584 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 16 18:48:52.872011 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:48:52.871965    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/efe7abc6-7f9b-4141-aaa5-fbe85db5cead-insights-runtime-extractor-tls podName:efe7abc6-7f9b-4141-aaa5-fbe85db5cead nodeName:}" failed. No retries permitted until 2026-09-16 18:49:00.87195213 +0000 UTC m=+271.201454520 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/efe7abc6-7f9b-4141-aaa5-fbe85db5cead-insights-runtime-extractor-tls") pod "insights-runtime-extractor-9gqhw" (UID: "efe7abc6-7f9b-4141-aaa5-fbe85db5cead") : secret "insights-runtime-extractor-tls" not found
Sep 16 18:48:52.872259 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:52.872238    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/efe7abc6-7f9b-4141-aaa5-fbe85db5cead-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9gqhw\" (UID: \"efe7abc6-7f9b-4141-aaa5-fbe85db5cead\") " pod="openshift-insights/insights-runtime-extractor-9gqhw"
Sep 16 18:48:52.907561 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:52.907532    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:53.888092 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:53.888058    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:54.096499 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:54.096470    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:54.490145 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:54.490092    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:54.685103 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:54.685064    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:55.288233 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:55.288200    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:55.686518 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:55.686420    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:55.882127 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:55.882093    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:56.082166 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:56.082126    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:56.273465 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:56.273432    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:56.300554 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:56.300524    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ec6fcb0d-227c-492a-923a-db5e74797aa6-service-ca-bundle\") pod \"router-default-7ff679ddf8-ctqk5\" (UID: \"ec6fcb0d-227c-492a-923a-db5e74797aa6\") " pod="openshift-ingress/router-default-7ff679ddf8-ctqk5"
Sep 16 18:48:56.300941 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:56.300683    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ec6fcb0d-227c-492a-923a-db5e74797aa6-metrics-certs\") pod \"router-default-7ff679ddf8-ctqk5\" (UID: \"ec6fcb0d-227c-492a-923a-db5e74797aa6\") " pod="openshift-ingress/router-default-7ff679ddf8-ctqk5"
Sep 16 18:48:56.300941 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:48:56.300703    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ec6fcb0d-227c-492a-923a-db5e74797aa6-service-ca-bundle podName:ec6fcb0d-227c-492a-923a-db5e74797aa6 nodeName:}" failed. No retries permitted until 2026-09-16 18:49:12.300687829 +0000 UTC m=+282.630190220 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/ec6fcb0d-227c-492a-923a-db5e74797aa6-service-ca-bundle") pod "router-default-7ff679ddf8-ctqk5" (UID: "ec6fcb0d-227c-492a-923a-db5e74797aa6") : configmap references non-existent config key: service-ca.crt
Sep 16 18:48:56.300941 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:48:56.300761    2584 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 16 18:48:56.300941 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:48:56.300805    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec6fcb0d-227c-492a-923a-db5e74797aa6-metrics-certs podName:ec6fcb0d-227c-492a-923a-db5e74797aa6 nodeName:}" failed. No retries permitted until 2026-09-16 18:49:12.300793987 +0000 UTC m=+282.630296378 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ec6fcb0d-227c-492a-923a-db5e74797aa6-metrics-certs") pod "router-default-7ff679ddf8-ctqk5" (UID: "ec6fcb0d-227c-492a-923a-db5e74797aa6") : secret "router-metrics-certs-default" not found
Sep 16 18:48:56.376297 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:56.376200    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:56.401447 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:56.401421    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/93861b72-565d-497a-a7d5-098fc010f83c-registry-tls\") pod \"image-registry-76c66cc6d5-zsjkq\" (UID: \"93861b72-565d-497a-a7d5-098fc010f83c\") " pod="openshift-image-registry/image-registry-76c66cc6d5-zsjkq"
Sep 16 18:48:56.403719 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:56.403697    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/93861b72-565d-497a-a7d5-098fc010f83c-registry-tls\") pod \"image-registry-76c66cc6d5-zsjkq\" (UID: \"93861b72-565d-497a-a7d5-098fc010f83c\") " pod="openshift-image-registry/image-registry-76c66cc6d5-zsjkq"
Sep 16 18:48:56.544817 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:56.544782    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-76c66cc6d5-zsjkq"
Sep 16 18:48:56.817910 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:56.817875    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-76c66cc6d5-zsjkq"]
Sep 16 18:48:56.820130 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:48:56.820106    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93861b72_565d_497a_a7d5_098fc010f83c.slice/crio-206a3e568da735e0cc8e4dc6d4c9b27953c8ff29221b901553e1867fadb01fad WatchSource:0}: Error finding container 206a3e568da735e0cc8e4dc6d4c9b27953c8ff29221b901553e1867fadb01fad: Status 404 returned error can't find the container with id 206a3e568da735e0cc8e4dc6d4c9b27953c8ff29221b901553e1867fadb01fad
Sep 16 18:48:56.887538 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:56.887504    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:56.893291 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:56.893263    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-76c66cc6d5-zsjkq" event={"ID":"93861b72-565d-497a-a7d5-098fc010f83c","Type":"ContainerStarted","Data":"c467eab7f9d522f45d41e5e83cc80a4aafe653ba783a5997d7c3654775e42882"}
Sep 16 18:48:56.893387 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:56.893312    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-76c66cc6d5-zsjkq" event={"ID":"93861b72-565d-497a-a7d5-098fc010f83c","Type":"ContainerStarted","Data":"206a3e568da735e0cc8e4dc6d4c9b27953c8ff29221b901553e1867fadb01fad"}
Sep 16 18:48:56.893387 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:56.893374    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-76c66cc6d5-zsjkq"
Sep 16 18:48:56.913285 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:56.913238    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-76c66cc6d5-zsjkq" podStartSLOduration=16.913223259 podStartE2EDuration="16.913223259s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:48:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:48:56.912445543 +0000 UTC m=+267.241947955" watchObservedRunningTime="2026-09-16 18:48:56.913223259 +0000 UTC m=+267.242725662"
Sep 16 18:48:57.486878 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:57.486845    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:57.688287 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:57.688257    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:57.891390 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:57.891332    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:58.286255 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:58.286223    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:58.676083 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:58.675996    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:58.770932 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:58.770905    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:59.082044 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:59.082016    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:59.685447 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:48:59.685412    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:00.154462 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:00.154429    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:00.287335 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:00.287300    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:00.487068 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:00.486984    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:00.692331 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:00.692299    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:00.937545 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:00.937513    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/efe7abc6-7f9b-4141-aaa5-fbe85db5cead-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9gqhw\" (UID: \"efe7abc6-7f9b-4141-aaa5-fbe85db5cead\") " pod="openshift-insights/insights-runtime-extractor-9gqhw"
Sep 16 18:49:00.939680 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:00.939653    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/efe7abc6-7f9b-4141-aaa5-fbe85db5cead-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9gqhw\" (UID: \"efe7abc6-7f9b-4141-aaa5-fbe85db5cead\") " pod="openshift-insights/insights-runtime-extractor-9gqhw"
Sep 16 18:49:00.995994 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:00.995945    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-9gqhw"
Sep 16 18:49:01.086696 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:01.086663    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:01.267549 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:01.267522    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-9gqhw"]
Sep 16 18:49:01.270486 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:49:01.270459    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podefe7abc6_7f9b_4141_aaa5_fbe85db5cead.slice/crio-9716d349d451666cb6c649287791b84ed452ed0eba6b3e504cdb27811d36c34d WatchSource:0}: Error finding container 9716d349d451666cb6c649287791b84ed452ed0eba6b3e504cdb27811d36c34d: Status 404 returned error can't find the container with id 9716d349d451666cb6c649287791b84ed452ed0eba6b3e504cdb27811d36c34d
Sep 16 18:49:01.682165 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:01.682083    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:01.911484 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:01.911453    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-9gqhw" event={"ID":"efe7abc6-7f9b-4141-aaa5-fbe85db5cead","Type":"ContainerStarted","Data":"cf486cc0354bad9583403424a0a17cf9a242c39456524a68c9ce085b59976b63"}
Sep 16 18:49:01.911852 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:01.911505    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-9gqhw" event={"ID":"efe7abc6-7f9b-4141-aaa5-fbe85db5cead","Type":"ContainerStarted","Data":"9716d349d451666cb6c649287791b84ed452ed0eba6b3e504cdb27811d36c34d"}
Sep 16 18:49:02.282248 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:02.282221    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:02.482705 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:02.482624    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:02.689478 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:02.689437    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:02.882695 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:02.882660    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:02.916548 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:02.916500    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-9gqhw" event={"ID":"efe7abc6-7f9b-4141-aaa5-fbe85db5cead","Type":"ContainerStarted","Data":"1ec2b56aef694a6fc418f2f746c265fafec0fac433d57e914ff87aa2ff7c0a91"}
Sep 16 18:49:03.111766 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:03.111738    2584 scope.go:117] "RemoveContainer" containerID="afa27c6465cfad7ad388ace7433d367335073c3373173449c41e3b1816dea9ef"
Sep 16 18:49:03.922210 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:03.922168    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-9gqhw" event={"ID":"efe7abc6-7f9b-4141-aaa5-fbe85db5cead","Type":"ContainerStarted","Data":"110bfcdcf4d262af85662ce96bae93c17f6657935688872f446d766a00697290"}
Sep 16 18:49:03.923963 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:03.923941    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-crjw7_ecf273b5-b465-4008-b035-f7ac0e740d58/console-operator/1.log"
Sep 16 18:49:03.924080 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:03.924014    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-crjw7" event={"ID":"ecf273b5-b465-4008-b035-f7ac0e740d58","Type":"ContainerStarted","Data":"8ee99161cf01187a89b54e68cca6a10b07082ae67da7de668200621c86cfb1bb"}
Sep 16 18:49:03.924258 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:03.924238    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-696884fb9c-crjw7"
Sep 16 18:49:03.940787 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:03.940733    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-9gqhw" podStartSLOduration=16.913940499 podStartE2EDuration="18.940712987s" totalImagesPullingTime="2.026772488s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 18:48:45 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:49:03.940437609 +0000 UTC m=+274.269940023" watchObservedRunningTime="2026-09-16 18:49:03.940712987 +0000 UTC m=+274.270215402"
Sep 16 18:49:03.956703 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:03.956654    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-696884fb9c-crjw7" podStartSLOduration=20.951946426 podStartE2EDuration="23.956637535s" totalImagesPullingTime="3.004691109s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:48:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:49:03.956150743 +0000 UTC m=+274.285653157" watchObservedRunningTime="2026-09-16 18:49:03.956637535 +0000 UTC m=+274.286140001"
Sep 16 18:49:04.128744 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:04.128712    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-696884fb9c-crjw7"
Sep 16 18:49:04.361886 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:04.361848    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:04.424191 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:04.424164    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:04.604307 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:04.604277    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:04.780544 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:04.780520    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:04.837412 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:04.837387    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:05.011867 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:05.011819    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:05.115546 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:05.115468    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:05.387195 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:05.387119    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:05.440566 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:05.440536    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:05.991980 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:05.991950    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:06.189772 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:06.189744    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:06.293256 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:06.293225    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:06.391800 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:06.391766    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:06.491046 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:06.491016    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:06.742506 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:06.742435    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-76c66cc6d5-zsjkq"]
Sep 16 18:49:06.810587 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:06.810536    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:06.912541 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:06.912506    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:07.408759 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:07.408729    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:07.509065 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:07.509034    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:10.611407 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:49:10.611355    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-tv99q" podUID="7a4aa47d-be1a-4b2c-bd88-3edc40ef5d6a"
Sep 16 18:49:10.946722 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:10.946654    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-tv99q"
Sep 16 18:49:12.322433 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:12.322376    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ec6fcb0d-227c-492a-923a-db5e74797aa6-service-ca-bundle\") pod \"router-default-7ff679ddf8-ctqk5\" (UID: \"ec6fcb0d-227c-492a-923a-db5e74797aa6\") " pod="openshift-ingress/router-default-7ff679ddf8-ctqk5"
Sep 16 18:49:12.322846 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:12.322544    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ec6fcb0d-227c-492a-923a-db5e74797aa6-metrics-certs\") pod \"router-default-7ff679ddf8-ctqk5\" (UID: \"ec6fcb0d-227c-492a-923a-db5e74797aa6\") " pod="openshift-ingress/router-default-7ff679ddf8-ctqk5"
Sep 16 18:49:12.323041 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:12.323019    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ec6fcb0d-227c-492a-923a-db5e74797aa6-service-ca-bundle\") pod \"router-default-7ff679ddf8-ctqk5\" (UID: \"ec6fcb0d-227c-492a-923a-db5e74797aa6\") " pod="openshift-ingress/router-default-7ff679ddf8-ctqk5"
Sep 16 18:49:12.324978 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:12.324958    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ec6fcb0d-227c-492a-923a-db5e74797aa6-metrics-certs\") pod \"router-default-7ff679ddf8-ctqk5\" (UID: \"ec6fcb0d-227c-492a-923a-db5e74797aa6\") " pod="openshift-ingress/router-default-7ff679ddf8-ctqk5"
Sep 16 18:49:12.617020 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:12.616921    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-7ff679ddf8-ctqk5"
Sep 16 18:49:12.888321 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:12.888219    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-7ff679ddf8-ctqk5"]
Sep 16 18:49:12.890782 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:49:12.890757    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec6fcb0d_227c_492a_923a_db5e74797aa6.slice/crio-2bbc560184fccbaab64b88b85c988c5c43c1f145ee6f35a7fddc8c9b887a81e1 WatchSource:0}: Error finding container 2bbc560184fccbaab64b88b85c988c5c43c1f145ee6f35a7fddc8c9b887a81e1: Status 404 returned error can't find the container with id 2bbc560184fccbaab64b88b85c988c5c43c1f145ee6f35a7fddc8c9b887a81e1
Sep 16 18:49:12.955303 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:12.955271    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-7ff679ddf8-ctqk5" event={"ID":"ec6fcb0d-227c-492a-923a-db5e74797aa6","Type":"ContainerStarted","Data":"98baf44f828f69a4a955f32450ec1ad036e7dd5ac4f442eb49a17ec7eaec398e"}
Sep 16 18:49:12.955481 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:12.955318    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-7ff679ddf8-ctqk5" event={"ID":"ec6fcb0d-227c-492a-923a-db5e74797aa6","Type":"ContainerStarted","Data":"2bbc560184fccbaab64b88b85c988c5c43c1f145ee6f35a7fddc8c9b887a81e1"}
Sep 16 18:49:12.975388 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:12.975341    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-7ff679ddf8-ctqk5" podStartSLOduration=32.975326328 podStartE2EDuration="32.975326328s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:48:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:49:12.973810519 +0000 UTC m=+283.303312942" watchObservedRunningTime="2026-09-16 18:49:12.975326328 +0000 UTC m=+283.304828741"
Sep 16 18:49:13.617384 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:13.617334    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-7ff679ddf8-ctqk5"
Sep 16 18:49:13.620087 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:13.620062    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-7ff679ddf8-ctqk5"
Sep 16 18:49:13.958379 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:13.958297    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-7ff679ddf8-ctqk5"
Sep 16 18:49:13.959499 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:13.959478    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-7ff679ddf8-ctqk5"
Sep 16 18:49:14.034214 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:14.034187    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7a4aa47d-be1a-4b2c-bd88-3edc40ef5d6a-metrics-tls\") pod \"dns-default-tv99q\" (UID: \"7a4aa47d-be1a-4b2c-bd88-3edc40ef5d6a\") " pod="openshift-dns/dns-default-tv99q"
Sep 16 18:49:14.036420 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:14.036394    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7a4aa47d-be1a-4b2c-bd88-3edc40ef5d6a-metrics-tls\") pod \"dns-default-tv99q\" (UID: \"7a4aa47d-be1a-4b2c-bd88-3edc40ef5d6a\") " pod="openshift-dns/dns-default-tv99q"
Sep 16 18:49:14.250552 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:14.250466    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-l7s4t\""
Sep 16 18:49:14.258784 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:14.258762    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-tv99q"
Sep 16 18:49:14.530568 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:14.530529    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-tv99q"]
Sep 16 18:49:14.534875 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:49:14.534844    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a4aa47d_be1a_4b2c_bd88_3edc40ef5d6a.slice/crio-73bc7a62abe7224d51d78dd6b1dc4f9ea51b3f58b6cf00072091e913b567ec28 WatchSource:0}: Error finding container 73bc7a62abe7224d51d78dd6b1dc4f9ea51b3f58b6cf00072091e913b567ec28: Status 404 returned error can't find the container with id 73bc7a62abe7224d51d78dd6b1dc4f9ea51b3f58b6cf00072091e913b567ec28
Sep 16 18:49:14.961889 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:14.961793    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-tv99q" event={"ID":"7a4aa47d-be1a-4b2c-bd88-3edc40ef5d6a","Type":"ContainerStarted","Data":"73bc7a62abe7224d51d78dd6b1dc4f9ea51b3f58b6cf00072091e913b567ec28"}
Sep 16 18:49:15.087322 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:15.087291    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:16.475412 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:16.475383    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:16.748650 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:16.748610    2584 patch_prober.go:28] interesting pod/image-registry-76c66cc6d5-zsjkq container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Sep 16 18:49:16.748782 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:16.748663    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-76c66cc6d5-zsjkq" podUID="93861b72-565d-497a-a7d5-098fc010f83c" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 16 18:49:16.970795 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:16.970760    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-tv99q" event={"ID":"7a4aa47d-be1a-4b2c-bd88-3edc40ef5d6a","Type":"ContainerStarted","Data":"f9300ec6a4257b998424a4ab4fca768aca326197acf5ac7ffcd47815c15fda1a"}
Sep 16 18:49:16.970979 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:16.970811    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-tv99q" event={"ID":"7a4aa47d-be1a-4b2c-bd88-3edc40ef5d6a","Type":"ContainerStarted","Data":"e13e4d3e8426a75a1168eab00ce77991e828f15ed0d0197468a3a38e236f74e2"}
Sep 16 18:49:16.970979 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:16.970878    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-tv99q"
Sep 16 18:49:16.988528 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:16.988442    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-tv99q" podStartSLOduration=250.944870679 podStartE2EDuration="4m12.988426838s" totalImagesPullingTime="2.043556159s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:45:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:49:16.988022999 +0000 UTC m=+287.317525415" watchObservedRunningTime="2026-09-16 18:49:16.988426838 +0000 UTC m=+287.317929251"
Sep 16 18:49:18.823937 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:18.823885    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-fc6456cd6-gk5g8"]
Sep 16 18:49:18.825877 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:18.825857    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-fc6456cd6-gk5g8"
Sep 16 18:49:18.828558 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:18.828529    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 16 18:49:18.829762 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:18.829737    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 16 18:49:18.829862 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:18.829764    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 16 18:49:18.829862 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:18.829797    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 16 18:49:18.829989 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:18.829746    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 16 18:49:18.829989 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:18.829886    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 16 18:49:18.829989 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:18.829898    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-wzshj\""
Sep 16 18:49:18.830102 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:18.830014    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 16 18:49:18.835322 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:18.835299    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-fc6456cd6-gk5g8"]
Sep 16 18:49:18.871298 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:18.871265    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1a01026a-ace9-487a-9e36-39376a917186-service-ca\") pod \"console-fc6456cd6-gk5g8\" (UID: \"1a01026a-ace9-487a-9e36-39376a917186\") " pod="openshift-console/console-fc6456cd6-gk5g8"
Sep 16 18:49:18.871298 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:18.871301    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1a01026a-ace9-487a-9e36-39376a917186-console-serving-cert\") pod \"console-fc6456cd6-gk5g8\" (UID: \"1a01026a-ace9-487a-9e36-39376a917186\") " pod="openshift-console/console-fc6456cd6-gk5g8"
Sep 16 18:49:18.871468 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:18.871327    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1a01026a-ace9-487a-9e36-39376a917186-oauth-serving-cert\") pod \"console-fc6456cd6-gk5g8\" (UID: \"1a01026a-ace9-487a-9e36-39376a917186\") " pod="openshift-console/console-fc6456cd6-gk5g8"
Sep 16 18:49:18.871468 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:18.871393    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1a01026a-ace9-487a-9e36-39376a917186-console-config\") pod \"console-fc6456cd6-gk5g8\" (UID: \"1a01026a-ace9-487a-9e36-39376a917186\") " pod="openshift-console/console-fc6456cd6-gk5g8"
Sep 16 18:49:18.871468 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:18.871428    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1a01026a-ace9-487a-9e36-39376a917186-console-oauth-config\") pod \"console-fc6456cd6-gk5g8\" (UID: \"1a01026a-ace9-487a-9e36-39376a917186\") " pod="openshift-console/console-fc6456cd6-gk5g8"
Sep 16 18:49:18.871468 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:18.871448    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2fpml\" (UniqueName: \"kubernetes.io/projected/1a01026a-ace9-487a-9e36-39376a917186-kube-api-access-2fpml\") pod \"console-fc6456cd6-gk5g8\" (UID: \"1a01026a-ace9-487a-9e36-39376a917186\") " pod="openshift-console/console-fc6456cd6-gk5g8"
Sep 16 18:49:18.972384 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:18.972347    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1a01026a-ace9-487a-9e36-39376a917186-console-oauth-config\") pod \"console-fc6456cd6-gk5g8\" (UID: \"1a01026a-ace9-487a-9e36-39376a917186\") " pod="openshift-console/console-fc6456cd6-gk5g8"
Sep 16 18:49:18.972384 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:18.972386    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2fpml\" (UniqueName: \"kubernetes.io/projected/1a01026a-ace9-487a-9e36-39376a917186-kube-api-access-2fpml\") pod \"console-fc6456cd6-gk5g8\" (UID: \"1a01026a-ace9-487a-9e36-39376a917186\") " pod="openshift-console/console-fc6456cd6-gk5g8"
Sep 16 18:49:18.972600 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:18.972442    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1a01026a-ace9-487a-9e36-39376a917186-service-ca\") pod \"console-fc6456cd6-gk5g8\" (UID: \"1a01026a-ace9-487a-9e36-39376a917186\") " pod="openshift-console/console-fc6456cd6-gk5g8"
Sep 16 18:49:18.972600 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:18.972473    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1a01026a-ace9-487a-9e36-39376a917186-console-serving-cert\") pod \"console-fc6456cd6-gk5g8\" (UID: \"1a01026a-ace9-487a-9e36-39376a917186\") " pod="openshift-console/console-fc6456cd6-gk5g8"
Sep 16 18:49:18.972600 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:18.972509    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1a01026a-ace9-487a-9e36-39376a917186-oauth-serving-cert\") pod \"console-fc6456cd6-gk5g8\" (UID: \"1a01026a-ace9-487a-9e36-39376a917186\") " pod="openshift-console/console-fc6456cd6-gk5g8"
Sep 16 18:49:18.972600 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:18.972547    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1a01026a-ace9-487a-9e36-39376a917186-console-config\") pod \"console-fc6456cd6-gk5g8\" (UID: \"1a01026a-ace9-487a-9e36-39376a917186\") " pod="openshift-console/console-fc6456cd6-gk5g8"
Sep 16 18:49:18.973268 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:18.973244    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1a01026a-ace9-487a-9e36-39376a917186-console-config\") pod \"console-fc6456cd6-gk5g8\" (UID: \"1a01026a-ace9-487a-9e36-39376a917186\") " pod="openshift-console/console-fc6456cd6-gk5g8"
Sep 16 18:49:18.973268 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:18.973258    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1a01026a-ace9-487a-9e36-39376a917186-oauth-serving-cert\") pod \"console-fc6456cd6-gk5g8\" (UID: \"1a01026a-ace9-487a-9e36-39376a917186\") " pod="openshift-console/console-fc6456cd6-gk5g8"
Sep 16 18:49:18.973268 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:18.973265    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1a01026a-ace9-487a-9e36-39376a917186-service-ca\") pod \"console-fc6456cd6-gk5g8\" (UID: \"1a01026a-ace9-487a-9e36-39376a917186\") " pod="openshift-console/console-fc6456cd6-gk5g8"
Sep 16 18:49:18.974950 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:18.974928    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1a01026a-ace9-487a-9e36-39376a917186-console-serving-cert\") pod \"console-fc6456cd6-gk5g8\" (UID: \"1a01026a-ace9-487a-9e36-39376a917186\") " pod="openshift-console/console-fc6456cd6-gk5g8"
Sep 16 18:49:18.975311 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:18.975295    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1a01026a-ace9-487a-9e36-39376a917186-console-oauth-config\") pod \"console-fc6456cd6-gk5g8\" (UID: \"1a01026a-ace9-487a-9e36-39376a917186\") " pod="openshift-console/console-fc6456cd6-gk5g8"
Sep 16 18:49:18.981291 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:18.981272    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2fpml\" (UniqueName: \"kubernetes.io/projected/1a01026a-ace9-487a-9e36-39376a917186-kube-api-access-2fpml\") pod \"console-fc6456cd6-gk5g8\" (UID: \"1a01026a-ace9-487a-9e36-39376a917186\") " pod="openshift-console/console-fc6456cd6-gk5g8"
Sep 16 18:49:19.134849 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:19.134750    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-fc6456cd6-gk5g8"
Sep 16 18:49:19.402932 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:19.402907    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-fc6456cd6-gk5g8"]
Sep 16 18:49:19.405306 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:49:19.405281    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a01026a_ace9_487a_9e36_39376a917186.slice/crio-f5cb8bb5c92f8a899fc847c33d8aad67d519a4de74d5948332b236e7ca19b610 WatchSource:0}: Error finding container f5cb8bb5c92f8a899fc847c33d8aad67d519a4de74d5948332b236e7ca19b610: Status 404 returned error can't find the container with id f5cb8bb5c92f8a899fc847c33d8aad67d519a4de74d5948332b236e7ca19b610
Sep 16 18:49:19.984148 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:19.984110    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-fc6456cd6-gk5g8" event={"ID":"1a01026a-ace9-487a-9e36-39376a917186","Type":"ContainerStarted","Data":"f5cb8bb5c92f8a899fc847c33d8aad67d519a4de74d5948332b236e7ca19b610"}
Sep 16 18:49:21.017901 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:21.017869    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6d964bb7dc-lznmb"]
Sep 16 18:49:21.019580 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:21.019558    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d964bb7dc-lznmb"
Sep 16 18:49:21.027694 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:21.027665    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:21.028647 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:21.028623    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 16 18:49:21.031262 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:21.031238    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6d964bb7dc-lznmb"]
Sep 16 18:49:21.089108 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:21.089064    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7c9aa940-b15d-44a7-9235-4504ccfd0615-console-serving-cert\") pod \"console-6d964bb7dc-lznmb\" (UID: \"7c9aa940-b15d-44a7-9235-4504ccfd0615\") " pod="openshift-console/console-6d964bb7dc-lznmb"
Sep 16 18:49:21.089108 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:21.089113    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mbwq2\" (UniqueName: \"kubernetes.io/projected/7c9aa940-b15d-44a7-9235-4504ccfd0615-kube-api-access-mbwq2\") pod \"console-6d964bb7dc-lznmb\" (UID: \"7c9aa940-b15d-44a7-9235-4504ccfd0615\") " pod="openshift-console/console-6d964bb7dc-lznmb"
Sep 16 18:49:21.089339 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:21.089208    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7c9aa940-b15d-44a7-9235-4504ccfd0615-console-oauth-config\") pod \"console-6d964bb7dc-lznmb\" (UID: \"7c9aa940-b15d-44a7-9235-4504ccfd0615\") " pod="openshift-console/console-6d964bb7dc-lznmb"
Sep 16 18:49:21.089339 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:21.089238    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7c9aa940-b15d-44a7-9235-4504ccfd0615-trusted-ca-bundle\") pod \"console-6d964bb7dc-lznmb\" (UID: \"7c9aa940-b15d-44a7-9235-4504ccfd0615\") " pod="openshift-console/console-6d964bb7dc-lznmb"
Sep 16 18:49:21.089339 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:21.089286    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7c9aa940-b15d-44a7-9235-4504ccfd0615-console-config\") pod \"console-6d964bb7dc-lznmb\" (UID: \"7c9aa940-b15d-44a7-9235-4504ccfd0615\") " pod="openshift-console/console-6d964bb7dc-lznmb"
Sep 16 18:49:21.089479 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:21.089338    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7c9aa940-b15d-44a7-9235-4504ccfd0615-oauth-serving-cert\") pod \"console-6d964bb7dc-lznmb\" (UID: \"7c9aa940-b15d-44a7-9235-4504ccfd0615\") " pod="openshift-console/console-6d964bb7dc-lznmb"
Sep 16 18:49:21.089479 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:21.089403    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7c9aa940-b15d-44a7-9235-4504ccfd0615-service-ca\") pod \"console-6d964bb7dc-lznmb\" (UID: \"7c9aa940-b15d-44a7-9235-4504ccfd0615\") " pod="openshift-console/console-6d964bb7dc-lznmb"
Sep 16 18:49:21.125147 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:21.125109    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:21.190204 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:21.190170    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mbwq2\" (UniqueName: \"kubernetes.io/projected/7c9aa940-b15d-44a7-9235-4504ccfd0615-kube-api-access-mbwq2\") pod \"console-6d964bb7dc-lznmb\" (UID: \"7c9aa940-b15d-44a7-9235-4504ccfd0615\") " pod="openshift-console/console-6d964bb7dc-lznmb"
Sep 16 18:49:21.190400 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:21.190218    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7c9aa940-b15d-44a7-9235-4504ccfd0615-console-oauth-config\") pod \"console-6d964bb7dc-lznmb\" (UID: \"7c9aa940-b15d-44a7-9235-4504ccfd0615\") " pod="openshift-console/console-6d964bb7dc-lznmb"
Sep 16 18:49:21.190400 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:21.190236    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7c9aa940-b15d-44a7-9235-4504ccfd0615-trusted-ca-bundle\") pod \"console-6d964bb7dc-lznmb\" (UID: \"7c9aa940-b15d-44a7-9235-4504ccfd0615\") " pod="openshift-console/console-6d964bb7dc-lznmb"
Sep 16 18:49:21.190400 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:21.190274    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7c9aa940-b15d-44a7-9235-4504ccfd0615-console-config\") pod \"console-6d964bb7dc-lznmb\" (UID: \"7c9aa940-b15d-44a7-9235-4504ccfd0615\") " pod="openshift-console/console-6d964bb7dc-lznmb"
Sep 16 18:49:21.190400 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:21.190325    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7c9aa940-b15d-44a7-9235-4504ccfd0615-oauth-serving-cert\") pod \"console-6d964bb7dc-lznmb\" (UID: \"7c9aa940-b15d-44a7-9235-4504ccfd0615\") " pod="openshift-console/console-6d964bb7dc-lznmb"
Sep 16 18:49:21.190400 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:21.190369    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7c9aa940-b15d-44a7-9235-4504ccfd0615-service-ca\") pod \"console-6d964bb7dc-lznmb\" (UID: \"7c9aa940-b15d-44a7-9235-4504ccfd0615\") " pod="openshift-console/console-6d964bb7dc-lznmb"
Sep 16 18:49:21.190657 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:21.190411    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7c9aa940-b15d-44a7-9235-4504ccfd0615-console-serving-cert\") pod \"console-6d964bb7dc-lznmb\" (UID: \"7c9aa940-b15d-44a7-9235-4504ccfd0615\") " pod="openshift-console/console-6d964bb7dc-lznmb"
Sep 16 18:49:21.191319 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:21.191288    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7c9aa940-b15d-44a7-9235-4504ccfd0615-oauth-serving-cert\") pod \"console-6d964bb7dc-lznmb\" (UID: \"7c9aa940-b15d-44a7-9235-4504ccfd0615\") " pod="openshift-console/console-6d964bb7dc-lznmb"
Sep 16 18:49:21.191319 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:21.191307    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7c9aa940-b15d-44a7-9235-4504ccfd0615-console-config\") pod \"console-6d964bb7dc-lznmb\" (UID: \"7c9aa940-b15d-44a7-9235-4504ccfd0615\") " pod="openshift-console/console-6d964bb7dc-lznmb"
Sep 16 18:49:21.191525 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:21.191385    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7c9aa940-b15d-44a7-9235-4504ccfd0615-service-ca\") pod \"console-6d964bb7dc-lznmb\" (UID: \"7c9aa940-b15d-44a7-9235-4504ccfd0615\") " pod="openshift-console/console-6d964bb7dc-lznmb"
Sep 16 18:49:21.191588 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:21.191551    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7c9aa940-b15d-44a7-9235-4504ccfd0615-trusted-ca-bundle\") pod \"console-6d964bb7dc-lznmb\" (UID: \"7c9aa940-b15d-44a7-9235-4504ccfd0615\") " pod="openshift-console/console-6d964bb7dc-lznmb"
Sep 16 18:49:21.193060 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:21.193037    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7c9aa940-b15d-44a7-9235-4504ccfd0615-console-serving-cert\") pod \"console-6d964bb7dc-lznmb\" (UID: \"7c9aa940-b15d-44a7-9235-4504ccfd0615\") " pod="openshift-console/console-6d964bb7dc-lznmb"
Sep 16 18:49:21.193143 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:21.193085    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7c9aa940-b15d-44a7-9235-4504ccfd0615-console-oauth-config\") pod \"console-6d964bb7dc-lznmb\" (UID: \"7c9aa940-b15d-44a7-9235-4504ccfd0615\") " pod="openshift-console/console-6d964bb7dc-lznmb"
Sep 16 18:49:21.198845 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:21.198806    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mbwq2\" (UniqueName: \"kubernetes.io/projected/7c9aa940-b15d-44a7-9235-4504ccfd0615-kube-api-access-mbwq2\") pod \"console-6d964bb7dc-lznmb\" (UID: \"7c9aa940-b15d-44a7-9235-4504ccfd0615\") " pod="openshift-console/console-6d964bb7dc-lznmb"
Sep 16 18:49:21.298319 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:21.298225    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:21.330853 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:21.330795    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d964bb7dc-lznmb"
Sep 16 18:49:21.362420 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:21.362385    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:22.317749 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:22.317695    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6d964bb7dc-lznmb"]
Sep 16 18:49:22.320165 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:49:22.320139    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c9aa940_b15d_44a7_9235_4504ccfd0615.slice/crio-8f0af2a169f0e432dd863e787cff0cb9269ad1062164af1738f431f1a9897942 WatchSource:0}: Error finding container 8f0af2a169f0e432dd863e787cff0cb9269ad1062164af1738f431f1a9897942: Status 404 returned error can't find the container with id 8f0af2a169f0e432dd863e787cff0cb9269ad1062164af1738f431f1a9897942
Sep 16 18:49:22.999306 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:22.999270    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-fc6456cd6-gk5g8" event={"ID":"1a01026a-ace9-487a-9e36-39376a917186","Type":"ContainerStarted","Data":"1d2bdb923dc7bc8ed38703cc3491da4535bb72b8477b2b11eed3cecd75e51485"}
Sep 16 18:49:23.000743 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:23.000715    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d964bb7dc-lznmb" event={"ID":"7c9aa940-b15d-44a7-9235-4504ccfd0615","Type":"ContainerStarted","Data":"38174231e1b23ccadba494e5d25dccff0cb7200a73133e1c2923d4d3e062cdb8"}
Sep 16 18:49:23.000850 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:23.000760    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d964bb7dc-lznmb" event={"ID":"7c9aa940-b15d-44a7-9235-4504ccfd0615","Type":"ContainerStarted","Data":"8f0af2a169f0e432dd863e787cff0cb9269ad1062164af1738f431f1a9897942"}
Sep 16 18:49:23.024771 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:23.024725    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-fc6456cd6-gk5g8" podStartSLOduration=2.353933444 podStartE2EDuration="5.024710393s" totalImagesPullingTime="2.670776949s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:49:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:49:23.023687001 +0000 UTC m=+293.353189437" watchObservedRunningTime="2026-09-16 18:49:23.024710393 +0000 UTC m=+293.354212806"
Sep 16 18:49:23.070213 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:23.070164    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6d964bb7dc-lznmb" podStartSLOduration=3.070146836 podStartE2EDuration="3.070146836s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:49:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:49:23.069490428 +0000 UTC m=+293.398992858" watchObservedRunningTime="2026-09-16 18:49:23.070146836 +0000 UTC m=+293.399649249"
Sep 16 18:49:23.112817 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:49:23.112786    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-bp5mv" podUID="9b96048a-0005-410b-a710-909d21fbfde8"
Sep 16 18:49:24.412906 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:24.412849    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:24.475206 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:24.475174    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:26.747488 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:26.747456    2584 patch_prober.go:28] interesting pod/image-registry-76c66cc6d5-zsjkq container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Sep 16 18:49:26.747876 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:26.747518    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-76c66cc6d5-zsjkq" podUID="93861b72-565d-497a-a7d5-098fc010f83c" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 16 18:49:26.976114 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:26.976086    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-tv99q"
Sep 16 18:49:27.284350 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:27.284311    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:27.366260 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:27.366195    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:27.492573 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:27.492534    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:28.206309 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:28.206264    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:28.270737 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:28.270708    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:28.760087 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:28.760057    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:28.862433 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:28.862402    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:28.916513 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:28.916486    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:29.136031 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:29.135958    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-fc6456cd6-gk5g8"
Sep 16 18:49:29.136031 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:29.136023    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-fc6456cd6-gk5g8"
Sep 16 18:49:29.140670 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:29.140649    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-fc6456cd6-gk5g8"
Sep 16 18:49:29.226640 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:29.226609    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:29.293619 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:29.293588    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:29.393401 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:29.393326    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:29.451242 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:29.451214    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:30.034025 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:30.033993    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-fc6456cd6-gk5g8"
Sep 16 18:49:30.038679 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:30.038652    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-crjw7_ecf273b5-b465-4008-b035-f7ac0e740d58/console-operator/1.log"
Sep 16 18:49:30.039324 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:30.039300    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-crjw7_ecf273b5-b465-4008-b035-f7ac0e740d58/console-operator/1.log"
Sep 16 18:49:30.051375 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:30.051352    2584 kubelet.go:1648] "Image garbage collection succeeded"
Sep 16 18:49:30.096987 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:30.096856    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:31.331704 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:31.331656    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6d964bb7dc-lznmb"
Sep 16 18:49:31.332273 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:31.331722    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6d964bb7dc-lznmb"
Sep 16 18:49:31.336758 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:31.336735    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6d964bb7dc-lznmb"
Sep 16 18:49:31.762954 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:31.762901    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-76c66cc6d5-zsjkq" podUID="93861b72-565d-497a-a7d5-098fc010f83c" containerName="registry" containerID="cri-o://c467eab7f9d522f45d41e5e83cc80a4aafe653ba783a5997d7c3654775e42882" gracePeriod=30
Sep 16 18:49:31.826411 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:31.826380    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:31.998132 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:31.998109    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-76c66cc6d5-zsjkq"
Sep 16 18:49:32.036536 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:32.036456    2584 generic.go:356] "Generic (PLEG): container finished" podID="93861b72-565d-497a-a7d5-098fc010f83c" containerID="c467eab7f9d522f45d41e5e83cc80a4aafe653ba783a5997d7c3654775e42882" exitCode=0
Sep 16 18:49:32.036536 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:32.036516    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-76c66cc6d5-zsjkq"
Sep 16 18:49:32.036714 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:32.036538    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-76c66cc6d5-zsjkq" event={"ID":"93861b72-565d-497a-a7d5-098fc010f83c","Type":"ContainerDied","Data":"c467eab7f9d522f45d41e5e83cc80a4aafe653ba783a5997d7c3654775e42882"}
Sep 16 18:49:32.036714 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:32.036604    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-76c66cc6d5-zsjkq" event={"ID":"93861b72-565d-497a-a7d5-098fc010f83c","Type":"ContainerDied","Data":"206a3e568da735e0cc8e4dc6d4c9b27953c8ff29221b901553e1867fadb01fad"}
Sep 16 18:49:32.036714 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:32.036625    2584 scope.go:117] "RemoveContainer" containerID="c467eab7f9d522f45d41e5e83cc80a4aafe653ba783a5997d7c3654775e42882"
Sep 16 18:49:32.041066 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:32.041038    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6d964bb7dc-lznmb"
Sep 16 18:49:32.044677 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:32.044660    2584 scope.go:117] "RemoveContainer" containerID="c467eab7f9d522f45d41e5e83cc80a4aafe653ba783a5997d7c3654775e42882"
Sep 16 18:49:32.045036 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:49:32.045017    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c467eab7f9d522f45d41e5e83cc80a4aafe653ba783a5997d7c3654775e42882\": container with ID starting with c467eab7f9d522f45d41e5e83cc80a4aafe653ba783a5997d7c3654775e42882 not found: ID does not exist" containerID="c467eab7f9d522f45d41e5e83cc80a4aafe653ba783a5997d7c3654775e42882"
Sep 16 18:49:32.045095 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:32.045045    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c467eab7f9d522f45d41e5e83cc80a4aafe653ba783a5997d7c3654775e42882"} err="failed to get container status \"c467eab7f9d522f45d41e5e83cc80a4aafe653ba783a5997d7c3654775e42882\": rpc error: code = NotFound desc = could not find container \"c467eab7f9d522f45d41e5e83cc80a4aafe653ba783a5997d7c3654775e42882\": container with ID starting with c467eab7f9d522f45d41e5e83cc80a4aafe653ba783a5997d7c3654775e42882 not found: ID does not exist"
Sep 16 18:49:32.080846 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:32.080794    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/93861b72-565d-497a-a7d5-098fc010f83c-registry-tls\") pod \"93861b72-565d-497a-a7d5-098fc010f83c\" (UID: \"93861b72-565d-497a-a7d5-098fc010f83c\") "
Sep 16 18:49:32.081010 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:32.080885    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sghhl\" (UniqueName: \"kubernetes.io/projected/93861b72-565d-497a-a7d5-098fc010f83c-kube-api-access-sghhl\") pod \"93861b72-565d-497a-a7d5-098fc010f83c\" (UID: \"93861b72-565d-497a-a7d5-098fc010f83c\") "
Sep 16 18:49:32.081010 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:32.080918    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/93861b72-565d-497a-a7d5-098fc010f83c-registry-certificates\") pod \"93861b72-565d-497a-a7d5-098fc010f83c\" (UID: \"93861b72-565d-497a-a7d5-098fc010f83c\") "
Sep 16 18:49:32.081010 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:32.080959    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/93861b72-565d-497a-a7d5-098fc010f83c-bound-sa-token\") pod \"93861b72-565d-497a-a7d5-098fc010f83c\" (UID: \"93861b72-565d-497a-a7d5-098fc010f83c\") "
Sep 16 18:49:32.081010 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:32.080996    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/93861b72-565d-497a-a7d5-098fc010f83c-image-registry-private-configuration\") pod \"93861b72-565d-497a-a7d5-098fc010f83c\" (UID: \"93861b72-565d-497a-a7d5-098fc010f83c\") "
Sep 16 18:49:32.081225 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:32.081032    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/93861b72-565d-497a-a7d5-098fc010f83c-ca-trust-extracted\") pod \"93861b72-565d-497a-a7d5-098fc010f83c\" (UID: \"93861b72-565d-497a-a7d5-098fc010f83c\") "
Sep 16 18:49:32.081225 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:32.081099    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/93861b72-565d-497a-a7d5-098fc010f83c-trusted-ca\") pod \"93861b72-565d-497a-a7d5-098fc010f83c\" (UID: \"93861b72-565d-497a-a7d5-098fc010f83c\") "
Sep 16 18:49:32.081225 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:32.081125    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/93861b72-565d-497a-a7d5-098fc010f83c-installation-pull-secrets\") pod \"93861b72-565d-497a-a7d5-098fc010f83c\" (UID: \"93861b72-565d-497a-a7d5-098fc010f83c\") "
Sep 16 18:49:32.081392 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:32.081365    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/93861b72-565d-497a-a7d5-098fc010f83c-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "93861b72-565d-497a-a7d5-098fc010f83c" (UID: "93861b72-565d-497a-a7d5-098fc010f83c"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:49:32.082091 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:32.081605    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/93861b72-565d-497a-a7d5-098fc010f83c-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "93861b72-565d-497a-a7d5-098fc010f83c" (UID: "93861b72-565d-497a-a7d5-098fc010f83c"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:49:32.083754 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:32.083721    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/93861b72-565d-497a-a7d5-098fc010f83c-kube-api-access-sghhl" (OuterVolumeSpecName: "kube-api-access-sghhl") pod "93861b72-565d-497a-a7d5-098fc010f83c" (UID: "93861b72-565d-497a-a7d5-098fc010f83c"). InnerVolumeSpecName "kube-api-access-sghhl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:49:32.083882 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:32.083753    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/93861b72-565d-497a-a7d5-098fc010f83c-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "93861b72-565d-497a-a7d5-098fc010f83c" (UID: "93861b72-565d-497a-a7d5-098fc010f83c"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:49:32.084027 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:32.083999    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/93861b72-565d-497a-a7d5-098fc010f83c-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "93861b72-565d-497a-a7d5-098fc010f83c" (UID: "93861b72-565d-497a-a7d5-098fc010f83c"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:49:32.084183 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:32.084150    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/93861b72-565d-497a-a7d5-098fc010f83c-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "93861b72-565d-497a-a7d5-098fc010f83c" (UID: "93861b72-565d-497a-a7d5-098fc010f83c"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:49:32.084183 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:32.084152    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/93861b72-565d-497a-a7d5-098fc010f83c-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "93861b72-565d-497a-a7d5-098fc010f83c" (UID: "93861b72-565d-497a-a7d5-098fc010f83c"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:49:32.088150 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:32.088015    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-fc6456cd6-gk5g8"]
Sep 16 18:49:32.094150 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:32.094114    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93861b72-565d-497a-a7d5-098fc010f83c-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "93861b72-565d-497a-a7d5-098fc010f83c" (UID: "93861b72-565d-497a-a7d5-098fc010f83c"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:49:32.182461 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:32.182426    2584 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/93861b72-565d-497a-a7d5-098fc010f83c-trusted-ca\") on node \"ip-10-0-139-182.ec2.internal\" DevicePath \"\""
Sep 16 18:49:32.182461 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:32.182454    2584 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/93861b72-565d-497a-a7d5-098fc010f83c-installation-pull-secrets\") on node \"ip-10-0-139-182.ec2.internal\" DevicePath \"\""
Sep 16 18:49:32.182461 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:32.182465    2584 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/93861b72-565d-497a-a7d5-098fc010f83c-registry-tls\") on node \"ip-10-0-139-182.ec2.internal\" DevicePath \"\""
Sep 16 18:49:32.182705 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:32.182474    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sghhl\" (UniqueName: \"kubernetes.io/projected/93861b72-565d-497a-a7d5-098fc010f83c-kube-api-access-sghhl\") on node \"ip-10-0-139-182.ec2.internal\" DevicePath \"\""
Sep 16 18:49:32.182705 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:32.182482    2584 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/93861b72-565d-497a-a7d5-098fc010f83c-registry-certificates\") on node \"ip-10-0-139-182.ec2.internal\" DevicePath \"\""
Sep 16 18:49:32.182705 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:32.182494    2584 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/93861b72-565d-497a-a7d5-098fc010f83c-bound-sa-token\") on node \"ip-10-0-139-182.ec2.internal\" DevicePath \"\""
Sep 16 18:49:32.182705 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:32.182502    2584 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/93861b72-565d-497a-a7d5-098fc010f83c-image-registry-private-configuration\") on node \"ip-10-0-139-182.ec2.internal\" DevicePath \"\""
Sep 16 18:49:32.182705 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:32.182511    2584 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/93861b72-565d-497a-a7d5-098fc010f83c-ca-trust-extracted\") on node \"ip-10-0-139-182.ec2.internal\" DevicePath \"\""
Sep 16 18:49:32.352201 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:32.352131    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-76c66cc6d5-zsjkq"]
Sep 16 18:49:32.355683 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:32.355658    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-76c66cc6d5-zsjkq"]
Sep 16 18:49:32.427235 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:32.427197    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:32.644793 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:32.644700    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:32.681686 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:32.681656    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-59fbdd77d-9h5nf"]
Sep 16 18:49:32.681921 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:32.681909    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="93861b72-565d-497a-a7d5-098fc010f83c" containerName="registry"
Sep 16 18:49:32.681962 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:32.681923    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="93861b72-565d-497a-a7d5-098fc010f83c" containerName="registry"
Sep 16 18:49:32.682011 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:32.681970    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="93861b72-565d-497a-a7d5-098fc010f83c" containerName="registry"
Sep 16 18:49:32.686884 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:32.686854    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-9h5nf"
Sep 16 18:49:32.691150 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:32.691127    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 16 18:49:32.691150 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:32.691145    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Sep 16 18:49:32.691335 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:32.691169    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 16 18:49:32.691335 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:32.691264    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 16 18:49:32.691335 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:32.691268    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-nr5xr\""
Sep 16 18:49:32.691504 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:32.691487    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Sep 16 18:49:32.691573 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:32.691551    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Sep 16 18:49:32.701052 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:32.701028    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-59fbdd77d-9h5nf"]
Sep 16 18:49:32.706961 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:32.706937    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-qbpp2"]
Sep 16 18:49:32.710143 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:32.710128    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-qbpp2"
Sep 16 18:49:32.712986 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:32.712969    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-rc6mq\""
Sep 16 18:49:32.713170 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:32.713157    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 16 18:49:32.713707 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:32.713692    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 16 18:49:32.713798 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:32.713726    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 16 18:49:32.738159 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:32.738131    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:32.785716 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:32.785683    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/3097f2a3-e38f-4abe-83dc-bcdc64d46c45-root\") pod \"node-exporter-qbpp2\" (UID: \"3097f2a3-e38f-4abe-83dc-bcdc64d46c45\") " pod="openshift-monitoring/node-exporter-qbpp2"
Sep 16 18:49:32.785716 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:32.785715    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/07294397-1424-41ee-b652-aec35f466d18-kube-state-metrics-tls\") pod \"kube-state-metrics-59fbdd77d-9h5nf\" (UID: \"07294397-1424-41ee-b652-aec35f466d18\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-9h5nf"
Sep 16 18:49:32.785917 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:32.785732    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3097f2a3-e38f-4abe-83dc-bcdc64d46c45-node-exporter-tls\") pod \"node-exporter-qbpp2\" (UID: \"3097f2a3-e38f-4abe-83dc-bcdc64d46c45\") " pod="openshift-monitoring/node-exporter-qbpp2"
Sep 16 18:49:32.785917 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:32.785785    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/07294397-1424-41ee-b652-aec35f466d18-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-59fbdd77d-9h5nf\" (UID: \"07294397-1424-41ee-b652-aec35f466d18\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-9h5nf"
Sep 16 18:49:32.785917 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:32.785843    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mwv8r\" (UniqueName: \"kubernetes.io/projected/07294397-1424-41ee-b652-aec35f466d18-kube-api-access-mwv8r\") pod \"kube-state-metrics-59fbdd77d-9h5nf\" (UID: \"07294397-1424-41ee-b652-aec35f466d18\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-9h5nf"
Sep 16 18:49:32.785917 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:32.785885    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/3097f2a3-e38f-4abe-83dc-bcdc64d46c45-node-exporter-accelerators-collector-config\") pod \"node-exporter-qbpp2\" (UID: \"3097f2a3-e38f-4abe-83dc-bcdc64d46c45\") " pod="openshift-monitoring/node-exporter-qbpp2"
Sep 16 18:49:32.785917 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:32.785906    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3097f2a3-e38f-4abe-83dc-bcdc64d46c45-sys\") pod \"node-exporter-qbpp2\" (UID: \"3097f2a3-e38f-4abe-83dc-bcdc64d46c45\") " pod="openshift-monitoring/node-exporter-qbpp2"
Sep 16 18:49:32.786079 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:32.785957    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/07294397-1424-41ee-b652-aec35f466d18-volume-directive-shadow\") pod \"kube-state-metrics-59fbdd77d-9h5nf\" (UID: \"07294397-1424-41ee-b652-aec35f466d18\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-9h5nf"
Sep 16 18:49:32.786079 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:32.786001    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3097f2a3-e38f-4abe-83dc-bcdc64d46c45-metrics-client-ca\") pod \"node-exporter-qbpp2\" (UID: \"3097f2a3-e38f-4abe-83dc-bcdc64d46c45\") " pod="openshift-monitoring/node-exporter-qbpp2"
Sep 16 18:49:32.786079 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:32.786041    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/07294397-1424-41ee-b652-aec35f466d18-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-59fbdd77d-9h5nf\" (UID: \"07294397-1424-41ee-b652-aec35f466d18\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-9h5nf"
Sep 16 18:49:32.786079 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:32.786060    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t8rnf\" (UniqueName: \"kubernetes.io/projected/3097f2a3-e38f-4abe-83dc-bcdc64d46c45-kube-api-access-t8rnf\") pod \"node-exporter-qbpp2\" (UID: \"3097f2a3-e38f-4abe-83dc-bcdc64d46c45\") " pod="openshift-monitoring/node-exporter-qbpp2"
Sep 16 18:49:32.786079 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:32.786077    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/07294397-1424-41ee-b652-aec35f466d18-metrics-client-ca\") pod \"kube-state-metrics-59fbdd77d-9h5nf\" (UID: \"07294397-1424-41ee-b652-aec35f466d18\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-9h5nf"
Sep 16 18:49:32.786225 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:32.786095    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3097f2a3-e38f-4abe-83dc-bcdc64d46c45-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-qbpp2\" (UID: \"3097f2a3-e38f-4abe-83dc-bcdc64d46c45\") " pod="openshift-monitoring/node-exporter-qbpp2"
Sep 16 18:49:32.786225 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:32.786160    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/3097f2a3-e38f-4abe-83dc-bcdc64d46c45-node-exporter-wtmp\") pod \"node-exporter-qbpp2\" (UID: \"3097f2a3-e38f-4abe-83dc-bcdc64d46c45\") " pod="openshift-monitoring/node-exporter-qbpp2"
Sep 16 18:49:32.786225 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:32.786176    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/3097f2a3-e38f-4abe-83dc-bcdc64d46c45-node-exporter-textfile\") pod \"node-exporter-qbpp2\" (UID: \"3097f2a3-e38f-4abe-83dc-bcdc64d46c45\") " pod="openshift-monitoring/node-exporter-qbpp2"
Sep 16 18:49:32.871078 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:32.871038    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:32.887178 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:32.887149    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/07294397-1424-41ee-b652-aec35f466d18-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-59fbdd77d-9h5nf\" (UID: \"07294397-1424-41ee-b652-aec35f466d18\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-9h5nf"
Sep 16 18:49:32.887289 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:32.887183    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t8rnf\" (UniqueName: \"kubernetes.io/projected/3097f2a3-e38f-4abe-83dc-bcdc64d46c45-kube-api-access-t8rnf\") pod \"node-exporter-qbpp2\" (UID: \"3097f2a3-e38f-4abe-83dc-bcdc64d46c45\") " pod="openshift-monitoring/node-exporter-qbpp2"
Sep 16 18:49:32.887289 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:32.887204    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/07294397-1424-41ee-b652-aec35f466d18-metrics-client-ca\") pod \"kube-state-metrics-59fbdd77d-9h5nf\" (UID: \"07294397-1424-41ee-b652-aec35f466d18\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-9h5nf"
Sep 16 18:49:32.887289 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:32.887230    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3097f2a3-e38f-4abe-83dc-bcdc64d46c45-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-qbpp2\" (UID: \"3097f2a3-e38f-4abe-83dc-bcdc64d46c45\") " pod="openshift-monitoring/node-exporter-qbpp2"
Sep 16 18:49:32.887289 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:32.887275    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/3097f2a3-e38f-4abe-83dc-bcdc64d46c45-node-exporter-wtmp\") pod \"node-exporter-qbpp2\" (UID: \"3097f2a3-e38f-4abe-83dc-bcdc64d46c45\") " pod="openshift-monitoring/node-exporter-qbpp2"
Sep 16 18:49:32.887466 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:32.887299    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/3097f2a3-e38f-4abe-83dc-bcdc64d46c45-node-exporter-textfile\") pod \"node-exporter-qbpp2\" (UID: \"3097f2a3-e38f-4abe-83dc-bcdc64d46c45\") " pod="openshift-monitoring/node-exporter-qbpp2"
Sep 16 18:49:32.887466 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:32.887330    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/3097f2a3-e38f-4abe-83dc-bcdc64d46c45-root\") pod \"node-exporter-qbpp2\" (UID: \"3097f2a3-e38f-4abe-83dc-bcdc64d46c45\") " pod="openshift-monitoring/node-exporter-qbpp2"
Sep 16 18:49:32.887466 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:32.887353    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/07294397-1424-41ee-b652-aec35f466d18-kube-state-metrics-tls\") pod \"kube-state-metrics-59fbdd77d-9h5nf\" (UID: \"07294397-1424-41ee-b652-aec35f466d18\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-9h5nf"
Sep 16 18:49:32.887466 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:32.887377    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3097f2a3-e38f-4abe-83dc-bcdc64d46c45-node-exporter-tls\") pod \"node-exporter-qbpp2\" (UID: \"3097f2a3-e38f-4abe-83dc-bcdc64d46c45\") " pod="openshift-monitoring/node-exporter-qbpp2"
Sep 16 18:49:32.887466 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:32.887453    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/07294397-1424-41ee-b652-aec35f466d18-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-59fbdd77d-9h5nf\" (UID: \"07294397-1424-41ee-b652-aec35f466d18\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-9h5nf"
Sep 16 18:49:32.887698 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:32.887478    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mwv8r\" (UniqueName: \"kubernetes.io/projected/07294397-1424-41ee-b652-aec35f466d18-kube-api-access-mwv8r\") pod \"kube-state-metrics-59fbdd77d-9h5nf\" (UID: \"07294397-1424-41ee-b652-aec35f466d18\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-9h5nf"
Sep 16 18:49:32.887698 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:32.887511    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/3097f2a3-e38f-4abe-83dc-bcdc64d46c45-node-exporter-wtmp\") pod \"node-exporter-qbpp2\" (UID: \"3097f2a3-e38f-4abe-83dc-bcdc64d46c45\") " pod="openshift-monitoring/node-exporter-qbpp2"
Sep 16 18:49:32.887698 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:32.887534    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/3097f2a3-e38f-4abe-83dc-bcdc64d46c45-node-exporter-accelerators-collector-config\") pod \"node-exporter-qbpp2\" (UID: \"3097f2a3-e38f-4abe-83dc-bcdc64d46c45\") " pod="openshift-monitoring/node-exporter-qbpp2"
Sep 16 18:49:32.887698 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:32.887567    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3097f2a3-e38f-4abe-83dc-bcdc64d46c45-sys\") pod \"node-exporter-qbpp2\" (UID: \"3097f2a3-e38f-4abe-83dc-bcdc64d46c45\") " pod="openshift-monitoring/node-exporter-qbpp2"
Sep 16 18:49:32.887698 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:32.887598    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/07294397-1424-41ee-b652-aec35f466d18-volume-directive-shadow\") pod \"kube-state-metrics-59fbdd77d-9h5nf\" (UID: \"07294397-1424-41ee-b652-aec35f466d18\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-9h5nf"
Sep 16 18:49:32.887698 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:32.887633    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3097f2a3-e38f-4abe-83dc-bcdc64d46c45-metrics-client-ca\") pod \"node-exporter-qbpp2\" (UID: \"3097f2a3-e38f-4abe-83dc-bcdc64d46c45\") " pod="openshift-monitoring/node-exporter-qbpp2"
Sep 16 18:49:32.887698 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:32.887680    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/3097f2a3-e38f-4abe-83dc-bcdc64d46c45-node-exporter-textfile\") pod \"node-exporter-qbpp2\" (UID: \"3097f2a3-e38f-4abe-83dc-bcdc64d46c45\") " pod="openshift-monitoring/node-exporter-qbpp2"
Sep 16 18:49:32.888073 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:32.887963    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/3097f2a3-e38f-4abe-83dc-bcdc64d46c45-root\") pod \"node-exporter-qbpp2\" (UID: \"3097f2a3-e38f-4abe-83dc-bcdc64d46c45\") " pod="openshift-monitoring/node-exporter-qbpp2"
Sep 16 18:49:32.888073 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:32.888028    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3097f2a3-e38f-4abe-83dc-bcdc64d46c45-sys\") pod \"node-exporter-qbpp2\" (UID: \"3097f2a3-e38f-4abe-83dc-bcdc64d46c45\") " pod="openshift-monitoring/node-exporter-qbpp2"
Sep 16 18:49:32.888073 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:32.888043    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/07294397-1424-41ee-b652-aec35f466d18-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-59fbdd77d-9h5nf\" (UID: \"07294397-1424-41ee-b652-aec35f466d18\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-9h5nf"
Sep 16 18:49:32.888218 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:32.888071    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/07294397-1424-41ee-b652-aec35f466d18-metrics-client-ca\") pod \"kube-state-metrics-59fbdd77d-9h5nf\" (UID: \"07294397-1424-41ee-b652-aec35f466d18\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-9h5nf"
Sep 16 18:49:32.888277 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:32.888253    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3097f2a3-e38f-4abe-83dc-bcdc64d46c45-metrics-client-ca\") pod \"node-exporter-qbpp2\" (UID: \"3097f2a3-e38f-4abe-83dc-bcdc64d46c45\") " pod="openshift-monitoring/node-exporter-qbpp2"
Sep 16 18:49:32.888336 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:32.888254    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/3097f2a3-e38f-4abe-83dc-bcdc64d46c45-node-exporter-accelerators-collector-config\") pod \"node-exporter-qbpp2\" (UID: \"3097f2a3-e38f-4abe-83dc-bcdc64d46c45\") " pod="openshift-monitoring/node-exporter-qbpp2"
Sep 16 18:49:32.888389 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:32.888328    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/07294397-1424-41ee-b652-aec35f466d18-volume-directive-shadow\") pod \"kube-state-metrics-59fbdd77d-9h5nf\" (UID: \"07294397-1424-41ee-b652-aec35f466d18\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-9h5nf"
Sep 16 18:49:32.889782 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:32.889762    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3097f2a3-e38f-4abe-83dc-bcdc64d46c45-node-exporter-tls\") pod \"node-exporter-qbpp2\" (UID: \"3097f2a3-e38f-4abe-83dc-bcdc64d46c45\") " pod="openshift-monitoring/node-exporter-qbpp2"
Sep 16 18:49:32.890147 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:32.890115    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3097f2a3-e38f-4abe-83dc-bcdc64d46c45-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-qbpp2\" (UID: \"3097f2a3-e38f-4abe-83dc-bcdc64d46c45\") " pod="openshift-monitoring/node-exporter-qbpp2"
Sep 16 18:49:32.890277 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:32.890262    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/07294397-1424-41ee-b652-aec35f466d18-kube-state-metrics-tls\") pod \"kube-state-metrics-59fbdd77d-9h5nf\" (UID: \"07294397-1424-41ee-b652-aec35f466d18\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-9h5nf"
Sep 16 18:49:32.890472 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:32.890451    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/07294397-1424-41ee-b652-aec35f466d18-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-59fbdd77d-9h5nf\" (UID: \"07294397-1424-41ee-b652-aec35f466d18\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-9h5nf"
Sep 16 18:49:32.897978 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:32.897929    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t8rnf\" (UniqueName: \"kubernetes.io/projected/3097f2a3-e38f-4abe-83dc-bcdc64d46c45-kube-api-access-t8rnf\") pod \"node-exporter-qbpp2\" (UID: \"3097f2a3-e38f-4abe-83dc-bcdc64d46c45\") " pod="openshift-monitoring/node-exporter-qbpp2"
Sep 16 18:49:32.898517 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:32.898495    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mwv8r\" (UniqueName: \"kubernetes.io/projected/07294397-1424-41ee-b652-aec35f466d18-kube-api-access-mwv8r\") pod \"kube-state-metrics-59fbdd77d-9h5nf\" (UID: \"07294397-1424-41ee-b652-aec35f466d18\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-9h5nf"
Sep 16 18:49:32.971151 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:32.971101    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:32.995875 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:32.995845    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-9h5nf"
Sep 16 18:49:33.019530 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:33.019499    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-qbpp2"
Sep 16 18:49:33.030699 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:49:33.030670    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3097f2a3_e38f_4abe_83dc_bcdc64d46c45.slice/crio-d60b534d9a5d2af428adb8ef41c54e80fa671d1d69d9eeabc47ca85d78e635c7 WatchSource:0}: Error finding container d60b534d9a5d2af428adb8ef41c54e80fa671d1d69d9eeabc47ca85d78e635c7: Status 404 returned error can't find the container with id d60b534d9a5d2af428adb8ef41c54e80fa671d1d69d9eeabc47ca85d78e635c7
Sep 16 18:49:33.032525 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:33.032508    2584 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 16 18:49:33.042267 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:33.042228    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qbpp2" event={"ID":"3097f2a3-e38f-4abe-83dc-bcdc64d46c45","Type":"ContainerStarted","Data":"d60b534d9a5d2af428adb8ef41c54e80fa671d1d69d9eeabc47ca85d78e635c7"}
Sep 16 18:49:33.072076 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:33.072041    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:33.270293 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:33.270265    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-59fbdd77d-9h5nf"]
Sep 16 18:49:33.272551 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:49:33.272526    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07294397_1424_41ee_b652_aec35f466d18.slice/crio-fb4938178b5a5b23ce703d0c73c38c5a85302d1bfee07720f7a713c7a92f065f WatchSource:0}: Error finding container fb4938178b5a5b23ce703d0c73c38c5a85302d1bfee07720f7a713c7a92f065f: Status 404 returned error can't find the container with id fb4938178b5a5b23ce703d0c73c38c5a85302d1bfee07720f7a713c7a92f065f
Sep 16 18:49:33.519206 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:33.519168    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:33.653572 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:33.653462    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:33.752496 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:33.752461    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:33.971744 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:33.971664    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:34.046251 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:34.046220    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-9h5nf" event={"ID":"07294397-1424-41ee-b652-aec35f466d18","Type":"ContainerStarted","Data":"fb4938178b5a5b23ce703d0c73c38c5a85302d1bfee07720f7a713c7a92f065f"}
Sep 16 18:49:34.116318 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:34.116284    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="93861b72-565d-497a-a7d5-098fc010f83c" path="/var/lib/kubelet/pods/93861b72-565d-497a-a7d5-098fc010f83c/volumes"
Sep 16 18:49:35.052446 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:35.052413    2584 generic.go:356] "Generic (PLEG): container finished" podID="3097f2a3-e38f-4abe-83dc-bcdc64d46c45" containerID="757cbdc1cedb06699d9471852f65e34992a79aef3d8ec9b883cc4cd7f52246ef" exitCode=0
Sep 16 18:49:35.052869 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:35.052486    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qbpp2" event={"ID":"3097f2a3-e38f-4abe-83dc-bcdc64d46c45","Type":"ContainerDied","Data":"757cbdc1cedb06699d9471852f65e34992a79aef3d8ec9b883cc4cd7f52246ef"}
Sep 16 18:49:35.055017 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:35.054987    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-9h5nf" event={"ID":"07294397-1424-41ee-b652-aec35f466d18","Type":"ContainerStarted","Data":"9f8761c203db133b0f04b8f1968ac473856d45dac07be8e9edd167a50211b3d2"}
Sep 16 18:49:35.055136 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:35.055045    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-9h5nf" event={"ID":"07294397-1424-41ee-b652-aec35f466d18","Type":"ContainerStarted","Data":"0e38e6540eeeec8ae776e360afdd813997ecaf609f73fd559f64399e37da2a5f"}
Sep 16 18:49:35.055136 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:35.055064    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-9h5nf" event={"ID":"07294397-1424-41ee-b652-aec35f466d18","Type":"ContainerStarted","Data":"1d9810b4d7beea7ae2ce119dc663ad267959c928569f3cc17968a484d7ba4a93"}
Sep 16 18:49:35.091027 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:35.090942    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-9h5nf" podStartSLOduration=1.5582703470000001 podStartE2EDuration="3.090919539s" totalImagesPullingTime="1.532649192s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:49:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:49:35.090230155 +0000 UTC m=+305.419732568" watchObservedRunningTime="2026-09-16 18:49:35.090919539 +0000 UTC m=+305.420421949"
Sep 16 18:49:35.423416 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:35.423333    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:35.525612 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:35.525577    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:35.627365 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:35.627335    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:35.670812 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:35.670777    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-6c9b6bc889-kslpj"]
Sep 16 18:49:35.674567 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:35.674512    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-6c9b6bc889-kslpj"
Sep 16 18:49:35.677420 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:35.677397    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Sep 16 18:49:35.677557 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:35.677397    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Sep 16 18:49:35.677557 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:35.677444    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-l94m9\""
Sep 16 18:49:35.677557 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:35.677456    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-apvgmonnjinub\""
Sep 16 18:49:35.677717 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:35.677703    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Sep 16 18:49:35.677769 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:35.677732    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Sep 16 18:49:35.677769 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:35.677742    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Sep 16 18:49:35.688892 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:35.688865    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-6c9b6bc889-kslpj"]
Sep 16 18:49:35.728571 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:35.728541    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:35.798082 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:35.798044    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:35.813153 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:35.813123    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/64f114cb-62c3-4547-8385-8259c4d9537d-secret-grpc-tls\") pod \"thanos-querier-6c9b6bc889-kslpj\" (UID: \"64f114cb-62c3-4547-8385-8259c4d9537d\") " pod="openshift-monitoring/thanos-querier-6c9b6bc889-kslpj"
Sep 16 18:49:35.813254 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:35.813159    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/64f114cb-62c3-4547-8385-8259c4d9537d-metrics-client-ca\") pod \"thanos-querier-6c9b6bc889-kslpj\" (UID: \"64f114cb-62c3-4547-8385-8259c4d9537d\") " pod="openshift-monitoring/thanos-querier-6c9b6bc889-kslpj"
Sep 16 18:49:35.813254 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:35.813180    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/64f114cb-62c3-4547-8385-8259c4d9537d-secret-thanos-querier-tls\") pod \"thanos-querier-6c9b6bc889-kslpj\" (UID: \"64f114cb-62c3-4547-8385-8259c4d9537d\") " pod="openshift-monitoring/thanos-querier-6c9b6bc889-kslpj"
Sep 16 18:49:35.813254 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:35.813229    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/64f114cb-62c3-4547-8385-8259c4d9537d-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6c9b6bc889-kslpj\" (UID: \"64f114cb-62c3-4547-8385-8259c4d9537d\") " pod="openshift-monitoring/thanos-querier-6c9b6bc889-kslpj"
Sep 16 18:49:35.813433 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:35.813268    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rr8lw\" (UniqueName: \"kubernetes.io/projected/64f114cb-62c3-4547-8385-8259c4d9537d-kube-api-access-rr8lw\") pod \"thanos-querier-6c9b6bc889-kslpj\" (UID: \"64f114cb-62c3-4547-8385-8259c4d9537d\") " pod="openshift-monitoring/thanos-querier-6c9b6bc889-kslpj"
Sep 16 18:49:35.813433 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:35.813312    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/64f114cb-62c3-4547-8385-8259c4d9537d-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6c9b6bc889-kslpj\" (UID: \"64f114cb-62c3-4547-8385-8259c4d9537d\") " pod="openshift-monitoring/thanos-querier-6c9b6bc889-kslpj"
Sep 16 18:49:35.813433 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:35.813408    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/64f114cb-62c3-4547-8385-8259c4d9537d-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6c9b6bc889-kslpj\" (UID: \"64f114cb-62c3-4547-8385-8259c4d9537d\") " pod="openshift-monitoring/thanos-querier-6c9b6bc889-kslpj"
Sep 16 18:49:35.813558 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:35.813444    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/64f114cb-62c3-4547-8385-8259c4d9537d-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6c9b6bc889-kslpj\" (UID: \"64f114cb-62c3-4547-8385-8259c4d9537d\") " pod="openshift-monitoring/thanos-querier-6c9b6bc889-kslpj"
Sep 16 18:49:35.914306 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:35.914264    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/64f114cb-62c3-4547-8385-8259c4d9537d-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6c9b6bc889-kslpj\" (UID: \"64f114cb-62c3-4547-8385-8259c4d9537d\") " pod="openshift-monitoring/thanos-querier-6c9b6bc889-kslpj"
Sep 16 18:49:35.914432 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:35.914318    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/64f114cb-62c3-4547-8385-8259c4d9537d-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6c9b6bc889-kslpj\" (UID: \"64f114cb-62c3-4547-8385-8259c4d9537d\") " pod="openshift-monitoring/thanos-querier-6c9b6bc889-kslpj"
Sep 16 18:49:35.914432 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:35.914390    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/64f114cb-62c3-4547-8385-8259c4d9537d-secret-grpc-tls\") pod \"thanos-querier-6c9b6bc889-kslpj\" (UID: \"64f114cb-62c3-4547-8385-8259c4d9537d\") " pod="openshift-monitoring/thanos-querier-6c9b6bc889-kslpj"
Sep 16 18:49:35.914432 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:35.914418    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/64f114cb-62c3-4547-8385-8259c4d9537d-metrics-client-ca\") pod \"thanos-querier-6c9b6bc889-kslpj\" (UID: \"64f114cb-62c3-4547-8385-8259c4d9537d\") " pod="openshift-monitoring/thanos-querier-6c9b6bc889-kslpj"
Sep 16 18:49:35.914603 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:35.914447    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/64f114cb-62c3-4547-8385-8259c4d9537d-secret-thanos-querier-tls\") pod \"thanos-querier-6c9b6bc889-kslpj\" (UID: \"64f114cb-62c3-4547-8385-8259c4d9537d\") " pod="openshift-monitoring/thanos-querier-6c9b6bc889-kslpj"
Sep 16 18:49:35.914603 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:35.914477    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/64f114cb-62c3-4547-8385-8259c4d9537d-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6c9b6bc889-kslpj\" (UID: \"64f114cb-62c3-4547-8385-8259c4d9537d\") " pod="openshift-monitoring/thanos-querier-6c9b6bc889-kslpj"
Sep 16 18:49:35.914603 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:35.914501    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rr8lw\" (UniqueName: \"kubernetes.io/projected/64f114cb-62c3-4547-8385-8259c4d9537d-kube-api-access-rr8lw\") pod \"thanos-querier-6c9b6bc889-kslpj\" (UID: \"64f114cb-62c3-4547-8385-8259c4d9537d\") " pod="openshift-monitoring/thanos-querier-6c9b6bc889-kslpj"
Sep 16 18:49:35.914603 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:35.914532    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/64f114cb-62c3-4547-8385-8259c4d9537d-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6c9b6bc889-kslpj\" (UID: \"64f114cb-62c3-4547-8385-8259c4d9537d\") " pod="openshift-monitoring/thanos-querier-6c9b6bc889-kslpj"
Sep 16 18:49:35.915343 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:35.915310    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/64f114cb-62c3-4547-8385-8259c4d9537d-metrics-client-ca\") pod \"thanos-querier-6c9b6bc889-kslpj\" (UID: \"64f114cb-62c3-4547-8385-8259c4d9537d\") " pod="openshift-monitoring/thanos-querier-6c9b6bc889-kslpj"
Sep 16 18:49:35.917178 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:35.917151    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/64f114cb-62c3-4547-8385-8259c4d9537d-secret-grpc-tls\") pod \"thanos-querier-6c9b6bc889-kslpj\" (UID: \"64f114cb-62c3-4547-8385-8259c4d9537d\") " pod="openshift-monitoring/thanos-querier-6c9b6bc889-kslpj"
Sep 16 18:49:35.922301 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:35.917522    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/64f114cb-62c3-4547-8385-8259c4d9537d-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6c9b6bc889-kslpj\" (UID: \"64f114cb-62c3-4547-8385-8259c4d9537d\") " pod="openshift-monitoring/thanos-querier-6c9b6bc889-kslpj"
Sep 16 18:49:35.922301 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:35.917264    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/64f114cb-62c3-4547-8385-8259c4d9537d-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6c9b6bc889-kslpj\" (UID: \"64f114cb-62c3-4547-8385-8259c4d9537d\") " pod="openshift-monitoring/thanos-querier-6c9b6bc889-kslpj"
Sep 16 18:49:35.922301 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:35.917869    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/64f114cb-62c3-4547-8385-8259c4d9537d-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6c9b6bc889-kslpj\" (UID: \"64f114cb-62c3-4547-8385-8259c4d9537d\") " pod="openshift-monitoring/thanos-querier-6c9b6bc889-kslpj"
Sep 16 18:49:35.922301 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:35.917964    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/64f114cb-62c3-4547-8385-8259c4d9537d-secret-thanos-querier-tls\") pod \"thanos-querier-6c9b6bc889-kslpj\" (UID: \"64f114cb-62c3-4547-8385-8259c4d9537d\") " pod="openshift-monitoring/thanos-querier-6c9b6bc889-kslpj"
Sep 16 18:49:35.922301 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:35.917963    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/64f114cb-62c3-4547-8385-8259c4d9537d-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6c9b6bc889-kslpj\" (UID: \"64f114cb-62c3-4547-8385-8259c4d9537d\") " pod="openshift-monitoring/thanos-querier-6c9b6bc889-kslpj"
Sep 16 18:49:35.923263 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:35.923240    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rr8lw\" (UniqueName: \"kubernetes.io/projected/64f114cb-62c3-4547-8385-8259c4d9537d-kube-api-access-rr8lw\") pod \"thanos-querier-6c9b6bc889-kslpj\" (UID: \"64f114cb-62c3-4547-8385-8259c4d9537d\") " pod="openshift-monitoring/thanos-querier-6c9b6bc889-kslpj"
Sep 16 18:49:35.983073 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:35.982984    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-6c9b6bc889-kslpj"
Sep 16 18:49:36.061075 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:36.061034    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qbpp2" event={"ID":"3097f2a3-e38f-4abe-83dc-bcdc64d46c45","Type":"ContainerStarted","Data":"4e489a5f01133507bb4d10b3b774a2418ebf0f4e6a39c69ecf4a07e6f9b1ef77"}
Sep 16 18:49:36.061432 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:36.061144    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qbpp2" event={"ID":"3097f2a3-e38f-4abe-83dc-bcdc64d46c45","Type":"ContainerStarted","Data":"219f99bc3d8feea9915d3132bfe81b03af751cd147df4d42007da81f0f03849b"}
Sep 16 18:49:36.081407 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:36.081355    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-qbpp2" podStartSLOduration=3.04920678 podStartE2EDuration="4.081338864s" totalImagesPullingTime="970.036049ms" totalInitContainerRuntime="62.096035ms" isStatefulPod=true podCreationTimestamp="2026-09-16 18:49:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:49:36.080258772 +0000 UTC m=+306.409761184" watchObservedRunningTime="2026-09-16 18:49:36.081338864 +0000 UTC m=+306.410841295"
Sep 16 18:49:36.110667 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:36.110638    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bp5mv"
Sep 16 18:49:36.167433 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:36.167392    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:36.259378 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:36.259210    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-6c9b6bc889-kslpj"]
Sep 16 18:49:36.261748 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:49:36.261719    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64f114cb_62c3_4547_8385_8259c4d9537d.slice/crio-abadc8829c575b9021a02f3435c84959d1fc319465974200bbc5ea5f62caee5d WatchSource:0}: Error finding container abadc8829c575b9021a02f3435c84959d1fc319465974200bbc5ea5f62caee5d: Status 404 returned error can't find the container with id abadc8829c575b9021a02f3435c84959d1fc319465974200bbc5ea5f62caee5d
Sep 16 18:49:36.261879 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:36.261795    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:36.362085 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:36.362045    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:37.013078 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:37.013040    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:37.067296 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:37.067263    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6c9b6bc889-kslpj" event={"ID":"64f114cb-62c3-4547-8385-8259c4d9537d","Type":"ContainerStarted","Data":"abadc8829c575b9021a02f3435c84959d1fc319465974200bbc5ea5f62caee5d"}
Sep 16 18:49:37.090787 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:37.090752    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:37.454485 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:37.454401    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:37.540779 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:37.540743    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:37.820909 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:37.820877    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-f578fdbcb-w4mp7"]
Sep 16 18:49:37.824438 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:37.824413    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-f578fdbcb-w4mp7"
Sep 16 18:49:37.833972 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:37.833944    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-f578fdbcb-w4mp7"]
Sep 16 18:49:37.897013 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:37.896978    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:37.931392 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:37.931351    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d3189590-9978-421a-8aa1-629838b0236d-console-serving-cert\") pod \"console-f578fdbcb-w4mp7\" (UID: \"d3189590-9978-421a-8aa1-629838b0236d\") " pod="openshift-console/console-f578fdbcb-w4mp7"
Sep 16 18:49:37.931585 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:37.931416    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xdg92\" (UniqueName: \"kubernetes.io/projected/d3189590-9978-421a-8aa1-629838b0236d-kube-api-access-xdg92\") pod \"console-f578fdbcb-w4mp7\" (UID: \"d3189590-9978-421a-8aa1-629838b0236d\") " pod="openshift-console/console-f578fdbcb-w4mp7"
Sep 16 18:49:37.931585 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:37.931451    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d3189590-9978-421a-8aa1-629838b0236d-service-ca\") pod \"console-f578fdbcb-w4mp7\" (UID: \"d3189590-9978-421a-8aa1-629838b0236d\") " pod="openshift-console/console-f578fdbcb-w4mp7"
Sep 16 18:49:37.931585 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:37.931490    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d3189590-9978-421a-8aa1-629838b0236d-console-config\") pod \"console-f578fdbcb-w4mp7\" (UID: \"d3189590-9978-421a-8aa1-629838b0236d\") " pod="openshift-console/console-f578fdbcb-w4mp7"
Sep 16 18:49:37.931585 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:37.931531    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d3189590-9978-421a-8aa1-629838b0236d-trusted-ca-bundle\") pod \"console-f578fdbcb-w4mp7\" (UID: \"d3189590-9978-421a-8aa1-629838b0236d\") " pod="openshift-console/console-f578fdbcb-w4mp7"
Sep 16 18:49:37.931585 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:37.931578    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d3189590-9978-421a-8aa1-629838b0236d-console-oauth-config\") pod \"console-f578fdbcb-w4mp7\" (UID: \"d3189590-9978-421a-8aa1-629838b0236d\") " pod="openshift-console/console-f578fdbcb-w4mp7"
Sep 16 18:49:37.931845 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:37.931602    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d3189590-9978-421a-8aa1-629838b0236d-oauth-serving-cert\") pod \"console-f578fdbcb-w4mp7\" (UID: \"d3189590-9978-421a-8aa1-629838b0236d\") " pod="openshift-console/console-f578fdbcb-w4mp7"
Sep 16 18:49:38.032261 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:38.032215    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xdg92\" (UniqueName: \"kubernetes.io/projected/d3189590-9978-421a-8aa1-629838b0236d-kube-api-access-xdg92\") pod \"console-f578fdbcb-w4mp7\" (UID: \"d3189590-9978-421a-8aa1-629838b0236d\") " pod="openshift-console/console-f578fdbcb-w4mp7"
Sep 16 18:49:38.032406 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:38.032276    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d3189590-9978-421a-8aa1-629838b0236d-service-ca\") pod \"console-f578fdbcb-w4mp7\" (UID: \"d3189590-9978-421a-8aa1-629838b0236d\") " pod="openshift-console/console-f578fdbcb-w4mp7"
Sep 16 18:49:38.032406 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:38.032317    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d3189590-9978-421a-8aa1-629838b0236d-console-config\") pod \"console-f578fdbcb-w4mp7\" (UID: \"d3189590-9978-421a-8aa1-629838b0236d\") " pod="openshift-console/console-f578fdbcb-w4mp7"
Sep 16 18:49:38.032406 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:38.032348    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d3189590-9978-421a-8aa1-629838b0236d-trusted-ca-bundle\") pod \"console-f578fdbcb-w4mp7\" (UID: \"d3189590-9978-421a-8aa1-629838b0236d\") " pod="openshift-console/console-f578fdbcb-w4mp7"
Sep 16 18:49:38.032406 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:38.032401    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d3189590-9978-421a-8aa1-629838b0236d-console-oauth-config\") pod \"console-f578fdbcb-w4mp7\" (UID: \"d3189590-9978-421a-8aa1-629838b0236d\") " pod="openshift-console/console-f578fdbcb-w4mp7"
Sep 16 18:49:38.032640 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:38.032425    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d3189590-9978-421a-8aa1-629838b0236d-oauth-serving-cert\") pod \"console-f578fdbcb-w4mp7\" (UID: \"d3189590-9978-421a-8aa1-629838b0236d\") " pod="openshift-console/console-f578fdbcb-w4mp7"
Sep 16 18:49:38.032693 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:38.032659    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d3189590-9978-421a-8aa1-629838b0236d-console-serving-cert\") pod \"console-f578fdbcb-w4mp7\" (UID: \"d3189590-9978-421a-8aa1-629838b0236d\") " pod="openshift-console/console-f578fdbcb-w4mp7"
Sep 16 18:49:38.033150 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:38.033118    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d3189590-9978-421a-8aa1-629838b0236d-service-ca\") pod \"console-f578fdbcb-w4mp7\" (UID: \"d3189590-9978-421a-8aa1-629838b0236d\") " pod="openshift-console/console-f578fdbcb-w4mp7"
Sep 16 18:49:38.033271 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:38.033188    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d3189590-9978-421a-8aa1-629838b0236d-console-config\") pod \"console-f578fdbcb-w4mp7\" (UID: \"d3189590-9978-421a-8aa1-629838b0236d\") " pod="openshift-console/console-f578fdbcb-w4mp7"
Sep 16 18:49:38.033342 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:38.033288    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d3189590-9978-421a-8aa1-629838b0236d-oauth-serving-cert\") pod \"console-f578fdbcb-w4mp7\" (UID: \"d3189590-9978-421a-8aa1-629838b0236d\") " pod="openshift-console/console-f578fdbcb-w4mp7"
Sep 16 18:49:38.033464 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:38.033442    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d3189590-9978-421a-8aa1-629838b0236d-trusted-ca-bundle\") pod \"console-f578fdbcb-w4mp7\" (UID: \"d3189590-9978-421a-8aa1-629838b0236d\") " pod="openshift-console/console-f578fdbcb-w4mp7"
Sep 16 18:49:38.035211 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:38.035191    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d3189590-9978-421a-8aa1-629838b0236d-console-serving-cert\") pod \"console-f578fdbcb-w4mp7\" (UID: \"d3189590-9978-421a-8aa1-629838b0236d\") " pod="openshift-console/console-f578fdbcb-w4mp7"
Sep 16 18:49:38.035291 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:38.035246    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d3189590-9978-421a-8aa1-629838b0236d-console-oauth-config\") pod \"console-f578fdbcb-w4mp7\" (UID: \"d3189590-9978-421a-8aa1-629838b0236d\") " pod="openshift-console/console-f578fdbcb-w4mp7"
Sep 16 18:49:38.040477 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:38.040453    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xdg92\" (UniqueName: \"kubernetes.io/projected/d3189590-9978-421a-8aa1-629838b0236d-kube-api-access-xdg92\") pod \"console-f578fdbcb-w4mp7\" (UID: \"d3189590-9978-421a-8aa1-629838b0236d\") " pod="openshift-console/console-f578fdbcb-w4mp7"
Sep 16 18:49:38.137341 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:38.137257    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-f578fdbcb-w4mp7"
Sep 16 18:49:38.440721 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:38.440691    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-f578fdbcb-w4mp7"]
Sep 16 18:49:38.444151 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:49:38.444121    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3189590_9978_421a_8aa1_629838b0236d.slice/crio-fd54a6d3cb8bdf3a34daa303ca9f699504757b64373597d6af5361d601fb4ba2 WatchSource:0}: Error finding container fd54a6d3cb8bdf3a34daa303ca9f699504757b64373597d6af5361d601fb4ba2: Status 404 returned error can't find the container with id fd54a6d3cb8bdf3a34daa303ca9f699504757b64373597d6af5361d601fb4ba2
Sep 16 18:49:38.924286 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:38.924245    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:39.077716 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:39.077674    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-f578fdbcb-w4mp7" event={"ID":"d3189590-9978-421a-8aa1-629838b0236d","Type":"ContainerStarted","Data":"3ec9fae3c7ac9021a1e4cf53b45df78e9fbbaffd704a3982cda0215afacb8501"}
Sep 16 18:49:39.077908 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:39.077757    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-f578fdbcb-w4mp7" event={"ID":"d3189590-9978-421a-8aa1-629838b0236d","Type":"ContainerStarted","Data":"fd54a6d3cb8bdf3a34daa303ca9f699504757b64373597d6af5361d601fb4ba2"}
Sep 16 18:49:39.079891 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:39.079863    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6c9b6bc889-kslpj" event={"ID":"64f114cb-62c3-4547-8385-8259c4d9537d","Type":"ContainerStarted","Data":"d64fd5d2879264a17c3fd8ec02bcd525b870fa5d04d22000fc9607da27cde849"}
Sep 16 18:49:39.079994 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:39.079920    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6c9b6bc889-kslpj" event={"ID":"64f114cb-62c3-4547-8385-8259c4d9537d","Type":"ContainerStarted","Data":"e7f7adff67f124f71e4bedd17c389095f274b6519fc11dc821d8605dda396f78"}
Sep 16 18:49:39.079994 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:39.079945    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6c9b6bc889-kslpj" event={"ID":"64f114cb-62c3-4547-8385-8259c4d9537d","Type":"ContainerStarted","Data":"02c67452c4b229f2fa289ea1e5b74c82cb7d73b5753c9d6b1bcd0561749c1a22"}
Sep 16 18:49:39.095225 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:39.095057    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-f578fdbcb-w4mp7" podStartSLOduration=2.095037231 podStartE2EDuration="2.095037231s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:49:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:49:39.094994122 +0000 UTC m=+309.424496536" watchObservedRunningTime="2026-09-16 18:49:39.095037231 +0000 UTC m=+309.424539644"
Sep 16 18:49:40.087044 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:40.087004    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6c9b6bc889-kslpj" event={"ID":"64f114cb-62c3-4547-8385-8259c4d9537d","Type":"ContainerStarted","Data":"213d1169653556d623d1868929acdf7c4c8e2fcf4c44d1ac188eeba964fa4a6a"}
Sep 16 18:49:40.087410 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:40.087083    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6c9b6bc889-kslpj" event={"ID":"64f114cb-62c3-4547-8385-8259c4d9537d","Type":"ContainerStarted","Data":"f316e578cf77b4a3f3049fcff2c1d08ee007b884493187d7754bf76f071ade5b"}
Sep 16 18:49:40.087410 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:40.087125    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6c9b6bc889-kslpj" event={"ID":"64f114cb-62c3-4547-8385-8259c4d9537d","Type":"ContainerStarted","Data":"af210bbffcd17a1429af1f8472fc9443bb5861ccc5feb1f4936f84f9a8292e2d"}
Sep 16 18:49:40.111440 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:40.111385    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-6c9b6bc889-kslpj" podStartSLOduration=2.183896125 podStartE2EDuration="5.1113684s" totalImagesPullingTime="2.927472275s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:49:35 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:49:40.108945782 +0000 UTC m=+310.438448194" watchObservedRunningTime="2026-09-16 18:49:40.1113684 +0000 UTC m=+310.440870812"
Sep 16 18:49:41.091190 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:41.091163    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-6c9b6bc889-kslpj"
Sep 16 18:49:41.623436 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:41.623399    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:41.722148 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:41.722113    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:41.839499 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:41.839464    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:41.941098 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:41.941017    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:42.001298 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:42.001263    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:43.118254 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:43.118218    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:43.170599 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:43.170573    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:43.816940 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:43.816905    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:43.918233 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:43.918195    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:45.300392 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:45.300357    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:47.101334 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:47.101307    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-6c9b6bc889-kslpj"
Sep 16 18:49:48.138104 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:48.138065    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-f578fdbcb-w4mp7"
Sep 16 18:49:48.138555 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:48.138120    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-f578fdbcb-w4mp7"
Sep 16 18:49:48.143050 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:48.143020    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-f578fdbcb-w4mp7"
Sep 16 18:49:49.124241 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:49.124213    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-f578fdbcb-w4mp7"
Sep 16 18:49:49.175265 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:49.175232    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6d964bb7dc-lznmb"]
Sep 16 18:49:49.289949 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:49.289913    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:49.387559 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:49.387489    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:49.441322 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:49.441288    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:50.993375 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:50.993344    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:51.052133 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:51.052090    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:57.109892 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:57.109809    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-fc6456cd6-gk5g8" podUID="1a01026a-ace9-487a-9e36-39376a917186" containerName="console" containerID="cri-o://1d2bdb923dc7bc8ed38703cc3491da4535bb72b8477b2b11eed3cecd75e51485" gracePeriod=15
Sep 16 18:49:57.340425 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:57.340400    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-fc6456cd6-gk5g8"
Sep 16 18:49:57.500449 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:57.500362    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1a01026a-ace9-487a-9e36-39376a917186-service-ca\") pod \"1a01026a-ace9-487a-9e36-39376a917186\" (UID: \"1a01026a-ace9-487a-9e36-39376a917186\") "
Sep 16 18:49:57.500449 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:57.500404    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1a01026a-ace9-487a-9e36-39376a917186-oauth-serving-cert\") pod \"1a01026a-ace9-487a-9e36-39376a917186\" (UID: \"1a01026a-ace9-487a-9e36-39376a917186\") "
Sep 16 18:49:57.500449 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:57.500441    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2fpml\" (UniqueName: \"kubernetes.io/projected/1a01026a-ace9-487a-9e36-39376a917186-kube-api-access-2fpml\") pod \"1a01026a-ace9-487a-9e36-39376a917186\" (UID: \"1a01026a-ace9-487a-9e36-39376a917186\") "
Sep 16 18:49:57.500686 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:57.500465    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1a01026a-ace9-487a-9e36-39376a917186-console-serving-cert\") pod \"1a01026a-ace9-487a-9e36-39376a917186\" (UID: \"1a01026a-ace9-487a-9e36-39376a917186\") "
Sep 16 18:49:57.500686 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:57.500488    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1a01026a-ace9-487a-9e36-39376a917186-console-oauth-config\") pod \"1a01026a-ace9-487a-9e36-39376a917186\" (UID: \"1a01026a-ace9-487a-9e36-39376a917186\") "
Sep 16 18:49:57.500686 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:57.500569    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1a01026a-ace9-487a-9e36-39376a917186-console-config\") pod \"1a01026a-ace9-487a-9e36-39376a917186\" (UID: \"1a01026a-ace9-487a-9e36-39376a917186\") "
Sep 16 18:49:57.500862 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:57.500791    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1a01026a-ace9-487a-9e36-39376a917186-service-ca" (OuterVolumeSpecName: "service-ca") pod "1a01026a-ace9-487a-9e36-39376a917186" (UID: "1a01026a-ace9-487a-9e36-39376a917186"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:49:57.500930 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:57.500883    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1a01026a-ace9-487a-9e36-39376a917186-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "1a01026a-ace9-487a-9e36-39376a917186" (UID: "1a01026a-ace9-487a-9e36-39376a917186"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:49:57.501064 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:57.501037    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1a01026a-ace9-487a-9e36-39376a917186-console-config" (OuterVolumeSpecName: "console-config") pod "1a01026a-ace9-487a-9e36-39376a917186" (UID: "1a01026a-ace9-487a-9e36-39376a917186"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:49:57.502806 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:57.502774    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1a01026a-ace9-487a-9e36-39376a917186-kube-api-access-2fpml" (OuterVolumeSpecName: "kube-api-access-2fpml") pod "1a01026a-ace9-487a-9e36-39376a917186" (UID: "1a01026a-ace9-487a-9e36-39376a917186"). InnerVolumeSpecName "kube-api-access-2fpml". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:49:57.502909 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:57.502800    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1a01026a-ace9-487a-9e36-39376a917186-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "1a01026a-ace9-487a-9e36-39376a917186" (UID: "1a01026a-ace9-487a-9e36-39376a917186"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:49:57.502909 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:57.502819    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1a01026a-ace9-487a-9e36-39376a917186-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "1a01026a-ace9-487a-9e36-39376a917186" (UID: "1a01026a-ace9-487a-9e36-39376a917186"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:49:57.601436 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:57.601405    2584 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1a01026a-ace9-487a-9e36-39376a917186-console-config\") on node \"ip-10-0-139-182.ec2.internal\" DevicePath \"\""
Sep 16 18:49:57.601436 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:57.601429    2584 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1a01026a-ace9-487a-9e36-39376a917186-service-ca\") on node \"ip-10-0-139-182.ec2.internal\" DevicePath \"\""
Sep 16 18:49:57.601436 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:57.601438    2584 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1a01026a-ace9-487a-9e36-39376a917186-oauth-serving-cert\") on node \"ip-10-0-139-182.ec2.internal\" DevicePath \"\""
Sep 16 18:49:57.601639 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:57.601447    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2fpml\" (UniqueName: \"kubernetes.io/projected/1a01026a-ace9-487a-9e36-39376a917186-kube-api-access-2fpml\") on node \"ip-10-0-139-182.ec2.internal\" DevicePath \"\""
Sep 16 18:49:57.601639 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:57.601456    2584 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1a01026a-ace9-487a-9e36-39376a917186-console-serving-cert\") on node \"ip-10-0-139-182.ec2.internal\" DevicePath \"\""
Sep 16 18:49:57.601639 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:57.601464    2584 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1a01026a-ace9-487a-9e36-39376a917186-console-oauth-config\") on node \"ip-10-0-139-182.ec2.internal\" DevicePath \"\""
Sep 16 18:49:58.152819 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:58.152786    2584 generic.go:356] "Generic (PLEG): container finished" podID="1a01026a-ace9-487a-9e36-39376a917186" containerID="1d2bdb923dc7bc8ed38703cc3491da4535bb72b8477b2b11eed3cecd75e51485" exitCode=0
Sep 16 18:49:58.153179 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:58.152840    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-fc6456cd6-gk5g8"
Sep 16 18:49:58.153179 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:58.152842    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-fc6456cd6-gk5g8" event={"ID":"1a01026a-ace9-487a-9e36-39376a917186","Type":"ContainerDied","Data":"1d2bdb923dc7bc8ed38703cc3491da4535bb72b8477b2b11eed3cecd75e51485"}
Sep 16 18:49:58.153179 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:58.152889    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-fc6456cd6-gk5g8" event={"ID":"1a01026a-ace9-487a-9e36-39376a917186","Type":"ContainerDied","Data":"f5cb8bb5c92f8a899fc847c33d8aad67d519a4de74d5948332b236e7ca19b610"}
Sep 16 18:49:58.153179 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:58.152912    2584 scope.go:117] "RemoveContainer" containerID="1d2bdb923dc7bc8ed38703cc3491da4535bb72b8477b2b11eed3cecd75e51485"
Sep 16 18:49:58.160809 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:58.160786    2584 scope.go:117] "RemoveContainer" containerID="1d2bdb923dc7bc8ed38703cc3491da4535bb72b8477b2b11eed3cecd75e51485"
Sep 16 18:49:58.161106 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:49:58.161086    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1d2bdb923dc7bc8ed38703cc3491da4535bb72b8477b2b11eed3cecd75e51485\": container with ID starting with 1d2bdb923dc7bc8ed38703cc3491da4535bb72b8477b2b11eed3cecd75e51485 not found: ID does not exist" containerID="1d2bdb923dc7bc8ed38703cc3491da4535bb72b8477b2b11eed3cecd75e51485"
Sep 16 18:49:58.161162 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:58.161114    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1d2bdb923dc7bc8ed38703cc3491da4535bb72b8477b2b11eed3cecd75e51485"} err="failed to get container status \"1d2bdb923dc7bc8ed38703cc3491da4535bb72b8477b2b11eed3cecd75e51485\": rpc error: code = NotFound desc = could not find container \"1d2bdb923dc7bc8ed38703cc3491da4535bb72b8477b2b11eed3cecd75e51485\": container with ID starting with 1d2bdb923dc7bc8ed38703cc3491da4535bb72b8477b2b11eed3cecd75e51485 not found: ID does not exist"
Sep 16 18:49:58.167798 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:58.167776    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-fc6456cd6-gk5g8"]
Sep 16 18:49:58.170054 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:49:58.170034    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-fc6456cd6-gk5g8"]
Sep 16 18:50:00.115480 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:50:00.115404    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1a01026a-ace9-487a-9e36-39376a917186" path="/var/lib/kubelet/pods/1a01026a-ace9-487a-9e36-39376a917186/volumes"
Sep 16 18:50:00.161600 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:50:00.161562    2584 generic.go:356] "Generic (PLEG): container finished" podID="65e92d96-65ad-4891-b693-1293389b7855" containerID="3e43e5ab2579e28cb9b49e1c0f33676301580da88e31f58e99a092772c3fac02" exitCode=0
Sep 16 18:50:00.161780 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:50:00.161614    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-qkzj5" event={"ID":"65e92d96-65ad-4891-b693-1293389b7855","Type":"ContainerDied","Data":"3e43e5ab2579e28cb9b49e1c0f33676301580da88e31f58e99a092772c3fac02"}
Sep 16 18:50:00.162017 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:50:00.162000    2584 scope.go:117] "RemoveContainer" containerID="3e43e5ab2579e28cb9b49e1c0f33676301580da88e31f58e99a092772c3fac02"
Sep 16 18:50:01.167780 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:50:01.167745    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-qkzj5" event={"ID":"65e92d96-65ad-4891-b693-1293389b7855","Type":"ContainerStarted","Data":"39f7d0271657d5c5d367c18632876592a1eb5d79d2a3fb9b03f5ea57426ddb1d"}
Sep 16 18:50:06.297143 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:50:06.297106    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:50:06.351435 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:50:06.351404    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:50:14.194897 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:50:14.194855    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6d964bb7dc-lznmb" podUID="7c9aa940-b15d-44a7-9235-4504ccfd0615" containerName="console" containerID="cri-o://38174231e1b23ccadba494e5d25dccff0cb7200a73133e1c2923d4d3e062cdb8" gracePeriod=15
Sep 16 18:50:14.438427 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:50:14.438402    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d964bb7dc-lznmb"
Sep 16 18:50:14.530321 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:50:14.530288    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7c9aa940-b15d-44a7-9235-4504ccfd0615-trusted-ca-bundle\") pod \"7c9aa940-b15d-44a7-9235-4504ccfd0615\" (UID: \"7c9aa940-b15d-44a7-9235-4504ccfd0615\") "
Sep 16 18:50:14.530520 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:50:14.530350    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7c9aa940-b15d-44a7-9235-4504ccfd0615-oauth-serving-cert\") pod \"7c9aa940-b15d-44a7-9235-4504ccfd0615\" (UID: \"7c9aa940-b15d-44a7-9235-4504ccfd0615\") "
Sep 16 18:50:14.530520 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:50:14.530381    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mbwq2\" (UniqueName: \"kubernetes.io/projected/7c9aa940-b15d-44a7-9235-4504ccfd0615-kube-api-access-mbwq2\") pod \"7c9aa940-b15d-44a7-9235-4504ccfd0615\" (UID: \"7c9aa940-b15d-44a7-9235-4504ccfd0615\") "
Sep 16 18:50:14.530520 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:50:14.530403    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7c9aa940-b15d-44a7-9235-4504ccfd0615-console-oauth-config\") pod \"7c9aa940-b15d-44a7-9235-4504ccfd0615\" (UID: \"7c9aa940-b15d-44a7-9235-4504ccfd0615\") "
Sep 16 18:50:14.530685 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:50:14.530603    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7c9aa940-b15d-44a7-9235-4504ccfd0615-console-serving-cert\") pod \"7c9aa940-b15d-44a7-9235-4504ccfd0615\" (UID: \"7c9aa940-b15d-44a7-9235-4504ccfd0615\") "
Sep 16 18:50:14.530685 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:50:14.530656    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7c9aa940-b15d-44a7-9235-4504ccfd0615-console-config\") pod \"7c9aa940-b15d-44a7-9235-4504ccfd0615\" (UID: \"7c9aa940-b15d-44a7-9235-4504ccfd0615\") "
Sep 16 18:50:14.530787 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:50:14.530690    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7c9aa940-b15d-44a7-9235-4504ccfd0615-service-ca\") pod \"7c9aa940-b15d-44a7-9235-4504ccfd0615\" (UID: \"7c9aa940-b15d-44a7-9235-4504ccfd0615\") "
Sep 16 18:50:14.530866 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:50:14.530807    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7c9aa940-b15d-44a7-9235-4504ccfd0615-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "7c9aa940-b15d-44a7-9235-4504ccfd0615" (UID: "7c9aa940-b15d-44a7-9235-4504ccfd0615"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:50:14.530922 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:50:14.530820    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7c9aa940-b15d-44a7-9235-4504ccfd0615-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "7c9aa940-b15d-44a7-9235-4504ccfd0615" (UID: "7c9aa940-b15d-44a7-9235-4504ccfd0615"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:50:14.531026 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:50:14.531001    2584 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7c9aa940-b15d-44a7-9235-4504ccfd0615-oauth-serving-cert\") on node \"ip-10-0-139-182.ec2.internal\" DevicePath \"\""
Sep 16 18:50:14.531228 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:50:14.531183    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7c9aa940-b15d-44a7-9235-4504ccfd0615-console-config" (OuterVolumeSpecName: "console-config") pod "7c9aa940-b15d-44a7-9235-4504ccfd0615" (UID: "7c9aa940-b15d-44a7-9235-4504ccfd0615"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:50:14.531228 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:50:14.531192    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7c9aa940-b15d-44a7-9235-4504ccfd0615-service-ca" (OuterVolumeSpecName: "service-ca") pod "7c9aa940-b15d-44a7-9235-4504ccfd0615" (UID: "7c9aa940-b15d-44a7-9235-4504ccfd0615"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:50:14.532679 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:50:14.532651    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7c9aa940-b15d-44a7-9235-4504ccfd0615-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "7c9aa940-b15d-44a7-9235-4504ccfd0615" (UID: "7c9aa940-b15d-44a7-9235-4504ccfd0615"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:50:14.532749 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:50:14.532675    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7c9aa940-b15d-44a7-9235-4504ccfd0615-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "7c9aa940-b15d-44a7-9235-4504ccfd0615" (UID: "7c9aa940-b15d-44a7-9235-4504ccfd0615"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:50:14.532749 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:50:14.532671    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7c9aa940-b15d-44a7-9235-4504ccfd0615-kube-api-access-mbwq2" (OuterVolumeSpecName: "kube-api-access-mbwq2") pod "7c9aa940-b15d-44a7-9235-4504ccfd0615" (UID: "7c9aa940-b15d-44a7-9235-4504ccfd0615"). InnerVolumeSpecName "kube-api-access-mbwq2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:50:14.631577 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:50:14.631540    2584 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7c9aa940-b15d-44a7-9235-4504ccfd0615-trusted-ca-bundle\") on node \"ip-10-0-139-182.ec2.internal\" DevicePath \"\""
Sep 16 18:50:14.631577 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:50:14.631570    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mbwq2\" (UniqueName: \"kubernetes.io/projected/7c9aa940-b15d-44a7-9235-4504ccfd0615-kube-api-access-mbwq2\") on node \"ip-10-0-139-182.ec2.internal\" DevicePath \"\""
Sep 16 18:50:14.631577 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:50:14.631581    2584 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7c9aa940-b15d-44a7-9235-4504ccfd0615-console-oauth-config\") on node \"ip-10-0-139-182.ec2.internal\" DevicePath \"\""
Sep 16 18:50:14.631795 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:50:14.631590    2584 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7c9aa940-b15d-44a7-9235-4504ccfd0615-console-serving-cert\") on node \"ip-10-0-139-182.ec2.internal\" DevicePath \"\""
Sep 16 18:50:14.631795 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:50:14.631600    2584 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7c9aa940-b15d-44a7-9235-4504ccfd0615-console-config\") on node \"ip-10-0-139-182.ec2.internal\" DevicePath \"\""
Sep 16 18:50:14.631795 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:50:14.631608    2584 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7c9aa940-b15d-44a7-9235-4504ccfd0615-service-ca\") on node \"ip-10-0-139-182.ec2.internal\" DevicePath \"\""
Sep 16 18:50:15.219539 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:50:15.219501    2584 generic.go:356] "Generic (PLEG): container finished" podID="7c9aa940-b15d-44a7-9235-4504ccfd0615" containerID="38174231e1b23ccadba494e5d25dccff0cb7200a73133e1c2923d4d3e062cdb8" exitCode=0
Sep 16 18:50:15.219949 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:50:15.219544    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d964bb7dc-lznmb"
Sep 16 18:50:15.219949 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:50:15.219569    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d964bb7dc-lznmb" event={"ID":"7c9aa940-b15d-44a7-9235-4504ccfd0615","Type":"ContainerDied","Data":"38174231e1b23ccadba494e5d25dccff0cb7200a73133e1c2923d4d3e062cdb8"}
Sep 16 18:50:15.219949 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:50:15.219607    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d964bb7dc-lznmb" event={"ID":"7c9aa940-b15d-44a7-9235-4504ccfd0615","Type":"ContainerDied","Data":"8f0af2a169f0e432dd863e787cff0cb9269ad1062164af1738f431f1a9897942"}
Sep 16 18:50:15.219949 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:50:15.219626    2584 scope.go:117] "RemoveContainer" containerID="38174231e1b23ccadba494e5d25dccff0cb7200a73133e1c2923d4d3e062cdb8"
Sep 16 18:50:15.228274 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:50:15.228256    2584 scope.go:117] "RemoveContainer" containerID="38174231e1b23ccadba494e5d25dccff0cb7200a73133e1c2923d4d3e062cdb8"
Sep 16 18:50:15.228541 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:50:15.228517    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"38174231e1b23ccadba494e5d25dccff0cb7200a73133e1c2923d4d3e062cdb8\": container with ID starting with 38174231e1b23ccadba494e5d25dccff0cb7200a73133e1c2923d4d3e062cdb8 not found: ID does not exist" containerID="38174231e1b23ccadba494e5d25dccff0cb7200a73133e1c2923d4d3e062cdb8"
Sep 16 18:50:15.228634 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:50:15.228546    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"38174231e1b23ccadba494e5d25dccff0cb7200a73133e1c2923d4d3e062cdb8"} err="failed to get container status \"38174231e1b23ccadba494e5d25dccff0cb7200a73133e1c2923d4d3e062cdb8\": rpc error: code = NotFound desc = could not find container \"38174231e1b23ccadba494e5d25dccff0cb7200a73133e1c2923d4d3e062cdb8\": container with ID starting with 38174231e1b23ccadba494e5d25dccff0cb7200a73133e1c2923d4d3e062cdb8 not found: ID does not exist"
Sep 16 18:50:15.243022 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:50:15.242995    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6d964bb7dc-lznmb"]
Sep 16 18:50:15.249816 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:50:15.249794    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6d964bb7dc-lznmb"]
Sep 16 18:50:16.115916 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:50:16.115885    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7c9aa940-b15d-44a7-9235-4504ccfd0615" path="/var/lib/kubelet/pods/7c9aa940-b15d-44a7-9235-4504ccfd0615/volumes"
Sep 16 18:50:20.240214 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:50:20.240179    2584 generic.go:356] "Generic (PLEG): container finished" podID="12570b86-13fd-4b6d-adbf-e2d515c0492c" containerID="6f9b34eb6f87767efde8c7fa4119f45761c14b1d8392d9317220fa3ab74a811a" exitCode=0
Sep 16 18:50:20.240729 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:50:20.240256    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-6xjxf" event={"ID":"12570b86-13fd-4b6d-adbf-e2d515c0492c","Type":"ContainerDied","Data":"6f9b34eb6f87767efde8c7fa4119f45761c14b1d8392d9317220fa3ab74a811a"}
Sep 16 18:50:20.240729 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:50:20.240681    2584 scope.go:117] "RemoveContainer" containerID="6f9b34eb6f87767efde8c7fa4119f45761c14b1d8392d9317220fa3ab74a811a"
Sep 16 18:50:21.246819 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:50:21.246780    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-6xjxf" event={"ID":"12570b86-13fd-4b6d-adbf-e2d515c0492c","Type":"ContainerStarted","Data":"d87c2cd53eb047481fc67411fb13d171f946855f0442c3f3bbc795b7d97b9a34"}
Sep 16 18:50:21.293759 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:50:21.293726    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:50:21.350659 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:50:21.350616    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:50:25.875984 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:50:25.875893    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:50:25.933617 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:50:25.933571    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:50:29.704933 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:50:29.704896    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:50:43.877493 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:50:43.877452    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9b96048a-0005-410b-a710-909d21fbfde8-metrics-certs\") pod \"network-metrics-daemon-bp5mv\" (UID: \"9b96048a-0005-410b-a710-909d21fbfde8\") " pod="openshift-multus/network-metrics-daemon-bp5mv"
Sep 16 18:50:43.879941 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:50:43.879915    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9b96048a-0005-410b-a710-909d21fbfde8-metrics-certs\") pod \"network-metrics-daemon-bp5mv\" (UID: \"9b96048a-0005-410b-a710-909d21fbfde8\") " pod="openshift-multus/network-metrics-daemon-bp5mv"
Sep 16 18:50:43.915008 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:50:43.914978    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-7prs7\""
Sep 16 18:50:43.922478 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:50:43.922457    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bp5mv"
Sep 16 18:50:44.193506 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:50:44.193428    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-bp5mv"]
Sep 16 18:50:44.197354 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:50:44.197328    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b96048a_0005_410b_a710_909d21fbfde8.slice/crio-091b99f09bbb61e212c0d020ca66f3cda39b6e00683dd9768274ae0812054956 WatchSource:0}: Error finding container 091b99f09bbb61e212c0d020ca66f3cda39b6e00683dd9768274ae0812054956: Status 404 returned error can't find the container with id 091b99f09bbb61e212c0d020ca66f3cda39b6e00683dd9768274ae0812054956
Sep 16 18:50:44.326811 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:50:44.326779    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-bp5mv" event={"ID":"9b96048a-0005-410b-a710-909d21fbfde8","Type":"ContainerStarted","Data":"091b99f09bbb61e212c0d020ca66f3cda39b6e00683dd9768274ae0812054956"}
Sep 16 18:50:45.332305 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:50:45.332272    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-bp5mv" event={"ID":"9b96048a-0005-410b-a710-909d21fbfde8","Type":"ContainerStarted","Data":"39e0f6fc81391701040fc38079f6198e2a951d1ae71ca9d0600c743f15b86a7d"}
Sep 16 18:50:46.337657 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:50:46.337616    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-bp5mv" event={"ID":"9b96048a-0005-410b-a710-909d21fbfde8","Type":"ContainerStarted","Data":"437bf124a178748a61880949fad383d213d8ec681c483593a31c44819b771131"}
Sep 16 18:50:46.359530 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:50:46.359479    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-bp5mv" podStartSLOduration=375.347404238 podStartE2EDuration="6m16.35946188s" totalImagesPullingTime="1.012057642s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:44:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:50:46.357711495 +0000 UTC m=+376.687213910" watchObservedRunningTime="2026-09-16 18:50:46.35946188 +0000 UTC m=+376.688964292"
Sep 16 18:50:46.476469 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:50:46.476423    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:50:46.549968 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:50:46.549937    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:50:51.972177 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:50:51.972132    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:50:52.247317 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:50:52.247226    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:50:54.592192 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:50:54.592161    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:50:54.649587 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:50:54.649559    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:51:00.093349 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:51:00.093314    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:51:00.155959 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:51:00.155926    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:51:00.424312 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:51:00.424227    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:51:01.035372 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:51:01.035336    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-84d5ff7d7-ks2c5"]
Sep 16 18:51:01.035761 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:51:01.035738    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c9aa940-b15d-44a7-9235-4504ccfd0615" containerName="console"
Sep 16 18:51:01.035761 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:51:01.035762    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c9aa940-b15d-44a7-9235-4504ccfd0615" containerName="console"
Sep 16 18:51:01.035974 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:51:01.035777    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1a01026a-ace9-487a-9e36-39376a917186" containerName="console"
Sep 16 18:51:01.035974 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:51:01.035786    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="1a01026a-ace9-487a-9e36-39376a917186" containerName="console"
Sep 16 18:51:01.035974 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:51:01.035871    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="1a01026a-ace9-487a-9e36-39376a917186" containerName="console"
Sep 16 18:51:01.035974 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:51:01.035901    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="7c9aa940-b15d-44a7-9235-4504ccfd0615" containerName="console"
Sep 16 18:51:01.040070 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:51:01.040046    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-84d5ff7d7-ks2c5"
Sep 16 18:51:01.057225 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:51:01.057199    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-84d5ff7d7-ks2c5"]
Sep 16 18:51:01.097871 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:51:01.097837    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:51:01.109117 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:51:01.109090    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/158d7548-0988-4248-aafd-eb56eb02fc52-console-oauth-config\") pod \"console-84d5ff7d7-ks2c5\" (UID: \"158d7548-0988-4248-aafd-eb56eb02fc52\") " pod="openshift-console/console-84d5ff7d7-ks2c5"
Sep 16 18:51:01.109250 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:51:01.109129    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/158d7548-0988-4248-aafd-eb56eb02fc52-service-ca\") pod \"console-84d5ff7d7-ks2c5\" (UID: \"158d7548-0988-4248-aafd-eb56eb02fc52\") " pod="openshift-console/console-84d5ff7d7-ks2c5"
Sep 16 18:51:01.109250 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:51:01.109157    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jhhdn\" (UniqueName: \"kubernetes.io/projected/158d7548-0988-4248-aafd-eb56eb02fc52-kube-api-access-jhhdn\") pod \"console-84d5ff7d7-ks2c5\" (UID: \"158d7548-0988-4248-aafd-eb56eb02fc52\") " pod="openshift-console/console-84d5ff7d7-ks2c5"
Sep 16 18:51:01.109250 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:51:01.109179    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/158d7548-0988-4248-aafd-eb56eb02fc52-console-serving-cert\") pod \"console-84d5ff7d7-ks2c5\" (UID: \"158d7548-0988-4248-aafd-eb56eb02fc52\") " pod="openshift-console/console-84d5ff7d7-ks2c5"
Sep 16 18:51:01.109250 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:51:01.109200    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/158d7548-0988-4248-aafd-eb56eb02fc52-oauth-serving-cert\") pod \"console-84d5ff7d7-ks2c5\" (UID: \"158d7548-0988-4248-aafd-eb56eb02fc52\") " pod="openshift-console/console-84d5ff7d7-ks2c5"
Sep 16 18:51:01.109250 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:51:01.109220    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/158d7548-0988-4248-aafd-eb56eb02fc52-console-config\") pod \"console-84d5ff7d7-ks2c5\" (UID: \"158d7548-0988-4248-aafd-eb56eb02fc52\") " pod="openshift-console/console-84d5ff7d7-ks2c5"
Sep 16 18:51:01.109426 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:51:01.109282    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/158d7548-0988-4248-aafd-eb56eb02fc52-trusted-ca-bundle\") pod \"console-84d5ff7d7-ks2c5\" (UID: \"158d7548-0988-4248-aafd-eb56eb02fc52\") " pod="openshift-console/console-84d5ff7d7-ks2c5"
Sep 16 18:51:01.167581 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:51:01.167551    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:51:01.210572 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:51:01.210540    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/158d7548-0988-4248-aafd-eb56eb02fc52-console-oauth-config\") pod \"console-84d5ff7d7-ks2c5\" (UID: \"158d7548-0988-4248-aafd-eb56eb02fc52\") " pod="openshift-console/console-84d5ff7d7-ks2c5"
Sep 16 18:51:01.210709 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:51:01.210583    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/158d7548-0988-4248-aafd-eb56eb02fc52-service-ca\") pod \"console-84d5ff7d7-ks2c5\" (UID: \"158d7548-0988-4248-aafd-eb56eb02fc52\") " pod="openshift-console/console-84d5ff7d7-ks2c5"
Sep 16 18:51:01.210709 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:51:01.210625    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jhhdn\" (UniqueName: \"kubernetes.io/projected/158d7548-0988-4248-aafd-eb56eb02fc52-kube-api-access-jhhdn\") pod \"console-84d5ff7d7-ks2c5\" (UID: \"158d7548-0988-4248-aafd-eb56eb02fc52\") " pod="openshift-console/console-84d5ff7d7-ks2c5"
Sep 16 18:51:01.210709 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:51:01.210642    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/158d7548-0988-4248-aafd-eb56eb02fc52-console-serving-cert\") pod \"console-84d5ff7d7-ks2c5\" (UID: \"158d7548-0988-4248-aafd-eb56eb02fc52\") " pod="openshift-console/console-84d5ff7d7-ks2c5"
Sep 16 18:51:01.210709 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:51:01.210657    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/158d7548-0988-4248-aafd-eb56eb02fc52-oauth-serving-cert\") pod \"console-84d5ff7d7-ks2c5\" (UID: \"158d7548-0988-4248-aafd-eb56eb02fc52\") " pod="openshift-console/console-84d5ff7d7-ks2c5"
Sep 16 18:51:01.210709 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:51:01.210675    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/158d7548-0988-4248-aafd-eb56eb02fc52-console-config\") pod \"console-84d5ff7d7-ks2c5\" (UID: \"158d7548-0988-4248-aafd-eb56eb02fc52\") " pod="openshift-console/console-84d5ff7d7-ks2c5"
Sep 16 18:51:01.210986 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:51:01.210722    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/158d7548-0988-4248-aafd-eb56eb02fc52-trusted-ca-bundle\") pod \"console-84d5ff7d7-ks2c5\" (UID: \"158d7548-0988-4248-aafd-eb56eb02fc52\") " pod="openshift-console/console-84d5ff7d7-ks2c5"
Sep 16 18:51:01.211551 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:51:01.211520    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/158d7548-0988-4248-aafd-eb56eb02fc52-oauth-serving-cert\") pod \"console-84d5ff7d7-ks2c5\" (UID: \"158d7548-0988-4248-aafd-eb56eb02fc52\") " pod="openshift-console/console-84d5ff7d7-ks2c5"
Sep 16 18:51:01.211686 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:51:01.211559    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/158d7548-0988-4248-aafd-eb56eb02fc52-service-ca\") pod \"console-84d5ff7d7-ks2c5\" (UID: \"158d7548-0988-4248-aafd-eb56eb02fc52\") " pod="openshift-console/console-84d5ff7d7-ks2c5"
Sep 16 18:51:01.211686 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:51:01.211621    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/158d7548-0988-4248-aafd-eb56eb02fc52-console-config\") pod \"console-84d5ff7d7-ks2c5\" (UID: \"158d7548-0988-4248-aafd-eb56eb02fc52\") " pod="openshift-console/console-84d5ff7d7-ks2c5"
Sep 16 18:51:01.211686 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:51:01.211653    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/158d7548-0988-4248-aafd-eb56eb02fc52-trusted-ca-bundle\") pod \"console-84d5ff7d7-ks2c5\" (UID: \"158d7548-0988-4248-aafd-eb56eb02fc52\") " pod="openshift-console/console-84d5ff7d7-ks2c5"
Sep 16 18:51:01.212992 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:51:01.212964    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/158d7548-0988-4248-aafd-eb56eb02fc52-console-oauth-config\") pod \"console-84d5ff7d7-ks2c5\" (UID: \"158d7548-0988-4248-aafd-eb56eb02fc52\") " pod="openshift-console/console-84d5ff7d7-ks2c5"
Sep 16 18:51:01.213147 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:51:01.213130    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/158d7548-0988-4248-aafd-eb56eb02fc52-console-serving-cert\") pod \"console-84d5ff7d7-ks2c5\" (UID: \"158d7548-0988-4248-aafd-eb56eb02fc52\") " pod="openshift-console/console-84d5ff7d7-ks2c5"
Sep 16 18:51:01.218469 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:51:01.218444    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jhhdn\" (UniqueName: \"kubernetes.io/projected/158d7548-0988-4248-aafd-eb56eb02fc52-kube-api-access-jhhdn\") pod \"console-84d5ff7d7-ks2c5\" (UID: \"158d7548-0988-4248-aafd-eb56eb02fc52\") " pod="openshift-console/console-84d5ff7d7-ks2c5"
Sep 16 18:51:01.349097 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:51:01.349007    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-84d5ff7d7-ks2c5"
Sep 16 18:51:01.621510 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:51:01.621431    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-84d5ff7d7-ks2c5"]
Sep 16 18:51:01.624214 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:51:01.624178    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod158d7548_0988_4248_aafd_eb56eb02fc52.slice/crio-0f9e8bd468e667868ae77ef6620fc99294002f3595612508feba28d10fbb29d0 WatchSource:0}: Error finding container 0f9e8bd468e667868ae77ef6620fc99294002f3595612508feba28d10fbb29d0: Status 404 returned error can't find the container with id 0f9e8bd468e667868ae77ef6620fc99294002f3595612508feba28d10fbb29d0
Sep 16 18:51:02.397931 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:51:02.397897    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-84d5ff7d7-ks2c5" event={"ID":"158d7548-0988-4248-aafd-eb56eb02fc52","Type":"ContainerStarted","Data":"4a1016451bc4cbf85fdb8ee357271083999ea54486e0f7a6c6848320bd857585"}
Sep 16 18:51:02.398312 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:51:02.397944    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-84d5ff7d7-ks2c5" event={"ID":"158d7548-0988-4248-aafd-eb56eb02fc52","Type":"ContainerStarted","Data":"0f9e8bd468e667868ae77ef6620fc99294002f3595612508feba28d10fbb29d0"}
Sep 16 18:51:02.417613 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:51:02.417563    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-84d5ff7d7-ks2c5" podStartSLOduration=1.417548223 podStartE2EDuration="1.417548223s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:51:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:51:02.416021092 +0000 UTC m=+392.745523515" watchObservedRunningTime="2026-09-16 18:51:02.417548223 +0000 UTC m=+392.747050635"
Sep 16 18:51:11.350183 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:51:11.350142    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-84d5ff7d7-ks2c5"
Sep 16 18:51:11.350573 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:51:11.350202    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-84d5ff7d7-ks2c5"
Sep 16 18:51:11.354799 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:51:11.354771    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-84d5ff7d7-ks2c5"
Sep 16 18:51:11.433889 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:51:11.433859    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-84d5ff7d7-ks2c5"
Sep 16 18:51:11.481217 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:51:11.481182    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-f578fdbcb-w4mp7"]
Sep 16 18:51:11.591610 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:51:11.591580    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:51:11.692173 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:51:11.692069    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:51:11.748844 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:51:11.748799    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:51:19.052934 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:51:19.052898    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:51:25.621501 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:51:25.621449    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:51:25.724403 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:51:25.724371    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:51:36.508041 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:51:36.507982    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-f578fdbcb-w4mp7" podUID="d3189590-9978-421a-8aa1-629838b0236d" containerName="console" containerID="cri-o://3ec9fae3c7ac9021a1e4cf53b45df78e9fbbaffd704a3982cda0215afacb8501" gracePeriod=15
Sep 16 18:51:36.749673 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:51:36.749649    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-f578fdbcb-w4mp7"
Sep 16 18:51:36.896741 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:51:36.896704    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d3189590-9978-421a-8aa1-629838b0236d-console-config\") pod \"d3189590-9978-421a-8aa1-629838b0236d\" (UID: \"d3189590-9978-421a-8aa1-629838b0236d\") "
Sep 16 18:51:36.896956 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:51:36.896757    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d3189590-9978-421a-8aa1-629838b0236d-trusted-ca-bundle\") pod \"d3189590-9978-421a-8aa1-629838b0236d\" (UID: \"d3189590-9978-421a-8aa1-629838b0236d\") "
Sep 16 18:51:36.896956 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:51:36.896791    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d3189590-9978-421a-8aa1-629838b0236d-console-oauth-config\") pod \"d3189590-9978-421a-8aa1-629838b0236d\" (UID: \"d3189590-9978-421a-8aa1-629838b0236d\") "
Sep 16 18:51:36.896956 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:51:36.896807    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d3189590-9978-421a-8aa1-629838b0236d-console-serving-cert\") pod \"d3189590-9978-421a-8aa1-629838b0236d\" (UID: \"d3189590-9978-421a-8aa1-629838b0236d\") "
Sep 16 18:51:36.896956 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:51:36.896859    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d3189590-9978-421a-8aa1-629838b0236d-oauth-serving-cert\") pod \"d3189590-9978-421a-8aa1-629838b0236d\" (UID: \"d3189590-9978-421a-8aa1-629838b0236d\") "
Sep 16 18:51:36.896956 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:51:36.896892    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d3189590-9978-421a-8aa1-629838b0236d-service-ca\") pod \"d3189590-9978-421a-8aa1-629838b0236d\" (UID: \"d3189590-9978-421a-8aa1-629838b0236d\") "
Sep 16 18:51:36.896956 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:51:36.896919    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xdg92\" (UniqueName: \"kubernetes.io/projected/d3189590-9978-421a-8aa1-629838b0236d-kube-api-access-xdg92\") pod \"d3189590-9978-421a-8aa1-629838b0236d\" (UID: \"d3189590-9978-421a-8aa1-629838b0236d\") "
Sep 16 18:51:36.897268 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:51:36.897161    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d3189590-9978-421a-8aa1-629838b0236d-console-config" (OuterVolumeSpecName: "console-config") pod "d3189590-9978-421a-8aa1-629838b0236d" (UID: "d3189590-9978-421a-8aa1-629838b0236d"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:51:36.897385 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:51:36.897357    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d3189590-9978-421a-8aa1-629838b0236d-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "d3189590-9978-421a-8aa1-629838b0236d" (UID: "d3189590-9978-421a-8aa1-629838b0236d"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:51:36.897481 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:51:36.897459    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d3189590-9978-421a-8aa1-629838b0236d-service-ca" (OuterVolumeSpecName: "service-ca") pod "d3189590-9978-421a-8aa1-629838b0236d" (UID: "d3189590-9978-421a-8aa1-629838b0236d"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:51:36.897526 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:51:36.897476    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d3189590-9978-421a-8aa1-629838b0236d-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "d3189590-9978-421a-8aa1-629838b0236d" (UID: "d3189590-9978-421a-8aa1-629838b0236d"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:51:36.899036 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:51:36.899014    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d3189590-9978-421a-8aa1-629838b0236d-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "d3189590-9978-421a-8aa1-629838b0236d" (UID: "d3189590-9978-421a-8aa1-629838b0236d"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:51:36.899453 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:51:36.899431    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d3189590-9978-421a-8aa1-629838b0236d-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "d3189590-9978-421a-8aa1-629838b0236d" (UID: "d3189590-9978-421a-8aa1-629838b0236d"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:51:36.899523 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:51:36.899433    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d3189590-9978-421a-8aa1-629838b0236d-kube-api-access-xdg92" (OuterVolumeSpecName: "kube-api-access-xdg92") pod "d3189590-9978-421a-8aa1-629838b0236d" (UID: "d3189590-9978-421a-8aa1-629838b0236d"). InnerVolumeSpecName "kube-api-access-xdg92". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:51:36.997512 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:51:36.997477    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xdg92\" (UniqueName: \"kubernetes.io/projected/d3189590-9978-421a-8aa1-629838b0236d-kube-api-access-xdg92\") on node \"ip-10-0-139-182.ec2.internal\" DevicePath \"\""
Sep 16 18:51:36.997512 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:51:36.997507    2584 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d3189590-9978-421a-8aa1-629838b0236d-console-config\") on node \"ip-10-0-139-182.ec2.internal\" DevicePath \"\""
Sep 16 18:51:36.997512 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:51:36.997517    2584 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d3189590-9978-421a-8aa1-629838b0236d-trusted-ca-bundle\") on node \"ip-10-0-139-182.ec2.internal\" DevicePath \"\""
Sep 16 18:51:36.997729 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:51:36.997525    2584 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d3189590-9978-421a-8aa1-629838b0236d-console-oauth-config\") on node \"ip-10-0-139-182.ec2.internal\" DevicePath \"\""
Sep 16 18:51:36.997729 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:51:36.997534    2584 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d3189590-9978-421a-8aa1-629838b0236d-console-serving-cert\") on node \"ip-10-0-139-182.ec2.internal\" DevicePath \"\""
Sep 16 18:51:36.997729 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:51:36.997543    2584 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d3189590-9978-421a-8aa1-629838b0236d-oauth-serving-cert\") on node \"ip-10-0-139-182.ec2.internal\" DevicePath \"\""
Sep 16 18:51:36.997729 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:51:36.997551    2584 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d3189590-9978-421a-8aa1-629838b0236d-service-ca\") on node \"ip-10-0-139-182.ec2.internal\" DevicePath \"\""
Sep 16 18:51:37.528334 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:51:37.528296    2584 generic.go:356] "Generic (PLEG): container finished" podID="d3189590-9978-421a-8aa1-629838b0236d" containerID="3ec9fae3c7ac9021a1e4cf53b45df78e9fbbaffd704a3982cda0215afacb8501" exitCode=0
Sep 16 18:51:37.528808 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:51:37.528342    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-f578fdbcb-w4mp7"
Sep 16 18:51:37.528808 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:51:37.528377    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-f578fdbcb-w4mp7" event={"ID":"d3189590-9978-421a-8aa1-629838b0236d","Type":"ContainerDied","Data":"3ec9fae3c7ac9021a1e4cf53b45df78e9fbbaffd704a3982cda0215afacb8501"}
Sep 16 18:51:37.528808 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:51:37.528430    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-f578fdbcb-w4mp7" event={"ID":"d3189590-9978-421a-8aa1-629838b0236d","Type":"ContainerDied","Data":"fd54a6d3cb8bdf3a34daa303ca9f699504757b64373597d6af5361d601fb4ba2"}
Sep 16 18:51:37.528808 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:51:37.528451    2584 scope.go:117] "RemoveContainer" containerID="3ec9fae3c7ac9021a1e4cf53b45df78e9fbbaffd704a3982cda0215afacb8501"
Sep 16 18:51:37.537107 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:51:37.537089    2584 scope.go:117] "RemoveContainer" containerID="3ec9fae3c7ac9021a1e4cf53b45df78e9fbbaffd704a3982cda0215afacb8501"
Sep 16 18:51:37.537386 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:51:37.537367    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3ec9fae3c7ac9021a1e4cf53b45df78e9fbbaffd704a3982cda0215afacb8501\": container with ID starting with 3ec9fae3c7ac9021a1e4cf53b45df78e9fbbaffd704a3982cda0215afacb8501 not found: ID does not exist" containerID="3ec9fae3c7ac9021a1e4cf53b45df78e9fbbaffd704a3982cda0215afacb8501"
Sep 16 18:51:37.537430 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:51:37.537393    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3ec9fae3c7ac9021a1e4cf53b45df78e9fbbaffd704a3982cda0215afacb8501"} err="failed to get container status \"3ec9fae3c7ac9021a1e4cf53b45df78e9fbbaffd704a3982cda0215afacb8501\": rpc error: code = NotFound desc = could not find container \"3ec9fae3c7ac9021a1e4cf53b45df78e9fbbaffd704a3982cda0215afacb8501\": container with ID starting with 3ec9fae3c7ac9021a1e4cf53b45df78e9fbbaffd704a3982cda0215afacb8501 not found: ID does not exist"
Sep 16 18:51:37.549756 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:51:37.549726    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-f578fdbcb-w4mp7"]
Sep 16 18:51:37.553543 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:51:37.553519    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-f578fdbcb-w4mp7"]
Sep 16 18:51:38.115378 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:51:38.115335    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d3189590-9978-421a-8aa1-629838b0236d" path="/var/lib/kubelet/pods/d3189590-9978-421a-8aa1-629838b0236d/volumes"
Sep 16 18:51:52.437950 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:51:52.437911    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:52:07.195842 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:52:07.195796    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:52:07.295796 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:52:07.295765    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:52:19.166198 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:52:19.166159    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:52:19.235826 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:52:19.235789    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:52:20.007461 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:52:20.007425    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7c845b4d74-lj9n5"]
Sep 16 18:52:20.008161 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:52:20.008129    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d3189590-9978-421a-8aa1-629838b0236d" containerName="console"
Sep 16 18:52:20.008161 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:52:20.008156    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="d3189590-9978-421a-8aa1-629838b0236d" containerName="console"
Sep 16 18:52:20.008374 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:52:20.008246    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="d3189590-9978-421a-8aa1-629838b0236d" containerName="console"
Sep 16 18:52:20.011383 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:52:20.011361    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c845b4d74-lj9n5"
Sep 16 18:52:20.026085 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:52:20.025999    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7c845b4d74-lj9n5"]
Sep 16 18:52:20.069564 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:52:20.069523    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:52:20.139172 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:52:20.139133    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0dd76b92-206d-42cf-a954-7ed474be9057-console-config\") pod \"console-7c845b4d74-lj9n5\" (UID: \"0dd76b92-206d-42cf-a954-7ed474be9057\") " pod="openshift-console/console-7c845b4d74-lj9n5"
Sep 16 18:52:20.139352 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:52:20.139187    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jtgmp\" (UniqueName: \"kubernetes.io/projected/0dd76b92-206d-42cf-a954-7ed474be9057-kube-api-access-jtgmp\") pod \"console-7c845b4d74-lj9n5\" (UID: \"0dd76b92-206d-42cf-a954-7ed474be9057\") " pod="openshift-console/console-7c845b4d74-lj9n5"
Sep 16 18:52:20.139352 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:52:20.139246    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0dd76b92-206d-42cf-a954-7ed474be9057-service-ca\") pod \"console-7c845b4d74-lj9n5\" (UID: \"0dd76b92-206d-42cf-a954-7ed474be9057\") " pod="openshift-console/console-7c845b4d74-lj9n5"
Sep 16 18:52:20.139352 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:52:20.139298    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0dd76b92-206d-42cf-a954-7ed474be9057-console-oauth-config\") pod \"console-7c845b4d74-lj9n5\" (UID: \"0dd76b92-206d-42cf-a954-7ed474be9057\") " pod="openshift-console/console-7c845b4d74-lj9n5"
Sep 16 18:52:20.139352 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:52:20.139325    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0dd76b92-206d-42cf-a954-7ed474be9057-oauth-serving-cert\") pod \"console-7c845b4d74-lj9n5\" (UID: \"0dd76b92-206d-42cf-a954-7ed474be9057\") " pod="openshift-console/console-7c845b4d74-lj9n5"
Sep 16 18:52:20.139527 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:52:20.139376    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0dd76b92-206d-42cf-a954-7ed474be9057-trusted-ca-bundle\") pod \"console-7c845b4d74-lj9n5\" (UID: \"0dd76b92-206d-42cf-a954-7ed474be9057\") " pod="openshift-console/console-7c845b4d74-lj9n5"
Sep 16 18:52:20.139527 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:52:20.139401    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0dd76b92-206d-42cf-a954-7ed474be9057-console-serving-cert\") pod \"console-7c845b4d74-lj9n5\" (UID: \"0dd76b92-206d-42cf-a954-7ed474be9057\") " pod="openshift-console/console-7c845b4d74-lj9n5"
Sep 16 18:52:20.169681 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:52:20.169628    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:52:20.240386 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:52:20.240351    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0dd76b92-206d-42cf-a954-7ed474be9057-trusted-ca-bundle\") pod \"console-7c845b4d74-lj9n5\" (UID: \"0dd76b92-206d-42cf-a954-7ed474be9057\") " pod="openshift-console/console-7c845b4d74-lj9n5"
Sep 16 18:52:20.240386 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:52:20.240392    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0dd76b92-206d-42cf-a954-7ed474be9057-console-serving-cert\") pod \"console-7c845b4d74-lj9n5\" (UID: \"0dd76b92-206d-42cf-a954-7ed474be9057\") " pod="openshift-console/console-7c845b4d74-lj9n5"
Sep 16 18:52:20.240604 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:52:20.240414    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0dd76b92-206d-42cf-a954-7ed474be9057-console-config\") pod \"console-7c845b4d74-lj9n5\" (UID: \"0dd76b92-206d-42cf-a954-7ed474be9057\") " pod="openshift-console/console-7c845b4d74-lj9n5"
Sep 16 18:52:20.240604 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:52:20.240452    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jtgmp\" (UniqueName: \"kubernetes.io/projected/0dd76b92-206d-42cf-a954-7ed474be9057-kube-api-access-jtgmp\") pod \"console-7c845b4d74-lj9n5\" (UID: \"0dd76b92-206d-42cf-a954-7ed474be9057\") " pod="openshift-console/console-7c845b4d74-lj9n5"
Sep 16 18:52:20.240604 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:52:20.240469    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0dd76b92-206d-42cf-a954-7ed474be9057-service-ca\") pod \"console-7c845b4d74-lj9n5\" (UID: \"0dd76b92-206d-42cf-a954-7ed474be9057\") " pod="openshift-console/console-7c845b4d74-lj9n5"
Sep 16 18:52:20.240604 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:52:20.240492    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0dd76b92-206d-42cf-a954-7ed474be9057-console-oauth-config\") pod \"console-7c845b4d74-lj9n5\" (UID: \"0dd76b92-206d-42cf-a954-7ed474be9057\") " pod="openshift-console/console-7c845b4d74-lj9n5"
Sep 16 18:52:20.240604 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:52:20.240509    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0dd76b92-206d-42cf-a954-7ed474be9057-oauth-serving-cert\") pod \"console-7c845b4d74-lj9n5\" (UID: \"0dd76b92-206d-42cf-a954-7ed474be9057\") " pod="openshift-console/console-7c845b4d74-lj9n5"
Sep 16 18:52:20.241243 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:52:20.241213    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0dd76b92-206d-42cf-a954-7ed474be9057-oauth-serving-cert\") pod \"console-7c845b4d74-lj9n5\" (UID: \"0dd76b92-206d-42cf-a954-7ed474be9057\") " pod="openshift-console/console-7c845b4d74-lj9n5"
Sep 16 18:52:20.241362 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:52:20.241337    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0dd76b92-206d-42cf-a954-7ed474be9057-console-config\") pod \"console-7c845b4d74-lj9n5\" (UID: \"0dd76b92-206d-42cf-a954-7ed474be9057\") " pod="openshift-console/console-7c845b4d74-lj9n5"
Sep 16 18:52:20.241422 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:52:20.241360    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0dd76b92-206d-42cf-a954-7ed474be9057-service-ca\") pod \"console-7c845b4d74-lj9n5\" (UID: \"0dd76b92-206d-42cf-a954-7ed474be9057\") " pod="openshift-console/console-7c845b4d74-lj9n5"
Sep 16 18:52:20.241485 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:52:20.241436    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0dd76b92-206d-42cf-a954-7ed474be9057-trusted-ca-bundle\") pod \"console-7c845b4d74-lj9n5\" (UID: \"0dd76b92-206d-42cf-a954-7ed474be9057\") " pod="openshift-console/console-7c845b4d74-lj9n5"
Sep 16 18:52:20.243153 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:52:20.243126    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0dd76b92-206d-42cf-a954-7ed474be9057-console-oauth-config\") pod \"console-7c845b4d74-lj9n5\" (UID: \"0dd76b92-206d-42cf-a954-7ed474be9057\") " pod="openshift-console/console-7c845b4d74-lj9n5"
Sep 16 18:52:20.243245 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:52:20.243133    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0dd76b92-206d-42cf-a954-7ed474be9057-console-serving-cert\") pod \"console-7c845b4d74-lj9n5\" (UID: \"0dd76b92-206d-42cf-a954-7ed474be9057\") " pod="openshift-console/console-7c845b4d74-lj9n5"
Sep 16 18:52:20.258056 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:52:20.257999    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jtgmp\" (UniqueName: \"kubernetes.io/projected/0dd76b92-206d-42cf-a954-7ed474be9057-kube-api-access-jtgmp\") pod \"console-7c845b4d74-lj9n5\" (UID: \"0dd76b92-206d-42cf-a954-7ed474be9057\") " pod="openshift-console/console-7c845b4d74-lj9n5"
Sep 16 18:52:20.321142 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:52:20.321111    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c845b4d74-lj9n5"
Sep 16 18:52:20.591804 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:52:20.591777    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7c845b4d74-lj9n5"]
Sep 16 18:52:20.594206 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:52:20.594171    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0dd76b92_206d_42cf_a954_7ed474be9057.slice/crio-379012025ce7f12eddeb90a05f1d4231d11b50cd31adde8adc4d48a49398c1ef WatchSource:0}: Error finding container 379012025ce7f12eddeb90a05f1d4231d11b50cd31adde8adc4d48a49398c1ef: Status 404 returned error can't find the container with id 379012025ce7f12eddeb90a05f1d4231d11b50cd31adde8adc4d48a49398c1ef
Sep 16 18:52:20.692262 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:52:20.692219    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c845b4d74-lj9n5" event={"ID":"0dd76b92-206d-42cf-a954-7ed474be9057","Type":"ContainerStarted","Data":"05d1ef0f2319e6131fe8d117c7b037615d0b687548d9140e73abc25080ba4a13"}
Sep 16 18:52:20.692479 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:52:20.692277    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c845b4d74-lj9n5" event={"ID":"0dd76b92-206d-42cf-a954-7ed474be9057","Type":"ContainerStarted","Data":"379012025ce7f12eddeb90a05f1d4231d11b50cd31adde8adc4d48a49398c1ef"}
Sep 16 18:52:20.712159 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:52:20.712105    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7c845b4d74-lj9n5" podStartSLOduration=1.7120826710000001 podStartE2EDuration="1.712082671s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:52:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:52:20.710935231 +0000 UTC m=+471.040437656" watchObservedRunningTime="2026-09-16 18:52:20.712082671 +0000 UTC m=+471.041585083"
Sep 16 18:52:30.321780 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:52:30.321681    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7c845b4d74-lj9n5"
Sep 16 18:52:30.321780 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:52:30.321752    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7c845b4d74-lj9n5"
Sep 16 18:52:30.326602 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:52:30.326578    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7c845b4d74-lj9n5"
Sep 16 18:52:30.732991 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:52:30.732911    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7c845b4d74-lj9n5"
Sep 16 18:52:30.781752 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:52:30.781716    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-84d5ff7d7-ks2c5"]
Sep 16 18:52:30.903542 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:52:30.903511    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:52:31.005629 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:52:31.005544    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:52:31.104254 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:52:31.104224    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:52:36.296305 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:52:36.296267    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:52:36.396464 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:52:36.396428    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:52:36.464356 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:52:36.464319    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:52:55.803509 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:52:55.803435    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-84d5ff7d7-ks2c5" podUID="158d7548-0988-4248-aafd-eb56eb02fc52" containerName="console" containerID="cri-o://4a1016451bc4cbf85fdb8ee357271083999ea54486e0f7a6c6848320bd857585" gracePeriod=15
Sep 16 18:52:56.042664 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:52:56.042637    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-84d5ff7d7-ks2c5"
Sep 16 18:52:56.225079 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:52:56.222755    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jhhdn\" (UniqueName: \"kubernetes.io/projected/158d7548-0988-4248-aafd-eb56eb02fc52-kube-api-access-jhhdn\") pod \"158d7548-0988-4248-aafd-eb56eb02fc52\" (UID: \"158d7548-0988-4248-aafd-eb56eb02fc52\") "
Sep 16 18:52:56.225079 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:52:56.222826    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/158d7548-0988-4248-aafd-eb56eb02fc52-trusted-ca-bundle\") pod \"158d7548-0988-4248-aafd-eb56eb02fc52\" (UID: \"158d7548-0988-4248-aafd-eb56eb02fc52\") "
Sep 16 18:52:56.225079 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:52:56.222939    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/158d7548-0988-4248-aafd-eb56eb02fc52-console-config\") pod \"158d7548-0988-4248-aafd-eb56eb02fc52\" (UID: \"158d7548-0988-4248-aafd-eb56eb02fc52\") "
Sep 16 18:52:56.225079 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:52:56.222981    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/158d7548-0988-4248-aafd-eb56eb02fc52-service-ca\") pod \"158d7548-0988-4248-aafd-eb56eb02fc52\" (UID: \"158d7548-0988-4248-aafd-eb56eb02fc52\") "
Sep 16 18:52:56.225079 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:52:56.223037    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/158d7548-0988-4248-aafd-eb56eb02fc52-console-oauth-config\") pod \"158d7548-0988-4248-aafd-eb56eb02fc52\" (UID: \"158d7548-0988-4248-aafd-eb56eb02fc52\") "
Sep 16 18:52:56.225079 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:52:56.223070    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/158d7548-0988-4248-aafd-eb56eb02fc52-console-serving-cert\") pod \"158d7548-0988-4248-aafd-eb56eb02fc52\" (UID: \"158d7548-0988-4248-aafd-eb56eb02fc52\") "
Sep 16 18:52:56.225079 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:52:56.223098    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/158d7548-0988-4248-aafd-eb56eb02fc52-oauth-serving-cert\") pod \"158d7548-0988-4248-aafd-eb56eb02fc52\" (UID: \"158d7548-0988-4248-aafd-eb56eb02fc52\") "
Sep 16 18:52:56.225079 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:52:56.224127    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/158d7548-0988-4248-aafd-eb56eb02fc52-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "158d7548-0988-4248-aafd-eb56eb02fc52" (UID: "158d7548-0988-4248-aafd-eb56eb02fc52"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:52:56.225079 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:52:56.224447    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/158d7548-0988-4248-aafd-eb56eb02fc52-console-config" (OuterVolumeSpecName: "console-config") pod "158d7548-0988-4248-aafd-eb56eb02fc52" (UID: "158d7548-0988-4248-aafd-eb56eb02fc52"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:52:56.225079 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:52:56.224751    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/158d7548-0988-4248-aafd-eb56eb02fc52-service-ca" (OuterVolumeSpecName: "service-ca") pod "158d7548-0988-4248-aafd-eb56eb02fc52" (UID: "158d7548-0988-4248-aafd-eb56eb02fc52"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:52:56.225712 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:52:56.225407    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/158d7548-0988-4248-aafd-eb56eb02fc52-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "158d7548-0988-4248-aafd-eb56eb02fc52" (UID: "158d7548-0988-4248-aafd-eb56eb02fc52"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:52:56.228391 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:52:56.226164    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/158d7548-0988-4248-aafd-eb56eb02fc52-kube-api-access-jhhdn" (OuterVolumeSpecName: "kube-api-access-jhhdn") pod "158d7548-0988-4248-aafd-eb56eb02fc52" (UID: "158d7548-0988-4248-aafd-eb56eb02fc52"). InnerVolumeSpecName "kube-api-access-jhhdn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:52:56.229868 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:52:56.229825    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/158d7548-0988-4248-aafd-eb56eb02fc52-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "158d7548-0988-4248-aafd-eb56eb02fc52" (UID: "158d7548-0988-4248-aafd-eb56eb02fc52"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:52:56.230007 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:52:56.229982    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/158d7548-0988-4248-aafd-eb56eb02fc52-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "158d7548-0988-4248-aafd-eb56eb02fc52" (UID: "158d7548-0988-4248-aafd-eb56eb02fc52"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:52:56.324174 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:52:56.324137    2584 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/158d7548-0988-4248-aafd-eb56eb02fc52-service-ca\") on node \"ip-10-0-139-182.ec2.internal\" DevicePath \"\""
Sep 16 18:52:56.324174 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:52:56.324171    2584 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/158d7548-0988-4248-aafd-eb56eb02fc52-console-oauth-config\") on node \"ip-10-0-139-182.ec2.internal\" DevicePath \"\""
Sep 16 18:52:56.324376 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:52:56.324185    2584 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/158d7548-0988-4248-aafd-eb56eb02fc52-console-serving-cert\") on node \"ip-10-0-139-182.ec2.internal\" DevicePath \"\""
Sep 16 18:52:56.324376 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:52:56.324198    2584 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/158d7548-0988-4248-aafd-eb56eb02fc52-oauth-serving-cert\") on node \"ip-10-0-139-182.ec2.internal\" DevicePath \"\""
Sep 16 18:52:56.324376 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:52:56.324212    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jhhdn\" (UniqueName: \"kubernetes.io/projected/158d7548-0988-4248-aafd-eb56eb02fc52-kube-api-access-jhhdn\") on node \"ip-10-0-139-182.ec2.internal\" DevicePath \"\""
Sep 16 18:52:56.324376 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:52:56.324223    2584 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/158d7548-0988-4248-aafd-eb56eb02fc52-trusted-ca-bundle\") on node \"ip-10-0-139-182.ec2.internal\" DevicePath \"\""
Sep 16 18:52:56.324376 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:52:56.324233    2584 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/158d7548-0988-4248-aafd-eb56eb02fc52-console-config\") on node \"ip-10-0-139-182.ec2.internal\" DevicePath \"\""
Sep 16 18:52:56.822437 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:52:56.822405    2584 generic.go:356] "Generic (PLEG): container finished" podID="158d7548-0988-4248-aafd-eb56eb02fc52" containerID="4a1016451bc4cbf85fdb8ee357271083999ea54486e0f7a6c6848320bd857585" exitCode=0
Sep 16 18:52:56.822882 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:52:56.822461    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-84d5ff7d7-ks2c5"
Sep 16 18:52:56.822882 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:52:56.822490    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-84d5ff7d7-ks2c5" event={"ID":"158d7548-0988-4248-aafd-eb56eb02fc52","Type":"ContainerDied","Data":"4a1016451bc4cbf85fdb8ee357271083999ea54486e0f7a6c6848320bd857585"}
Sep 16 18:52:56.822882 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:52:56.822566    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-84d5ff7d7-ks2c5" event={"ID":"158d7548-0988-4248-aafd-eb56eb02fc52","Type":"ContainerDied","Data":"0f9e8bd468e667868ae77ef6620fc99294002f3595612508feba28d10fbb29d0"}
Sep 16 18:52:56.822882 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:52:56.822587    2584 scope.go:117] "RemoveContainer" containerID="4a1016451bc4cbf85fdb8ee357271083999ea54486e0f7a6c6848320bd857585"
Sep 16 18:52:56.831088 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:52:56.831073    2584 scope.go:117] "RemoveContainer" containerID="4a1016451bc4cbf85fdb8ee357271083999ea54486e0f7a6c6848320bd857585"
Sep 16 18:52:56.831403 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:52:56.831383    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4a1016451bc4cbf85fdb8ee357271083999ea54486e0f7a6c6848320bd857585\": container with ID starting with 4a1016451bc4cbf85fdb8ee357271083999ea54486e0f7a6c6848320bd857585 not found: ID does not exist" containerID="4a1016451bc4cbf85fdb8ee357271083999ea54486e0f7a6c6848320bd857585"
Sep 16 18:52:56.831481 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:52:56.831416    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4a1016451bc4cbf85fdb8ee357271083999ea54486e0f7a6c6848320bd857585"} err="failed to get container status \"4a1016451bc4cbf85fdb8ee357271083999ea54486e0f7a6c6848320bd857585\": rpc error: code = NotFound desc = could not find container \"4a1016451bc4cbf85fdb8ee357271083999ea54486e0f7a6c6848320bd857585\": container with ID starting with 4a1016451bc4cbf85fdb8ee357271083999ea54486e0f7a6c6848320bd857585 not found: ID does not exist"
Sep 16 18:52:56.843804 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:52:56.843770    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-84d5ff7d7-ks2c5"]
Sep 16 18:52:56.848143 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:52:56.848119    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-84d5ff7d7-ks2c5"]
Sep 16 18:52:58.115316 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:52:58.115279    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="158d7548-0988-4248-aafd-eb56eb02fc52" path="/var/lib/kubelet/pods/158d7548-0988-4248-aafd-eb56eb02fc52/volumes"
Sep 16 18:53:43.933317 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:53:43.933284    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:54:30.065182 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:54:30.065150    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-crjw7_ecf273b5-b465-4008-b035-f7ac0e740d58/console-operator/1.log"
Sep 16 18:54:30.066348 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:54:30.066325    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-crjw7_ecf273b5-b465-4008-b035-f7ac0e740d58/console-operator/1.log"
Sep 16 18:54:31.809541 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:54:31.809432    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:54:32.837236 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:54:32.837201    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1a6fmj4"]
Sep 16 18:54:32.837593 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:54:32.837456    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="158d7548-0988-4248-aafd-eb56eb02fc52" containerName="console"
Sep 16 18:54:32.837593 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:54:32.837466    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="158d7548-0988-4248-aafd-eb56eb02fc52" containerName="console"
Sep 16 18:54:32.837593 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:54:32.837521    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="158d7548-0988-4248-aafd-eb56eb02fc52" containerName="console"
Sep 16 18:54:32.840448 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:54:32.840427    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1a6fmj4"
Sep 16 18:54:32.843266 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:54:32.843247    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 16 18:54:32.844617 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:54:32.844600    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 16 18:54:32.844702 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:54:32.844619    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-c7g5n\""
Sep 16 18:54:32.857204 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:54:32.857175    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1a6fmj4"]
Sep 16 18:54:32.866142 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:54:32.866115    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/586200cf-4d2a-400c-bd0c-a399ffb63afe-bundle\") pod \"1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1a6fmj4\" (UID: \"586200cf-4d2a-400c-bd0c-a399ffb63afe\") " pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1a6fmj4"
Sep 16 18:54:32.866245 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:54:32.866150    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/586200cf-4d2a-400c-bd0c-a399ffb63afe-util\") pod \"1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1a6fmj4\" (UID: \"586200cf-4d2a-400c-bd0c-a399ffb63afe\") " pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1a6fmj4"
Sep 16 18:54:32.866245 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:54:32.866174    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t6ngc\" (UniqueName: \"kubernetes.io/projected/586200cf-4d2a-400c-bd0c-a399ffb63afe-kube-api-access-t6ngc\") pod \"1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1a6fmj4\" (UID: \"586200cf-4d2a-400c-bd0c-a399ffb63afe\") " pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1a6fmj4"
Sep 16 18:54:32.870251 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:54:32.870231    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:54:32.967304 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:54:32.967269    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/586200cf-4d2a-400c-bd0c-a399ffb63afe-bundle\") pod \"1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1a6fmj4\" (UID: \"586200cf-4d2a-400c-bd0c-a399ffb63afe\") " pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1a6fmj4"
Sep 16 18:54:32.967441 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:54:32.967311    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/586200cf-4d2a-400c-bd0c-a399ffb63afe-util\") pod \"1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1a6fmj4\" (UID: \"586200cf-4d2a-400c-bd0c-a399ffb63afe\") " pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1a6fmj4"
Sep 16 18:54:32.967441 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:54:32.967344    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t6ngc\" (UniqueName: \"kubernetes.io/projected/586200cf-4d2a-400c-bd0c-a399ffb63afe-kube-api-access-t6ngc\") pod \"1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1a6fmj4\" (UID: \"586200cf-4d2a-400c-bd0c-a399ffb63afe\") " pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1a6fmj4"
Sep 16 18:54:32.967731 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:54:32.967710    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/586200cf-4d2a-400c-bd0c-a399ffb63afe-util\") pod \"1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1a6fmj4\" (UID: \"586200cf-4d2a-400c-bd0c-a399ffb63afe\") " pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1a6fmj4"
Sep 16 18:54:32.967774 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:54:32.967735    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/586200cf-4d2a-400c-bd0c-a399ffb63afe-bundle\") pod \"1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1a6fmj4\" (UID: \"586200cf-4d2a-400c-bd0c-a399ffb63afe\") " pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1a6fmj4"
Sep 16 18:54:32.976041 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:54:32.976018    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t6ngc\" (UniqueName: \"kubernetes.io/projected/586200cf-4d2a-400c-bd0c-a399ffb63afe-kube-api-access-t6ngc\") pod \"1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1a6fmj4\" (UID: \"586200cf-4d2a-400c-bd0c-a399ffb63afe\") " pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1a6fmj4"
Sep 16 18:54:33.149252 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:54:33.149150    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1a6fmj4"
Sep 16 18:54:33.418996 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:54:33.418926    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1a6fmj4"]
Sep 16 18:54:33.422579 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:54:33.422551    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod586200cf_4d2a_400c_bd0c_a399ffb63afe.slice/crio-d3aa7bfe3ac487c14658d3b544f8f1e8f1d17fde3ba13fd1f2286ff4c6f7d938 WatchSource:0}: Error finding container d3aa7bfe3ac487c14658d3b544f8f1e8f1d17fde3ba13fd1f2286ff4c6f7d938: Status 404 returned error can't find the container with id d3aa7bfe3ac487c14658d3b544f8f1e8f1d17fde3ba13fd1f2286ff4c6f7d938
Sep 16 18:54:33.425061 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:54:33.425041    2584 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 16 18:54:34.184286 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:54:34.184236    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1a6fmj4" event={"ID":"586200cf-4d2a-400c-bd0c-a399ffb63afe","Type":"ContainerStarted","Data":"d3aa7bfe3ac487c14658d3b544f8f1e8f1d17fde3ba13fd1f2286ff4c6f7d938"}
Sep 16 18:54:36.833763 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:54:36.833722    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:54:40.215048 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:54:40.215011    2584 generic.go:356] "Generic (PLEG): container finished" podID="586200cf-4d2a-400c-bd0c-a399ffb63afe" containerID="2b81c9817041c38ec3ab561a62c174925ebc9c4e254596eb1c1b5fedeee22770" exitCode=0
Sep 16 18:54:40.215433 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:54:40.215099    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1a6fmj4" event={"ID":"586200cf-4d2a-400c-bd0c-a399ffb63afe","Type":"ContainerDied","Data":"2b81c9817041c38ec3ab561a62c174925ebc9c4e254596eb1c1b5fedeee22770"}
Sep 16 18:54:43.228874 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:54:43.228776    2584 generic.go:356] "Generic (PLEG): container finished" podID="586200cf-4d2a-400c-bd0c-a399ffb63afe" containerID="d1806b57ae942532c12a6dc8426044b4663bb05871aef53d8ad8085f072c99ca" exitCode=0
Sep 16 18:54:43.229214 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:54:43.228866    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1a6fmj4" event={"ID":"586200cf-4d2a-400c-bd0c-a399ffb63afe","Type":"ContainerDied","Data":"d1806b57ae942532c12a6dc8426044b4663bb05871aef53d8ad8085f072c99ca"}
Sep 16 18:54:50.258191 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:54:50.258158    2584 generic.go:356] "Generic (PLEG): container finished" podID="586200cf-4d2a-400c-bd0c-a399ffb63afe" containerID="b61d2d8a42be88309a456474c441d19ed531d2b17e8b0c0a0894029d83a2a501" exitCode=0
Sep 16 18:54:50.258552 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:54:50.258248    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1a6fmj4" event={"ID":"586200cf-4d2a-400c-bd0c-a399ffb63afe","Type":"ContainerDied","Data":"b61d2d8a42be88309a456474c441d19ed531d2b17e8b0c0a0894029d83a2a501"}
Sep 16 18:54:51.384876 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:54:51.384855    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1a6fmj4"
Sep 16 18:54:51.522154 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:54:51.522083    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/586200cf-4d2a-400c-bd0c-a399ffb63afe-util\") pod \"586200cf-4d2a-400c-bd0c-a399ffb63afe\" (UID: \"586200cf-4d2a-400c-bd0c-a399ffb63afe\") "
Sep 16 18:54:51.522154 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:54:51.522144    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/586200cf-4d2a-400c-bd0c-a399ffb63afe-bundle\") pod \"586200cf-4d2a-400c-bd0c-a399ffb63afe\" (UID: \"586200cf-4d2a-400c-bd0c-a399ffb63afe\") "
Sep 16 18:54:51.522324 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:54:51.522179    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t6ngc\" (UniqueName: \"kubernetes.io/projected/586200cf-4d2a-400c-bd0c-a399ffb63afe-kube-api-access-t6ngc\") pod \"586200cf-4d2a-400c-bd0c-a399ffb63afe\" (UID: \"586200cf-4d2a-400c-bd0c-a399ffb63afe\") "
Sep 16 18:54:51.522776 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:54:51.522741    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/586200cf-4d2a-400c-bd0c-a399ffb63afe-bundle" (OuterVolumeSpecName: "bundle") pod "586200cf-4d2a-400c-bd0c-a399ffb63afe" (UID: "586200cf-4d2a-400c-bd0c-a399ffb63afe"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:54:51.524484 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:54:51.524454    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/586200cf-4d2a-400c-bd0c-a399ffb63afe-kube-api-access-t6ngc" (OuterVolumeSpecName: "kube-api-access-t6ngc") pod "586200cf-4d2a-400c-bd0c-a399ffb63afe" (UID: "586200cf-4d2a-400c-bd0c-a399ffb63afe"). InnerVolumeSpecName "kube-api-access-t6ngc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:54:51.528877 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:54:51.526981    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/586200cf-4d2a-400c-bd0c-a399ffb63afe-util" (OuterVolumeSpecName: "util") pod "586200cf-4d2a-400c-bd0c-a399ffb63afe" (UID: "586200cf-4d2a-400c-bd0c-a399ffb63afe"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:54:51.622746 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:54:51.622711    2584 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/586200cf-4d2a-400c-bd0c-a399ffb63afe-bundle\") on node \"ip-10-0-139-182.ec2.internal\" DevicePath \"\""
Sep 16 18:54:51.622918 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:54:51.622761    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t6ngc\" (UniqueName: \"kubernetes.io/projected/586200cf-4d2a-400c-bd0c-a399ffb63afe-kube-api-access-t6ngc\") on node \"ip-10-0-139-182.ec2.internal\" DevicePath \"\""
Sep 16 18:54:51.622918 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:54:51.622770    2584 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/586200cf-4d2a-400c-bd0c-a399ffb63afe-util\") on node \"ip-10-0-139-182.ec2.internal\" DevicePath \"\""
Sep 16 18:54:52.266922 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:54:52.266880    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1a6fmj4" event={"ID":"586200cf-4d2a-400c-bd0c-a399ffb63afe","Type":"ContainerDied","Data":"d3aa7bfe3ac487c14658d3b544f8f1e8f1d17fde3ba13fd1f2286ff4c6f7d938"}
Sep 16 18:54:52.267097 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:54:52.266943    2584 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d3aa7bfe3ac487c14658d3b544f8f1e8f1d17fde3ba13fd1f2286ff4c6f7d938"
Sep 16 18:54:52.267097 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:54:52.266961    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1a6fmj4"
Sep 16 18:54:54.213519 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:54:54.213407    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:54:54.343253 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:54:54.343205    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:55:00.368646 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:55:00.368609    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:55:00.648630 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:55:00.648550    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:55:00.655369 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:55:00.655343    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-operator-579c8958f7-gv7bx"]
Sep 16 18:55:00.655612 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:55:00.655600    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="586200cf-4d2a-400c-bd0c-a399ffb63afe" containerName="extract"
Sep 16 18:55:00.655668 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:55:00.655613    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="586200cf-4d2a-400c-bd0c-a399ffb63afe" containerName="extract"
Sep 16 18:55:00.655668 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:55:00.655629    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="586200cf-4d2a-400c-bd0c-a399ffb63afe" containerName="util"
Sep 16 18:55:00.655668 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:55:00.655634    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="586200cf-4d2a-400c-bd0c-a399ffb63afe" containerName="util"
Sep 16 18:55:00.655668 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:55:00.655651    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="586200cf-4d2a-400c-bd0c-a399ffb63afe" containerName="pull"
Sep 16 18:55:00.655668 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:55:00.655656    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="586200cf-4d2a-400c-bd0c-a399ffb63afe" containerName="pull"
Sep 16 18:55:00.655843 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:55:00.655698    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="586200cf-4d2a-400c-bd0c-a399ffb63afe" containerName="extract"
Sep 16 18:55:00.658493 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:55:00.658478    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-operator-579c8958f7-gv7bx"
Sep 16 18:55:00.661437 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:55:00.661416    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-operator-certs\""
Sep 16 18:55:00.661560 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:55:00.661419    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"kedaorg-certs\""
Sep 16 18:55:00.661710 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:55:00.661696    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"openshift-service-ca.crt\""
Sep 16 18:55:00.662676 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:55:00.662648    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"keda-ocp-cabundle\""
Sep 16 18:55:00.662767 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:55:00.662645    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-operator-dockercfg-7jlxm\""
Sep 16 18:55:00.662767 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:55:00.662725    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"kube-root-ca.crt\""
Sep 16 18:55:00.667528 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:55:00.667507    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-operator-579c8958f7-gv7bx"]
Sep 16 18:55:00.800158 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:55:00.800126    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5vlws\" (UniqueName: \"kubernetes.io/projected/7916c050-3248-46f8-b607-3655e271829e-kube-api-access-5vlws\") pod \"keda-operator-579c8958f7-gv7bx\" (UID: \"7916c050-3248-46f8-b607-3655e271829e\") " pod="openshift-keda/keda-operator-579c8958f7-gv7bx"
Sep 16 18:55:00.800383 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:55:00.800161    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/7916c050-3248-46f8-b607-3655e271829e-cabundle0\") pod \"keda-operator-579c8958f7-gv7bx\" (UID: \"7916c050-3248-46f8-b607-3655e271829e\") " pod="openshift-keda/keda-operator-579c8958f7-gv7bx"
Sep 16 18:55:00.800383 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:55:00.800249    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/7916c050-3248-46f8-b607-3655e271829e-certificates\") pod \"keda-operator-579c8958f7-gv7bx\" (UID: \"7916c050-3248-46f8-b607-3655e271829e\") " pod="openshift-keda/keda-operator-579c8958f7-gv7bx"
Sep 16 18:55:00.846576 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:55:00.846542    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:55:00.854249 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:55:00.854221    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-metrics-apiserver-77857b7b98-k2hjs"]
Sep 16 18:55:00.857063 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:55:00.857047    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-metrics-apiserver-77857b7b98-k2hjs"
Sep 16 18:55:00.859841 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:55:00.859817    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-metrics-apiserver-certs\""
Sep 16 18:55:00.868347 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:55:00.868321    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-metrics-apiserver-77857b7b98-k2hjs"]
Sep 16 18:55:00.901077 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:55:00.900998    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5vlws\" (UniqueName: \"kubernetes.io/projected/7916c050-3248-46f8-b607-3655e271829e-kube-api-access-5vlws\") pod \"keda-operator-579c8958f7-gv7bx\" (UID: \"7916c050-3248-46f8-b607-3655e271829e\") " pod="openshift-keda/keda-operator-579c8958f7-gv7bx"
Sep 16 18:55:00.901077 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:55:00.901035    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/7916c050-3248-46f8-b607-3655e271829e-cabundle0\") pod \"keda-operator-579c8958f7-gv7bx\" (UID: \"7916c050-3248-46f8-b607-3655e271829e\") " pod="openshift-keda/keda-operator-579c8958f7-gv7bx"
Sep 16 18:55:00.901077 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:55:00.901070    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/7916c050-3248-46f8-b607-3655e271829e-certificates\") pod \"keda-operator-579c8958f7-gv7bx\" (UID: \"7916c050-3248-46f8-b607-3655e271829e\") " pod="openshift-keda/keda-operator-579c8958f7-gv7bx"
Sep 16 18:55:00.901317 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:55:00.901196    2584 secret.go:281] references non-existent secret key: ca.crt
Sep 16 18:55:00.901317 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:55:00.901212    2584 projected.go:279] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: ca.crt
Sep 16 18:55:00.901317 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:55:00.901222    2584 projected.go:196] Error preparing data for projected volume certificates for pod openshift-keda/keda-operator-579c8958f7-gv7bx: references non-existent secret key: ca.crt
Sep 16 18:55:00.901317 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:55:00.901294    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7916c050-3248-46f8-b607-3655e271829e-certificates podName:7916c050-3248-46f8-b607-3655e271829e nodeName:}" failed. No retries permitted until 2026-09-16 18:55:01.401273645 +0000 UTC m=+631.730776037 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/7916c050-3248-46f8-b607-3655e271829e-certificates") pod "keda-operator-579c8958f7-gv7bx" (UID: "7916c050-3248-46f8-b607-3655e271829e") : references non-existent secret key: ca.crt
Sep 16 18:55:00.901719 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:55:00.901702    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/7916c050-3248-46f8-b607-3655e271829e-cabundle0\") pod \"keda-operator-579c8958f7-gv7bx\" (UID: \"7916c050-3248-46f8-b607-3655e271829e\") " pod="openshift-keda/keda-operator-579c8958f7-gv7bx"
Sep 16 18:55:00.908498 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:55:00.908459    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:55:00.910025 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:55:00.910003    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5vlws\" (UniqueName: \"kubernetes.io/projected/7916c050-3248-46f8-b607-3655e271829e-kube-api-access-5vlws\") pod \"keda-operator-579c8958f7-gv7bx\" (UID: \"7916c050-3248-46f8-b607-3655e271829e\") " pod="openshift-keda/keda-operator-579c8958f7-gv7bx"
Sep 16 18:55:01.002163 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:55:01.002132    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/26f53c11-5eb4-4bfa-aeca-9b55fa0d30e6-temp-vol\") pod \"keda-metrics-apiserver-77857b7b98-k2hjs\" (UID: \"26f53c11-5eb4-4bfa-aeca-9b55fa0d30e6\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-k2hjs"
Sep 16 18:55:01.002333 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:55:01.002179    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j6nv9\" (UniqueName: \"kubernetes.io/projected/26f53c11-5eb4-4bfa-aeca-9b55fa0d30e6-kube-api-access-j6nv9\") pod \"keda-metrics-apiserver-77857b7b98-k2hjs\" (UID: \"26f53c11-5eb4-4bfa-aeca-9b55fa0d30e6\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-k2hjs"
Sep 16 18:55:01.002333 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:55:01.002302    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/26f53c11-5eb4-4bfa-aeca-9b55fa0d30e6-certificates\") pod \"keda-metrics-apiserver-77857b7b98-k2hjs\" (UID: \"26f53c11-5eb4-4bfa-aeca-9b55fa0d30e6\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-k2hjs"
Sep 16 18:55:01.091050 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:55:01.091016    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:55:01.103139 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:55:01.103100    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/26f53c11-5eb4-4bfa-aeca-9b55fa0d30e6-certificates\") pod \"keda-metrics-apiserver-77857b7b98-k2hjs\" (UID: \"26f53c11-5eb4-4bfa-aeca-9b55fa0d30e6\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-k2hjs"
Sep 16 18:55:01.103258 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:55:01.103172    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/26f53c11-5eb4-4bfa-aeca-9b55fa0d30e6-temp-vol\") pod \"keda-metrics-apiserver-77857b7b98-k2hjs\" (UID: \"26f53c11-5eb4-4bfa-aeca-9b55fa0d30e6\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-k2hjs"
Sep 16 18:55:01.103258 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:55:01.103203    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j6nv9\" (UniqueName: \"kubernetes.io/projected/26f53c11-5eb4-4bfa-aeca-9b55fa0d30e6-kube-api-access-j6nv9\") pod \"keda-metrics-apiserver-77857b7b98-k2hjs\" (UID: \"26f53c11-5eb4-4bfa-aeca-9b55fa0d30e6\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-k2hjs"
Sep 16 18:55:01.103258 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:55:01.103248    2584 secret.go:281] references non-existent secret key: tls.crt
Sep 16 18:55:01.103367 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:55:01.103270    2584 projected.go:279] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: tls.crt
Sep 16 18:55:01.103367 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:55:01.103290    2584 projected.go:196] Error preparing data for projected volume certificates for pod openshift-keda/keda-metrics-apiserver-77857b7b98-k2hjs: references non-existent secret key: tls.crt
Sep 16 18:55:01.103367 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:55:01.103358    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/26f53c11-5eb4-4bfa-aeca-9b55fa0d30e6-certificates podName:26f53c11-5eb4-4bfa-aeca-9b55fa0d30e6 nodeName:}" failed. No retries permitted until 2026-09-16 18:55:01.603340625 +0000 UTC m=+631.932843020 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/26f53c11-5eb4-4bfa-aeca-9b55fa0d30e6-certificates") pod "keda-metrics-apiserver-77857b7b98-k2hjs" (UID: "26f53c11-5eb4-4bfa-aeca-9b55fa0d30e6") : references non-existent secret key: tls.crt
Sep 16 18:55:01.103631 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:55:01.103614    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/26f53c11-5eb4-4bfa-aeca-9b55fa0d30e6-temp-vol\") pod \"keda-metrics-apiserver-77857b7b98-k2hjs\" (UID: \"26f53c11-5eb4-4bfa-aeca-9b55fa0d30e6\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-k2hjs"
Sep 16 18:55:01.113423 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:55:01.113400    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j6nv9\" (UniqueName: \"kubernetes.io/projected/26f53c11-5eb4-4bfa-aeca-9b55fa0d30e6-kube-api-access-j6nv9\") pod \"keda-metrics-apiserver-77857b7b98-k2hjs\" (UID: \"26f53c11-5eb4-4bfa-aeca-9b55fa0d30e6\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-k2hjs"
Sep 16 18:55:01.238577 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:55:01.238489    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:55:01.406000 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:55:01.405963    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/7916c050-3248-46f8-b607-3655e271829e-certificates\") pod \"keda-operator-579c8958f7-gv7bx\" (UID: \"7916c050-3248-46f8-b607-3655e271829e\") " pod="openshift-keda/keda-operator-579c8958f7-gv7bx"
Sep 16 18:55:01.408357 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:55:01.408333    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/7916c050-3248-46f8-b607-3655e271829e-certificates\") pod \"keda-operator-579c8958f7-gv7bx\" (UID: \"7916c050-3248-46f8-b607-3655e271829e\") " pod="openshift-keda/keda-operator-579c8958f7-gv7bx"
Sep 16 18:55:01.454113 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:55:01.454077    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:55:01.559163 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:55:01.559130    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:55:01.568961 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:55:01.568933    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-operator-579c8958f7-gv7bx"
Sep 16 18:55:01.608210 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:55:01.608176    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/26f53c11-5eb4-4bfa-aeca-9b55fa0d30e6-certificates\") pod \"keda-metrics-apiserver-77857b7b98-k2hjs\" (UID: \"26f53c11-5eb4-4bfa-aeca-9b55fa0d30e6\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-k2hjs"
Sep 16 18:55:01.610626 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:55:01.610597    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/26f53c11-5eb4-4bfa-aeca-9b55fa0d30e6-certificates\") pod \"keda-metrics-apiserver-77857b7b98-k2hjs\" (UID: \"26f53c11-5eb4-4bfa-aeca-9b55fa0d30e6\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-k2hjs"
Sep 16 18:55:01.766515 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:55:01.766478    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-metrics-apiserver-77857b7b98-k2hjs"
Sep 16 18:55:01.840189 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:55:01.840164    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-operator-579c8958f7-gv7bx"]
Sep 16 18:55:01.842200 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:55:01.842172    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7916c050_3248_46f8_b607_3655e271829e.slice/crio-5ebffe2b96ad53bf0a0a1931cfb120465952498e13ed868c21b1617554bd287a WatchSource:0}: Error finding container 5ebffe2b96ad53bf0a0a1931cfb120465952498e13ed868c21b1617554bd287a: Status 404 returned error can't find the container with id 5ebffe2b96ad53bf0a0a1931cfb120465952498e13ed868c21b1617554bd287a
Sep 16 18:55:01.880546 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:55:01.880516    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:55:02.039173 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:55:02.039138    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-metrics-apiserver-77857b7b98-k2hjs"]
Sep 16 18:55:02.042318 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:55:02.042291    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26f53c11_5eb4_4bfa_aeca_9b55fa0d30e6.slice/crio-26f191a2b9e78c4685e564a6fc63efae7451027f81699c8396fe50d1d76fe4d9 WatchSource:0}: Error finding container 26f191a2b9e78c4685e564a6fc63efae7451027f81699c8396fe50d1d76fe4d9: Status 404 returned error can't find the container with id 26f191a2b9e78c4685e564a6fc63efae7451027f81699c8396fe50d1d76fe4d9
Sep 16 18:55:02.086117 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:55:02.086033    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:55:02.198440 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:55:02.198408    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:55:02.296093 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:55:02.296062    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:55:02.304040 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:55:02.304011    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-operator-579c8958f7-gv7bx" event={"ID":"7916c050-3248-46f8-b607-3655e271829e","Type":"ContainerStarted","Data":"5ebffe2b96ad53bf0a0a1931cfb120465952498e13ed868c21b1617554bd287a"}
Sep 16 18:55:02.305173 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:55:02.305153    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-metrics-apiserver-77857b7b98-k2hjs" event={"ID":"26f53c11-5eb4-4bfa-aeca-9b55fa0d30e6","Type":"ContainerStarted","Data":"26f191a2b9e78c4685e564a6fc63efae7451027f81699c8396fe50d1d76fe4d9"}
Sep 16 18:55:06.324957 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:55:06.324913    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-operator-579c8958f7-gv7bx" event={"ID":"7916c050-3248-46f8-b607-3655e271829e","Type":"ContainerStarted","Data":"1cfdf4d48dc4a8795fab2424ffaaa35c70bf97ae491d507b15c75229539a85f6"}
Sep 16 18:55:06.325392 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:55:06.325004    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-operator-579c8958f7-gv7bx"
Sep 16 18:55:06.326724 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:55:06.326693    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-metrics-apiserver-77857b7b98-k2hjs" event={"ID":"26f53c11-5eb4-4bfa-aeca-9b55fa0d30e6","Type":"ContainerStarted","Data":"5b0f0402afedc80db1e6c2c4c59643e9222f7b5ff2abcf4ccdc0574618f5f0be"}
Sep 16 18:55:06.326859 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:55:06.326781    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-metrics-apiserver-77857b7b98-k2hjs"
Sep 16 18:55:06.343239 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:55:06.343189    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/keda-operator-579c8958f7-gv7bx" podStartSLOduration=2.058974081 podStartE2EDuration="6.34317518s" totalImagesPullingTime="4.284201099s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:55:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:55:06.341312418 +0000 UTC m=+636.670814832" watchObservedRunningTime="2026-09-16 18:55:06.34317518 +0000 UTC m=+636.672677593"
Sep 16 18:55:06.358468 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:55:06.358419    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/keda-metrics-apiserver-77857b7b98-k2hjs" podStartSLOduration=2.274134978 podStartE2EDuration="6.358404786s" totalImagesPullingTime="4.084269808s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:55:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:55:06.357643149 +0000 UTC m=+636.687145563" watchObservedRunningTime="2026-09-16 18:55:06.358404786 +0000 UTC m=+636.687907198"
Sep 16 18:55:17.335728 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:55:17.335698    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-metrics-apiserver-77857b7b98-k2hjs"
Sep 16 18:55:27.333372 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:55:27.333343    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-operator-579c8958f7-gv7bx"
Sep 16 18:55:27.372383 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:55:27.372344    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:55:39.953930 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:55:39.953894    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:55:46.117155 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:55:46.117119    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:55:46.172627 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:55:46.172585    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:55:47.966373 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:55:47.966343    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:55:48.026409 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:55:48.026376    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:55:52.542582 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:55:52.542544    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:55:52.643645 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:55:52.643608    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:56:17.884945 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:56:17.884896    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:56:17.942940 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:56:17.942902    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/kserve-controller-manager-75c4c6c467-phcpd"]
Sep 16 18:56:17.945715 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:56:17.945692    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-75c4c6c467-phcpd"
Sep 16 18:56:17.948860 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:56:17.948817    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"kserve-webhook-server-cert\""
Sep 16 18:56:17.948964 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:56:17.948844    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"kserve-controller-manager-dockercfg-pml2l\""
Sep 16 18:56:17.950061 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:56:17.950035    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"openshift-service-ca.crt\""
Sep 16 18:56:17.950166 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:56:17.950035    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"kube-root-ca.crt\""
Sep 16 18:56:17.960053 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:56:17.960031    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-75c4c6c467-phcpd"]
Sep 16 18:56:17.982888 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:56:17.982860    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:56:17.984023 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:56:17.983999    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/seaweedfs-64568bcd49-67qr4"]
Sep 16 18:56:17.986111 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:56:17.986088    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/seaweedfs-64568bcd49-67qr4"
Sep 16 18:56:17.988760 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:56:17.988743    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"default-dockercfg-wb9h8\""
Sep 16 18:56:17.988958 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:56:17.988937    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"mlpipeline-s3-artifact\""
Sep 16 18:56:17.995606 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:56:17.995581    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/seaweedfs-64568bcd49-67qr4"]
Sep 16 18:56:18.044120 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:56:18.044088    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:56:18.122165 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:56:18.122133    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/a19cc48d-affd-49a6-a918-1d860c4f9896-data\") pod \"seaweedfs-64568bcd49-67qr4\" (UID: \"a19cc48d-affd-49a6-a918-1d860c4f9896\") " pod="kserve/seaweedfs-64568bcd49-67qr4"
Sep 16 18:56:18.122306 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:56:18.122198    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lsfz9\" (UniqueName: \"kubernetes.io/projected/d96a41db-bc12-42fa-a5ea-08f42efae57c-kube-api-access-lsfz9\") pod \"kserve-controller-manager-75c4c6c467-phcpd\" (UID: \"d96a41db-bc12-42fa-a5ea-08f42efae57c\") " pod="kserve/kserve-controller-manager-75c4c6c467-phcpd"
Sep 16 18:56:18.122306 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:56:18.122250    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d96a41db-bc12-42fa-a5ea-08f42efae57c-cert\") pod \"kserve-controller-manager-75c4c6c467-phcpd\" (UID: \"d96a41db-bc12-42fa-a5ea-08f42efae57c\") " pod="kserve/kserve-controller-manager-75c4c6c467-phcpd"
Sep 16 18:56:18.122383 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:56:18.122330    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hhzlr\" (UniqueName: \"kubernetes.io/projected/a19cc48d-affd-49a6-a918-1d860c4f9896-kube-api-access-hhzlr\") pod \"seaweedfs-64568bcd49-67qr4\" (UID: \"a19cc48d-affd-49a6-a918-1d860c4f9896\") " pod="kserve/seaweedfs-64568bcd49-67qr4"
Sep 16 18:56:18.145330 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:56:18.145271    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:56:18.223531 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:56:18.223502    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lsfz9\" (UniqueName: \"kubernetes.io/projected/d96a41db-bc12-42fa-a5ea-08f42efae57c-kube-api-access-lsfz9\") pod \"kserve-controller-manager-75c4c6c467-phcpd\" (UID: \"d96a41db-bc12-42fa-a5ea-08f42efae57c\") " pod="kserve/kserve-controller-manager-75c4c6c467-phcpd"
Sep 16 18:56:18.223646 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:56:18.223533    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d96a41db-bc12-42fa-a5ea-08f42efae57c-cert\") pod \"kserve-controller-manager-75c4c6c467-phcpd\" (UID: \"d96a41db-bc12-42fa-a5ea-08f42efae57c\") " pod="kserve/kserve-controller-manager-75c4c6c467-phcpd"
Sep 16 18:56:18.223646 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:56:18.223566    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hhzlr\" (UniqueName: \"kubernetes.io/projected/a19cc48d-affd-49a6-a918-1d860c4f9896-kube-api-access-hhzlr\") pod \"seaweedfs-64568bcd49-67qr4\" (UID: \"a19cc48d-affd-49a6-a918-1d860c4f9896\") " pod="kserve/seaweedfs-64568bcd49-67qr4"
Sep 16 18:56:18.223646 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:56:18.223591    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/a19cc48d-affd-49a6-a918-1d860c4f9896-data\") pod \"seaweedfs-64568bcd49-67qr4\" (UID: \"a19cc48d-affd-49a6-a918-1d860c4f9896\") " pod="kserve/seaweedfs-64568bcd49-67qr4"
Sep 16 18:56:18.223921 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:56:18.223906    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/a19cc48d-affd-49a6-a918-1d860c4f9896-data\") pod \"seaweedfs-64568bcd49-67qr4\" (UID: \"a19cc48d-affd-49a6-a918-1d860c4f9896\") " pod="kserve/seaweedfs-64568bcd49-67qr4"
Sep 16 18:56:18.225935 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:56:18.225909    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d96a41db-bc12-42fa-a5ea-08f42efae57c-cert\") pod \"kserve-controller-manager-75c4c6c467-phcpd\" (UID: \"d96a41db-bc12-42fa-a5ea-08f42efae57c\") " pod="kserve/kserve-controller-manager-75c4c6c467-phcpd"
Sep 16 18:56:18.234064 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:56:18.234035    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lsfz9\" (UniqueName: \"kubernetes.io/projected/d96a41db-bc12-42fa-a5ea-08f42efae57c-kube-api-access-lsfz9\") pod \"kserve-controller-manager-75c4c6c467-phcpd\" (UID: \"d96a41db-bc12-42fa-a5ea-08f42efae57c\") " pod="kserve/kserve-controller-manager-75c4c6c467-phcpd"
Sep 16 18:56:18.234173 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:56:18.234155    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hhzlr\" (UniqueName: \"kubernetes.io/projected/a19cc48d-affd-49a6-a918-1d860c4f9896-kube-api-access-hhzlr\") pod \"seaweedfs-64568bcd49-67qr4\" (UID: \"a19cc48d-affd-49a6-a918-1d860c4f9896\") " pod="kserve/seaweedfs-64568bcd49-67qr4"
Sep 16 18:56:18.256526 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:56:18.256498    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-75c4c6c467-phcpd"
Sep 16 18:56:18.273472 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:56:18.273451    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:56:18.295264 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:56:18.295002    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/seaweedfs-64568bcd49-67qr4"
Sep 16 18:56:18.513106 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:56:18.513082    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:56:18.530424 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:56:18.530398    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-75c4c6c467-phcpd"]
Sep 16 18:56:18.532856 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:56:18.532822    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd96a41db_bc12_42fa_a5ea_08f42efae57c.slice/crio-b86309e738c7ac269c6dbdaea333eb6e2e4c75ba8941da854b6fb0c4c2aa83e7 WatchSource:0}: Error finding container b86309e738c7ac269c6dbdaea333eb6e2e4c75ba8941da854b6fb0c4c2aa83e7: Status 404 returned error can't find the container with id b86309e738c7ac269c6dbdaea333eb6e2e4c75ba8941da854b6fb0c4c2aa83e7
Sep 16 18:56:18.566582 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:56:18.566552    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/seaweedfs-64568bcd49-67qr4"]
Sep 16 18:56:18.569203 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:56:18.569177    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda19cc48d_affd_49a6_a918_1d860c4f9896.slice/crio-a63ccdc7e42bd7818cfd40bc83b0ea6268a51e9635de8f5224678ca0d13563b7 WatchSource:0}: Error finding container a63ccdc7e42bd7818cfd40bc83b0ea6268a51e9635de8f5224678ca0d13563b7: Status 404 returned error can't find the container with id a63ccdc7e42bd7818cfd40bc83b0ea6268a51e9635de8f5224678ca0d13563b7
Sep 16 18:56:18.571272 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:56:18.571250    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:56:18.613182 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:56:18.613155    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-75c4c6c467-phcpd" event={"ID":"d96a41db-bc12-42fa-a5ea-08f42efae57c","Type":"ContainerStarted","Data":"b86309e738c7ac269c6dbdaea333eb6e2e4c75ba8941da854b6fb0c4c2aa83e7"}
Sep 16 18:56:18.614168 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:56:18.614152    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/seaweedfs-64568bcd49-67qr4" event={"ID":"a19cc48d-affd-49a6-a918-1d860c4f9896","Type":"ContainerStarted","Data":"a63ccdc7e42bd7818cfd40bc83b0ea6268a51e9635de8f5224678ca0d13563b7"}
Sep 16 18:56:18.629780 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:56:18.629756    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:56:22.634967 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:56:22.634878    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-75c4c6c467-phcpd" event={"ID":"d96a41db-bc12-42fa-a5ea-08f42efae57c","Type":"ContainerStarted","Data":"f9ddfa7fd012a06b7a52572efebfd554e5d953e090498f74b213d8cf23899423"}
Sep 16 18:56:22.635364 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:56:22.634976    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/kserve-controller-manager-75c4c6c467-phcpd"
Sep 16 18:56:22.636335 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:56:22.636311    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/seaweedfs-64568bcd49-67qr4" event={"ID":"a19cc48d-affd-49a6-a918-1d860c4f9896","Type":"ContainerStarted","Data":"b3d446dc8935bd04feeffd49acbe38cf2ff082d38d1d63886a06fd811b3ce79e"}
Sep 16 18:56:22.636436 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:56:22.636426    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/seaweedfs-64568bcd49-67qr4"
Sep 16 18:56:22.653435 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:56:22.653373    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/kserve-controller-manager-75c4c6c467-phcpd" podStartSLOduration=1.963250087 podStartE2EDuration="5.65335949s" totalImagesPullingTime="3.690109403s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:56:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:56:22.652020974 +0000 UTC m=+712.981523386" watchObservedRunningTime="2026-09-16 18:56:22.65335949 +0000 UTC m=+712.982861939"
Sep 16 18:56:22.670328 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:56:22.670240    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/seaweedfs-64568bcd49-67qr4" podStartSLOduration=1.951816263 podStartE2EDuration="5.670224328s" totalImagesPullingTime="3.718408065s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:56:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:56:22.668905785 +0000 UTC m=+712.998408197" watchObservedRunningTime="2026-09-16 18:56:22.670224328 +0000 UTC m=+712.999726740"
Sep 16 18:56:28.642919 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:56:28.642890    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/seaweedfs-64568bcd49-67qr4"
Sep 16 18:56:53.644888 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:56:53.644849    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/kserve-controller-manager-75c4c6c467-phcpd"
Sep 16 18:57:00.479205 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:57:00.479172    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve/kserve-controller-manager-75c4c6c467-phcpd"]
Sep 16 18:57:00.479624 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:57:00.479395    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve/kserve-controller-manager-75c4c6c467-phcpd" podUID="d96a41db-bc12-42fa-a5ea-08f42efae57c" containerName="manager" containerID="cri-o://f9ddfa7fd012a06b7a52572efebfd554e5d953e090498f74b213d8cf23899423" gracePeriod=10
Sep 16 18:57:00.502912 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:57:00.502879    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/kserve-controller-manager-75c4c6c467-jfjnj"]
Sep 16 18:57:00.504720 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:57:00.504706    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-75c4c6c467-jfjnj"
Sep 16 18:57:00.515232 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:57:00.515206    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-75c4c6c467-jfjnj"]
Sep 16 18:57:00.662000 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:57:00.661963    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b5be7327-d711-4e51-a1cc-01421b6f209b-cert\") pod \"kserve-controller-manager-75c4c6c467-jfjnj\" (UID: \"b5be7327-d711-4e51-a1cc-01421b6f209b\") " pod="kserve/kserve-controller-manager-75c4c6c467-jfjnj"
Sep 16 18:57:00.662158 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:57:00.662042    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9lqhf\" (UniqueName: \"kubernetes.io/projected/b5be7327-d711-4e51-a1cc-01421b6f209b-kube-api-access-9lqhf\") pod \"kserve-controller-manager-75c4c6c467-jfjnj\" (UID: \"b5be7327-d711-4e51-a1cc-01421b6f209b\") " pod="kserve/kserve-controller-manager-75c4c6c467-jfjnj"
Sep 16 18:57:00.707201 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:57:00.707177    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-75c4c6c467-phcpd"
Sep 16 18:57:00.762527 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:57:00.762494    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9lqhf\" (UniqueName: \"kubernetes.io/projected/b5be7327-d711-4e51-a1cc-01421b6f209b-kube-api-access-9lqhf\") pod \"kserve-controller-manager-75c4c6c467-jfjnj\" (UID: \"b5be7327-d711-4e51-a1cc-01421b6f209b\") " pod="kserve/kserve-controller-manager-75c4c6c467-jfjnj"
Sep 16 18:57:00.762705 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:57:00.762593    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b5be7327-d711-4e51-a1cc-01421b6f209b-cert\") pod \"kserve-controller-manager-75c4c6c467-jfjnj\" (UID: \"b5be7327-d711-4e51-a1cc-01421b6f209b\") " pod="kserve/kserve-controller-manager-75c4c6c467-jfjnj"
Sep 16 18:57:00.764812 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:57:00.764785    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b5be7327-d711-4e51-a1cc-01421b6f209b-cert\") pod \"kserve-controller-manager-75c4c6c467-jfjnj\" (UID: \"b5be7327-d711-4e51-a1cc-01421b6f209b\") " pod="kserve/kserve-controller-manager-75c4c6c467-jfjnj"
Sep 16 18:57:00.772241 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:57:00.772221    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9lqhf\" (UniqueName: \"kubernetes.io/projected/b5be7327-d711-4e51-a1cc-01421b6f209b-kube-api-access-9lqhf\") pod \"kserve-controller-manager-75c4c6c467-jfjnj\" (UID: \"b5be7327-d711-4e51-a1cc-01421b6f209b\") " pod="kserve/kserve-controller-manager-75c4c6c467-jfjnj"
Sep 16 18:57:00.782113 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:57:00.782083    2584 generic.go:356] "Generic (PLEG): container finished" podID="d96a41db-bc12-42fa-a5ea-08f42efae57c" containerID="f9ddfa7fd012a06b7a52572efebfd554e5d953e090498f74b213d8cf23899423" exitCode=0
Sep 16 18:57:00.782229 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:57:00.782133    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-75c4c6c467-phcpd"
Sep 16 18:57:00.782229 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:57:00.782134    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-75c4c6c467-phcpd" event={"ID":"d96a41db-bc12-42fa-a5ea-08f42efae57c","Type":"ContainerDied","Data":"f9ddfa7fd012a06b7a52572efebfd554e5d953e090498f74b213d8cf23899423"}
Sep 16 18:57:00.782337 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:57:00.782316    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-75c4c6c467-phcpd" event={"ID":"d96a41db-bc12-42fa-a5ea-08f42efae57c","Type":"ContainerDied","Data":"b86309e738c7ac269c6dbdaea333eb6e2e4c75ba8941da854b6fb0c4c2aa83e7"}
Sep 16 18:57:00.782403 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:57:00.782360    2584 scope.go:117] "RemoveContainer" containerID="f9ddfa7fd012a06b7a52572efebfd554e5d953e090498f74b213d8cf23899423"
Sep 16 18:57:00.789917 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:57:00.789900    2584 scope.go:117] "RemoveContainer" containerID="f9ddfa7fd012a06b7a52572efebfd554e5d953e090498f74b213d8cf23899423"
Sep 16 18:57:00.790182 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:57:00.790158    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f9ddfa7fd012a06b7a52572efebfd554e5d953e090498f74b213d8cf23899423\": container with ID starting with f9ddfa7fd012a06b7a52572efebfd554e5d953e090498f74b213d8cf23899423 not found: ID does not exist" containerID="f9ddfa7fd012a06b7a52572efebfd554e5d953e090498f74b213d8cf23899423"
Sep 16 18:57:00.790228 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:57:00.790183    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f9ddfa7fd012a06b7a52572efebfd554e5d953e090498f74b213d8cf23899423"} err="failed to get container status \"f9ddfa7fd012a06b7a52572efebfd554e5d953e090498f74b213d8cf23899423\": rpc error: code = NotFound desc = could not find container \"f9ddfa7fd012a06b7a52572efebfd554e5d953e090498f74b213d8cf23899423\": container with ID starting with f9ddfa7fd012a06b7a52572efebfd554e5d953e090498f74b213d8cf23899423 not found: ID does not exist"
Sep 16 18:57:00.854218 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:57:00.854184    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-75c4c6c467-jfjnj"
Sep 16 18:57:00.863622 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:57:00.863601    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d96a41db-bc12-42fa-a5ea-08f42efae57c-cert\") pod \"d96a41db-bc12-42fa-a5ea-08f42efae57c\" (UID: \"d96a41db-bc12-42fa-a5ea-08f42efae57c\") "
Sep 16 18:57:00.863736 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:57:00.863652    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lsfz9\" (UniqueName: \"kubernetes.io/projected/d96a41db-bc12-42fa-a5ea-08f42efae57c-kube-api-access-lsfz9\") pod \"d96a41db-bc12-42fa-a5ea-08f42efae57c\" (UID: \"d96a41db-bc12-42fa-a5ea-08f42efae57c\") "
Sep 16 18:57:00.865657 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:57:00.865626    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d96a41db-bc12-42fa-a5ea-08f42efae57c-kube-api-access-lsfz9" (OuterVolumeSpecName: "kube-api-access-lsfz9") pod "d96a41db-bc12-42fa-a5ea-08f42efae57c" (UID: "d96a41db-bc12-42fa-a5ea-08f42efae57c"). InnerVolumeSpecName "kube-api-access-lsfz9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:57:00.865738 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:57:00.865642    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d96a41db-bc12-42fa-a5ea-08f42efae57c-cert" (OuterVolumeSpecName: "cert") pod "d96a41db-bc12-42fa-a5ea-08f42efae57c" (UID: "d96a41db-bc12-42fa-a5ea-08f42efae57c"). InnerVolumeSpecName "cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:57:00.964270 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:57:00.964242    2584 reconciler_common.go:299] "Volume detached for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d96a41db-bc12-42fa-a5ea-08f42efae57c-cert\") on node \"ip-10-0-139-182.ec2.internal\" DevicePath \"\""
Sep 16 18:57:00.964270 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:57:00.964267    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lsfz9\" (UniqueName: \"kubernetes.io/projected/d96a41db-bc12-42fa-a5ea-08f42efae57c-kube-api-access-lsfz9\") on node \"ip-10-0-139-182.ec2.internal\" DevicePath \"\""
Sep 16 18:57:01.103589 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:57:01.103554    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve/kserve-controller-manager-75c4c6c467-phcpd"]
Sep 16 18:57:01.109180 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:57:01.109155    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve/kserve-controller-manager-75c4c6c467-phcpd"]
Sep 16 18:57:01.120168 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:57:01.120140    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-75c4c6c467-jfjnj"]
Sep 16 18:57:01.122962 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:57:01.122937    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5be7327_d711_4e51_a1cc_01421b6f209b.slice/crio-35abf34b88958e1a95b9a70a6a6004056093247f6e2d24d295f85cad7f921599 WatchSource:0}: Error finding container 35abf34b88958e1a95b9a70a6a6004056093247f6e2d24d295f85cad7f921599: Status 404 returned error can't find the container with id 35abf34b88958e1a95b9a70a6a6004056093247f6e2d24d295f85cad7f921599
Sep 16 18:57:01.788432 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:57:01.788403    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-75c4c6c467-jfjnj" event={"ID":"b5be7327-d711-4e51-a1cc-01421b6f209b","Type":"ContainerStarted","Data":"b79553ed33b7fb0589af9ad1eb97f2ca6936618e62692c0ab5c3fea0791fb904"}
Sep 16 18:57:01.788759 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:57:01.788449    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-75c4c6c467-jfjnj" event={"ID":"b5be7327-d711-4e51-a1cc-01421b6f209b","Type":"ContainerStarted","Data":"35abf34b88958e1a95b9a70a6a6004056093247f6e2d24d295f85cad7f921599"}
Sep 16 18:57:01.788759 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:57:01.788514    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/kserve-controller-manager-75c4c6c467-jfjnj"
Sep 16 18:57:01.807227 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:57:01.807181    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/kserve-controller-manager-75c4c6c467-jfjnj" podStartSLOduration=1.210938134 podStartE2EDuration="1.80716712s" totalImagesPullingTime="596.228986ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:57:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:57:01.805411697 +0000 UTC m=+752.134914134" watchObservedRunningTime="2026-09-16 18:57:01.80716712 +0000 UTC m=+752.136669533"
Sep 16 18:57:02.116070 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:57:02.116039    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d96a41db-bc12-42fa-a5ea-08f42efae57c" path="/var/lib/kubelet/pods/d96a41db-bc12-42fa-a5ea-08f42efae57c/volumes"
Sep 16 18:57:32.797312 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:57:32.797276    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/kserve-controller-manager-75c4c6c467-jfjnj"
Sep 16 18:57:34.067702 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:57:34.067659    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:57:34.108225 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:57:34.108192    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/model-serving-api-7b954ffc95-mxzbw"]
Sep 16 18:57:34.108565 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:57:34.108552    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d96a41db-bc12-42fa-a5ea-08f42efae57c" containerName="manager"
Sep 16 18:57:34.108614 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:57:34.108568    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="d96a41db-bc12-42fa-a5ea-08f42efae57c" containerName="manager"
Sep 16 18:57:34.108648 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:57:34.108616    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="d96a41db-bc12-42fa-a5ea-08f42efae57c" containerName="manager"
Sep 16 18:57:34.112780 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:57:34.112756    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/model-serving-api-7b954ffc95-mxzbw"
Sep 16 18:57:34.115509 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:57:34.115489    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"model-serving-api-tls\""
Sep 16 18:57:34.115629 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:57:34.115606    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"model-serving-api-dockercfg-lsljc\""
Sep 16 18:57:34.123802 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:57:34.123782    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/model-serving-api-7b954ffc95-mxzbw"]
Sep 16 18:57:34.125425 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:57:34.125391    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vd68d\" (UniqueName: \"kubernetes.io/projected/ad86938c-bf26-4b41-8615-27995751a6d4-kube-api-access-vd68d\") pod \"model-serving-api-7b954ffc95-mxzbw\" (UID: \"ad86938c-bf26-4b41-8615-27995751a6d4\") " pod="kserve/model-serving-api-7b954ffc95-mxzbw"
Sep 16 18:57:34.125530 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:57:34.125432    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ad86938c-bf26-4b41-8615-27995751a6d4-tls-certs\") pod \"model-serving-api-7b954ffc95-mxzbw\" (UID: \"ad86938c-bf26-4b41-8615-27995751a6d4\") " pod="kserve/model-serving-api-7b954ffc95-mxzbw"
Sep 16 18:57:34.125662 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:57:34.125644    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/odh-model-controller-7b966d7984-5nnrz"]
Sep 16 18:57:34.128683 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:57:34.128667    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/odh-model-controller-7b966d7984-5nnrz"
Sep 16 18:57:34.131672 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:57:34.131656    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"odh-model-controller-dockercfg-69pfw\""
Sep 16 18:57:34.131910 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:57:34.131895    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"odh-model-controller-webhook-cert\""
Sep 16 18:57:34.132113 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:57:34.132093    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"odh-model-controller-metrics-tls\""
Sep 16 18:57:34.136295 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:57:34.136273    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/odh-model-controller-7b966d7984-5nnrz"]
Sep 16 18:57:34.168310 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:57:34.168278    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:57:34.226668 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:57:34.226635    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0915599e-1581-4f0d-b575-6bc3be699f89-cert\") pod \"odh-model-controller-7b966d7984-5nnrz\" (UID: \"0915599e-1581-4f0d-b575-6bc3be699f89\") " pod="kserve/odh-model-controller-7b966d7984-5nnrz"
Sep 16 18:57:34.226864 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:57:34.226674    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vd68d\" (UniqueName: \"kubernetes.io/projected/ad86938c-bf26-4b41-8615-27995751a6d4-kube-api-access-vd68d\") pod \"model-serving-api-7b954ffc95-mxzbw\" (UID: \"ad86938c-bf26-4b41-8615-27995751a6d4\") " pod="kserve/model-serving-api-7b954ffc95-mxzbw"
Sep 16 18:57:34.226864 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:57:34.226693    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ad86938c-bf26-4b41-8615-27995751a6d4-tls-certs\") pod \"model-serving-api-7b954ffc95-mxzbw\" (UID: \"ad86938c-bf26-4b41-8615-27995751a6d4\") " pod="kserve/model-serving-api-7b954ffc95-mxzbw"
Sep 16 18:57:34.226864 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:57:34.226748    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0915599e-1581-4f0d-b575-6bc3be699f89-metrics-certs\") pod \"odh-model-controller-7b966d7984-5nnrz\" (UID: \"0915599e-1581-4f0d-b575-6bc3be699f89\") " pod="kserve/odh-model-controller-7b966d7984-5nnrz"
Sep 16 18:57:34.226864 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:57:34.226775    2584 secret.go:189] Couldn't get secret kserve/model-serving-api-tls: secret "model-serving-api-tls" not found
Sep 16 18:57:34.226864 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:57:34.226780    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gd5v9\" (UniqueName: \"kubernetes.io/projected/0915599e-1581-4f0d-b575-6bc3be699f89-kube-api-access-gd5v9\") pod \"odh-model-controller-7b966d7984-5nnrz\" (UID: \"0915599e-1581-4f0d-b575-6bc3be699f89\") " pod="kserve/odh-model-controller-7b966d7984-5nnrz"
Sep 16 18:57:34.226864 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:57:34.226826    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad86938c-bf26-4b41-8615-27995751a6d4-tls-certs podName:ad86938c-bf26-4b41-8615-27995751a6d4 nodeName:}" failed. No retries permitted until 2026-09-16 18:57:34.726811661 +0000 UTC m=+785.056314053 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/ad86938c-bf26-4b41-8615-27995751a6d4-tls-certs") pod "model-serving-api-7b954ffc95-mxzbw" (UID: "ad86938c-bf26-4b41-8615-27995751a6d4") : secret "model-serving-api-tls" not found
Sep 16 18:57:34.236961 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:57:34.236938    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vd68d\" (UniqueName: \"kubernetes.io/projected/ad86938c-bf26-4b41-8615-27995751a6d4-kube-api-access-vd68d\") pod \"model-serving-api-7b954ffc95-mxzbw\" (UID: \"ad86938c-bf26-4b41-8615-27995751a6d4\") " pod="kserve/model-serving-api-7b954ffc95-mxzbw"
Sep 16 18:57:34.328156 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:57:34.328065    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0915599e-1581-4f0d-b575-6bc3be699f89-cert\") pod \"odh-model-controller-7b966d7984-5nnrz\" (UID: \"0915599e-1581-4f0d-b575-6bc3be699f89\") " pod="kserve/odh-model-controller-7b966d7984-5nnrz"
Sep 16 18:57:34.328156 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:57:34.328133    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0915599e-1581-4f0d-b575-6bc3be699f89-metrics-certs\") pod \"odh-model-controller-7b966d7984-5nnrz\" (UID: \"0915599e-1581-4f0d-b575-6bc3be699f89\") " pod="kserve/odh-model-controller-7b966d7984-5nnrz"
Sep 16 18:57:34.328156 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:57:34.328154    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gd5v9\" (UniqueName: \"kubernetes.io/projected/0915599e-1581-4f0d-b575-6bc3be699f89-kube-api-access-gd5v9\") pod \"odh-model-controller-7b966d7984-5nnrz\" (UID: \"0915599e-1581-4f0d-b575-6bc3be699f89\") " pod="kserve/odh-model-controller-7b966d7984-5nnrz"
Sep 16 18:57:34.328370 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:57:34.328216    2584 secret.go:189] Couldn't get secret kserve/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Sep 16 18:57:34.328370 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:57:34.328283    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0915599e-1581-4f0d-b575-6bc3be699f89-cert podName:0915599e-1581-4f0d-b575-6bc3be699f89 nodeName:}" failed. No retries permitted until 2026-09-16 18:57:34.828267871 +0000 UTC m=+785.157770263 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/0915599e-1581-4f0d-b575-6bc3be699f89-cert") pod "odh-model-controller-7b966d7984-5nnrz" (UID: "0915599e-1581-4f0d-b575-6bc3be699f89") : secret "odh-model-controller-webhook-cert" not found
Sep 16 18:57:34.330646 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:57:34.330623    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0915599e-1581-4f0d-b575-6bc3be699f89-metrics-certs\") pod \"odh-model-controller-7b966d7984-5nnrz\" (UID: \"0915599e-1581-4f0d-b575-6bc3be699f89\") " pod="kserve/odh-model-controller-7b966d7984-5nnrz"
Sep 16 18:57:34.337898 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:57:34.337876    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gd5v9\" (UniqueName: \"kubernetes.io/projected/0915599e-1581-4f0d-b575-6bc3be699f89-kube-api-access-gd5v9\") pod \"odh-model-controller-7b966d7984-5nnrz\" (UID: \"0915599e-1581-4f0d-b575-6bc3be699f89\") " pod="kserve/odh-model-controller-7b966d7984-5nnrz"
Sep 16 18:57:34.357310 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:57:34.357280    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:57:34.457496 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:57:34.457468    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:57:34.640691 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:57:34.640588    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:57:34.731502 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:57:34.731467    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ad86938c-bf26-4b41-8615-27995751a6d4-tls-certs\") pod \"model-serving-api-7b954ffc95-mxzbw\" (UID: \"ad86938c-bf26-4b41-8615-27995751a6d4\") " pod="kserve/model-serving-api-7b954ffc95-mxzbw"
Sep 16 18:57:34.733940 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:57:34.733920    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ad86938c-bf26-4b41-8615-27995751a6d4-tls-certs\") pod \"model-serving-api-7b954ffc95-mxzbw\" (UID: \"ad86938c-bf26-4b41-8615-27995751a6d4\") " pod="kserve/model-serving-api-7b954ffc95-mxzbw"
Sep 16 18:57:34.832015 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:57:34.831966    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0915599e-1581-4f0d-b575-6bc3be699f89-cert\") pod \"odh-model-controller-7b966d7984-5nnrz\" (UID: \"0915599e-1581-4f0d-b575-6bc3be699f89\") " pod="kserve/odh-model-controller-7b966d7984-5nnrz"
Sep 16 18:57:34.834364 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:57:34.834343    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0915599e-1581-4f0d-b575-6bc3be699f89-cert\") pod \"odh-model-controller-7b966d7984-5nnrz\" (UID: \"0915599e-1581-4f0d-b575-6bc3be699f89\") " pod="kserve/odh-model-controller-7b966d7984-5nnrz"
Sep 16 18:57:35.023885 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:57:35.023844    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/model-serving-api-7b954ffc95-mxzbw"
Sep 16 18:57:35.039558 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:57:35.039529    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/odh-model-controller-7b966d7984-5nnrz"
Sep 16 18:57:35.301577 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:57:35.301494    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/model-serving-api-7b954ffc95-mxzbw"]
Sep 16 18:57:35.304266 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:57:35.304235    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad86938c_bf26_4b41_8615_27995751a6d4.slice/crio-c2db5422e5e7c3d57cc2df81b30a0f8c1eda95c372d7a17c4c1e295ed929cf22 WatchSource:0}: Error finding container c2db5422e5e7c3d57cc2df81b30a0f8c1eda95c372d7a17c4c1e295ed929cf22: Status 404 returned error can't find the container with id c2db5422e5e7c3d57cc2df81b30a0f8c1eda95c372d7a17c4c1e295ed929cf22
Sep 16 18:57:35.322710 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:57:35.322689    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/odh-model-controller-7b966d7984-5nnrz"]
Sep 16 18:57:35.324647 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:57:35.324625    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0915599e_1581_4f0d_b575_6bc3be699f89.slice/crio-5e2a7ab57c59cbf0d2d8a314037b8e2a3701ef9cec33ea60c0f02d843f88a1d6 WatchSource:0}: Error finding container 5e2a7ab57c59cbf0d2d8a314037b8e2a3701ef9cec33ea60c0f02d843f88a1d6: Status 404 returned error can't find the container with id 5e2a7ab57c59cbf0d2d8a314037b8e2a3701ef9cec33ea60c0f02d843f88a1d6
Sep 16 18:57:35.382165 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:57:35.382132    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:57:35.443111 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:57:35.443050    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:57:35.914318 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:57:35.914275    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/odh-model-controller-7b966d7984-5nnrz" event={"ID":"0915599e-1581-4f0d-b575-6bc3be699f89","Type":"ContainerStarted","Data":"5e2a7ab57c59cbf0d2d8a314037b8e2a3701ef9cec33ea60c0f02d843f88a1d6"}
Sep 16 18:57:35.915678 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:57:35.915646    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/model-serving-api-7b954ffc95-mxzbw" event={"ID":"ad86938c-bf26-4b41-8615-27995751a6d4","Type":"ContainerStarted","Data":"c2db5422e5e7c3d57cc2df81b30a0f8c1eda95c372d7a17c4c1e295ed929cf22"}
Sep 16 18:57:38.538461 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:57:38.538410    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:57:39.934518 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:57:39.934477    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/model-serving-api-7b954ffc95-mxzbw" event={"ID":"ad86938c-bf26-4b41-8615-27995751a6d4","Type":"ContainerStarted","Data":"bf2eeb5e57726ab3688c335b32fbc15c5d76f23f86aa17999a54453e42d6d596"}
Sep 16 18:57:39.934970 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:57:39.934587    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/model-serving-api-7b954ffc95-mxzbw"
Sep 16 18:57:39.936185 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:57:39.936163    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/odh-model-controller-7b966d7984-5nnrz" event={"ID":"0915599e-1581-4f0d-b575-6bc3be699f89","Type":"ContainerStarted","Data":"70f30a9fa3bb6a8a524a26afe22b1864ff8415be610c7b6d5da18948195ee105"}
Sep 16 18:57:39.936270 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:57:39.936208    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/odh-model-controller-7b966d7984-5nnrz"
Sep 16 18:57:39.953707 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:57:39.953648    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/model-serving-api-7b954ffc95-mxzbw" podStartSLOduration=2.241467883 podStartE2EDuration="5.953627096s" totalImagesPullingTime="3.712159213s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:57:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:57:39.951266752 +0000 UTC m=+790.280769168" watchObservedRunningTime="2026-09-16 18:57:39.953627096 +0000 UTC m=+790.283129514"
Sep 16 18:57:39.967009 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:57:39.966966    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/odh-model-controller-7b966d7984-5nnrz" podStartSLOduration=2.3137322510000002 podStartE2EDuration="5.966950726s" totalImagesPullingTime="3.653218475s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:57:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:57:39.966512294 +0000 UTC m=+790.296014699" watchObservedRunningTime="2026-09-16 18:57:39.966950726 +0000 UTC m=+790.296453139"
Sep 16 18:57:50.943551 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:57:50.943514    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/odh-model-controller-7b966d7984-5nnrz"
Sep 16 18:57:50.945192 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:57:50.945170    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/model-serving-api-7b954ffc95-mxzbw"
Sep 16 18:59:08.939626 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:59:08.939582    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:59:09.027727 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:59:09.027691    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:59:20.644798 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:59:20.644767    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:59:20.677982 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:59:20.677941    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-658c9-predictor-6b7979b5bc-gvx7k"]
Sep 16 18:59:20.681876 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:59:20.681854    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-658c9-predictor-6b7979b5bc-gvx7k"
Sep 16 18:59:20.684566 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:59:20.684543    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Sep 16 18:59:20.684673 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:59:20.684642    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-dgjbs\""
Sep 16 18:59:20.685875 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:59:20.685852    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-658c9-predictor-serving-cert\""
Sep 16 18:59:20.685982 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:59:20.685897    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Sep 16 18:59:20.685982 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:59:20.685900    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-658c9-kube-rbac-proxy-sar-config\""
Sep 16 18:59:20.690757 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:59:20.690734    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-658c9-predictor-6b7979b5bc-gvx7k"]
Sep 16 18:59:20.702711 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:59:20.702541    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rjxkx\" (UniqueName: \"kubernetes.io/projected/d187efb1-1ed6-46ba-b217-2b86ed506ba4-kube-api-access-rjxkx\") pod \"success-200-isvc-658c9-predictor-6b7979b5bc-gvx7k\" (UID: \"d187efb1-1ed6-46ba-b217-2b86ed506ba4\") " pod="kserve-ci-e2e-test/success-200-isvc-658c9-predictor-6b7979b5bc-gvx7k"
Sep 16 18:59:20.702711 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:59:20.702629    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d187efb1-1ed6-46ba-b217-2b86ed506ba4-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-658c9-predictor-6b7979b5bc-gvx7k\" (UID: \"d187efb1-1ed6-46ba-b217-2b86ed506ba4\") " pod="kserve-ci-e2e-test/success-200-isvc-658c9-predictor-6b7979b5bc-gvx7k"
Sep 16 18:59:20.702711 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:59:20.702664    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d187efb1-1ed6-46ba-b217-2b86ed506ba4-proxy-tls\") pod \"success-200-isvc-658c9-predictor-6b7979b5bc-gvx7k\" (UID: \"d187efb1-1ed6-46ba-b217-2b86ed506ba4\") " pod="kserve-ci-e2e-test/success-200-isvc-658c9-predictor-6b7979b5bc-gvx7k"
Sep 16 18:59:20.803397 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:59:20.803356    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d187efb1-1ed6-46ba-b217-2b86ed506ba4-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-658c9-predictor-6b7979b5bc-gvx7k\" (UID: \"d187efb1-1ed6-46ba-b217-2b86ed506ba4\") " pod="kserve-ci-e2e-test/success-200-isvc-658c9-predictor-6b7979b5bc-gvx7k"
Sep 16 18:59:20.803397 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:59:20.803395    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d187efb1-1ed6-46ba-b217-2b86ed506ba4-proxy-tls\") pod \"success-200-isvc-658c9-predictor-6b7979b5bc-gvx7k\" (UID: \"d187efb1-1ed6-46ba-b217-2b86ed506ba4\") " pod="kserve-ci-e2e-test/success-200-isvc-658c9-predictor-6b7979b5bc-gvx7k"
Sep 16 18:59:20.803592 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:59:20.803568    2584 secret.go:189] Couldn't get secret kserve-ci-e2e-test/success-200-isvc-658c9-predictor-serving-cert: secret "success-200-isvc-658c9-predictor-serving-cert" not found
Sep 16 18:59:20.803631 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:59:20.803598    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rjxkx\" (UniqueName: \"kubernetes.io/projected/d187efb1-1ed6-46ba-b217-2b86ed506ba4-kube-api-access-rjxkx\") pod \"success-200-isvc-658c9-predictor-6b7979b5bc-gvx7k\" (UID: \"d187efb1-1ed6-46ba-b217-2b86ed506ba4\") " pod="kserve-ci-e2e-test/success-200-isvc-658c9-predictor-6b7979b5bc-gvx7k"
Sep 16 18:59:20.803673 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:59:20.803654    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d187efb1-1ed6-46ba-b217-2b86ed506ba4-proxy-tls podName:d187efb1-1ed6-46ba-b217-2b86ed506ba4 nodeName:}" failed. No retries permitted until 2026-09-16 18:59:21.303636022 +0000 UTC m=+891.633138413 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/d187efb1-1ed6-46ba-b217-2b86ed506ba4-proxy-tls") pod "success-200-isvc-658c9-predictor-6b7979b5bc-gvx7k" (UID: "d187efb1-1ed6-46ba-b217-2b86ed506ba4") : secret "success-200-isvc-658c9-predictor-serving-cert" not found
Sep 16 18:59:20.804064 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:59:20.804045    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d187efb1-1ed6-46ba-b217-2b86ed506ba4-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-658c9-predictor-6b7979b5bc-gvx7k\" (UID: \"d187efb1-1ed6-46ba-b217-2b86ed506ba4\") " pod="kserve-ci-e2e-test/success-200-isvc-658c9-predictor-6b7979b5bc-gvx7k"
Sep 16 18:59:20.814466 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:59:20.814440    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rjxkx\" (UniqueName: \"kubernetes.io/projected/d187efb1-1ed6-46ba-b217-2b86ed506ba4-kube-api-access-rjxkx\") pod \"success-200-isvc-658c9-predictor-6b7979b5bc-gvx7k\" (UID: \"d187efb1-1ed6-46ba-b217-2b86ed506ba4\") " pod="kserve-ci-e2e-test/success-200-isvc-658c9-predictor-6b7979b5bc-gvx7k"
Sep 16 18:59:20.943047 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:59:20.942961    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:59:21.126308 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:59:21.126252    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:59:21.226273 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:59:21.226185    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:59:21.295299 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:59:21.295264    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:59:21.307155 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:59:21.307116    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d187efb1-1ed6-46ba-b217-2b86ed506ba4-proxy-tls\") pod \"success-200-isvc-658c9-predictor-6b7979b5bc-gvx7k\" (UID: \"d187efb1-1ed6-46ba-b217-2b86ed506ba4\") " pod="kserve-ci-e2e-test/success-200-isvc-658c9-predictor-6b7979b5bc-gvx7k"
Sep 16 18:59:21.307356 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:59:21.307337    2584 secret.go:189] Couldn't get secret kserve-ci-e2e-test/success-200-isvc-658c9-predictor-serving-cert: secret "success-200-isvc-658c9-predictor-serving-cert" not found
Sep 16 18:59:21.307467 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:59:21.307429    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d187efb1-1ed6-46ba-b217-2b86ed506ba4-proxy-tls podName:d187efb1-1ed6-46ba-b217-2b86ed506ba4 nodeName:}" failed. No retries permitted until 2026-09-16 18:59:22.307407667 +0000 UTC m=+892.636910071 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/d187efb1-1ed6-46ba-b217-2b86ed506ba4-proxy-tls") pod "success-200-isvc-658c9-predictor-6b7979b5bc-gvx7k" (UID: "d187efb1-1ed6-46ba-b217-2b86ed506ba4") : secret "success-200-isvc-658c9-predictor-serving-cert" not found
Sep 16 18:59:21.357974 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:59:21.357946    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:59:21.373808 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:59:21.373776    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-799ddc8799-hjrgh"]
Sep 16 18:59:21.377542 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:59:21.377522    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-799ddc8799-hjrgh"
Sep 16 18:59:21.380429 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:59:21.380405    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-graph-2-predictor-serving-cert\""
Sep 16 18:59:21.380540 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:59:21.380429    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\""
Sep 16 18:59:21.380954 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:59:21.380938    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"odh-kserve-custom-ca-bundle\""
Sep 16 18:59:21.381248 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:59:21.381226    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-graph-2-kube-rbac-proxy-sar-config\""
Sep 16 18:59:21.388579 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:59:21.388559    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-799ddc8799-hjrgh"]
Sep 16 18:59:21.407590 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:59:21.407561    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jgshn\" (UniqueName: \"kubernetes.io/projected/83dc260d-a788-4eb1-a55d-7132c5114b8f-kube-api-access-jgshn\") pod \"isvc-sklearn-graph-2-predictor-799ddc8799-hjrgh\" (UID: \"83dc260d-a788-4eb1-a55d-7132c5114b8f\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-799ddc8799-hjrgh"
Sep 16 18:59:21.407745 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:59:21.407609    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/83dc260d-a788-4eb1-a55d-7132c5114b8f-kserve-provision-location\") pod \"isvc-sklearn-graph-2-predictor-799ddc8799-hjrgh\" (UID: \"83dc260d-a788-4eb1-a55d-7132c5114b8f\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-799ddc8799-hjrgh"
Sep 16 18:59:21.407745 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:59:21.407649    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/83dc260d-a788-4eb1-a55d-7132c5114b8f-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-2-predictor-799ddc8799-hjrgh\" (UID: \"83dc260d-a788-4eb1-a55d-7132c5114b8f\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-799ddc8799-hjrgh"
Sep 16 18:59:21.407745 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:59:21.407695    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/83dc260d-a788-4eb1-a55d-7132c5114b8f-cabundle-cert\") pod \"isvc-sklearn-graph-2-predictor-799ddc8799-hjrgh\" (UID: \"83dc260d-a788-4eb1-a55d-7132c5114b8f\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-799ddc8799-hjrgh"
Sep 16 18:59:21.407745 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:59:21.407731    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/83dc260d-a788-4eb1-a55d-7132c5114b8f-proxy-tls\") pod \"isvc-sklearn-graph-2-predictor-799ddc8799-hjrgh\" (UID: \"83dc260d-a788-4eb1-a55d-7132c5114b8f\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-799ddc8799-hjrgh"
Sep 16 18:59:21.425255 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:59:21.425214    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:59:21.497265 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:59:21.497183    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:59:21.508557 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:59:21.508534    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/83dc260d-a788-4eb1-a55d-7132c5114b8f-cabundle-cert\") pod \"isvc-sklearn-graph-2-predictor-799ddc8799-hjrgh\" (UID: \"83dc260d-a788-4eb1-a55d-7132c5114b8f\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-799ddc8799-hjrgh"
Sep 16 18:59:21.508677 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:59:21.508586    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/83dc260d-a788-4eb1-a55d-7132c5114b8f-proxy-tls\") pod \"isvc-sklearn-graph-2-predictor-799ddc8799-hjrgh\" (UID: \"83dc260d-a788-4eb1-a55d-7132c5114b8f\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-799ddc8799-hjrgh"
Sep 16 18:59:21.508734 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:59:21.508651    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jgshn\" (UniqueName: \"kubernetes.io/projected/83dc260d-a788-4eb1-a55d-7132c5114b8f-kube-api-access-jgshn\") pod \"isvc-sklearn-graph-2-predictor-799ddc8799-hjrgh\" (UID: \"83dc260d-a788-4eb1-a55d-7132c5114b8f\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-799ddc8799-hjrgh"
Sep 16 18:59:21.508734 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:59:21.508722    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/83dc260d-a788-4eb1-a55d-7132c5114b8f-kserve-provision-location\") pod \"isvc-sklearn-graph-2-predictor-799ddc8799-hjrgh\" (UID: \"83dc260d-a788-4eb1-a55d-7132c5114b8f\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-799ddc8799-hjrgh"
Sep 16 18:59:21.508869 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:59:21.508756    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/83dc260d-a788-4eb1-a55d-7132c5114b8f-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-2-predictor-799ddc8799-hjrgh\" (UID: \"83dc260d-a788-4eb1-a55d-7132c5114b8f\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-799ddc8799-hjrgh"
Sep 16 18:59:21.508869 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:59:21.508779    2584 secret.go:189] Couldn't get secret kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-serving-cert: secret "isvc-sklearn-graph-2-predictor-serving-cert" not found
Sep 16 18:59:21.508972 ip-10-0-139-182 kubenswrapper[2584]: E0916 18:59:21.508874    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/83dc260d-a788-4eb1-a55d-7132c5114b8f-proxy-tls podName:83dc260d-a788-4eb1-a55d-7132c5114b8f nodeName:}" failed. No retries permitted until 2026-09-16 18:59:22.008857541 +0000 UTC m=+892.338359941 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/83dc260d-a788-4eb1-a55d-7132c5114b8f-proxy-tls") pod "isvc-sklearn-graph-2-predictor-799ddc8799-hjrgh" (UID: "83dc260d-a788-4eb1-a55d-7132c5114b8f") : secret "isvc-sklearn-graph-2-predictor-serving-cert" not found
Sep 16 18:59:21.509187 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:59:21.509164    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/83dc260d-a788-4eb1-a55d-7132c5114b8f-kserve-provision-location\") pod \"isvc-sklearn-graph-2-predictor-799ddc8799-hjrgh\" (UID: \"83dc260d-a788-4eb1-a55d-7132c5114b8f\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-799ddc8799-hjrgh"
Sep 16 18:59:21.509363 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:59:21.509347    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/83dc260d-a788-4eb1-a55d-7132c5114b8f-cabundle-cert\") pod \"isvc-sklearn-graph-2-predictor-799ddc8799-hjrgh\" (UID: \"83dc260d-a788-4eb1-a55d-7132c5114b8f\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-799ddc8799-hjrgh"
Sep 16 18:59:21.509408 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:59:21.509370    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/83dc260d-a788-4eb1-a55d-7132c5114b8f-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-2-predictor-799ddc8799-hjrgh\" (UID: \"83dc260d-a788-4eb1-a55d-7132c5114b8f\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-799ddc8799-hjrgh"
Sep 16 18:59:21.519584 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:59:21.519558    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jgshn\" (UniqueName: \"kubernetes.io/projected/83dc260d-a788-4eb1-a55d-7132c5114b8f-kube-api-access-jgshn\") pod \"isvc-sklearn-graph-2-predictor-799ddc8799-hjrgh\" (UID: \"83dc260d-a788-4eb1-a55d-7132c5114b8f\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-799ddc8799-hjrgh"
Sep 16 18:59:21.742486 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:59:21.742453    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:59:21.858199 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:59:21.858161    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:59:22.012808 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:59:22.012771    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/83dc260d-a788-4eb1-a55d-7132c5114b8f-proxy-tls\") pod \"isvc-sklearn-graph-2-predictor-799ddc8799-hjrgh\" (UID: \"83dc260d-a788-4eb1-a55d-7132c5114b8f\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-799ddc8799-hjrgh"
Sep 16 18:59:22.015132 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:59:22.015109    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/83dc260d-a788-4eb1-a55d-7132c5114b8f-proxy-tls\") pod \"isvc-sklearn-graph-2-predictor-799ddc8799-hjrgh\" (UID: \"83dc260d-a788-4eb1-a55d-7132c5114b8f\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-799ddc8799-hjrgh"
Sep 16 18:59:22.289638 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:59:22.289595    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-799ddc8799-hjrgh"
Sep 16 18:59:22.314113 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:59:22.314087    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d187efb1-1ed6-46ba-b217-2b86ed506ba4-proxy-tls\") pod \"success-200-isvc-658c9-predictor-6b7979b5bc-gvx7k\" (UID: \"d187efb1-1ed6-46ba-b217-2b86ed506ba4\") " pod="kserve-ci-e2e-test/success-200-isvc-658c9-predictor-6b7979b5bc-gvx7k"
Sep 16 18:59:22.316588 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:59:22.316560    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d187efb1-1ed6-46ba-b217-2b86ed506ba4-proxy-tls\") pod \"success-200-isvc-658c9-predictor-6b7979b5bc-gvx7k\" (UID: \"d187efb1-1ed6-46ba-b217-2b86ed506ba4\") " pod="kserve-ci-e2e-test/success-200-isvc-658c9-predictor-6b7979b5bc-gvx7k"
Sep 16 18:59:22.513262 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:59:22.513222    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-658c9-predictor-6b7979b5bc-gvx7k"
Sep 16 18:59:22.564927 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:59:22.564900    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-799ddc8799-hjrgh"]
Sep 16 18:59:22.567073 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:59:22.566938    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83dc260d_a788_4eb1_a55d_7132c5114b8f.slice/crio-9bcc3ea89a47259ee0c655d51eb5c498841e117661f3156d39f7ed6e4c8b153b WatchSource:0}: Error finding container 9bcc3ea89a47259ee0c655d51eb5c498841e117661f3156d39f7ed6e4c8b153b: Status 404 returned error can't find the container with id 9bcc3ea89a47259ee0c655d51eb5c498841e117661f3156d39f7ed6e4c8b153b
Sep 16 18:59:22.785805 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:59:22.785726    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-658c9-predictor-6b7979b5bc-gvx7k"]
Sep 16 18:59:22.788800 ip-10-0-139-182 kubenswrapper[2584]: W0916 18:59:22.788776    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd187efb1_1ed6_46ba_b217_2b86ed506ba4.slice/crio-b0b48522d1404fb0a0975d035a815e3832d477cdaea24b7927bf414a719bbc2b WatchSource:0}: Error finding container b0b48522d1404fb0a0975d035a815e3832d477cdaea24b7927bf414a719bbc2b: Status 404 returned error can't find the container with id b0b48522d1404fb0a0975d035a815e3832d477cdaea24b7927bf414a719bbc2b
Sep 16 18:59:23.344223 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:59:23.344186    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-799ddc8799-hjrgh" event={"ID":"83dc260d-a788-4eb1-a55d-7132c5114b8f","Type":"ContainerStarted","Data":"9bcc3ea89a47259ee0c655d51eb5c498841e117661f3156d39f7ed6e4c8b153b"}
Sep 16 18:59:23.347479 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:59:23.347446    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-658c9-predictor-6b7979b5bc-gvx7k" event={"ID":"d187efb1-1ed6-46ba-b217-2b86ed506ba4","Type":"ContainerStarted","Data":"b0b48522d1404fb0a0975d035a815e3832d477cdaea24b7927bf414a719bbc2b"}
Sep 16 18:59:27.501990 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:59:27.501944    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 18:59:36.049473 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:59:36.049437    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\""
Sep 16 18:59:36.066392 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:59:36.066360    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-crjw7_ecf273b5-b465-4008-b035-f7ac0e740d58/console-operator/1.log"
Sep 16 18:59:36.066668 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:59:36.066647    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-crjw7_ecf273b5-b465-4008-b035-f7ac0e740d58/console-operator/1.log"
Sep 16 18:59:36.204128 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:59:36.204110    2584 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 16 18:59:36.419408 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:59:36.419310    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-799ddc8799-hjrgh" event={"ID":"83dc260d-a788-4eb1-a55d-7132c5114b8f","Type":"ContainerStarted","Data":"e6be8383f57e45b5a3e14fd79441fc74d1ed33182e0e481819d32c8a71b45920"}
Sep 16 18:59:36.420869 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:59:36.420845    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-658c9-predictor-6b7979b5bc-gvx7k" event={"ID":"d187efb1-1ed6-46ba-b217-2b86ed506ba4","Type":"ContainerStarted","Data":"1a7b87620c0f47c27aa4409b697f34f18c78721d484d8918fcd849b419352f43"}
Sep 16 18:59:39.438462 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:59:39.438424    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-658c9-predictor-6b7979b5bc-gvx7k" event={"ID":"d187efb1-1ed6-46ba-b217-2b86ed506ba4","Type":"ContainerStarted","Data":"faac47a1e79f32f7ba1efcb8b9da9a16f1092fd8480293d6750919d6268c9274"}
Sep 16 18:59:39.438888 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:59:39.438538    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-658c9-predictor-6b7979b5bc-gvx7k"
Sep 16 18:59:39.457391 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:59:39.457335    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-658c9-predictor-6b7979b5bc-gvx7k" podStartSLOduration=3.80163353 podStartE2EDuration="19.457320272s" totalImagesPullingTime="15.655686742s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:59:20 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:59:39.456187646 +0000 UTC m=+909.785690065" watchObservedRunningTime="2026-09-16 18:59:39.457320272 +0000 UTC m=+909.786822685"
Sep 16 18:59:40.444454 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:59:40.444418    2584 generic.go:356] "Generic (PLEG): container finished" podID="83dc260d-a788-4eb1-a55d-7132c5114b8f" containerID="e6be8383f57e45b5a3e14fd79441fc74d1ed33182e0e481819d32c8a71b45920" exitCode=0
Sep 16 18:59:40.445024 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:59:40.444487    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-799ddc8799-hjrgh" event={"ID":"83dc260d-a788-4eb1-a55d-7132c5114b8f","Type":"ContainerDied","Data":"e6be8383f57e45b5a3e14fd79441fc74d1ed33182e0e481819d32c8a71b45920"}
Sep 16 18:59:40.445024 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:59:40.444772    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-658c9-predictor-6b7979b5bc-gvx7k"
Sep 16 18:59:40.446537 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:59:40.446506    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-658c9-predictor-6b7979b5bc-gvx7k" podUID="d187efb1-1ed6-46ba-b217-2b86ed506ba4" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.30:8080: connect: connection refused"
Sep 16 18:59:41.450094 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:59:41.450047    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-658c9-predictor-6b7979b5bc-gvx7k" podUID="d187efb1-1ed6-46ba-b217-2b86ed506ba4" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.30:8080: connect: connection refused"
Sep 16 18:59:46.454318 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:59:46.454288    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-658c9-predictor-6b7979b5bc-gvx7k"
Sep 16 18:59:46.454710 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:59:46.454685    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-658c9-predictor-6b7979b5bc-gvx7k" podUID="d187efb1-1ed6-46ba-b217-2b86ed506ba4" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.30:8080: connect: connection refused"
Sep 16 18:59:47.480804 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:59:47.480769    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-799ddc8799-hjrgh" event={"ID":"83dc260d-a788-4eb1-a55d-7132c5114b8f","Type":"ContainerStarted","Data":"fbe95d802cda97354e101d955f9b23286b9757b06c6d20a2d8ac49512fa63519"}
Sep 16 18:59:47.481206 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:59:47.480853    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-799ddc8799-hjrgh" event={"ID":"83dc260d-a788-4eb1-a55d-7132c5114b8f","Type":"ContainerStarted","Data":"1d54c6e771855882ae08e598a780153939746f5f5871493d62a78e260f92be2e"}
Sep 16 18:59:47.481206 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:59:47.481099    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-799ddc8799-hjrgh"
Sep 16 18:59:47.499978 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:59:47.499921    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-799ddc8799-hjrgh" podStartSLOduration=3.362420792 podStartE2EDuration="26.499902326s" totalImagesPullingTime="19.521600423s" totalInitContainerRuntime="3.615881111s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:59:21 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:59:47.49907938 +0000 UTC m=+917.828581805" watchObservedRunningTime="2026-09-16 18:59:47.499902326 +0000 UTC m=+917.829404738"
Sep 16 18:59:48.485361 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:59:48.485325    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-799ddc8799-hjrgh"
Sep 16 18:59:48.486721 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:59:48.486689    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-799ddc8799-hjrgh" podUID="83dc260d-a788-4eb1-a55d-7132c5114b8f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.31:8080: connect: connection refused"
Sep 16 18:59:49.489412 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:59:49.489368    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-799ddc8799-hjrgh" podUID="83dc260d-a788-4eb1-a55d-7132c5114b8f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.31:8080: connect: connection refused"
Sep 16 18:59:54.494565 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:59:54.494535    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-799ddc8799-hjrgh"
Sep 16 18:59:54.495082 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:59:54.495054    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-799ddc8799-hjrgh" podUID="83dc260d-a788-4eb1-a55d-7132c5114b8f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.31:8080: connect: connection refused"
Sep 16 18:59:56.455759 ip-10-0-139-182 kubenswrapper[2584]: I0916 18:59:56.455715    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-658c9-predictor-6b7979b5bc-gvx7k" podUID="d187efb1-1ed6-46ba-b217-2b86ed506ba4" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.30:8080: connect: connection refused"
Sep 16 19:00:04.495859 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:00:04.495804    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-799ddc8799-hjrgh" podUID="83dc260d-a788-4eb1-a55d-7132c5114b8f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.31:8080: connect: connection refused"
Sep 16 19:00:06.455524 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:00:06.455481    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-658c9-predictor-6b7979b5bc-gvx7k" podUID="d187efb1-1ed6-46ba-b217-2b86ed506ba4" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.30:8080: connect: connection refused"
Sep 16 19:00:14.495087 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:00:14.495045    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-799ddc8799-hjrgh" podUID="83dc260d-a788-4eb1-a55d-7132c5114b8f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.31:8080: connect: connection refused"
Sep 16 19:00:16.455054 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:00:16.455007    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-658c9-predictor-6b7979b5bc-gvx7k" podUID="d187efb1-1ed6-46ba-b217-2b86ed506ba4" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.30:8080: connect: connection refused"
Sep 16 19:00:24.495647 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:00:24.495604    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-799ddc8799-hjrgh" podUID="83dc260d-a788-4eb1-a55d-7132c5114b8f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.31:8080: connect: connection refused"
Sep 16 19:00:26.455649 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:00:26.455605    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-658c9-predictor-6b7979b5bc-gvx7k"
Sep 16 19:00:28.538189 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:00:28.538156    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 19:00:34.495215 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:00:34.495171    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-799ddc8799-hjrgh" podUID="83dc260d-a788-4eb1-a55d-7132c5114b8f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.31:8080: connect: connection refused"
Sep 16 19:00:44.495064 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:00:44.495027    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-799ddc8799-hjrgh" podUID="83dc260d-a788-4eb1-a55d-7132c5114b8f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.31:8080: connect: connection refused"
Sep 16 19:00:50.824693 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:00:50.824651    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/switch-graph-658c9-84496b65dc-jv894"]
Sep 16 19:00:50.828158 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:00:50.828138    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-658c9-84496b65dc-jv894"
Sep 16 19:00:50.831246 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:00:50.831222    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"switch-graph-658c9-kube-rbac-proxy-sar-config\""
Sep 16 19:00:50.831600 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:00:50.831578    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"switch-graph-658c9-serving-cert\""
Sep 16 19:00:50.841122 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:00:50.841097    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/switch-graph-658c9-84496b65dc-jv894"]
Sep 16 19:00:50.842134 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:00:50.842104    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 19:00:50.982986 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:00:50.982955    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6413f28e-243d-48cf-9983-7519b048ca6f-openshift-service-ca-bundle\") pod \"switch-graph-658c9-84496b65dc-jv894\" (UID: \"6413f28e-243d-48cf-9983-7519b048ca6f\") " pod="kserve-ci-e2e-test/switch-graph-658c9-84496b65dc-jv894"
Sep 16 19:00:50.983154 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:00:50.983019    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/6413f28e-243d-48cf-9983-7519b048ca6f-proxy-tls\") pod \"switch-graph-658c9-84496b65dc-jv894\" (UID: \"6413f28e-243d-48cf-9983-7519b048ca6f\") " pod="kserve-ci-e2e-test/switch-graph-658c9-84496b65dc-jv894"
Sep 16 19:00:51.084174 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:00:51.084083    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/6413f28e-243d-48cf-9983-7519b048ca6f-proxy-tls\") pod \"switch-graph-658c9-84496b65dc-jv894\" (UID: \"6413f28e-243d-48cf-9983-7519b048ca6f\") " pod="kserve-ci-e2e-test/switch-graph-658c9-84496b65dc-jv894"
Sep 16 19:00:51.084174 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:00:51.084159    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6413f28e-243d-48cf-9983-7519b048ca6f-openshift-service-ca-bundle\") pod \"switch-graph-658c9-84496b65dc-jv894\" (UID: \"6413f28e-243d-48cf-9983-7519b048ca6f\") " pod="kserve-ci-e2e-test/switch-graph-658c9-84496b65dc-jv894"
Sep 16 19:00:51.084743 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:00:51.084713    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6413f28e-243d-48cf-9983-7519b048ca6f-openshift-service-ca-bundle\") pod \"switch-graph-658c9-84496b65dc-jv894\" (UID: \"6413f28e-243d-48cf-9983-7519b048ca6f\") " pod="kserve-ci-e2e-test/switch-graph-658c9-84496b65dc-jv894"
Sep 16 19:00:51.086672 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:00:51.086656    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/6413f28e-243d-48cf-9983-7519b048ca6f-proxy-tls\") pod \"switch-graph-658c9-84496b65dc-jv894\" (UID: \"6413f28e-243d-48cf-9983-7519b048ca6f\") " pod="kserve-ci-e2e-test/switch-graph-658c9-84496b65dc-jv894"
Sep 16 19:00:51.117124 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:00:51.117096    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 19:00:51.137933 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:00:51.137911    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-658c9-84496b65dc-jv894"
Sep 16 19:00:51.417280 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:00:51.417251    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/switch-graph-658c9-84496b65dc-jv894"]
Sep 16 19:00:51.419263 ip-10-0-139-182 kubenswrapper[2584]: W0916 19:00:51.419239    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6413f28e_243d_48cf_9983_7519b048ca6f.slice/crio-f1149a4aece1ce80979976673e4575249180cba0ec622c0b7ca09bf2eb3cd5e1 WatchSource:0}: Error finding container f1149a4aece1ce80979976673e4575249180cba0ec622c0b7ca09bf2eb3cd5e1: Status 404 returned error can't find the container with id f1149a4aece1ce80979976673e4575249180cba0ec622c0b7ca09bf2eb3cd5e1
Sep 16 19:00:51.748550 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:00:51.748466    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-658c9-84496b65dc-jv894" event={"ID":"6413f28e-243d-48cf-9983-7519b048ca6f","Type":"ContainerStarted","Data":"f1149a4aece1ce80979976673e4575249180cba0ec622c0b7ca09bf2eb3cd5e1"}
Sep 16 19:00:53.757756 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:00:53.757722    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-658c9-84496b65dc-jv894" event={"ID":"6413f28e-243d-48cf-9983-7519b048ca6f","Type":"ContainerStarted","Data":"74797f5a64293bdf5fe073126cb2a2ed96497bdb8ac77d0945af70d35ab3e9dc"}
Sep 16 19:00:53.758193 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:00:53.757787    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/switch-graph-658c9-84496b65dc-jv894"
Sep 16 19:00:53.783876 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:00:53.783814    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/switch-graph-658c9-84496b65dc-jv894" podStartSLOduration=1.662379943 podStartE2EDuration="3.783799872s" totalImagesPullingTime="2.121419929s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 19:00:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 19:00:53.783412735 +0000 UTC m=+984.112915148" watchObservedRunningTime="2026-09-16 19:00:53.783799872 +0000 UTC m=+984.113302285"
Sep 16 19:00:54.495356 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:00:54.495312    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-799ddc8799-hjrgh" podUID="83dc260d-a788-4eb1-a55d-7132c5114b8f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.31:8080: connect: connection refused"
Sep 16 19:00:56.111622 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:00:56.111583    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-799ddc8799-hjrgh" podUID="83dc260d-a788-4eb1-a55d-7132c5114b8f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.31:8080: connect: connection refused"
Sep 16 19:00:59.767579 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:00:59.767550    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/switch-graph-658c9-84496b65dc-jv894"
Sep 16 19:01:00.916572 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:00.916535    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/switch-graph-658c9-84496b65dc-jv894"]
Sep 16 19:01:00.916963 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:00.916784    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/switch-graph-658c9-84496b65dc-jv894" podUID="6413f28e-243d-48cf-9983-7519b048ca6f" containerName="switch-graph-658c9" containerID="cri-o://74797f5a64293bdf5fe073126cb2a2ed96497bdb8ac77d0945af70d35ab3e9dc" gracePeriod=30
Sep 16 19:01:00.954012 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:00.953979    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 19:01:01.116602 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:01.116569    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-658c9-predictor-6b7979b5bc-gvx7k"]
Sep 16 19:01:01.116801 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:01.116780    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-658c9-predictor-6b7979b5bc-gvx7k" podUID="d187efb1-1ed6-46ba-b217-2b86ed506ba4" containerName="kserve-container" containerID="cri-o://1a7b87620c0f47c27aa4409b697f34f18c78721d484d8918fcd849b419352f43" gracePeriod=30
Sep 16 19:01:01.116958 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:01.116915    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-658c9-predictor-6b7979b5bc-gvx7k" podUID="d187efb1-1ed6-46ba-b217-2b86ed506ba4" containerName="kube-rbac-proxy" containerID="cri-o://faac47a1e79f32f7ba1efcb8b9da9a16f1092fd8480293d6750919d6268c9274" gracePeriod=30
Sep 16 19:01:01.144434 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:01.144332    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 19:01:01.244133 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:01.244098    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 19:01:01.302346 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:01.302312    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-93efe-predictor-85b4cff8bf-k95wm"]
Sep 16 19:01:01.305909 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:01.305890    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-93efe-predictor-85b4cff8bf-k95wm"
Sep 16 19:01:01.308932 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:01.308903    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-93efe-predictor-serving-cert\""
Sep 16 19:01:01.308932 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:01.308919    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-93efe-kube-rbac-proxy-sar-config\""
Sep 16 19:01:01.318162 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:01.318133    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-93efe-predictor-85b4cff8bf-k95wm"]
Sep 16 19:01:01.323730 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:01.323707    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 19:01:01.365891 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:01.365823    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/771df157-671d-4a80-aeea-88aca624578a-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-93efe-predictor-85b4cff8bf-k95wm\" (UID: \"771df157-671d-4a80-aeea-88aca624578a\") " pod="kserve-ci-e2e-test/success-200-isvc-93efe-predictor-85b4cff8bf-k95wm"
Sep 16 19:01:01.366030 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:01.365945    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7whxt\" (UniqueName: \"kubernetes.io/projected/771df157-671d-4a80-aeea-88aca624578a-kube-api-access-7whxt\") pod \"success-200-isvc-93efe-predictor-85b4cff8bf-k95wm\" (UID: \"771df157-671d-4a80-aeea-88aca624578a\") " pod="kserve-ci-e2e-test/success-200-isvc-93efe-predictor-85b4cff8bf-k95wm"
Sep 16 19:01:01.366030 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:01.365967    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/771df157-671d-4a80-aeea-88aca624578a-proxy-tls\") pod \"success-200-isvc-93efe-predictor-85b4cff8bf-k95wm\" (UID: \"771df157-671d-4a80-aeea-88aca624578a\") " pod="kserve-ci-e2e-test/success-200-isvc-93efe-predictor-85b4cff8bf-k95wm"
Sep 16 19:01:01.419689 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:01.419645    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 19:01:01.450778 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:01.450733    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-658c9-predictor-6b7979b5bc-gvx7k" podUID="d187efb1-1ed6-46ba-b217-2b86ed506ba4" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.30:8643/healthz\": dial tcp 10.133.0.30:8643: connect: connection refused"
Sep 16 19:01:01.467031 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:01.466998    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/771df157-671d-4a80-aeea-88aca624578a-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-93efe-predictor-85b4cff8bf-k95wm\" (UID: \"771df157-671d-4a80-aeea-88aca624578a\") " pod="kserve-ci-e2e-test/success-200-isvc-93efe-predictor-85b4cff8bf-k95wm"
Sep 16 19:01:01.467175 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:01.467084    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7whxt\" (UniqueName: \"kubernetes.io/projected/771df157-671d-4a80-aeea-88aca624578a-kube-api-access-7whxt\") pod \"success-200-isvc-93efe-predictor-85b4cff8bf-k95wm\" (UID: \"771df157-671d-4a80-aeea-88aca624578a\") " pod="kserve-ci-e2e-test/success-200-isvc-93efe-predictor-85b4cff8bf-k95wm"
Sep 16 19:01:01.467175 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:01.467104    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/771df157-671d-4a80-aeea-88aca624578a-proxy-tls\") pod \"success-200-isvc-93efe-predictor-85b4cff8bf-k95wm\" (UID: \"771df157-671d-4a80-aeea-88aca624578a\") " pod="kserve-ci-e2e-test/success-200-isvc-93efe-predictor-85b4cff8bf-k95wm"
Sep 16 19:01:01.467665 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:01.467643    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/771df157-671d-4a80-aeea-88aca624578a-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-93efe-predictor-85b4cff8bf-k95wm\" (UID: \"771df157-671d-4a80-aeea-88aca624578a\") " pod="kserve-ci-e2e-test/success-200-isvc-93efe-predictor-85b4cff8bf-k95wm"
Sep 16 19:01:01.469500 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:01.469479    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/771df157-671d-4a80-aeea-88aca624578a-proxy-tls\") pod \"success-200-isvc-93efe-predictor-85b4cff8bf-k95wm\" (UID: \"771df157-671d-4a80-aeea-88aca624578a\") " pod="kserve-ci-e2e-test/success-200-isvc-93efe-predictor-85b4cff8bf-k95wm"
Sep 16 19:01:01.480102 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:01.480081    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7whxt\" (UniqueName: \"kubernetes.io/projected/771df157-671d-4a80-aeea-88aca624578a-kube-api-access-7whxt\") pod \"success-200-isvc-93efe-predictor-85b4cff8bf-k95wm\" (UID: \"771df157-671d-4a80-aeea-88aca624578a\") " pod="kserve-ci-e2e-test/success-200-isvc-93efe-predictor-85b4cff8bf-k95wm"
Sep 16 19:01:01.503900 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:01.503822    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 19:01:01.602527 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:01.602491    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 19:01:01.616449 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:01.616419    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-93efe-predictor-85b4cff8bf-k95wm"
Sep 16 19:01:01.793424 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:01.793390    2584 generic.go:356] "Generic (PLEG): container finished" podID="d187efb1-1ed6-46ba-b217-2b86ed506ba4" containerID="faac47a1e79f32f7ba1efcb8b9da9a16f1092fd8480293d6750919d6268c9274" exitCode=2
Sep 16 19:01:01.793592 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:01.793443    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-658c9-predictor-6b7979b5bc-gvx7k" event={"ID":"d187efb1-1ed6-46ba-b217-2b86ed506ba4","Type":"ContainerDied","Data":"faac47a1e79f32f7ba1efcb8b9da9a16f1092fd8480293d6750919d6268c9274"}
Sep 16 19:01:01.897883 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:01.897855    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-93efe-predictor-85b4cff8bf-k95wm"]
Sep 16 19:01:01.899486 ip-10-0-139-182 kubenswrapper[2584]: W0916 19:01:01.899462    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod771df157_671d_4a80_aeea_88aca624578a.slice/crio-7f3aebb5b8373f58d500b75082499f4df9e7b483899e47dc44620fc679c0aca7 WatchSource:0}: Error finding container 7f3aebb5b8373f58d500b75082499f4df9e7b483899e47dc44620fc679c0aca7: Status 404 returned error can't find the container with id 7f3aebb5b8373f58d500b75082499f4df9e7b483899e47dc44620fc679c0aca7
Sep 16 19:01:02.799393 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:02.799347    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-93efe-predictor-85b4cff8bf-k95wm" event={"ID":"771df157-671d-4a80-aeea-88aca624578a","Type":"ContainerStarted","Data":"9be062ac6b74f816fd05e6355e30f7eaf4ea266d45622076a8f7a156611a62a7"}
Sep 16 19:01:02.799914 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:02.799424    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-93efe-predictor-85b4cff8bf-k95wm" event={"ID":"771df157-671d-4a80-aeea-88aca624578a","Type":"ContainerStarted","Data":"3c0f4d4fb04a891cda2799ecc1eabbcfb524bfeabaeb2cf325fff84dc45c28c9"}
Sep 16 19:01:02.799914 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:02.799458    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-93efe-predictor-85b4cff8bf-k95wm"
Sep 16 19:01:02.799914 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:02.799489    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-93efe-predictor-85b4cff8bf-k95wm" event={"ID":"771df157-671d-4a80-aeea-88aca624578a","Type":"ContainerStarted","Data":"7f3aebb5b8373f58d500b75082499f4df9e7b483899e47dc44620fc679c0aca7"}
Sep 16 19:01:02.799914 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:02.799521    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-93efe-predictor-85b4cff8bf-k95wm"
Sep 16 19:01:02.801104 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:02.801076    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-93efe-predictor-85b4cff8bf-k95wm" podUID="771df157-671d-4a80-aeea-88aca624578a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.33:8080: connect: connection refused"
Sep 16 19:01:02.819692 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:02.819643    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-93efe-predictor-85b4cff8bf-k95wm" podStartSLOduration=1.819627697 podStartE2EDuration="1.819627697s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 19:01:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 19:01:02.818308008 +0000 UTC m=+993.147810422" watchObservedRunningTime="2026-09-16 19:01:02.819627697 +0000 UTC m=+993.149130173"
Sep 16 19:01:03.803870 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:03.803816    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-93efe-predictor-85b4cff8bf-k95wm" podUID="771df157-671d-4a80-aeea-88aca624578a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.33:8080: connect: connection refused"
Sep 16 19:01:04.263983 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:04.263959    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-658c9-predictor-6b7979b5bc-gvx7k"
Sep 16 19:01:04.291179 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:04.291150    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rjxkx\" (UniqueName: \"kubernetes.io/projected/d187efb1-1ed6-46ba-b217-2b86ed506ba4-kube-api-access-rjxkx\") pod \"d187efb1-1ed6-46ba-b217-2b86ed506ba4\" (UID: \"d187efb1-1ed6-46ba-b217-2b86ed506ba4\") "
Sep 16 19:01:04.291325 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:04.291236    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d187efb1-1ed6-46ba-b217-2b86ed506ba4-kube-rbac-proxy-sar-config\") pod \"d187efb1-1ed6-46ba-b217-2b86ed506ba4\" (UID: \"d187efb1-1ed6-46ba-b217-2b86ed506ba4\") "
Sep 16 19:01:04.291325 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:04.291306    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d187efb1-1ed6-46ba-b217-2b86ed506ba4-proxy-tls\") pod \"d187efb1-1ed6-46ba-b217-2b86ed506ba4\" (UID: \"d187efb1-1ed6-46ba-b217-2b86ed506ba4\") "
Sep 16 19:01:04.291603 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:04.291578    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d187efb1-1ed6-46ba-b217-2b86ed506ba4-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "d187efb1-1ed6-46ba-b217-2b86ed506ba4" (UID: "d187efb1-1ed6-46ba-b217-2b86ed506ba4"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 19:01:04.291697 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:04.291674    2584 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d187efb1-1ed6-46ba-b217-2b86ed506ba4-kube-rbac-proxy-sar-config\") on node \"ip-10-0-139-182.ec2.internal\" DevicePath \"\""
Sep 16 19:01:04.293534 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:04.293512    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d187efb1-1ed6-46ba-b217-2b86ed506ba4-kube-api-access-rjxkx" (OuterVolumeSpecName: "kube-api-access-rjxkx") pod "d187efb1-1ed6-46ba-b217-2b86ed506ba4" (UID: "d187efb1-1ed6-46ba-b217-2b86ed506ba4"). InnerVolumeSpecName "kube-api-access-rjxkx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 19:01:04.293610 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:04.293528    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d187efb1-1ed6-46ba-b217-2b86ed506ba4-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "d187efb1-1ed6-46ba-b217-2b86ed506ba4" (UID: "d187efb1-1ed6-46ba-b217-2b86ed506ba4"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 19:01:04.392192 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:04.392119    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rjxkx\" (UniqueName: \"kubernetes.io/projected/d187efb1-1ed6-46ba-b217-2b86ed506ba4-kube-api-access-rjxkx\") on node \"ip-10-0-139-182.ec2.internal\" DevicePath \"\""
Sep 16 19:01:04.392192 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:04.392144    2584 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d187efb1-1ed6-46ba-b217-2b86ed506ba4-proxy-tls\") on node \"ip-10-0-139-182.ec2.internal\" DevicePath \"\""
Sep 16 19:01:04.766518 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:04.766483    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-658c9-84496b65dc-jv894" podUID="6413f28e-243d-48cf-9983-7519b048ca6f" containerName="switch-graph-658c9" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 16 19:01:04.814608 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:04.814572    2584 generic.go:356] "Generic (PLEG): container finished" podID="d187efb1-1ed6-46ba-b217-2b86ed506ba4" containerID="1a7b87620c0f47c27aa4409b697f34f18c78721d484d8918fcd849b419352f43" exitCode=0
Sep 16 19:01:04.815058 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:04.814632    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-658c9-predictor-6b7979b5bc-gvx7k"
Sep 16 19:01:04.815058 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:04.814633    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-658c9-predictor-6b7979b5bc-gvx7k" event={"ID":"d187efb1-1ed6-46ba-b217-2b86ed506ba4","Type":"ContainerDied","Data":"1a7b87620c0f47c27aa4409b697f34f18c78721d484d8918fcd849b419352f43"}
Sep 16 19:01:04.815058 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:04.814778    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-658c9-predictor-6b7979b5bc-gvx7k" event={"ID":"d187efb1-1ed6-46ba-b217-2b86ed506ba4","Type":"ContainerDied","Data":"b0b48522d1404fb0a0975d035a815e3832d477cdaea24b7927bf414a719bbc2b"}
Sep 16 19:01:04.815058 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:04.814820    2584 scope.go:117] "RemoveContainer" containerID="faac47a1e79f32f7ba1efcb8b9da9a16f1092fd8480293d6750919d6268c9274"
Sep 16 19:01:04.825125 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:04.825077    2584 scope.go:117] "RemoveContainer" containerID="1a7b87620c0f47c27aa4409b697f34f18c78721d484d8918fcd849b419352f43"
Sep 16 19:01:04.833911 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:04.833819    2584 scope.go:117] "RemoveContainer" containerID="faac47a1e79f32f7ba1efcb8b9da9a16f1092fd8480293d6750919d6268c9274"
Sep 16 19:01:04.834272 ip-10-0-139-182 kubenswrapper[2584]: E0916 19:01:04.834243    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"faac47a1e79f32f7ba1efcb8b9da9a16f1092fd8480293d6750919d6268c9274\": container with ID starting with faac47a1e79f32f7ba1efcb8b9da9a16f1092fd8480293d6750919d6268c9274 not found: ID does not exist" containerID="faac47a1e79f32f7ba1efcb8b9da9a16f1092fd8480293d6750919d6268c9274"
Sep 16 19:01:04.834319 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:04.834288    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"faac47a1e79f32f7ba1efcb8b9da9a16f1092fd8480293d6750919d6268c9274"} err="failed to get container status \"faac47a1e79f32f7ba1efcb8b9da9a16f1092fd8480293d6750919d6268c9274\": rpc error: code = NotFound desc = could not find container \"faac47a1e79f32f7ba1efcb8b9da9a16f1092fd8480293d6750919d6268c9274\": container with ID starting with faac47a1e79f32f7ba1efcb8b9da9a16f1092fd8480293d6750919d6268c9274 not found: ID does not exist"
Sep 16 19:01:04.834319 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:04.834312    2584 scope.go:117] "RemoveContainer" containerID="1a7b87620c0f47c27aa4409b697f34f18c78721d484d8918fcd849b419352f43"
Sep 16 19:01:04.834608 ip-10-0-139-182 kubenswrapper[2584]: E0916 19:01:04.834590    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1a7b87620c0f47c27aa4409b697f34f18c78721d484d8918fcd849b419352f43\": container with ID starting with 1a7b87620c0f47c27aa4409b697f34f18c78721d484d8918fcd849b419352f43 not found: ID does not exist" containerID="1a7b87620c0f47c27aa4409b697f34f18c78721d484d8918fcd849b419352f43"
Sep 16 19:01:04.834679 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:04.834610    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1a7b87620c0f47c27aa4409b697f34f18c78721d484d8918fcd849b419352f43"} err="failed to get container status \"1a7b87620c0f47c27aa4409b697f34f18c78721d484d8918fcd849b419352f43\": rpc error: code = NotFound desc = could not find container \"1a7b87620c0f47c27aa4409b697f34f18c78721d484d8918fcd849b419352f43\": container with ID starting with 1a7b87620c0f47c27aa4409b697f34f18c78721d484d8918fcd849b419352f43 not found: ID does not exist"
Sep 16 19:01:04.856293 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:04.856266    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-658c9-predictor-6b7979b5bc-gvx7k"]
Sep 16 19:01:04.867624 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:04.867594    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-658c9-predictor-6b7979b5bc-gvx7k"]
Sep 16 19:01:06.117393 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:06.117360    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d187efb1-1ed6-46ba-b217-2b86ed506ba4" path="/var/lib/kubelet/pods/d187efb1-1ed6-46ba-b217-2b86ed506ba4/volumes"
Sep 16 19:01:06.117774 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:06.117708    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-799ddc8799-hjrgh"
Sep 16 19:01:08.808465 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:08.808438    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-93efe-predictor-85b4cff8bf-k95wm"
Sep 16 19:01:08.808935 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:08.808910    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-93efe-predictor-85b4cff8bf-k95wm" podUID="771df157-671d-4a80-aeea-88aca624578a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.33:8080: connect: connection refused"
Sep 16 19:01:09.766076 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:09.766041    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-658c9-84496b65dc-jv894" podUID="6413f28e-243d-48cf-9983-7519b048ca6f" containerName="switch-graph-658c9" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 16 19:01:14.159445 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:14.159410    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 19:01:14.766414 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:14.766369    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-658c9-84496b65dc-jv894" podUID="6413f28e-243d-48cf-9983-7519b048ca6f" containerName="switch-graph-658c9" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 16 19:01:14.766578 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:14.766475    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/switch-graph-658c9-84496b65dc-jv894"
Sep 16 19:01:18.808924 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:18.808884    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-93efe-predictor-85b4cff8bf-k95wm" podUID="771df157-671d-4a80-aeea-88aca624578a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.33:8080: connect: connection refused"
Sep 16 19:01:19.765968 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:19.765931    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-658c9-84496b65dc-jv894" podUID="6413f28e-243d-48cf-9983-7519b048ca6f" containerName="switch-graph-658c9" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 16 19:01:24.550907 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:24.550872    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 19:01:24.766663 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:24.766625    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-658c9-84496b65dc-jv894" podUID="6413f28e-243d-48cf-9983-7519b048ca6f" containerName="switch-graph-658c9" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 16 19:01:28.809190 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:28.809151    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-93efe-predictor-85b4cff8bf-k95wm" podUID="771df157-671d-4a80-aeea-88aca624578a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.33:8080: connect: connection refused"
Sep 16 19:01:29.766376 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:29.766340    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-658c9-84496b65dc-jv894" podUID="6413f28e-243d-48cf-9983-7519b048ca6f" containerName="switch-graph-658c9" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 16 19:01:30.750142 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:30.750105    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/model-chainer-5fdf84b867-44hvl"]
Sep 16 19:01:30.750494 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:30.750459    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d187efb1-1ed6-46ba-b217-2b86ed506ba4" containerName="kube-rbac-proxy"
Sep 16 19:01:30.750494 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:30.750475    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="d187efb1-1ed6-46ba-b217-2b86ed506ba4" containerName="kube-rbac-proxy"
Sep 16 19:01:30.750494 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:30.750494    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d187efb1-1ed6-46ba-b217-2b86ed506ba4" containerName="kserve-container"
Sep 16 19:01:30.750618 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:30.750499    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="d187efb1-1ed6-46ba-b217-2b86ed506ba4" containerName="kserve-container"
Sep 16 19:01:30.750618 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:30.750567    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="d187efb1-1ed6-46ba-b217-2b86ed506ba4" containerName="kserve-container"
Sep 16 19:01:30.750618 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:30.750575    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="d187efb1-1ed6-46ba-b217-2b86ed506ba4" containerName="kube-rbac-proxy"
Sep 16 19:01:30.752709 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:30.752686    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 19:01:30.754806 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:30.754789    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-5fdf84b867-44hvl"
Sep 16 19:01:30.757602 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:30.757582    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"model-chainer-kube-rbac-proxy-sar-config\""
Sep 16 19:01:30.757767 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:30.757749    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"model-chainer-serving-cert\""
Sep 16 19:01:30.763734 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:30.763712    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/model-chainer-5fdf84b867-44hvl"]
Sep 16 19:01:30.814629 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:30.814586    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/037f0caf-e2d5-4e4e-a252-417a0a69d8b6-openshift-service-ca-bundle\") pod \"model-chainer-5fdf84b867-44hvl\" (UID: \"037f0caf-e2d5-4e4e-a252-417a0a69d8b6\") " pod="kserve-ci-e2e-test/model-chainer-5fdf84b867-44hvl"
Sep 16 19:01:30.814788 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:30.814725    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/037f0caf-e2d5-4e4e-a252-417a0a69d8b6-proxy-tls\") pod \"model-chainer-5fdf84b867-44hvl\" (UID: \"037f0caf-e2d5-4e4e-a252-417a0a69d8b6\") " pod="kserve-ci-e2e-test/model-chainer-5fdf84b867-44hvl"
Sep 16 19:01:30.915437 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:30.915398    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/037f0caf-e2d5-4e4e-a252-417a0a69d8b6-openshift-service-ca-bundle\") pod \"model-chainer-5fdf84b867-44hvl\" (UID: \"037f0caf-e2d5-4e4e-a252-417a0a69d8b6\") " pod="kserve-ci-e2e-test/model-chainer-5fdf84b867-44hvl"
Sep 16 19:01:30.915626 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:30.915515    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/037f0caf-e2d5-4e4e-a252-417a0a69d8b6-proxy-tls\") pod \"model-chainer-5fdf84b867-44hvl\" (UID: \"037f0caf-e2d5-4e4e-a252-417a0a69d8b6\") " pod="kserve-ci-e2e-test/model-chainer-5fdf84b867-44hvl"
Sep 16 19:01:30.915698 ip-10-0-139-182 kubenswrapper[2584]: E0916 19:01:30.915657    2584 secret.go:189] Couldn't get secret kserve-ci-e2e-test/model-chainer-serving-cert: secret "model-chainer-serving-cert" not found
Sep 16 19:01:30.915752 ip-10-0-139-182 kubenswrapper[2584]: E0916 19:01:30.915739    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/037f0caf-e2d5-4e4e-a252-417a0a69d8b6-proxy-tls podName:037f0caf-e2d5-4e4e-a252-417a0a69d8b6 nodeName:}" failed. No retries permitted until 2026-09-16 19:01:31.415717269 +0000 UTC m=+1021.745219677 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/037f0caf-e2d5-4e4e-a252-417a0a69d8b6-proxy-tls") pod "model-chainer-5fdf84b867-44hvl" (UID: "037f0caf-e2d5-4e4e-a252-417a0a69d8b6") : secret "model-chainer-serving-cert" not found
Sep 16 19:01:30.916096 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:30.916074    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/037f0caf-e2d5-4e4e-a252-417a0a69d8b6-openshift-service-ca-bundle\") pod \"model-chainer-5fdf84b867-44hvl\" (UID: \"037f0caf-e2d5-4e4e-a252-417a0a69d8b6\") " pod="kserve-ci-e2e-test/model-chainer-5fdf84b867-44hvl"
Sep 16 19:01:30.939322 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:30.939292    2584 generic.go:356] "Generic (PLEG): container finished" podID="6413f28e-243d-48cf-9983-7519b048ca6f" containerID="74797f5a64293bdf5fe073126cb2a2ed96497bdb8ac77d0945af70d35ab3e9dc" exitCode=0
Sep 16 19:01:30.939425 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:30.939362    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-658c9-84496b65dc-jv894" event={"ID":"6413f28e-243d-48cf-9983-7519b048ca6f","Type":"ContainerDied","Data":"74797f5a64293bdf5fe073126cb2a2ed96497bdb8ac77d0945af70d35ab3e9dc"}
Sep 16 19:01:31.065209 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:31.065180    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-658c9-84496b65dc-jv894"
Sep 16 19:01:31.117065 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:31.117032    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/6413f28e-243d-48cf-9983-7519b048ca6f-proxy-tls\") pod \"6413f28e-243d-48cf-9983-7519b048ca6f\" (UID: \"6413f28e-243d-48cf-9983-7519b048ca6f\") "
Sep 16 19:01:31.117228 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:31.117075    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6413f28e-243d-48cf-9983-7519b048ca6f-openshift-service-ca-bundle\") pod \"6413f28e-243d-48cf-9983-7519b048ca6f\" (UID: \"6413f28e-243d-48cf-9983-7519b048ca6f\") "
Sep 16 19:01:31.117495 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:31.117469    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6413f28e-243d-48cf-9983-7519b048ca6f-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "6413f28e-243d-48cf-9983-7519b048ca6f" (UID: "6413f28e-243d-48cf-9983-7519b048ca6f"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 19:01:31.119130 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:31.119108    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6413f28e-243d-48cf-9983-7519b048ca6f-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "6413f28e-243d-48cf-9983-7519b048ca6f" (UID: "6413f28e-243d-48cf-9983-7519b048ca6f"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 19:01:31.217810 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:31.217780    2584 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/6413f28e-243d-48cf-9983-7519b048ca6f-proxy-tls\") on node \"ip-10-0-139-182.ec2.internal\" DevicePath \"\""
Sep 16 19:01:31.217810 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:31.217807    2584 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6413f28e-243d-48cf-9983-7519b048ca6f-openshift-service-ca-bundle\") on node \"ip-10-0-139-182.ec2.internal\" DevicePath \"\""
Sep 16 19:01:31.219605 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:31.219574    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 19:01:31.420509 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:31.420425    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/037f0caf-e2d5-4e4e-a252-417a0a69d8b6-proxy-tls\") pod \"model-chainer-5fdf84b867-44hvl\" (UID: \"037f0caf-e2d5-4e4e-a252-417a0a69d8b6\") " pod="kserve-ci-e2e-test/model-chainer-5fdf84b867-44hvl"
Sep 16 19:01:31.422883 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:31.422861    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/037f0caf-e2d5-4e4e-a252-417a0a69d8b6-proxy-tls\") pod \"model-chainer-5fdf84b867-44hvl\" (UID: \"037f0caf-e2d5-4e4e-a252-417a0a69d8b6\") " pod="kserve-ci-e2e-test/model-chainer-5fdf84b867-44hvl"
Sep 16 19:01:31.665396 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:31.665357    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-5fdf84b867-44hvl"
Sep 16 19:01:31.934287 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:31.934214    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/model-chainer-5fdf84b867-44hvl"]
Sep 16 19:01:31.939655 ip-10-0-139-182 kubenswrapper[2584]: W0916 19:01:31.939626    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod037f0caf_e2d5_4e4e_a252_417a0a69d8b6.slice/crio-ea7d543259157677d5e4c270cd23565bc5b645f9c19e30c8cd2d851cbce04c4e WatchSource:0}: Error finding container ea7d543259157677d5e4c270cd23565bc5b645f9c19e30c8cd2d851cbce04c4e: Status 404 returned error can't find the container with id ea7d543259157677d5e4c270cd23565bc5b645f9c19e30c8cd2d851cbce04c4e
Sep 16 19:01:31.944847 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:31.944805    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-658c9-84496b65dc-jv894"
Sep 16 19:01:31.944847 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:31.944816    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-658c9-84496b65dc-jv894" event={"ID":"6413f28e-243d-48cf-9983-7519b048ca6f","Type":"ContainerDied","Data":"f1149a4aece1ce80979976673e4575249180cba0ec622c0b7ca09bf2eb3cd5e1"}
Sep 16 19:01:31.944971 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:31.944906    2584 scope.go:117] "RemoveContainer" containerID="74797f5a64293bdf5fe073126cb2a2ed96497bdb8ac77d0945af70d35ab3e9dc"
Sep 16 19:01:31.966541 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:31.966519    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/switch-graph-658c9-84496b65dc-jv894"]
Sep 16 19:01:31.970282 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:31.970259    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/switch-graph-658c9-84496b65dc-jv894"]
Sep 16 19:01:32.116489 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:32.116447    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6413f28e-243d-48cf-9983-7519b048ca6f" path="/var/lib/kubelet/pods/6413f28e-243d-48cf-9983-7519b048ca6f/volumes"
Sep 16 19:01:32.951905 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:32.951859    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-5fdf84b867-44hvl" event={"ID":"037f0caf-e2d5-4e4e-a252-417a0a69d8b6","Type":"ContainerStarted","Data":"ba7981998ee84069dcc56c288078135b3fecf901e1724735a9565896b84c6b3e"}
Sep 16 19:01:32.952333 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:32.951927    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-5fdf84b867-44hvl" event={"ID":"037f0caf-e2d5-4e4e-a252-417a0a69d8b6","Type":"ContainerStarted","Data":"ea7d543259157677d5e4c270cd23565bc5b645f9c19e30c8cd2d851cbce04c4e"}
Sep 16 19:01:32.952333 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:32.952032    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/model-chainer-5fdf84b867-44hvl"
Sep 16 19:01:32.969599 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:32.969404    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/model-chainer-5fdf84b867-44hvl" podStartSLOduration=2.96939032 podStartE2EDuration="2.96939032s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 19:01:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 19:01:32.968907518 +0000 UTC m=+1023.298409943" watchObservedRunningTime="2026-09-16 19:01:32.96939032 +0000 UTC m=+1023.298892733"
Sep 16 19:01:38.809158 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:38.809121    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-93efe-predictor-85b4cff8bf-k95wm" podUID="771df157-671d-4a80-aeea-88aca624578a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.33:8080: connect: connection refused"
Sep 16 19:01:38.962573 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:38.962545    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/model-chainer-5fdf84b867-44hvl"
Sep 16 19:01:40.846807 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:40.846766    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/model-chainer-5fdf84b867-44hvl"]
Sep 16 19:01:40.847266 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:40.847007    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/model-chainer-5fdf84b867-44hvl" podUID="037f0caf-e2d5-4e4e-a252-417a0a69d8b6" containerName="model-chainer" containerID="cri-o://ba7981998ee84069dcc56c288078135b3fecf901e1724735a9565896b84c6b3e" gracePeriod=30
Sep 16 19:01:40.869129 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:40.869098    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 19:01:40.973468 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:40.973433    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-799ddc8799-hjrgh"]
Sep 16 19:01:40.973744 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:40.973718    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-799ddc8799-hjrgh" podUID="83dc260d-a788-4eb1-a55d-7132c5114b8f" containerName="kserve-container" containerID="cri-o://1d54c6e771855882ae08e598a780153939746f5f5871493d62a78e260f92be2e" gracePeriod=30
Sep 16 19:01:40.973810 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:40.973760    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-799ddc8799-hjrgh" podUID="83dc260d-a788-4eb1-a55d-7132c5114b8f" containerName="kube-rbac-proxy" containerID="cri-o://fbe95d802cda97354e101d955f9b23286b9757b06c6d20a2d8ac49512fa63519" gracePeriod=30
Sep 16 19:01:41.005770 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:41.005742    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 19:01:41.104547 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:41.104439    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 19:01:41.163669 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:41.163634    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-cb9a3-predictor-55b48b4985-z5s2w"]
Sep 16 19:01:41.164009 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:41.163997    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6413f28e-243d-48cf-9983-7519b048ca6f" containerName="switch-graph-658c9"
Sep 16 19:01:41.164068 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:41.164011    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="6413f28e-243d-48cf-9983-7519b048ca6f" containerName="switch-graph-658c9"
Sep 16 19:01:41.164105 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:41.164071    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="6413f28e-243d-48cf-9983-7519b048ca6f" containerName="switch-graph-658c9"
Sep 16 19:01:41.167004 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:41.166982    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-cb9a3-predictor-55b48b4985-z5s2w"
Sep 16 19:01:41.169990 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:41.169969    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-cb9a3-predictor-serving-cert\""
Sep 16 19:01:41.170108 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:41.169969    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-cb9a3-kube-rbac-proxy-sar-config\""
Sep 16 19:01:41.180697 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:41.180661    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-cb9a3-predictor-55b48b4985-z5s2w"]
Sep 16 19:01:41.190763 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:41.190739    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 19:01:41.306700 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:41.306658    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/86ef3559-67b7-4535-b7c6-2673417ceb8e-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-cb9a3-predictor-55b48b4985-z5s2w\" (UID: \"86ef3559-67b7-4535-b7c6-2673417ceb8e\") " pod="kserve-ci-e2e-test/success-200-isvc-cb9a3-predictor-55b48b4985-z5s2w"
Sep 16 19:01:41.306700 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:41.306703    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-brczs\" (UniqueName: \"kubernetes.io/projected/86ef3559-67b7-4535-b7c6-2673417ceb8e-kube-api-access-brczs\") pod \"success-200-isvc-cb9a3-predictor-55b48b4985-z5s2w\" (UID: \"86ef3559-67b7-4535-b7c6-2673417ceb8e\") " pod="kserve-ci-e2e-test/success-200-isvc-cb9a3-predictor-55b48b4985-z5s2w"
Sep 16 19:01:41.306967 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:41.306849    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/86ef3559-67b7-4535-b7c6-2673417ceb8e-proxy-tls\") pod \"success-200-isvc-cb9a3-predictor-55b48b4985-z5s2w\" (UID: \"86ef3559-67b7-4535-b7c6-2673417ceb8e\") " pod="kserve-ci-e2e-test/success-200-isvc-cb9a3-predictor-55b48b4985-z5s2w"
Sep 16 19:01:41.307853 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:41.307804    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 19:01:41.407692 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:41.407610    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-brczs\" (UniqueName: \"kubernetes.io/projected/86ef3559-67b7-4535-b7c6-2673417ceb8e-kube-api-access-brczs\") pod \"success-200-isvc-cb9a3-predictor-55b48b4985-z5s2w\" (UID: \"86ef3559-67b7-4535-b7c6-2673417ceb8e\") " pod="kserve-ci-e2e-test/success-200-isvc-cb9a3-predictor-55b48b4985-z5s2w"
Sep 16 19:01:41.407867 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:41.407707    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/86ef3559-67b7-4535-b7c6-2673417ceb8e-proxy-tls\") pod \"success-200-isvc-cb9a3-predictor-55b48b4985-z5s2w\" (UID: \"86ef3559-67b7-4535-b7c6-2673417ceb8e\") " pod="kserve-ci-e2e-test/success-200-isvc-cb9a3-predictor-55b48b4985-z5s2w"
Sep 16 19:01:41.407867 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:41.407728    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/86ef3559-67b7-4535-b7c6-2673417ceb8e-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-cb9a3-predictor-55b48b4985-z5s2w\" (UID: \"86ef3559-67b7-4535-b7c6-2673417ceb8e\") " pod="kserve-ci-e2e-test/success-200-isvc-cb9a3-predictor-55b48b4985-z5s2w"
Sep 16 19:01:41.408419 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:41.408399    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/86ef3559-67b7-4535-b7c6-2673417ceb8e-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-cb9a3-predictor-55b48b4985-z5s2w\" (UID: \"86ef3559-67b7-4535-b7c6-2673417ceb8e\") " pod="kserve-ci-e2e-test/success-200-isvc-cb9a3-predictor-55b48b4985-z5s2w"
Sep 16 19:01:41.410613 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:41.410591    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/86ef3559-67b7-4535-b7c6-2673417ceb8e-proxy-tls\") pod \"success-200-isvc-cb9a3-predictor-55b48b4985-z5s2w\" (UID: \"86ef3559-67b7-4535-b7c6-2673417ceb8e\") " pod="kserve-ci-e2e-test/success-200-isvc-cb9a3-predictor-55b48b4985-z5s2w"
Sep 16 19:01:41.418275 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:41.418246    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-brczs\" (UniqueName: \"kubernetes.io/projected/86ef3559-67b7-4535-b7c6-2673417ceb8e-kube-api-access-brczs\") pod \"success-200-isvc-cb9a3-predictor-55b48b4985-z5s2w\" (UID: \"86ef3559-67b7-4535-b7c6-2673417ceb8e\") " pod="kserve-ci-e2e-test/success-200-isvc-cb9a3-predictor-55b48b4985-z5s2w"
Sep 16 19:01:41.462844 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:41.462807    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 19:01:41.477594 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:41.477560    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-cb9a3-predictor-55b48b4985-z5s2w"
Sep 16 19:01:41.757892 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:41.757867    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-cb9a3-predictor-55b48b4985-z5s2w"]
Sep 16 19:01:41.760484 ip-10-0-139-182 kubenswrapper[2584]: W0916 19:01:41.760459    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86ef3559_67b7_4535_b7c6_2673417ceb8e.slice/crio-8688f97f3ca3321f06e8b5de6c7ae3d52ce1c0d6ee0fb5165ff34f8a6e65be3a WatchSource:0}: Error finding container 8688f97f3ca3321f06e8b5de6c7ae3d52ce1c0d6ee0fb5165ff34f8a6e65be3a: Status 404 returned error can't find the container with id 8688f97f3ca3321f06e8b5de6c7ae3d52ce1c0d6ee0fb5165ff34f8a6e65be3a
Sep 16 19:01:41.784935 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:41.784909    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 19:01:41.998982 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:41.998951    2584 generic.go:356] "Generic (PLEG): container finished" podID="83dc260d-a788-4eb1-a55d-7132c5114b8f" containerID="fbe95d802cda97354e101d955f9b23286b9757b06c6d20a2d8ac49512fa63519" exitCode=2
Sep 16 19:01:41.999564 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:41.999018    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-799ddc8799-hjrgh" event={"ID":"83dc260d-a788-4eb1-a55d-7132c5114b8f","Type":"ContainerDied","Data":"fbe95d802cda97354e101d955f9b23286b9757b06c6d20a2d8ac49512fa63519"}
Sep 16 19:01:42.001020 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:42.000994    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-cb9a3-predictor-55b48b4985-z5s2w" event={"ID":"86ef3559-67b7-4535-b7c6-2673417ceb8e","Type":"ContainerStarted","Data":"0e73cb0b6cec9a2fd1b9367617783f1321288e79bf6d29421e68f47eeef2afcb"}
Sep 16 19:01:42.001146 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:42.001042    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-cb9a3-predictor-55b48b4985-z5s2w" event={"ID":"86ef3559-67b7-4535-b7c6-2673417ceb8e","Type":"ContainerStarted","Data":"e3216e71f3a1a54694b4348fb63f5af136c9065c1464b1b1918d2ba6771100b0"}
Sep 16 19:01:42.001146 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:42.001060    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-cb9a3-predictor-55b48b4985-z5s2w" event={"ID":"86ef3559-67b7-4535-b7c6-2673417ceb8e","Type":"ContainerStarted","Data":"8688f97f3ca3321f06e8b5de6c7ae3d52ce1c0d6ee0fb5165ff34f8a6e65be3a"}
Sep 16 19:01:42.001256 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:42.001230    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-cb9a3-predictor-55b48b4985-z5s2w"
Sep 16 19:01:42.001404 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:42.001384    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-cb9a3-predictor-55b48b4985-z5s2w"
Sep 16 19:01:42.002749 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:42.002727    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-cb9a3-predictor-55b48b4985-z5s2w" podUID="86ef3559-67b7-4535-b7c6-2673417ceb8e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.35:8080: connect: connection refused"
Sep 16 19:01:42.029605 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:42.029560    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-cb9a3-predictor-55b48b4985-z5s2w" podStartSLOduration=1.02954643 podStartE2EDuration="1.02954643s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 19:01:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 19:01:42.02787562 +0000 UTC m=+1032.357378033" watchObservedRunningTime="2026-09-16 19:01:42.02954643 +0000 UTC m=+1032.359048842"
Sep 16 19:01:43.005966 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:43.005927    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-cb9a3-predictor-55b48b4985-z5s2w" podUID="86ef3559-67b7-4535-b7c6-2673417ceb8e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.35:8080: connect: connection refused"
Sep 16 19:01:43.961742 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:43.961706    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-5fdf84b867-44hvl" podUID="037f0caf-e2d5-4e4e-a252-417a0a69d8b6" containerName="model-chainer" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 16 19:01:44.010110 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:44.010065    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-cb9a3-predictor-55b48b4985-z5s2w" podUID="86ef3559-67b7-4535-b7c6-2673417ceb8e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.35:8080: connect: connection refused"
Sep 16 19:01:44.490567 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:44.490527    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-799ddc8799-hjrgh" podUID="83dc260d-a788-4eb1-a55d-7132c5114b8f" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.31:8643/healthz\": dial tcp 10.133.0.31:8643: connect: connection refused"
Sep 16 19:01:45.914071 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:45.914045    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-799ddc8799-hjrgh"
Sep 16 19:01:46.020350 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:46.020317    2584 generic.go:356] "Generic (PLEG): container finished" podID="83dc260d-a788-4eb1-a55d-7132c5114b8f" containerID="1d54c6e771855882ae08e598a780153939746f5f5871493d62a78e260f92be2e" exitCode=0
Sep 16 19:01:46.020535 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:46.020375    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-799ddc8799-hjrgh"
Sep 16 19:01:46.020535 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:46.020375    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-799ddc8799-hjrgh" event={"ID":"83dc260d-a788-4eb1-a55d-7132c5114b8f","Type":"ContainerDied","Data":"1d54c6e771855882ae08e598a780153939746f5f5871493d62a78e260f92be2e"}
Sep 16 19:01:46.020535 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:46.020516    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-799ddc8799-hjrgh" event={"ID":"83dc260d-a788-4eb1-a55d-7132c5114b8f","Type":"ContainerDied","Data":"9bcc3ea89a47259ee0c655d51eb5c498841e117661f3156d39f7ed6e4c8b153b"}
Sep 16 19:01:46.020713 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:46.020558    2584 scope.go:117] "RemoveContainer" containerID="fbe95d802cda97354e101d955f9b23286b9757b06c6d20a2d8ac49512fa63519"
Sep 16 19:01:46.028847 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:46.028816    2584 scope.go:117] "RemoveContainer" containerID="1d54c6e771855882ae08e598a780153939746f5f5871493d62a78e260f92be2e"
Sep 16 19:01:46.038533 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:46.038504    2584 scope.go:117] "RemoveContainer" containerID="e6be8383f57e45b5a3e14fd79441fc74d1ed33182e0e481819d32c8a71b45920"
Sep 16 19:01:46.046087 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:46.046053    2584 scope.go:117] "RemoveContainer" containerID="fbe95d802cda97354e101d955f9b23286b9757b06c6d20a2d8ac49512fa63519"
Sep 16 19:01:46.046379 ip-10-0-139-182 kubenswrapper[2584]: E0916 19:01:46.046359    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fbe95d802cda97354e101d955f9b23286b9757b06c6d20a2d8ac49512fa63519\": container with ID starting with fbe95d802cda97354e101d955f9b23286b9757b06c6d20a2d8ac49512fa63519 not found: ID does not exist" containerID="fbe95d802cda97354e101d955f9b23286b9757b06c6d20a2d8ac49512fa63519"
Sep 16 19:01:46.046460 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:46.046391    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fbe95d802cda97354e101d955f9b23286b9757b06c6d20a2d8ac49512fa63519"} err="failed to get container status \"fbe95d802cda97354e101d955f9b23286b9757b06c6d20a2d8ac49512fa63519\": rpc error: code = NotFound desc = could not find container \"fbe95d802cda97354e101d955f9b23286b9757b06c6d20a2d8ac49512fa63519\": container with ID starting with fbe95d802cda97354e101d955f9b23286b9757b06c6d20a2d8ac49512fa63519 not found: ID does not exist"
Sep 16 19:01:46.046460 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:46.046417    2584 scope.go:117] "RemoveContainer" containerID="1d54c6e771855882ae08e598a780153939746f5f5871493d62a78e260f92be2e"
Sep 16 19:01:46.046704 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:46.046679    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/83dc260d-a788-4eb1-a55d-7132c5114b8f-kserve-provision-location\") pod \"83dc260d-a788-4eb1-a55d-7132c5114b8f\" (UID: \"83dc260d-a788-4eb1-a55d-7132c5114b8f\") "
Sep 16 19:01:46.046798 ip-10-0-139-182 kubenswrapper[2584]: E0916 19:01:46.046701    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1d54c6e771855882ae08e598a780153939746f5f5871493d62a78e260f92be2e\": container with ID starting with 1d54c6e771855882ae08e598a780153939746f5f5871493d62a78e260f92be2e not found: ID does not exist" containerID="1d54c6e771855882ae08e598a780153939746f5f5871493d62a78e260f92be2e"
Sep 16 19:01:46.046798 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:46.046724    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1d54c6e771855882ae08e598a780153939746f5f5871493d62a78e260f92be2e"} err="failed to get container status \"1d54c6e771855882ae08e598a780153939746f5f5871493d62a78e260f92be2e\": rpc error: code = NotFound desc = could not find container \"1d54c6e771855882ae08e598a780153939746f5f5871493d62a78e260f92be2e\": container with ID starting with 1d54c6e771855882ae08e598a780153939746f5f5871493d62a78e260f92be2e not found: ID does not exist"
Sep 16 19:01:46.046798 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:46.046735    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/83dc260d-a788-4eb1-a55d-7132c5114b8f-proxy-tls\") pod \"83dc260d-a788-4eb1-a55d-7132c5114b8f\" (UID: \"83dc260d-a788-4eb1-a55d-7132c5114b8f\") "
Sep 16 19:01:46.046798 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:46.046742    2584 scope.go:117] "RemoveContainer" containerID="e6be8383f57e45b5a3e14fd79441fc74d1ed33182e0e481819d32c8a71b45920"
Sep 16 19:01:46.046798 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:46.046768    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/83dc260d-a788-4eb1-a55d-7132c5114b8f-cabundle-cert\") pod \"83dc260d-a788-4eb1-a55d-7132c5114b8f\" (UID: \"83dc260d-a788-4eb1-a55d-7132c5114b8f\") "
Sep 16 19:01:46.047088 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:46.046961    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jgshn\" (UniqueName: \"kubernetes.io/projected/83dc260d-a788-4eb1-a55d-7132c5114b8f-kube-api-access-jgshn\") pod \"83dc260d-a788-4eb1-a55d-7132c5114b8f\" (UID: \"83dc260d-a788-4eb1-a55d-7132c5114b8f\") "
Sep 16 19:01:46.047088 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:46.047035    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/83dc260d-a788-4eb1-a55d-7132c5114b8f-kube-rbac-proxy-sar-config\") pod \"83dc260d-a788-4eb1-a55d-7132c5114b8f\" (UID: \"83dc260d-a788-4eb1-a55d-7132c5114b8f\") "
Sep 16 19:01:46.047344 ip-10-0-139-182 kubenswrapper[2584]: E0916 19:01:46.047286    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e6be8383f57e45b5a3e14fd79441fc74d1ed33182e0e481819d32c8a71b45920\": container with ID starting with e6be8383f57e45b5a3e14fd79441fc74d1ed33182e0e481819d32c8a71b45920 not found: ID does not exist" containerID="e6be8383f57e45b5a3e14fd79441fc74d1ed33182e0e481819d32c8a71b45920"
Sep 16 19:01:46.047344 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:46.047320    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e6be8383f57e45b5a3e14fd79441fc74d1ed33182e0e481819d32c8a71b45920"} err="failed to get container status \"e6be8383f57e45b5a3e14fd79441fc74d1ed33182e0e481819d32c8a71b45920\": rpc error: code = NotFound desc = could not find container \"e6be8383f57e45b5a3e14fd79441fc74d1ed33182e0e481819d32c8a71b45920\": container with ID starting with e6be8383f57e45b5a3e14fd79441fc74d1ed33182e0e481819d32c8a71b45920 not found: ID does not exist"
Sep 16 19:01:46.047489 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:46.047389    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83dc260d-a788-4eb1-a55d-7132c5114b8f-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "83dc260d-a788-4eb1-a55d-7132c5114b8f" (UID: "83dc260d-a788-4eb1-a55d-7132c5114b8f"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:01:46.047634 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:46.047605    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/83dc260d-a788-4eb1-a55d-7132c5114b8f-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "83dc260d-a788-4eb1-a55d-7132c5114b8f" (UID: "83dc260d-a788-4eb1-a55d-7132c5114b8f"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 19:01:46.047711 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:46.047695    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/83dc260d-a788-4eb1-a55d-7132c5114b8f-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "83dc260d-a788-4eb1-a55d-7132c5114b8f" (UID: "83dc260d-a788-4eb1-a55d-7132c5114b8f"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 19:01:46.049363 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:46.049330    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/83dc260d-a788-4eb1-a55d-7132c5114b8f-kube-api-access-jgshn" (OuterVolumeSpecName: "kube-api-access-jgshn") pod "83dc260d-a788-4eb1-a55d-7132c5114b8f" (UID: "83dc260d-a788-4eb1-a55d-7132c5114b8f"). InnerVolumeSpecName "kube-api-access-jgshn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 19:01:46.049460 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:46.049373    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/83dc260d-a788-4eb1-a55d-7132c5114b8f-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "83dc260d-a788-4eb1-a55d-7132c5114b8f" (UID: "83dc260d-a788-4eb1-a55d-7132c5114b8f"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 19:01:46.148171 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:46.148141    2584 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/83dc260d-a788-4eb1-a55d-7132c5114b8f-kube-rbac-proxy-sar-config\") on node \"ip-10-0-139-182.ec2.internal\" DevicePath \"\""
Sep 16 19:01:46.148171 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:46.148168    2584 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/83dc260d-a788-4eb1-a55d-7132c5114b8f-kserve-provision-location\") on node \"ip-10-0-139-182.ec2.internal\" DevicePath \"\""
Sep 16 19:01:46.148340 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:46.148182    2584 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/83dc260d-a788-4eb1-a55d-7132c5114b8f-proxy-tls\") on node \"ip-10-0-139-182.ec2.internal\" DevicePath \"\""
Sep 16 19:01:46.148340 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:46.148194    2584 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/83dc260d-a788-4eb1-a55d-7132c5114b8f-cabundle-cert\") on node \"ip-10-0-139-182.ec2.internal\" DevicePath \"\""
Sep 16 19:01:46.148340 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:46.148203    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jgshn\" (UniqueName: \"kubernetes.io/projected/83dc260d-a788-4eb1-a55d-7132c5114b8f-kube-api-access-jgshn\") on node \"ip-10-0-139-182.ec2.internal\" DevicePath \"\""
Sep 16 19:01:46.350719 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:46.350634    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-799ddc8799-hjrgh"]
Sep 16 19:01:46.356813 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:46.356783    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-799ddc8799-hjrgh"]
Sep 16 19:01:48.116008 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:48.115975    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="83dc260d-a788-4eb1-a55d-7132c5114b8f" path="/var/lib/kubelet/pods/83dc260d-a788-4eb1-a55d-7132c5114b8f/volumes"
Sep 16 19:01:48.809489 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:48.809458    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-93efe-predictor-85b4cff8bf-k95wm"
Sep 16 19:01:48.961089 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:48.961051    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-5fdf84b867-44hvl" podUID="037f0caf-e2d5-4e4e-a252-417a0a69d8b6" containerName="model-chainer" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 16 19:01:49.014938 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:49.014908    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-cb9a3-predictor-55b48b4985-z5s2w"
Sep 16 19:01:49.015351 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:49.015318    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-cb9a3-predictor-55b48b4985-z5s2w" podUID="86ef3559-67b7-4535-b7c6-2673417ceb8e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.35:8080: connect: connection refused"
Sep 16 19:01:53.961509 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:53.961472    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-5fdf84b867-44hvl" podUID="037f0caf-e2d5-4e4e-a252-417a0a69d8b6" containerName="model-chainer" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 16 19:01:53.961934 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:53.961571    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/model-chainer-5fdf84b867-44hvl"
Sep 16 19:01:58.961824 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:58.961782    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-5fdf84b867-44hvl" podUID="037f0caf-e2d5-4e4e-a252-417a0a69d8b6" containerName="model-chainer" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 16 19:01:59.015676 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:01:59.015635    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-cb9a3-predictor-55b48b4985-z5s2w" podUID="86ef3559-67b7-4535-b7c6-2673417ceb8e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.35:8080: connect: connection refused"
Sep 16 19:02:01.211479 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:02:01.211443    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/switch-graph-93efe-7846bd77df-h9n8c"]
Sep 16 19:02:01.211853 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:02:01.211770    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="83dc260d-a788-4eb1-a55d-7132c5114b8f" containerName="kserve-container"
Sep 16 19:02:01.211853 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:02:01.211782    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="83dc260d-a788-4eb1-a55d-7132c5114b8f" containerName="kserve-container"
Sep 16 19:02:01.211853 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:02:01.211790    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="83dc260d-a788-4eb1-a55d-7132c5114b8f" containerName="kube-rbac-proxy"
Sep 16 19:02:01.211853 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:02:01.211795    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="83dc260d-a788-4eb1-a55d-7132c5114b8f" containerName="kube-rbac-proxy"
Sep 16 19:02:01.211853 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:02:01.211813    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="83dc260d-a788-4eb1-a55d-7132c5114b8f" containerName="storage-initializer"
Sep 16 19:02:01.211853 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:02:01.211819    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="83dc260d-a788-4eb1-a55d-7132c5114b8f" containerName="storage-initializer"
Sep 16 19:02:01.212057 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:02:01.211883    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="83dc260d-a788-4eb1-a55d-7132c5114b8f" containerName="kserve-container"
Sep 16 19:02:01.212057 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:02:01.211894    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="83dc260d-a788-4eb1-a55d-7132c5114b8f" containerName="kube-rbac-proxy"
Sep 16 19:02:01.214698 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:02:01.214677    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-93efe-7846bd77df-h9n8c"
Sep 16 19:02:01.217404 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:02:01.217387    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"switch-graph-93efe-kube-rbac-proxy-sar-config\""
Sep 16 19:02:01.217501 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:02:01.217387    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"switch-graph-93efe-serving-cert\""
Sep 16 19:02:01.225289 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:02:01.225265    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/switch-graph-93efe-7846bd77df-h9n8c"]
Sep 16 19:02:01.227930 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:02:01.227906    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 19:02:01.268194 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:02:01.268159    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9ec73f37-d895-442d-a2d4-f3dfe3eee6c1-openshift-service-ca-bundle\") pod \"switch-graph-93efe-7846bd77df-h9n8c\" (UID: \"9ec73f37-d895-442d-a2d4-f3dfe3eee6c1\") " pod="kserve-ci-e2e-test/switch-graph-93efe-7846bd77df-h9n8c"
Sep 16 19:02:01.268334 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:02:01.268279    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9ec73f37-d895-442d-a2d4-f3dfe3eee6c1-proxy-tls\") pod \"switch-graph-93efe-7846bd77df-h9n8c\" (UID: \"9ec73f37-d895-442d-a2d4-f3dfe3eee6c1\") " pod="kserve-ci-e2e-test/switch-graph-93efe-7846bd77df-h9n8c"
Sep 16 19:02:01.369484 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:02:01.369450    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9ec73f37-d895-442d-a2d4-f3dfe3eee6c1-openshift-service-ca-bundle\") pod \"switch-graph-93efe-7846bd77df-h9n8c\" (UID: \"9ec73f37-d895-442d-a2d4-f3dfe3eee6c1\") " pod="kserve-ci-e2e-test/switch-graph-93efe-7846bd77df-h9n8c"
Sep 16 19:02:01.369484 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:02:01.369488    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9ec73f37-d895-442d-a2d4-f3dfe3eee6c1-proxy-tls\") pod \"switch-graph-93efe-7846bd77df-h9n8c\" (UID: \"9ec73f37-d895-442d-a2d4-f3dfe3eee6c1\") " pod="kserve-ci-e2e-test/switch-graph-93efe-7846bd77df-h9n8c"
Sep 16 19:02:01.369682 ip-10-0-139-182 kubenswrapper[2584]: E0916 19:02:01.369579    2584 secret.go:189] Couldn't get secret kserve-ci-e2e-test/switch-graph-93efe-serving-cert: secret "switch-graph-93efe-serving-cert" not found
Sep 16 19:02:01.369682 ip-10-0-139-182 kubenswrapper[2584]: E0916 19:02:01.369643    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9ec73f37-d895-442d-a2d4-f3dfe3eee6c1-proxy-tls podName:9ec73f37-d895-442d-a2d4-f3dfe3eee6c1 nodeName:}" failed. No retries permitted until 2026-09-16 19:02:01.869625869 +0000 UTC m=+1052.199128259 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/9ec73f37-d895-442d-a2d4-f3dfe3eee6c1-proxy-tls") pod "switch-graph-93efe-7846bd77df-h9n8c" (UID: "9ec73f37-d895-442d-a2d4-f3dfe3eee6c1") : secret "switch-graph-93efe-serving-cert" not found
Sep 16 19:02:01.370122 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:02:01.370100    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9ec73f37-d895-442d-a2d4-f3dfe3eee6c1-openshift-service-ca-bundle\") pod \"switch-graph-93efe-7846bd77df-h9n8c\" (UID: \"9ec73f37-d895-442d-a2d4-f3dfe3eee6c1\") " pod="kserve-ci-e2e-test/switch-graph-93efe-7846bd77df-h9n8c"
Sep 16 19:02:01.493862 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:02:01.493758    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 19:02:01.873855 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:02:01.873800    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9ec73f37-d895-442d-a2d4-f3dfe3eee6c1-proxy-tls\") pod \"switch-graph-93efe-7846bd77df-h9n8c\" (UID: \"9ec73f37-d895-442d-a2d4-f3dfe3eee6c1\") " pod="kserve-ci-e2e-test/switch-graph-93efe-7846bd77df-h9n8c"
Sep 16 19:02:01.876214 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:02:01.876185    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9ec73f37-d895-442d-a2d4-f3dfe3eee6c1-proxy-tls\") pod \"switch-graph-93efe-7846bd77df-h9n8c\" (UID: \"9ec73f37-d895-442d-a2d4-f3dfe3eee6c1\") " pod="kserve-ci-e2e-test/switch-graph-93efe-7846bd77df-h9n8c"
Sep 16 19:02:02.125019 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:02:02.124946    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-93efe-7846bd77df-h9n8c"
Sep 16 19:02:02.395398 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:02:02.395373    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/switch-graph-93efe-7846bd77df-h9n8c"]
Sep 16 19:02:02.397748 ip-10-0-139-182 kubenswrapper[2584]: W0916 19:02:02.397721    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ec73f37_d895_442d_a2d4_f3dfe3eee6c1.slice/crio-0423e42615977ee16a4835086e62833281a7266bd81eb2c442516cd5fb6def09 WatchSource:0}: Error finding container 0423e42615977ee16a4835086e62833281a7266bd81eb2c442516cd5fb6def09: Status 404 returned error can't find the container with id 0423e42615977ee16a4835086e62833281a7266bd81eb2c442516cd5fb6def09
Sep 16 19:02:03.100270 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:02:03.100234    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-93efe-7846bd77df-h9n8c" event={"ID":"9ec73f37-d895-442d-a2d4-f3dfe3eee6c1","Type":"ContainerStarted","Data":"4a6fd92f63c2fe778eb67313d0a01697ffdb53a6cd967b6ad0104385107efd8a"}
Sep 16 19:02:03.100461 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:02:03.100290    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-93efe-7846bd77df-h9n8c" event={"ID":"9ec73f37-d895-442d-a2d4-f3dfe3eee6c1","Type":"ContainerStarted","Data":"0423e42615977ee16a4835086e62833281a7266bd81eb2c442516cd5fb6def09"}
Sep 16 19:02:03.100461 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:02:03.100318    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/switch-graph-93efe-7846bd77df-h9n8c"
Sep 16 19:02:03.117310 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:02:03.117260    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/switch-graph-93efe-7846bd77df-h9n8c" podStartSLOduration=2.117245131 podStartE2EDuration="2.117245131s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 19:02:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 19:02:03.116226554 +0000 UTC m=+1053.445728958" watchObservedRunningTime="2026-09-16 19:02:03.117245131 +0000 UTC m=+1053.446747541"
Sep 16 19:02:03.961369 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:02:03.961330    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-5fdf84b867-44hvl" podUID="037f0caf-e2d5-4e4e-a252-417a0a69d8b6" containerName="model-chainer" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 16 19:02:08.960978 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:02:08.960937    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-5fdf84b867-44hvl" podUID="037f0caf-e2d5-4e4e-a252-417a0a69d8b6" containerName="model-chainer" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 16 19:02:09.015807 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:02:09.015767    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-cb9a3-predictor-55b48b4985-z5s2w" podUID="86ef3559-67b7-4535-b7c6-2673417ceb8e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.35:8080: connect: connection refused"
Sep 16 19:02:09.110133 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:02:09.110104    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/switch-graph-93efe-7846bd77df-h9n8c"
Sep 16 19:02:10.992843 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:02:10.992812    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-5fdf84b867-44hvl"
Sep 16 19:02:11.045594 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:02:11.045566    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/037f0caf-e2d5-4e4e-a252-417a0a69d8b6-openshift-service-ca-bundle\") pod \"037f0caf-e2d5-4e4e-a252-417a0a69d8b6\" (UID: \"037f0caf-e2d5-4e4e-a252-417a0a69d8b6\") "
Sep 16 19:02:11.045761 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:02:11.045616    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/037f0caf-e2d5-4e4e-a252-417a0a69d8b6-proxy-tls\") pod \"037f0caf-e2d5-4e4e-a252-417a0a69d8b6\" (UID: \"037f0caf-e2d5-4e4e-a252-417a0a69d8b6\") "
Sep 16 19:02:11.045977 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:02:11.045954    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/037f0caf-e2d5-4e4e-a252-417a0a69d8b6-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "037f0caf-e2d5-4e4e-a252-417a0a69d8b6" (UID: "037f0caf-e2d5-4e4e-a252-417a0a69d8b6"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 19:02:11.047692 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:02:11.047667    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/037f0caf-e2d5-4e4e-a252-417a0a69d8b6-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "037f0caf-e2d5-4e4e-a252-417a0a69d8b6" (UID: "037f0caf-e2d5-4e4e-a252-417a0a69d8b6"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 19:02:11.141799 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:02:11.141711    2584 generic.go:356] "Generic (PLEG): container finished" podID="037f0caf-e2d5-4e4e-a252-417a0a69d8b6" containerID="ba7981998ee84069dcc56c288078135b3fecf901e1724735a9565896b84c6b3e" exitCode=0
Sep 16 19:02:11.141799 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:02:11.141750    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-5fdf84b867-44hvl"
Sep 16 19:02:11.141799 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:02:11.141788    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-5fdf84b867-44hvl" event={"ID":"037f0caf-e2d5-4e4e-a252-417a0a69d8b6","Type":"ContainerDied","Data":"ba7981998ee84069dcc56c288078135b3fecf901e1724735a9565896b84c6b3e"}
Sep 16 19:02:11.142062 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:02:11.141864    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-5fdf84b867-44hvl" event={"ID":"037f0caf-e2d5-4e4e-a252-417a0a69d8b6","Type":"ContainerDied","Data":"ea7d543259157677d5e4c270cd23565bc5b645f9c19e30c8cd2d851cbce04c4e"}
Sep 16 19:02:11.142062 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:02:11.141895    2584 scope.go:117] "RemoveContainer" containerID="ba7981998ee84069dcc56c288078135b3fecf901e1724735a9565896b84c6b3e"
Sep 16 19:02:11.147103 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:02:11.146804    2584 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/037f0caf-e2d5-4e4e-a252-417a0a69d8b6-proxy-tls\") on node \"ip-10-0-139-182.ec2.internal\" DevicePath \"\""
Sep 16 19:02:11.147103 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:02:11.146864    2584 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/037f0caf-e2d5-4e4e-a252-417a0a69d8b6-openshift-service-ca-bundle\") on node \"ip-10-0-139-182.ec2.internal\" DevicePath \"\""
Sep 16 19:02:11.152576 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:02:11.152547    2584 scope.go:117] "RemoveContainer" containerID="ba7981998ee84069dcc56c288078135b3fecf901e1724735a9565896b84c6b3e"
Sep 16 19:02:11.152911 ip-10-0-139-182 kubenswrapper[2584]: E0916 19:02:11.152887    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ba7981998ee84069dcc56c288078135b3fecf901e1724735a9565896b84c6b3e\": container with ID starting with ba7981998ee84069dcc56c288078135b3fecf901e1724735a9565896b84c6b3e not found: ID does not exist" containerID="ba7981998ee84069dcc56c288078135b3fecf901e1724735a9565896b84c6b3e"
Sep 16 19:02:11.152977 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:02:11.152927    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ba7981998ee84069dcc56c288078135b3fecf901e1724735a9565896b84c6b3e"} err="failed to get container status \"ba7981998ee84069dcc56c288078135b3fecf901e1724735a9565896b84c6b3e\": rpc error: code = NotFound desc = could not find container \"ba7981998ee84069dcc56c288078135b3fecf901e1724735a9565896b84c6b3e\": container with ID starting with ba7981998ee84069dcc56c288078135b3fecf901e1724735a9565896b84c6b3e not found: ID does not exist"
Sep 16 19:02:11.171307 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:02:11.171273    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/model-chainer-5fdf84b867-44hvl"]
Sep 16 19:02:11.174817 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:02:11.174792    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/model-chainer-5fdf84b867-44hvl"]
Sep 16 19:02:12.116477 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:02:12.116442    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="037f0caf-e2d5-4e4e-a252-417a0a69d8b6" path="/var/lib/kubelet/pods/037f0caf-e2d5-4e4e-a252-417a0a69d8b6/volumes"
Sep 16 19:02:19.016084 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:02:19.016038    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-cb9a3-predictor-55b48b4985-z5s2w" podUID="86ef3559-67b7-4535-b7c6-2673417ceb8e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.35:8080: connect: connection refused"
Sep 16 19:02:27.179320 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:02:27.179233    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 19:02:27.233382 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:02:27.233354    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 19:02:29.016652 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:02:29.016621    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-cb9a3-predictor-55b48b4985-z5s2w"
Sep 16 19:02:41.068524 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:02:41.068489    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/sequence-graph-cb9a3-5ff6b444fd-5fq6s"]
Sep 16 19:02:41.068931 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:02:41.068843    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="037f0caf-e2d5-4e4e-a252-417a0a69d8b6" containerName="model-chainer"
Sep 16 19:02:41.068931 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:02:41.068854    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="037f0caf-e2d5-4e4e-a252-417a0a69d8b6" containerName="model-chainer"
Sep 16 19:02:41.069021 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:02:41.068938    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="037f0caf-e2d5-4e4e-a252-417a0a69d8b6" containerName="model-chainer"
Sep 16 19:02:41.073118 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:02:41.073098    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-cb9a3-5ff6b444fd-5fq6s"
Sep 16 19:02:41.076033 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:02:41.076007    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"sequence-graph-cb9a3-serving-cert\""
Sep 16 19:02:41.076172 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:02:41.076103    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"sequence-graph-cb9a3-kube-rbac-proxy-sar-config\""
Sep 16 19:02:41.076335 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:02:41.076313    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 19:02:41.080983 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:02:41.080961    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-cb9a3-5ff6b444fd-5fq6s"]
Sep 16 19:02:41.206958 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:02:41.206920    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/abd27f76-1d33-41f7-ad59-bc0b0836de34-openshift-service-ca-bundle\") pod \"sequence-graph-cb9a3-5ff6b444fd-5fq6s\" (UID: \"abd27f76-1d33-41f7-ad59-bc0b0836de34\") " pod="kserve-ci-e2e-test/sequence-graph-cb9a3-5ff6b444fd-5fq6s"
Sep 16 19:02:41.207119 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:02:41.206992    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/abd27f76-1d33-41f7-ad59-bc0b0836de34-proxy-tls\") pod \"sequence-graph-cb9a3-5ff6b444fd-5fq6s\" (UID: \"abd27f76-1d33-41f7-ad59-bc0b0836de34\") " pod="kserve-ci-e2e-test/sequence-graph-cb9a3-5ff6b444fd-5fq6s"
Sep 16 19:02:41.303599 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:02:41.303565    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 19:02:41.308217 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:02:41.308199    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/abd27f76-1d33-41f7-ad59-bc0b0836de34-proxy-tls\") pod \"sequence-graph-cb9a3-5ff6b444fd-5fq6s\" (UID: \"abd27f76-1d33-41f7-ad59-bc0b0836de34\") " pod="kserve-ci-e2e-test/sequence-graph-cb9a3-5ff6b444fd-5fq6s"
Sep 16 19:02:41.308284 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:02:41.308249    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/abd27f76-1d33-41f7-ad59-bc0b0836de34-openshift-service-ca-bundle\") pod \"sequence-graph-cb9a3-5ff6b444fd-5fq6s\" (UID: \"abd27f76-1d33-41f7-ad59-bc0b0836de34\") " pod="kserve-ci-e2e-test/sequence-graph-cb9a3-5ff6b444fd-5fq6s"
Sep 16 19:02:41.308893 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:02:41.308875    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/abd27f76-1d33-41f7-ad59-bc0b0836de34-openshift-service-ca-bundle\") pod \"sequence-graph-cb9a3-5ff6b444fd-5fq6s\" (UID: \"abd27f76-1d33-41f7-ad59-bc0b0836de34\") " pod="kserve-ci-e2e-test/sequence-graph-cb9a3-5ff6b444fd-5fq6s"
Sep 16 19:02:41.310645 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:02:41.310625    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/abd27f76-1d33-41f7-ad59-bc0b0836de34-proxy-tls\") pod \"sequence-graph-cb9a3-5ff6b444fd-5fq6s\" (UID: \"abd27f76-1d33-41f7-ad59-bc0b0836de34\") " pod="kserve-ci-e2e-test/sequence-graph-cb9a3-5ff6b444fd-5fq6s"
Sep 16 19:02:41.384373 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:02:41.384300    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-cb9a3-5ff6b444fd-5fq6s"
Sep 16 19:02:41.654012 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:02:41.653986    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-cb9a3-5ff6b444fd-5fq6s"]
Sep 16 19:02:41.656084 ip-10-0-139-182 kubenswrapper[2584]: W0916 19:02:41.656048    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podabd27f76_1d33_41f7_ad59_bc0b0836de34.slice/crio-87d67eea490a5de9e78e0bb66d0ebcf0f62356cb337502798d3dfe3e752fdb12 WatchSource:0}: Error finding container 87d67eea490a5de9e78e0bb66d0ebcf0f62356cb337502798d3dfe3e752fdb12: Status 404 returned error can't find the container with id 87d67eea490a5de9e78e0bb66d0ebcf0f62356cb337502798d3dfe3e752fdb12
Sep 16 19:02:42.279614 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:02:42.279575    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-cb9a3-5ff6b444fd-5fq6s" event={"ID":"abd27f76-1d33-41f7-ad59-bc0b0836de34","Type":"ContainerStarted","Data":"b57e307975e20764563897d2bce7db0e54100389e2f95e683df55bc1d5f7130b"}
Sep 16 19:02:42.280103 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:02:42.279630    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-cb9a3-5ff6b444fd-5fq6s" event={"ID":"abd27f76-1d33-41f7-ad59-bc0b0836de34","Type":"ContainerStarted","Data":"87d67eea490a5de9e78e0bb66d0ebcf0f62356cb337502798d3dfe3e752fdb12"}
Sep 16 19:02:42.280103 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:02:42.279739    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/sequence-graph-cb9a3-5ff6b444fd-5fq6s"
Sep 16 19:02:42.296631 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:02:42.296582    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/sequence-graph-cb9a3-5ff6b444fd-5fq6s" podStartSLOduration=1.296567338 podStartE2EDuration="1.296567338s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 19:02:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 19:02:42.294680215 +0000 UTC m=+1092.624182628" watchObservedRunningTime="2026-09-16 19:02:42.296567338 +0000 UTC m=+1092.626069751"
Sep 16 19:02:48.289661 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:02:48.289635    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/sequence-graph-cb9a3-5ff6b444fd-5fq6s"
Sep 16 19:03:34.435558 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:03:34.435516    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 19:03:37.006695 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:03:37.006655    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 19:03:37.107996 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:03:37.107956    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 19:04:36.096121 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:04:36.096012    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-crjw7_ecf273b5-b465-4008-b035-f7ac0e740d58/console-operator/1.log"
Sep 16 19:04:36.099948 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:04:36.097108    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-crjw7_ecf273b5-b465-4008-b035-f7ac0e740d58/console-operator/1.log"
Sep 16 19:08:31.633556 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:08:31.633465    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 19:09:36.125212 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:09:36.125084    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-crjw7_ecf273b5-b465-4008-b035-f7ac0e740d58/console-operator/1.log"
Sep 16 19:09:36.129206 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:09:36.126705    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-crjw7_ecf273b5-b465-4008-b035-f7ac0e740d58/console-operator/1.log"
Sep 16 19:10:04.856484 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:04.856393    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-54df7c6d-nl4g9"]
Sep 16 19:10:04.860125 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:04.860098    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-54df7c6d-nl4g9"
Sep 16 19:10:04.882578 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:04.882536    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-54df7c6d-nl4g9"]
Sep 16 19:10:04.938885 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:04.938853    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 19:10:04.963288 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:04.963248    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3e77e0b9-7198-4a61-9c43-f84eccf663dd-oauth-serving-cert\") pod \"console-54df7c6d-nl4g9\" (UID: \"3e77e0b9-7198-4a61-9c43-f84eccf663dd\") " pod="openshift-console/console-54df7c6d-nl4g9"
Sep 16 19:10:04.963472 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:04.963323    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3e77e0b9-7198-4a61-9c43-f84eccf663dd-console-config\") pod \"console-54df7c6d-nl4g9\" (UID: \"3e77e0b9-7198-4a61-9c43-f84eccf663dd\") " pod="openshift-console/console-54df7c6d-nl4g9"
Sep 16 19:10:04.963472 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:04.963350    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3e77e0b9-7198-4a61-9c43-f84eccf663dd-console-serving-cert\") pod \"console-54df7c6d-nl4g9\" (UID: \"3e77e0b9-7198-4a61-9c43-f84eccf663dd\") " pod="openshift-console/console-54df7c6d-nl4g9"
Sep 16 19:10:04.963472 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:04.963366    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zlm8b\" (UniqueName: \"kubernetes.io/projected/3e77e0b9-7198-4a61-9c43-f84eccf663dd-kube-api-access-zlm8b\") pod \"console-54df7c6d-nl4g9\" (UID: \"3e77e0b9-7198-4a61-9c43-f84eccf663dd\") " pod="openshift-console/console-54df7c6d-nl4g9"
Sep 16 19:10:04.963646 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:04.963471    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3e77e0b9-7198-4a61-9c43-f84eccf663dd-console-oauth-config\") pod \"console-54df7c6d-nl4g9\" (UID: \"3e77e0b9-7198-4a61-9c43-f84eccf663dd\") " pod="openshift-console/console-54df7c6d-nl4g9"
Sep 16 19:10:04.963646 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:04.963510    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3e77e0b9-7198-4a61-9c43-f84eccf663dd-trusted-ca-bundle\") pod \"console-54df7c6d-nl4g9\" (UID: \"3e77e0b9-7198-4a61-9c43-f84eccf663dd\") " pod="openshift-console/console-54df7c6d-nl4g9"
Sep 16 19:10:04.963646 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:04.963573    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3e77e0b9-7198-4a61-9c43-f84eccf663dd-service-ca\") pod \"console-54df7c6d-nl4g9\" (UID: \"3e77e0b9-7198-4a61-9c43-f84eccf663dd\") " pod="openshift-console/console-54df7c6d-nl4g9"
Sep 16 19:10:04.996949 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:04.996908    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 19:10:05.065121 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:05.065079    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3e77e0b9-7198-4a61-9c43-f84eccf663dd-service-ca\") pod \"console-54df7c6d-nl4g9\" (UID: \"3e77e0b9-7198-4a61-9c43-f84eccf663dd\") " pod="openshift-console/console-54df7c6d-nl4g9"
Sep 16 19:10:05.065121 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:05.065133    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3e77e0b9-7198-4a61-9c43-f84eccf663dd-oauth-serving-cert\") pod \"console-54df7c6d-nl4g9\" (UID: \"3e77e0b9-7198-4a61-9c43-f84eccf663dd\") " pod="openshift-console/console-54df7c6d-nl4g9"
Sep 16 19:10:05.065389 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:05.065176    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3e77e0b9-7198-4a61-9c43-f84eccf663dd-console-config\") pod \"console-54df7c6d-nl4g9\" (UID: \"3e77e0b9-7198-4a61-9c43-f84eccf663dd\") " pod="openshift-console/console-54df7c6d-nl4g9"
Sep 16 19:10:05.065389 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:05.065200    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3e77e0b9-7198-4a61-9c43-f84eccf663dd-console-serving-cert\") pod \"console-54df7c6d-nl4g9\" (UID: \"3e77e0b9-7198-4a61-9c43-f84eccf663dd\") " pod="openshift-console/console-54df7c6d-nl4g9"
Sep 16 19:10:05.065389 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:05.065222    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zlm8b\" (UniqueName: \"kubernetes.io/projected/3e77e0b9-7198-4a61-9c43-f84eccf663dd-kube-api-access-zlm8b\") pod \"console-54df7c6d-nl4g9\" (UID: \"3e77e0b9-7198-4a61-9c43-f84eccf663dd\") " pod="openshift-console/console-54df7c6d-nl4g9"
Sep 16 19:10:05.065389 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:05.065264    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3e77e0b9-7198-4a61-9c43-f84eccf663dd-console-oauth-config\") pod \"console-54df7c6d-nl4g9\" (UID: \"3e77e0b9-7198-4a61-9c43-f84eccf663dd\") " pod="openshift-console/console-54df7c6d-nl4g9"
Sep 16 19:10:05.065389 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:05.065290    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3e77e0b9-7198-4a61-9c43-f84eccf663dd-trusted-ca-bundle\") pod \"console-54df7c6d-nl4g9\" (UID: \"3e77e0b9-7198-4a61-9c43-f84eccf663dd\") " pod="openshift-console/console-54df7c6d-nl4g9"
Sep 16 19:10:05.066004 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:05.065976    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3e77e0b9-7198-4a61-9c43-f84eccf663dd-console-config\") pod \"console-54df7c6d-nl4g9\" (UID: \"3e77e0b9-7198-4a61-9c43-f84eccf663dd\") " pod="openshift-console/console-54df7c6d-nl4g9"
Sep 16 19:10:05.066137 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:05.065980    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3e77e0b9-7198-4a61-9c43-f84eccf663dd-service-ca\") pod \"console-54df7c6d-nl4g9\" (UID: \"3e77e0b9-7198-4a61-9c43-f84eccf663dd\") " pod="openshift-console/console-54df7c6d-nl4g9"
Sep 16 19:10:05.066137 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:05.066052    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3e77e0b9-7198-4a61-9c43-f84eccf663dd-oauth-serving-cert\") pod \"console-54df7c6d-nl4g9\" (UID: \"3e77e0b9-7198-4a61-9c43-f84eccf663dd\") " pod="openshift-console/console-54df7c6d-nl4g9"
Sep 16 19:10:05.066494 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:05.066475    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3e77e0b9-7198-4a61-9c43-f84eccf663dd-trusted-ca-bundle\") pod \"console-54df7c6d-nl4g9\" (UID: \"3e77e0b9-7198-4a61-9c43-f84eccf663dd\") " pod="openshift-console/console-54df7c6d-nl4g9"
Sep 16 19:10:05.067911 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:05.067880    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3e77e0b9-7198-4a61-9c43-f84eccf663dd-console-oauth-config\") pod \"console-54df7c6d-nl4g9\" (UID: \"3e77e0b9-7198-4a61-9c43-f84eccf663dd\") " pod="openshift-console/console-54df7c6d-nl4g9"
Sep 16 19:10:05.068046 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:05.067954    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3e77e0b9-7198-4a61-9c43-f84eccf663dd-console-serving-cert\") pod \"console-54df7c6d-nl4g9\" (UID: \"3e77e0b9-7198-4a61-9c43-f84eccf663dd\") " pod="openshift-console/console-54df7c6d-nl4g9"
Sep 16 19:10:05.074475 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:05.074444    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zlm8b\" (UniqueName: \"kubernetes.io/projected/3e77e0b9-7198-4a61-9c43-f84eccf663dd-kube-api-access-zlm8b\") pod \"console-54df7c6d-nl4g9\" (UID: \"3e77e0b9-7198-4a61-9c43-f84eccf663dd\") " pod="openshift-console/console-54df7c6d-nl4g9"
Sep 16 19:10:05.169442 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:05.169343    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-54df7c6d-nl4g9"
Sep 16 19:10:05.449013 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:05.448988    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-54df7c6d-nl4g9"]
Sep 16 19:10:05.451384 ip-10-0-139-182 kubenswrapper[2584]: W0916 19:10:05.451351    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e77e0b9_7198_4a61_9c43_f84eccf663dd.slice/crio-2a1a996adc9caaf730b6b38f157962e2ee342343e7366895c102cf207783063a WatchSource:0}: Error finding container 2a1a996adc9caaf730b6b38f157962e2ee342343e7366895c102cf207783063a: Status 404 returned error can't find the container with id 2a1a996adc9caaf730b6b38f157962e2ee342343e7366895c102cf207783063a
Sep 16 19:10:06.295351 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:06.295315    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-54df7c6d-nl4g9" event={"ID":"3e77e0b9-7198-4a61-9c43-f84eccf663dd","Type":"ContainerStarted","Data":"8873bbf3cf10d5d61bec029b6a41545386226d530a5285d13f9e4cd28702dfe0"}
Sep 16 19:10:06.295778 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:06.295365    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-54df7c6d-nl4g9" event={"ID":"3e77e0b9-7198-4a61-9c43-f84eccf663dd","Type":"ContainerStarted","Data":"2a1a996adc9caaf730b6b38f157962e2ee342343e7366895c102cf207783063a"}
Sep 16 19:10:06.316706 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:06.316639    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-54df7c6d-nl4g9" podStartSLOduration=2.316621075 podStartE2EDuration="2.316621075s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 19:10:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 19:10:06.314666289 +0000 UTC m=+1536.644168713" watchObservedRunningTime="2026-09-16 19:10:06.316621075 +0000 UTC m=+1536.646123488"
Sep 16 19:10:15.170482 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:15.170440    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-54df7c6d-nl4g9"
Sep 16 19:10:15.171106 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:15.170512    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-54df7c6d-nl4g9"
Sep 16 19:10:15.175315 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:15.175291    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-54df7c6d-nl4g9"
Sep 16 19:10:15.340616 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:15.340583    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-54df7c6d-nl4g9"
Sep 16 19:10:15.389551 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:15.389507    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7c845b4d74-lj9n5"]
Sep 16 19:10:15.543708 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:15.543672    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 19:10:15.644308 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:15.644271    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 19:10:15.745380 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:15.745345    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 19:10:15.932869 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:15.932757    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/switch-graph-93efe-7846bd77df-h9n8c"]
Sep 16 19:10:15.933045 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:15.932997    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/switch-graph-93efe-7846bd77df-h9n8c" podUID="9ec73f37-d895-442d-a2d4-f3dfe3eee6c1" containerName="switch-graph-93efe" containerID="cri-o://4a6fd92f63c2fe778eb67313d0a01697ffdb53a6cd967b6ad0104385107efd8a" gracePeriod=30
Sep 16 19:10:16.067854 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:16.067794    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-93efe-predictor-85b4cff8bf-k95wm"]
Sep 16 19:10:16.068096 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:16.068071    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-93efe-predictor-85b4cff8bf-k95wm" podUID="771df157-671d-4a80-aeea-88aca624578a" containerName="kserve-container" containerID="cri-o://3c0f4d4fb04a891cda2799ecc1eabbcfb524bfeabaeb2cf325fff84dc45c28c9" gracePeriod=30
Sep 16 19:10:16.068197 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:16.068166    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-93efe-predictor-85b4cff8bf-k95wm" podUID="771df157-671d-4a80-aeea-88aca624578a" containerName="kube-rbac-proxy" containerID="cri-o://9be062ac6b74f816fd05e6355e30f7eaf4ea266d45622076a8f7a156611a62a7" gracePeriod=30
Sep 16 19:10:16.154714 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:16.154679    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-016d3-predictor-698bb76d65-bphfw"]
Sep 16 19:10:16.159644 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:16.159619    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-016d3-predictor-698bb76d65-bphfw"
Sep 16 19:10:16.162429 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:16.162406    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-016d3-kube-rbac-proxy-sar-config\""
Sep 16 19:10:16.162572 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:16.162451    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-016d3-predictor-serving-cert\""
Sep 16 19:10:16.169966 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:16.169920    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-016d3-predictor-698bb76d65-bphfw"]
Sep 16 19:10:16.267346 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:16.267313    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8b84fe5f-226c-478a-bd7a-c5d815fba70f-proxy-tls\") pod \"success-200-isvc-016d3-predictor-698bb76d65-bphfw\" (UID: \"8b84fe5f-226c-478a-bd7a-c5d815fba70f\") " pod="kserve-ci-e2e-test/success-200-isvc-016d3-predictor-698bb76d65-bphfw"
Sep 16 19:10:16.267346 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:16.267349    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/8b84fe5f-226c-478a-bd7a-c5d815fba70f-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-016d3-predictor-698bb76d65-bphfw\" (UID: \"8b84fe5f-226c-478a-bd7a-c5d815fba70f\") " pod="kserve-ci-e2e-test/success-200-isvc-016d3-predictor-698bb76d65-bphfw"
Sep 16 19:10:16.267758 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:16.267372    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6r268\" (UniqueName: \"kubernetes.io/projected/8b84fe5f-226c-478a-bd7a-c5d815fba70f-kube-api-access-6r268\") pod \"success-200-isvc-016d3-predictor-698bb76d65-bphfw\" (UID: \"8b84fe5f-226c-478a-bd7a-c5d815fba70f\") " pod="kserve-ci-e2e-test/success-200-isvc-016d3-predictor-698bb76d65-bphfw"
Sep 16 19:10:16.349060 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:16.349025    2584 generic.go:356] "Generic (PLEG): container finished" podID="771df157-671d-4a80-aeea-88aca624578a" containerID="9be062ac6b74f816fd05e6355e30f7eaf4ea266d45622076a8f7a156611a62a7" exitCode=2
Sep 16 19:10:16.349217 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:16.349102    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-93efe-predictor-85b4cff8bf-k95wm" event={"ID":"771df157-671d-4a80-aeea-88aca624578a","Type":"ContainerDied","Data":"9be062ac6b74f816fd05e6355e30f7eaf4ea266d45622076a8f7a156611a62a7"}
Sep 16 19:10:16.368584 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:16.368547    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8b84fe5f-226c-478a-bd7a-c5d815fba70f-proxy-tls\") pod \"success-200-isvc-016d3-predictor-698bb76d65-bphfw\" (UID: \"8b84fe5f-226c-478a-bd7a-c5d815fba70f\") " pod="kserve-ci-e2e-test/success-200-isvc-016d3-predictor-698bb76d65-bphfw"
Sep 16 19:10:16.368584 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:16.368586    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/8b84fe5f-226c-478a-bd7a-c5d815fba70f-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-016d3-predictor-698bb76d65-bphfw\" (UID: \"8b84fe5f-226c-478a-bd7a-c5d815fba70f\") " pod="kserve-ci-e2e-test/success-200-isvc-016d3-predictor-698bb76d65-bphfw"
Sep 16 19:10:16.368798 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:16.368605    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6r268\" (UniqueName: \"kubernetes.io/projected/8b84fe5f-226c-478a-bd7a-c5d815fba70f-kube-api-access-6r268\") pod \"success-200-isvc-016d3-predictor-698bb76d65-bphfw\" (UID: \"8b84fe5f-226c-478a-bd7a-c5d815fba70f\") " pod="kserve-ci-e2e-test/success-200-isvc-016d3-predictor-698bb76d65-bphfw"
Sep 16 19:10:16.369232 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:16.369212    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/8b84fe5f-226c-478a-bd7a-c5d815fba70f-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-016d3-predictor-698bb76d65-bphfw\" (UID: \"8b84fe5f-226c-478a-bd7a-c5d815fba70f\") " pod="kserve-ci-e2e-test/success-200-isvc-016d3-predictor-698bb76d65-bphfw"
Sep 16 19:10:16.371028 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:16.371004    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8b84fe5f-226c-478a-bd7a-c5d815fba70f-proxy-tls\") pod \"success-200-isvc-016d3-predictor-698bb76d65-bphfw\" (UID: \"8b84fe5f-226c-478a-bd7a-c5d815fba70f\") " pod="kserve-ci-e2e-test/success-200-isvc-016d3-predictor-698bb76d65-bphfw"
Sep 16 19:10:16.377580 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:16.377555    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6r268\" (UniqueName: \"kubernetes.io/projected/8b84fe5f-226c-478a-bd7a-c5d815fba70f-kube-api-access-6r268\") pod \"success-200-isvc-016d3-predictor-698bb76d65-bphfw\" (UID: \"8b84fe5f-226c-478a-bd7a-c5d815fba70f\") " pod="kserve-ci-e2e-test/success-200-isvc-016d3-predictor-698bb76d65-bphfw"
Sep 16 19:10:16.387594 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:16.387562    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 19:10:16.474732 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:16.474704    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-016d3-predictor-698bb76d65-bphfw"
Sep 16 19:10:16.489291 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:16.489257    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 19:10:16.746515 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:16.746491    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-016d3-predictor-698bb76d65-bphfw"]
Sep 16 19:10:16.748481 ip-10-0-139-182 kubenswrapper[2584]: W0916 19:10:16.748446    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b84fe5f_226c_478a_bd7a_c5d815fba70f.slice/crio-945844f991192ff380e8ccc0ca7543ac7414f1e33db7ac8256705d91f4e31bab WatchSource:0}: Error finding container 945844f991192ff380e8ccc0ca7543ac7414f1e33db7ac8256705d91f4e31bab: Status 404 returned error can't find the container with id 945844f991192ff380e8ccc0ca7543ac7414f1e33db7ac8256705d91f4e31bab
Sep 16 19:10:17.089460 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:17.089387    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 19:10:17.357060 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:17.356979    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-016d3-predictor-698bb76d65-bphfw" event={"ID":"8b84fe5f-226c-478a-bd7a-c5d815fba70f","Type":"ContainerStarted","Data":"a840db21c3366603cf43b8c201c202c0ea001930a8e4b4a94d7801d5c7382159"}
Sep 16 19:10:17.357060 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:17.357045    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-016d3-predictor-698bb76d65-bphfw" event={"ID":"8b84fe5f-226c-478a-bd7a-c5d815fba70f","Type":"ContainerStarted","Data":"cd8056f266b6a4308e52cdf2f6a5f3cfad7d128ca77b0556efb4d2f4a9f58e12"}
Sep 16 19:10:17.357435 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:17.357074    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-016d3-predictor-698bb76d65-bphfw" event={"ID":"8b84fe5f-226c-478a-bd7a-c5d815fba70f","Type":"ContainerStarted","Data":"945844f991192ff380e8ccc0ca7543ac7414f1e33db7ac8256705d91f4e31bab"}
Sep 16 19:10:17.357435 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:17.357221    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-016d3-predictor-698bb76d65-bphfw"
Sep 16 19:10:17.377094 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:17.377044    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-016d3-predictor-698bb76d65-bphfw" podStartSLOduration=1.377027435 podStartE2EDuration="1.377027435s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 19:10:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 19:10:17.375854746 +0000 UTC m=+1547.705357153" watchObservedRunningTime="2026-09-16 19:10:17.377027435 +0000 UTC m=+1547.706529848"
Sep 16 19:10:18.361848 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:18.361791    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-016d3-predictor-698bb76d65-bphfw"
Sep 16 19:10:18.363271 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:18.363240    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-016d3-predictor-698bb76d65-bphfw" podUID="8b84fe5f-226c-478a-bd7a-c5d815fba70f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.39:8080: connect: connection refused"
Sep 16 19:10:18.805068 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:18.805029    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-93efe-predictor-85b4cff8bf-k95wm" podUID="771df157-671d-4a80-aeea-88aca624578a" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.33:8643/healthz\": dial tcp 10.133.0.33:8643: connect: connection refused"
Sep 16 19:10:18.809313 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:18.809288    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-93efe-predictor-85b4cff8bf-k95wm" podUID="771df157-671d-4a80-aeea-88aca624578a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.33:8080: connect: connection refused"
Sep 16 19:10:19.109073 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:19.109036    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-93efe-7846bd77df-h9n8c" podUID="9ec73f37-d895-442d-a2d4-f3dfe3eee6c1" containerName="switch-graph-93efe" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 16 19:10:19.223418 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:19.223364    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-93efe-predictor-85b4cff8bf-k95wm"
Sep 16 19:10:19.367758 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:19.367669    2584 generic.go:356] "Generic (PLEG): container finished" podID="771df157-671d-4a80-aeea-88aca624578a" containerID="3c0f4d4fb04a891cda2799ecc1eabbcfb524bfeabaeb2cf325fff84dc45c28c9" exitCode=0
Sep 16 19:10:19.367758 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:19.367727    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-93efe-predictor-85b4cff8bf-k95wm"
Sep 16 19:10:19.368302 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:19.367760    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-93efe-predictor-85b4cff8bf-k95wm" event={"ID":"771df157-671d-4a80-aeea-88aca624578a","Type":"ContainerDied","Data":"3c0f4d4fb04a891cda2799ecc1eabbcfb524bfeabaeb2cf325fff84dc45c28c9"}
Sep 16 19:10:19.368302 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:19.367844    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-93efe-predictor-85b4cff8bf-k95wm" event={"ID":"771df157-671d-4a80-aeea-88aca624578a","Type":"ContainerDied","Data":"7f3aebb5b8373f58d500b75082499f4df9e7b483899e47dc44620fc679c0aca7"}
Sep 16 19:10:19.368302 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:19.367889    2584 scope.go:117] "RemoveContainer" containerID="9be062ac6b74f816fd05e6355e30f7eaf4ea266d45622076a8f7a156611a62a7"
Sep 16 19:10:19.368302 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:19.368205    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-016d3-predictor-698bb76d65-bphfw" podUID="8b84fe5f-226c-478a-bd7a-c5d815fba70f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.39:8080: connect: connection refused"
Sep 16 19:10:19.376028 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:19.376011    2584 scope.go:117] "RemoveContainer" containerID="3c0f4d4fb04a891cda2799ecc1eabbcfb524bfeabaeb2cf325fff84dc45c28c9"
Sep 16 19:10:19.383346 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:19.383331    2584 scope.go:117] "RemoveContainer" containerID="9be062ac6b74f816fd05e6355e30f7eaf4ea266d45622076a8f7a156611a62a7"
Sep 16 19:10:19.383627 ip-10-0-139-182 kubenswrapper[2584]: E0916 19:10:19.383611    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9be062ac6b74f816fd05e6355e30f7eaf4ea266d45622076a8f7a156611a62a7\": container with ID starting with 9be062ac6b74f816fd05e6355e30f7eaf4ea266d45622076a8f7a156611a62a7 not found: ID does not exist" containerID="9be062ac6b74f816fd05e6355e30f7eaf4ea266d45622076a8f7a156611a62a7"
Sep 16 19:10:19.383679 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:19.383635    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9be062ac6b74f816fd05e6355e30f7eaf4ea266d45622076a8f7a156611a62a7"} err="failed to get container status \"9be062ac6b74f816fd05e6355e30f7eaf4ea266d45622076a8f7a156611a62a7\": rpc error: code = NotFound desc = could not find container \"9be062ac6b74f816fd05e6355e30f7eaf4ea266d45622076a8f7a156611a62a7\": container with ID starting with 9be062ac6b74f816fd05e6355e30f7eaf4ea266d45622076a8f7a156611a62a7 not found: ID does not exist"
Sep 16 19:10:19.383679 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:19.383651    2584 scope.go:117] "RemoveContainer" containerID="3c0f4d4fb04a891cda2799ecc1eabbcfb524bfeabaeb2cf325fff84dc45c28c9"
Sep 16 19:10:19.383880 ip-10-0-139-182 kubenswrapper[2584]: E0916 19:10:19.383861    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3c0f4d4fb04a891cda2799ecc1eabbcfb524bfeabaeb2cf325fff84dc45c28c9\": container with ID starting with 3c0f4d4fb04a891cda2799ecc1eabbcfb524bfeabaeb2cf325fff84dc45c28c9 not found: ID does not exist" containerID="3c0f4d4fb04a891cda2799ecc1eabbcfb524bfeabaeb2cf325fff84dc45c28c9"
Sep 16 19:10:19.383989 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:19.383882    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3c0f4d4fb04a891cda2799ecc1eabbcfb524bfeabaeb2cf325fff84dc45c28c9"} err="failed to get container status \"3c0f4d4fb04a891cda2799ecc1eabbcfb524bfeabaeb2cf325fff84dc45c28c9\": rpc error: code = NotFound desc = could not find container \"3c0f4d4fb04a891cda2799ecc1eabbcfb524bfeabaeb2cf325fff84dc45c28c9\": container with ID starting with 3c0f4d4fb04a891cda2799ecc1eabbcfb524bfeabaeb2cf325fff84dc45c28c9 not found: ID does not exist"
Sep 16 19:10:19.395196 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:19.395173    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7whxt\" (UniqueName: \"kubernetes.io/projected/771df157-671d-4a80-aeea-88aca624578a-kube-api-access-7whxt\") pod \"771df157-671d-4a80-aeea-88aca624578a\" (UID: \"771df157-671d-4a80-aeea-88aca624578a\") "
Sep 16 19:10:19.395359 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:19.395215    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/771df157-671d-4a80-aeea-88aca624578a-kube-rbac-proxy-sar-config\") pod \"771df157-671d-4a80-aeea-88aca624578a\" (UID: \"771df157-671d-4a80-aeea-88aca624578a\") "
Sep 16 19:10:19.395359 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:19.395252    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/771df157-671d-4a80-aeea-88aca624578a-proxy-tls\") pod \"771df157-671d-4a80-aeea-88aca624578a\" (UID: \"771df157-671d-4a80-aeea-88aca624578a\") "
Sep 16 19:10:19.395629 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:19.395600    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/771df157-671d-4a80-aeea-88aca624578a-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "771df157-671d-4a80-aeea-88aca624578a" (UID: "771df157-671d-4a80-aeea-88aca624578a"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 19:10:19.397404 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:19.397382    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/771df157-671d-4a80-aeea-88aca624578a-kube-api-access-7whxt" (OuterVolumeSpecName: "kube-api-access-7whxt") pod "771df157-671d-4a80-aeea-88aca624578a" (UID: "771df157-671d-4a80-aeea-88aca624578a"). InnerVolumeSpecName "kube-api-access-7whxt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 19:10:19.397481 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:19.397381    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/771df157-671d-4a80-aeea-88aca624578a-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "771df157-671d-4a80-aeea-88aca624578a" (UID: "771df157-671d-4a80-aeea-88aca624578a"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 19:10:19.496119 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:19.496079    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7whxt\" (UniqueName: \"kubernetes.io/projected/771df157-671d-4a80-aeea-88aca624578a-kube-api-access-7whxt\") on node \"ip-10-0-139-182.ec2.internal\" DevicePath \"\""
Sep 16 19:10:19.496119 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:19.496111    2584 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/771df157-671d-4a80-aeea-88aca624578a-kube-rbac-proxy-sar-config\") on node \"ip-10-0-139-182.ec2.internal\" DevicePath \"\""
Sep 16 19:10:19.496119 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:19.496124    2584 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/771df157-671d-4a80-aeea-88aca624578a-proxy-tls\") on node \"ip-10-0-139-182.ec2.internal\" DevicePath \"\""
Sep 16 19:10:19.692263 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:19.692224    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-93efe-predictor-85b4cff8bf-k95wm"]
Sep 16 19:10:19.697859 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:19.697814    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-93efe-predictor-85b4cff8bf-k95wm"]
Sep 16 19:10:20.116807 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:20.116771    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="771df157-671d-4a80-aeea-88aca624578a" path="/var/lib/kubelet/pods/771df157-671d-4a80-aeea-88aca624578a/volumes"
Sep 16 19:10:24.109179 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:24.109142    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-93efe-7846bd77df-h9n8c" podUID="9ec73f37-d895-442d-a2d4-f3dfe3eee6c1" containerName="switch-graph-93efe" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 16 19:10:24.372559 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:24.372466    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-016d3-predictor-698bb76d65-bphfw"
Sep 16 19:10:24.373068 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:24.373039    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-016d3-predictor-698bb76d65-bphfw" podUID="8b84fe5f-226c-478a-bd7a-c5d815fba70f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.39:8080: connect: connection refused"
Sep 16 19:10:29.108503 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:29.108465    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-93efe-7846bd77df-h9n8c" podUID="9ec73f37-d895-442d-a2d4-f3dfe3eee6c1" containerName="switch-graph-93efe" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 16 19:10:29.108893 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:29.108581    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/switch-graph-93efe-7846bd77df-h9n8c"
Sep 16 19:10:34.109132 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:34.109090    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-93efe-7846bd77df-h9n8c" podUID="9ec73f37-d895-442d-a2d4-f3dfe3eee6c1" containerName="switch-graph-93efe" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 16 19:10:34.373904 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:34.373796    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-016d3-predictor-698bb76d65-bphfw" podUID="8b84fe5f-226c-478a-bd7a-c5d815fba70f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.39:8080: connect: connection refused"
Sep 16 19:10:39.109291 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:39.109245    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-93efe-7846bd77df-h9n8c" podUID="9ec73f37-d895-442d-a2d4-f3dfe3eee6c1" containerName="switch-graph-93efe" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 16 19:10:40.411601 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:40.411551    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7c845b4d74-lj9n5" podUID="0dd76b92-206d-42cf-a954-7ed474be9057" containerName="console" containerID="cri-o://05d1ef0f2319e6131fe8d117c7b037615d0b687548d9140e73abc25080ba4a13" gracePeriod=15
Sep 16 19:10:40.659944 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:40.659913    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c845b4d74-lj9n5"
Sep 16 19:10:40.773467 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:40.773433    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0dd76b92-206d-42cf-a954-7ed474be9057-console-oauth-config\") pod \"0dd76b92-206d-42cf-a954-7ed474be9057\" (UID: \"0dd76b92-206d-42cf-a954-7ed474be9057\") "
Sep 16 19:10:40.773670 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:40.773475    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0dd76b92-206d-42cf-a954-7ed474be9057-console-serving-cert\") pod \"0dd76b92-206d-42cf-a954-7ed474be9057\" (UID: \"0dd76b92-206d-42cf-a954-7ed474be9057\") "
Sep 16 19:10:40.773670 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:40.773495    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jtgmp\" (UniqueName: \"kubernetes.io/projected/0dd76b92-206d-42cf-a954-7ed474be9057-kube-api-access-jtgmp\") pod \"0dd76b92-206d-42cf-a954-7ed474be9057\" (UID: \"0dd76b92-206d-42cf-a954-7ed474be9057\") "
Sep 16 19:10:40.773670 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:40.773539    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0dd76b92-206d-42cf-a954-7ed474be9057-service-ca\") pod \"0dd76b92-206d-42cf-a954-7ed474be9057\" (UID: \"0dd76b92-206d-42cf-a954-7ed474be9057\") "
Sep 16 19:10:40.773670 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:40.773573    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0dd76b92-206d-42cf-a954-7ed474be9057-console-config\") pod \"0dd76b92-206d-42cf-a954-7ed474be9057\" (UID: \"0dd76b92-206d-42cf-a954-7ed474be9057\") "
Sep 16 19:10:40.773670 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:40.773608    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0dd76b92-206d-42cf-a954-7ed474be9057-trusted-ca-bundle\") pod \"0dd76b92-206d-42cf-a954-7ed474be9057\" (UID: \"0dd76b92-206d-42cf-a954-7ed474be9057\") "
Sep 16 19:10:40.773670 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:40.773646    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0dd76b92-206d-42cf-a954-7ed474be9057-oauth-serving-cert\") pod \"0dd76b92-206d-42cf-a954-7ed474be9057\" (UID: \"0dd76b92-206d-42cf-a954-7ed474be9057\") "
Sep 16 19:10:40.774080 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:40.774050    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0dd76b92-206d-42cf-a954-7ed474be9057-service-ca" (OuterVolumeSpecName: "service-ca") pod "0dd76b92-206d-42cf-a954-7ed474be9057" (UID: "0dd76b92-206d-42cf-a954-7ed474be9057"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 19:10:40.774080 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:40.774063    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0dd76b92-206d-42cf-a954-7ed474be9057-console-config" (OuterVolumeSpecName: "console-config") pod "0dd76b92-206d-42cf-a954-7ed474be9057" (UID: "0dd76b92-206d-42cf-a954-7ed474be9057"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 19:10:40.774196 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:40.774068    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0dd76b92-206d-42cf-a954-7ed474be9057-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "0dd76b92-206d-42cf-a954-7ed474be9057" (UID: "0dd76b92-206d-42cf-a954-7ed474be9057"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 19:10:40.774196 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:40.774101    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0dd76b92-206d-42cf-a954-7ed474be9057-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "0dd76b92-206d-42cf-a954-7ed474be9057" (UID: "0dd76b92-206d-42cf-a954-7ed474be9057"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 19:10:40.775777 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:40.775752    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0dd76b92-206d-42cf-a954-7ed474be9057-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "0dd76b92-206d-42cf-a954-7ed474be9057" (UID: "0dd76b92-206d-42cf-a954-7ed474be9057"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 19:10:40.775916 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:40.775778    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0dd76b92-206d-42cf-a954-7ed474be9057-kube-api-access-jtgmp" (OuterVolumeSpecName: "kube-api-access-jtgmp") pod "0dd76b92-206d-42cf-a954-7ed474be9057" (UID: "0dd76b92-206d-42cf-a954-7ed474be9057"). InnerVolumeSpecName "kube-api-access-jtgmp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 19:10:40.775916 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:40.775842    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0dd76b92-206d-42cf-a954-7ed474be9057-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "0dd76b92-206d-42cf-a954-7ed474be9057" (UID: "0dd76b92-206d-42cf-a954-7ed474be9057"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 19:10:40.874649 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:40.874603    2584 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0dd76b92-206d-42cf-a954-7ed474be9057-service-ca\") on node \"ip-10-0-139-182.ec2.internal\" DevicePath \"\""
Sep 16 19:10:40.874649 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:40.874635    2584 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0dd76b92-206d-42cf-a954-7ed474be9057-console-config\") on node \"ip-10-0-139-182.ec2.internal\" DevicePath \"\""
Sep 16 19:10:40.874649 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:40.874648    2584 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0dd76b92-206d-42cf-a954-7ed474be9057-trusted-ca-bundle\") on node \"ip-10-0-139-182.ec2.internal\" DevicePath \"\""
Sep 16 19:10:40.874649 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:40.874658    2584 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0dd76b92-206d-42cf-a954-7ed474be9057-oauth-serving-cert\") on node \"ip-10-0-139-182.ec2.internal\" DevicePath \"\""
Sep 16 19:10:40.874958 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:40.874671    2584 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0dd76b92-206d-42cf-a954-7ed474be9057-console-oauth-config\") on node \"ip-10-0-139-182.ec2.internal\" DevicePath \"\""
Sep 16 19:10:40.874958 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:40.874679    2584 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0dd76b92-206d-42cf-a954-7ed474be9057-console-serving-cert\") on node \"ip-10-0-139-182.ec2.internal\" DevicePath \"\""
Sep 16 19:10:40.874958 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:40.874687    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jtgmp\" (UniqueName: \"kubernetes.io/projected/0dd76b92-206d-42cf-a954-7ed474be9057-kube-api-access-jtgmp\") on node \"ip-10-0-139-182.ec2.internal\" DevicePath \"\""
Sep 16 19:10:41.477780 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:41.477734    2584 generic.go:356] "Generic (PLEG): container finished" podID="0dd76b92-206d-42cf-a954-7ed474be9057" containerID="05d1ef0f2319e6131fe8d117c7b037615d0b687548d9140e73abc25080ba4a13" exitCode=0
Sep 16 19:10:41.478314 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:41.477770    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c845b4d74-lj9n5" event={"ID":"0dd76b92-206d-42cf-a954-7ed474be9057","Type":"ContainerDied","Data":"05d1ef0f2319e6131fe8d117c7b037615d0b687548d9140e73abc25080ba4a13"}
Sep 16 19:10:41.478314 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:41.477823    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c845b4d74-lj9n5"
Sep 16 19:10:41.478314 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:41.477881    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c845b4d74-lj9n5" event={"ID":"0dd76b92-206d-42cf-a954-7ed474be9057","Type":"ContainerDied","Data":"379012025ce7f12eddeb90a05f1d4231d11b50cd31adde8adc4d48a49398c1ef"}
Sep 16 19:10:41.478314 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:41.477903    2584 scope.go:117] "RemoveContainer" containerID="05d1ef0f2319e6131fe8d117c7b037615d0b687548d9140e73abc25080ba4a13"
Sep 16 19:10:41.487776 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:41.487752    2584 scope.go:117] "RemoveContainer" containerID="05d1ef0f2319e6131fe8d117c7b037615d0b687548d9140e73abc25080ba4a13"
Sep 16 19:10:41.488135 ip-10-0-139-182 kubenswrapper[2584]: E0916 19:10:41.488116    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"05d1ef0f2319e6131fe8d117c7b037615d0b687548d9140e73abc25080ba4a13\": container with ID starting with 05d1ef0f2319e6131fe8d117c7b037615d0b687548d9140e73abc25080ba4a13 not found: ID does not exist" containerID="05d1ef0f2319e6131fe8d117c7b037615d0b687548d9140e73abc25080ba4a13"
Sep 16 19:10:41.488215 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:41.488143    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"05d1ef0f2319e6131fe8d117c7b037615d0b687548d9140e73abc25080ba4a13"} err="failed to get container status \"05d1ef0f2319e6131fe8d117c7b037615d0b687548d9140e73abc25080ba4a13\": rpc error: code = NotFound desc = could not find container \"05d1ef0f2319e6131fe8d117c7b037615d0b687548d9140e73abc25080ba4a13\": container with ID starting with 05d1ef0f2319e6131fe8d117c7b037615d0b687548d9140e73abc25080ba4a13 not found: ID does not exist"
Sep 16 19:10:41.500344 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:41.500309    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7c845b4d74-lj9n5"]
Sep 16 19:10:41.504456 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:41.504421    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7c845b4d74-lj9n5"]
Sep 16 19:10:42.115920 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:42.115885    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0dd76b92-206d-42cf-a954-7ed474be9057" path="/var/lib/kubelet/pods/0dd76b92-206d-42cf-a954-7ed474be9057/volumes"
Sep 16 19:10:44.108643 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:44.108597    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-93efe-7846bd77df-h9n8c" podUID="9ec73f37-d895-442d-a2d4-f3dfe3eee6c1" containerName="switch-graph-93efe" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 16 19:10:44.373778 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:44.373677    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-016d3-predictor-698bb76d65-bphfw" podUID="8b84fe5f-226c-478a-bd7a-c5d815fba70f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.39:8080: connect: connection refused"
Sep 16 19:10:46.090579 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:46.090549    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-93efe-7846bd77df-h9n8c"
Sep 16 19:10:46.223234 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:46.223129    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9ec73f37-d895-442d-a2d4-f3dfe3eee6c1-openshift-service-ca-bundle\") pod \"9ec73f37-d895-442d-a2d4-f3dfe3eee6c1\" (UID: \"9ec73f37-d895-442d-a2d4-f3dfe3eee6c1\") "
Sep 16 19:10:46.223234 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:46.223183    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9ec73f37-d895-442d-a2d4-f3dfe3eee6c1-proxy-tls\") pod \"9ec73f37-d895-442d-a2d4-f3dfe3eee6c1\" (UID: \"9ec73f37-d895-442d-a2d4-f3dfe3eee6c1\") "
Sep 16 19:10:46.223513 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:46.223487    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9ec73f37-d895-442d-a2d4-f3dfe3eee6c1-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "9ec73f37-d895-442d-a2d4-f3dfe3eee6c1" (UID: "9ec73f37-d895-442d-a2d4-f3dfe3eee6c1"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 19:10:46.223886 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:46.223749    2584 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9ec73f37-d895-442d-a2d4-f3dfe3eee6c1-openshift-service-ca-bundle\") on node \"ip-10-0-139-182.ec2.internal\" DevicePath \"\""
Sep 16 19:10:46.225434 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:46.225412    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9ec73f37-d895-442d-a2d4-f3dfe3eee6c1-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "9ec73f37-d895-442d-a2d4-f3dfe3eee6c1" (UID: "9ec73f37-d895-442d-a2d4-f3dfe3eee6c1"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 19:10:46.324270 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:46.324233    2584 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9ec73f37-d895-442d-a2d4-f3dfe3eee6c1-proxy-tls\") on node \"ip-10-0-139-182.ec2.internal\" DevicePath \"\""
Sep 16 19:10:46.504686 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:46.504593    2584 generic.go:356] "Generic (PLEG): container finished" podID="9ec73f37-d895-442d-a2d4-f3dfe3eee6c1" containerID="4a6fd92f63c2fe778eb67313d0a01697ffdb53a6cd967b6ad0104385107efd8a" exitCode=0
Sep 16 19:10:46.504686 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:46.504644    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-93efe-7846bd77df-h9n8c"
Sep 16 19:10:46.504686 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:46.504660    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-93efe-7846bd77df-h9n8c" event={"ID":"9ec73f37-d895-442d-a2d4-f3dfe3eee6c1","Type":"ContainerDied","Data":"4a6fd92f63c2fe778eb67313d0a01697ffdb53a6cd967b6ad0104385107efd8a"}
Sep 16 19:10:46.505017 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:46.504702    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-93efe-7846bd77df-h9n8c" event={"ID":"9ec73f37-d895-442d-a2d4-f3dfe3eee6c1","Type":"ContainerDied","Data":"0423e42615977ee16a4835086e62833281a7266bd81eb2c442516cd5fb6def09"}
Sep 16 19:10:46.505017 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:46.504724    2584 scope.go:117] "RemoveContainer" containerID="4a6fd92f63c2fe778eb67313d0a01697ffdb53a6cd967b6ad0104385107efd8a"
Sep 16 19:10:46.514294 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:46.514266    2584 scope.go:117] "RemoveContainer" containerID="4a6fd92f63c2fe778eb67313d0a01697ffdb53a6cd967b6ad0104385107efd8a"
Sep 16 19:10:46.514623 ip-10-0-139-182 kubenswrapper[2584]: E0916 19:10:46.514599    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4a6fd92f63c2fe778eb67313d0a01697ffdb53a6cd967b6ad0104385107efd8a\": container with ID starting with 4a6fd92f63c2fe778eb67313d0a01697ffdb53a6cd967b6ad0104385107efd8a not found: ID does not exist" containerID="4a6fd92f63c2fe778eb67313d0a01697ffdb53a6cd967b6ad0104385107efd8a"
Sep 16 19:10:46.514695 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:46.514637    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4a6fd92f63c2fe778eb67313d0a01697ffdb53a6cd967b6ad0104385107efd8a"} err="failed to get container status \"4a6fd92f63c2fe778eb67313d0a01697ffdb53a6cd967b6ad0104385107efd8a\": rpc error: code = NotFound desc = could not find container \"4a6fd92f63c2fe778eb67313d0a01697ffdb53a6cd967b6ad0104385107efd8a\": container with ID starting with 4a6fd92f63c2fe778eb67313d0a01697ffdb53a6cd967b6ad0104385107efd8a not found: ID does not exist"
Sep 16 19:10:46.527505 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:46.527469    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/switch-graph-93efe-7846bd77df-h9n8c"]
Sep 16 19:10:46.533013 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:46.532982    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/switch-graph-93efe-7846bd77df-h9n8c"]
Sep 16 19:10:48.116614 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:48.116576    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9ec73f37-d895-442d-a2d4-f3dfe3eee6c1" path="/var/lib/kubelet/pods/9ec73f37-d895-442d-a2d4-f3dfe3eee6c1/volumes"
Sep 16 19:10:54.373492 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:54.373448    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-016d3-predictor-698bb76d65-bphfw" podUID="8b84fe5f-226c-478a-bd7a-c5d815fba70f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.39:8080: connect: connection refused"
Sep 16 19:10:55.739537 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:55.739491    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-cb9a3-5ff6b444fd-5fq6s"]
Sep 16 19:10:55.740055 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:55.739749    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/sequence-graph-cb9a3-5ff6b444fd-5fq6s" podUID="abd27f76-1d33-41f7-ad59-bc0b0836de34" containerName="sequence-graph-cb9a3" containerID="cri-o://b57e307975e20764563897d2bce7db0e54100389e2f95e683df55bc1d5f7130b" gracePeriod=30
Sep 16 19:10:55.768618 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:55.768575    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 19:10:55.859869 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:55.859804    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-cb9a3-predictor-55b48b4985-z5s2w"]
Sep 16 19:10:55.860110 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:55.860080    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-cb9a3-predictor-55b48b4985-z5s2w" podUID="86ef3559-67b7-4535-b7c6-2673417ceb8e" containerName="kserve-container" containerID="cri-o://e3216e71f3a1a54694b4348fb63f5af136c9065c1464b1b1918d2ba6771100b0" gracePeriod=30
Sep 16 19:10:55.860190 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:55.860111    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-cb9a3-predictor-55b48b4985-z5s2w" podUID="86ef3559-67b7-4535-b7c6-2673417ceb8e" containerName="kube-rbac-proxy" containerID="cri-o://0e73cb0b6cec9a2fd1b9367617783f1321288e79bf6d29421e68f47eeef2afcb" gracePeriod=30
Sep 16 19:10:55.908911 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:55.908860    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 19:10:55.998979 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:55.998884    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 19:10:56.003171 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:56.003142    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-7ea69-predictor-6fbd78b44f-btvsd"]
Sep 16 19:10:56.003523 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:56.003505    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="771df157-671d-4a80-aeea-88aca624578a" containerName="kserve-container"
Sep 16 19:10:56.003523 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:56.003526    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="771df157-671d-4a80-aeea-88aca624578a" containerName="kserve-container"
Sep 16 19:10:56.003648 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:56.003541    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ec73f37-d895-442d-a2d4-f3dfe3eee6c1" containerName="switch-graph-93efe"
Sep 16 19:10:56.003648 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:56.003551    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ec73f37-d895-442d-a2d4-f3dfe3eee6c1" containerName="switch-graph-93efe"
Sep 16 19:10:56.003648 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:56.003570    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0dd76b92-206d-42cf-a954-7ed474be9057" containerName="console"
Sep 16 19:10:56.003648 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:56.003580    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="0dd76b92-206d-42cf-a954-7ed474be9057" containerName="console"
Sep 16 19:10:56.003648 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:56.003606    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="771df157-671d-4a80-aeea-88aca624578a" containerName="kube-rbac-proxy"
Sep 16 19:10:56.003648 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:56.003614    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="771df157-671d-4a80-aeea-88aca624578a" containerName="kube-rbac-proxy"
Sep 16 19:10:56.003961 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:56.003696    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="0dd76b92-206d-42cf-a954-7ed474be9057" containerName="console"
Sep 16 19:10:56.003961 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:56.003709    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="771df157-671d-4a80-aeea-88aca624578a" containerName="kserve-container"
Sep 16 19:10:56.003961 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:56.003718    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="771df157-671d-4a80-aeea-88aca624578a" containerName="kube-rbac-proxy"
Sep 16 19:10:56.003961 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:56.003724    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ec73f37-d895-442d-a2d4-f3dfe3eee6c1" containerName="switch-graph-93efe"
Sep 16 19:10:56.006767 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:56.006744    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-7ea69-predictor-6fbd78b44f-btvsd"
Sep 16 19:10:56.009485 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:56.009463    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-7ea69-kube-rbac-proxy-sar-config\""
Sep 16 19:10:56.009595 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:56.009494    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-7ea69-predictor-serving-cert\""
Sep 16 19:10:56.017356 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:56.017322    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-7ea69-predictor-6fbd78b44f-btvsd"]
Sep 16 19:10:56.069173 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:56.069123    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 19:10:56.107627 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:56.107589    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3049e3b2-f9e2-475b-8602-6ba3e585511b-proxy-tls\") pod \"success-200-isvc-7ea69-predictor-6fbd78b44f-btvsd\" (UID: \"3049e3b2-f9e2-475b-8602-6ba3e585511b\") " pod="kserve-ci-e2e-test/success-200-isvc-7ea69-predictor-6fbd78b44f-btvsd"
Sep 16 19:10:56.107817 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:56.107666    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zswd8\" (UniqueName: \"kubernetes.io/projected/3049e3b2-f9e2-475b-8602-6ba3e585511b-kube-api-access-zswd8\") pod \"success-200-isvc-7ea69-predictor-6fbd78b44f-btvsd\" (UID: \"3049e3b2-f9e2-475b-8602-6ba3e585511b\") " pod="kserve-ci-e2e-test/success-200-isvc-7ea69-predictor-6fbd78b44f-btvsd"
Sep 16 19:10:56.107817 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:56.107696    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/3049e3b2-f9e2-475b-8602-6ba3e585511b-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-7ea69-predictor-6fbd78b44f-btvsd\" (UID: \"3049e3b2-f9e2-475b-8602-6ba3e585511b\") " pod="kserve-ci-e2e-test/success-200-isvc-7ea69-predictor-6fbd78b44f-btvsd"
Sep 16 19:10:56.135442 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:56.135398    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 19:10:56.208547 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:56.208509    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3049e3b2-f9e2-475b-8602-6ba3e585511b-proxy-tls\") pod \"success-200-isvc-7ea69-predictor-6fbd78b44f-btvsd\" (UID: \"3049e3b2-f9e2-475b-8602-6ba3e585511b\") " pod="kserve-ci-e2e-test/success-200-isvc-7ea69-predictor-6fbd78b44f-btvsd"
Sep 16 19:10:56.208704 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:56.208581    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zswd8\" (UniqueName: \"kubernetes.io/projected/3049e3b2-f9e2-475b-8602-6ba3e585511b-kube-api-access-zswd8\") pod \"success-200-isvc-7ea69-predictor-6fbd78b44f-btvsd\" (UID: \"3049e3b2-f9e2-475b-8602-6ba3e585511b\") " pod="kserve-ci-e2e-test/success-200-isvc-7ea69-predictor-6fbd78b44f-btvsd"
Sep 16 19:10:56.208704 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:56.208609    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/3049e3b2-f9e2-475b-8602-6ba3e585511b-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-7ea69-predictor-6fbd78b44f-btvsd\" (UID: \"3049e3b2-f9e2-475b-8602-6ba3e585511b\") " pod="kserve-ci-e2e-test/success-200-isvc-7ea69-predictor-6fbd78b44f-btvsd"
Sep 16 19:10:56.209322 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:56.209295    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/3049e3b2-f9e2-475b-8602-6ba3e585511b-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-7ea69-predictor-6fbd78b44f-btvsd\" (UID: \"3049e3b2-f9e2-475b-8602-6ba3e585511b\") " pod="kserve-ci-e2e-test/success-200-isvc-7ea69-predictor-6fbd78b44f-btvsd"
Sep 16 19:10:56.211140 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:56.211113    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3049e3b2-f9e2-475b-8602-6ba3e585511b-proxy-tls\") pod \"success-200-isvc-7ea69-predictor-6fbd78b44f-btvsd\" (UID: \"3049e3b2-f9e2-475b-8602-6ba3e585511b\") " pod="kserve-ci-e2e-test/success-200-isvc-7ea69-predictor-6fbd78b44f-btvsd"
Sep 16 19:10:56.216900 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:56.216870    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zswd8\" (UniqueName: \"kubernetes.io/projected/3049e3b2-f9e2-475b-8602-6ba3e585511b-kube-api-access-zswd8\") pod \"success-200-isvc-7ea69-predictor-6fbd78b44f-btvsd\" (UID: \"3049e3b2-f9e2-475b-8602-6ba3e585511b\") " pod="kserve-ci-e2e-test/success-200-isvc-7ea69-predictor-6fbd78b44f-btvsd"
Sep 16 19:10:56.319019 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:56.318967    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-7ea69-predictor-6fbd78b44f-btvsd"
Sep 16 19:10:56.393521 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:56.393457    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 19:10:56.556114 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:56.556078    2584 generic.go:356] "Generic (PLEG): container finished" podID="86ef3559-67b7-4535-b7c6-2673417ceb8e" containerID="0e73cb0b6cec9a2fd1b9367617783f1321288e79bf6d29421e68f47eeef2afcb" exitCode=2
Sep 16 19:10:56.556291 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:56.556130    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-cb9a3-predictor-55b48b4985-z5s2w" event={"ID":"86ef3559-67b7-4535-b7c6-2673417ceb8e","Type":"ContainerDied","Data":"0e73cb0b6cec9a2fd1b9367617783f1321288e79bf6d29421e68f47eeef2afcb"}
Sep 16 19:10:56.594847 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:56.594806    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-7ea69-predictor-6fbd78b44f-btvsd"]
Sep 16 19:10:56.597102 ip-10-0-139-182 kubenswrapper[2584]: W0916 19:10:56.597069    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3049e3b2_f9e2_475b_8602_6ba3e585511b.slice/crio-608230e140c46d3c99027a4ac6be225405fd852ecc3054a096f1e27ed91b76c2 WatchSource:0}: Error finding container 608230e140c46d3c99027a4ac6be225405fd852ecc3054a096f1e27ed91b76c2: Status 404 returned error can't find the container with id 608230e140c46d3c99027a4ac6be225405fd852ecc3054a096f1e27ed91b76c2
Sep 16 19:10:57.562466 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:57.562427    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-7ea69-predictor-6fbd78b44f-btvsd" event={"ID":"3049e3b2-f9e2-475b-8602-6ba3e585511b","Type":"ContainerStarted","Data":"a0c57ec2e17878d6f95afab0d815c0111897088b71df3b06723330ad62d7fa98"}
Sep 16 19:10:57.562887 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:57.562488    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-7ea69-predictor-6fbd78b44f-btvsd" event={"ID":"3049e3b2-f9e2-475b-8602-6ba3e585511b","Type":"ContainerStarted","Data":"31afc00aad77e3dbee3f68c435d01b81b555cf81cc03ec65967c83d8482a1c4a"}
Sep 16 19:10:57.562887 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:57.562520    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-7ea69-predictor-6fbd78b44f-btvsd"
Sep 16 19:10:57.562887 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:57.562541    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-7ea69-predictor-6fbd78b44f-btvsd" event={"ID":"3049e3b2-f9e2-475b-8602-6ba3e585511b","Type":"ContainerStarted","Data":"608230e140c46d3c99027a4ac6be225405fd852ecc3054a096f1e27ed91b76c2"}
Sep 16 19:10:57.580558 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:57.580497    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-7ea69-predictor-6fbd78b44f-btvsd" podStartSLOduration=2.580476694 podStartE2EDuration="2.580476694s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 19:10:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 19:10:57.579208711 +0000 UTC m=+1587.908711126" watchObservedRunningTime="2026-09-16 19:10:57.580476694 +0000 UTC m=+1587.909979107"
Sep 16 19:10:58.288549 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:58.288507    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-cb9a3-5ff6b444fd-5fq6s" podUID="abd27f76-1d33-41f7-ad59-bc0b0836de34" containerName="sequence-graph-cb9a3" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 16 19:10:58.567698 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:58.567605    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-7ea69-predictor-6fbd78b44f-btvsd"
Sep 16 19:10:58.569309 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:58.569277    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-7ea69-predictor-6fbd78b44f-btvsd" podUID="3049e3b2-f9e2-475b-8602-6ba3e585511b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.40:8080: connect: connection refused"
Sep 16 19:10:59.010788 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:59.010747    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-cb9a3-predictor-55b48b4985-z5s2w" podUID="86ef3559-67b7-4535-b7c6-2673417ceb8e" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.35:8643/healthz\": dial tcp 10.133.0.35:8643: connect: connection refused"
Sep 16 19:10:59.016201 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:59.016168    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-cb9a3-predictor-55b48b4985-z5s2w" podUID="86ef3559-67b7-4535-b7c6-2673417ceb8e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.35:8080: connect: connection refused"
Sep 16 19:10:59.508942 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:59.508912    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-cb9a3-predictor-55b48b4985-z5s2w"
Sep 16 19:10:59.579908 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:59.579869    2584 generic.go:356] "Generic (PLEG): container finished" podID="86ef3559-67b7-4535-b7c6-2673417ceb8e" containerID="e3216e71f3a1a54694b4348fb63f5af136c9065c1464b1b1918d2ba6771100b0" exitCode=0
Sep 16 19:10:59.580347 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:59.579914    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-cb9a3-predictor-55b48b4985-z5s2w" event={"ID":"86ef3559-67b7-4535-b7c6-2673417ceb8e","Type":"ContainerDied","Data":"e3216e71f3a1a54694b4348fb63f5af136c9065c1464b1b1918d2ba6771100b0"}
Sep 16 19:10:59.580347 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:59.579941    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-cb9a3-predictor-55b48b4985-z5s2w"
Sep 16 19:10:59.580347 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:59.580011    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-cb9a3-predictor-55b48b4985-z5s2w" event={"ID":"86ef3559-67b7-4535-b7c6-2673417ceb8e","Type":"ContainerDied","Data":"8688f97f3ca3321f06e8b5de6c7ae3d52ce1c0d6ee0fb5165ff34f8a6e65be3a"}
Sep 16 19:10:59.580347 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:59.580043    2584 scope.go:117] "RemoveContainer" containerID="0e73cb0b6cec9a2fd1b9367617783f1321288e79bf6d29421e68f47eeef2afcb"
Sep 16 19:10:59.580576 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:59.580547    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-7ea69-predictor-6fbd78b44f-btvsd" podUID="3049e3b2-f9e2-475b-8602-6ba3e585511b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.40:8080: connect: connection refused"
Sep 16 19:10:59.590775 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:59.590750    2584 scope.go:117] "RemoveContainer" containerID="e3216e71f3a1a54694b4348fb63f5af136c9065c1464b1b1918d2ba6771100b0"
Sep 16 19:10:59.599301 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:59.599277    2584 scope.go:117] "RemoveContainer" containerID="0e73cb0b6cec9a2fd1b9367617783f1321288e79bf6d29421e68f47eeef2afcb"
Sep 16 19:10:59.599638 ip-10-0-139-182 kubenswrapper[2584]: E0916 19:10:59.599620    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0e73cb0b6cec9a2fd1b9367617783f1321288e79bf6d29421e68f47eeef2afcb\": container with ID starting with 0e73cb0b6cec9a2fd1b9367617783f1321288e79bf6d29421e68f47eeef2afcb not found: ID does not exist" containerID="0e73cb0b6cec9a2fd1b9367617783f1321288e79bf6d29421e68f47eeef2afcb"
Sep 16 19:10:59.599687 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:59.599648    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0e73cb0b6cec9a2fd1b9367617783f1321288e79bf6d29421e68f47eeef2afcb"} err="failed to get container status \"0e73cb0b6cec9a2fd1b9367617783f1321288e79bf6d29421e68f47eeef2afcb\": rpc error: code = NotFound desc = could not find container \"0e73cb0b6cec9a2fd1b9367617783f1321288e79bf6d29421e68f47eeef2afcb\": container with ID starting with 0e73cb0b6cec9a2fd1b9367617783f1321288e79bf6d29421e68f47eeef2afcb not found: ID does not exist"
Sep 16 19:10:59.599687 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:59.599665    2584 scope.go:117] "RemoveContainer" containerID="e3216e71f3a1a54694b4348fb63f5af136c9065c1464b1b1918d2ba6771100b0"
Sep 16 19:10:59.599942 ip-10-0-139-182 kubenswrapper[2584]: E0916 19:10:59.599923    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e3216e71f3a1a54694b4348fb63f5af136c9065c1464b1b1918d2ba6771100b0\": container with ID starting with e3216e71f3a1a54694b4348fb63f5af136c9065c1464b1b1918d2ba6771100b0 not found: ID does not exist" containerID="e3216e71f3a1a54694b4348fb63f5af136c9065c1464b1b1918d2ba6771100b0"
Sep 16 19:10:59.600001 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:59.599947    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e3216e71f3a1a54694b4348fb63f5af136c9065c1464b1b1918d2ba6771100b0"} err="failed to get container status \"e3216e71f3a1a54694b4348fb63f5af136c9065c1464b1b1918d2ba6771100b0\": rpc error: code = NotFound desc = could not find container \"e3216e71f3a1a54694b4348fb63f5af136c9065c1464b1b1918d2ba6771100b0\": container with ID starting with e3216e71f3a1a54694b4348fb63f5af136c9065c1464b1b1918d2ba6771100b0 not found: ID does not exist"
Sep 16 19:10:59.638113 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:59.638076    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-brczs\" (UniqueName: \"kubernetes.io/projected/86ef3559-67b7-4535-b7c6-2673417ceb8e-kube-api-access-brczs\") pod \"86ef3559-67b7-4535-b7c6-2673417ceb8e\" (UID: \"86ef3559-67b7-4535-b7c6-2673417ceb8e\") "
Sep 16 19:10:59.638304 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:59.638126    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/86ef3559-67b7-4535-b7c6-2673417ceb8e-proxy-tls\") pod \"86ef3559-67b7-4535-b7c6-2673417ceb8e\" (UID: \"86ef3559-67b7-4535-b7c6-2673417ceb8e\") "
Sep 16 19:10:59.638304 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:59.638164    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/86ef3559-67b7-4535-b7c6-2673417ceb8e-kube-rbac-proxy-sar-config\") pod \"86ef3559-67b7-4535-b7c6-2673417ceb8e\" (UID: \"86ef3559-67b7-4535-b7c6-2673417ceb8e\") "
Sep 16 19:10:59.638603 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:59.638579    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/86ef3559-67b7-4535-b7c6-2673417ceb8e-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "86ef3559-67b7-4535-b7c6-2673417ceb8e" (UID: "86ef3559-67b7-4535-b7c6-2673417ceb8e"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 19:10:59.640354 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:59.640320    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/86ef3559-67b7-4535-b7c6-2673417ceb8e-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "86ef3559-67b7-4535-b7c6-2673417ceb8e" (UID: "86ef3559-67b7-4535-b7c6-2673417ceb8e"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 19:10:59.640501 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:59.640372    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/86ef3559-67b7-4535-b7c6-2673417ceb8e-kube-api-access-brczs" (OuterVolumeSpecName: "kube-api-access-brczs") pod "86ef3559-67b7-4535-b7c6-2673417ceb8e" (UID: "86ef3559-67b7-4535-b7c6-2673417ceb8e"). InnerVolumeSpecName "kube-api-access-brczs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 19:10:59.739634 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:59.739578    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-brczs\" (UniqueName: \"kubernetes.io/projected/86ef3559-67b7-4535-b7c6-2673417ceb8e-kube-api-access-brczs\") on node \"ip-10-0-139-182.ec2.internal\" DevicePath \"\""
Sep 16 19:10:59.739634 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:59.739630    2584 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/86ef3559-67b7-4535-b7c6-2673417ceb8e-proxy-tls\") on node \"ip-10-0-139-182.ec2.internal\" DevicePath \"\""
Sep 16 19:10:59.739634 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:59.739642    2584 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/86ef3559-67b7-4535-b7c6-2673417ceb8e-kube-rbac-proxy-sar-config\") on node \"ip-10-0-139-182.ec2.internal\" DevicePath \"\""
Sep 16 19:10:59.904210 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:59.904171    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-cb9a3-predictor-55b48b4985-z5s2w"]
Sep 16 19:10:59.908067 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:10:59.908036    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-cb9a3-predictor-55b48b4985-z5s2w"]
Sep 16 19:11:00.124822 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:00.124769    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="86ef3559-67b7-4535-b7c6-2673417ceb8e" path="/var/lib/kubelet/pods/86ef3559-67b7-4535-b7c6-2673417ceb8e/volumes"
Sep 16 19:11:03.287848 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:03.287792    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-cb9a3-5ff6b444fd-5fq6s" podUID="abd27f76-1d33-41f7-ad59-bc0b0836de34" containerName="sequence-graph-cb9a3" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 16 19:11:04.374080 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:04.374042    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-016d3-predictor-698bb76d65-bphfw"
Sep 16 19:11:04.585411 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:04.585376    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-7ea69-predictor-6fbd78b44f-btvsd"
Sep 16 19:11:04.585889 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:04.585860    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-7ea69-predictor-6fbd78b44f-btvsd" podUID="3049e3b2-f9e2-475b-8602-6ba3e585511b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.40:8080: connect: connection refused"
Sep 16 19:11:08.288122 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:08.288084    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-cb9a3-5ff6b444fd-5fq6s" podUID="abd27f76-1d33-41f7-ad59-bc0b0836de34" containerName="sequence-graph-cb9a3" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 16 19:11:08.288573 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:08.288191    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/sequence-graph-cb9a3-5ff6b444fd-5fq6s"
Sep 16 19:11:13.288043 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:13.288002    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-cb9a3-5ff6b444fd-5fq6s" podUID="abd27f76-1d33-41f7-ad59-bc0b0836de34" containerName="sequence-graph-cb9a3" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 16 19:11:13.337360 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:13.337317    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 19:11:14.586599 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:14.586558    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-7ea69-predictor-6fbd78b44f-btvsd" podUID="3049e3b2-f9e2-475b-8602-6ba3e585511b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.40:8080: connect: connection refused"
Sep 16 19:11:16.171643 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:16.171613    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-016d3-58bf87cfb-7x29f"]
Sep 16 19:11:16.172009 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:16.171958    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="86ef3559-67b7-4535-b7c6-2673417ceb8e" containerName="kserve-container"
Sep 16 19:11:16.172009 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:16.171970    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="86ef3559-67b7-4535-b7c6-2673417ceb8e" containerName="kserve-container"
Sep 16 19:11:16.172009 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:16.171977    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="86ef3559-67b7-4535-b7c6-2673417ceb8e" containerName="kube-rbac-proxy"
Sep 16 19:11:16.172009 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:16.171983    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="86ef3559-67b7-4535-b7c6-2673417ceb8e" containerName="kube-rbac-proxy"
Sep 16 19:11:16.172143 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:16.172046    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="86ef3559-67b7-4535-b7c6-2673417ceb8e" containerName="kube-rbac-proxy"
Sep 16 19:11:16.172143 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:16.172055    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="86ef3559-67b7-4535-b7c6-2673417ceb8e" containerName="kserve-container"
Sep 16 19:11:16.174997 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:16.174977    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-016d3-58bf87cfb-7x29f"
Sep 16 19:11:16.177503 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:16.177485    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"ensemble-graph-016d3-serving-cert\""
Sep 16 19:11:16.177567 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:16.177535    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"ensemble-graph-016d3-kube-rbac-proxy-sar-config\""
Sep 16 19:11:16.184732 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:16.184702    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-016d3-58bf87cfb-7x29f"]
Sep 16 19:11:16.185610 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:16.185590    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/86cf374c-6afd-472c-b160-cad83527d6ae-proxy-tls\") pod \"ensemble-graph-016d3-58bf87cfb-7x29f\" (UID: \"86cf374c-6afd-472c-b160-cad83527d6ae\") " pod="kserve-ci-e2e-test/ensemble-graph-016d3-58bf87cfb-7x29f"
Sep 16 19:11:16.185713 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:16.185627    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/86cf374c-6afd-472c-b160-cad83527d6ae-openshift-service-ca-bundle\") pod \"ensemble-graph-016d3-58bf87cfb-7x29f\" (UID: \"86cf374c-6afd-472c-b160-cad83527d6ae\") " pod="kserve-ci-e2e-test/ensemble-graph-016d3-58bf87cfb-7x29f"
Sep 16 19:11:16.286284 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:16.286246    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/86cf374c-6afd-472c-b160-cad83527d6ae-proxy-tls\") pod \"ensemble-graph-016d3-58bf87cfb-7x29f\" (UID: \"86cf374c-6afd-472c-b160-cad83527d6ae\") " pod="kserve-ci-e2e-test/ensemble-graph-016d3-58bf87cfb-7x29f"
Sep 16 19:11:16.286469 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:16.286308    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/86cf374c-6afd-472c-b160-cad83527d6ae-openshift-service-ca-bundle\") pod \"ensemble-graph-016d3-58bf87cfb-7x29f\" (UID: \"86cf374c-6afd-472c-b160-cad83527d6ae\") " pod="kserve-ci-e2e-test/ensemble-graph-016d3-58bf87cfb-7x29f"
Sep 16 19:11:16.286469 ip-10-0-139-182 kubenswrapper[2584]: E0916 19:11:16.286385    2584 secret.go:189] Couldn't get secret kserve-ci-e2e-test/ensemble-graph-016d3-serving-cert: secret "ensemble-graph-016d3-serving-cert" not found
Sep 16 19:11:16.286469 ip-10-0-139-182 kubenswrapper[2584]: E0916 19:11:16.286454    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/86cf374c-6afd-472c-b160-cad83527d6ae-proxy-tls podName:86cf374c-6afd-472c-b160-cad83527d6ae nodeName:}" failed. No retries permitted until 2026-09-16 19:11:16.786437428 +0000 UTC m=+1607.115939826 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/86cf374c-6afd-472c-b160-cad83527d6ae-proxy-tls") pod "ensemble-graph-016d3-58bf87cfb-7x29f" (UID: "86cf374c-6afd-472c-b160-cad83527d6ae") : secret "ensemble-graph-016d3-serving-cert" not found
Sep 16 19:11:16.286924 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:16.286904    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/86cf374c-6afd-472c-b160-cad83527d6ae-openshift-service-ca-bundle\") pod \"ensemble-graph-016d3-58bf87cfb-7x29f\" (UID: \"86cf374c-6afd-472c-b160-cad83527d6ae\") " pod="kserve-ci-e2e-test/ensemble-graph-016d3-58bf87cfb-7x29f"
Sep 16 19:11:16.791173 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:16.791135    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/86cf374c-6afd-472c-b160-cad83527d6ae-proxy-tls\") pod \"ensemble-graph-016d3-58bf87cfb-7x29f\" (UID: \"86cf374c-6afd-472c-b160-cad83527d6ae\") " pod="kserve-ci-e2e-test/ensemble-graph-016d3-58bf87cfb-7x29f"
Sep 16 19:11:16.793539 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:16.793515    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/86cf374c-6afd-472c-b160-cad83527d6ae-proxy-tls\") pod \"ensemble-graph-016d3-58bf87cfb-7x29f\" (UID: \"86cf374c-6afd-472c-b160-cad83527d6ae\") " pod="kserve-ci-e2e-test/ensemble-graph-016d3-58bf87cfb-7x29f"
Sep 16 19:11:16.876448 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:16.876406    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 19:11:17.086379 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:17.086288    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-016d3-58bf87cfb-7x29f"
Sep 16 19:11:17.356145 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:17.356084    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-016d3-58bf87cfb-7x29f"]
Sep 16 19:11:17.358703 ip-10-0-139-182 kubenswrapper[2584]: W0916 19:11:17.358667    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86cf374c_6afd_472c_b160_cad83527d6ae.slice/crio-4866f42727824a1be2ffdc4d15279f67312b6ea763360d3d2cf2b978b104a26d WatchSource:0}: Error finding container 4866f42727824a1be2ffdc4d15279f67312b6ea763360d3d2cf2b978b104a26d: Status 404 returned error can't find the container with id 4866f42727824a1be2ffdc4d15279f67312b6ea763360d3d2cf2b978b104a26d
Sep 16 19:11:17.635440 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:17.635351    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 19:11:17.673628 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:17.673590    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-016d3-58bf87cfb-7x29f" event={"ID":"86cf374c-6afd-472c-b160-cad83527d6ae","Type":"ContainerStarted","Data":"d747621583b2c3fa148e3d0230a6030226fe5749ccd074d765f359c7192214a6"}
Sep 16 19:11:17.673807 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:17.673651    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-016d3-58bf87cfb-7x29f" event={"ID":"86cf374c-6afd-472c-b160-cad83527d6ae","Type":"ContainerStarted","Data":"4866f42727824a1be2ffdc4d15279f67312b6ea763360d3d2cf2b978b104a26d"}
Sep 16 19:11:17.673807 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:17.673690    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/ensemble-graph-016d3-58bf87cfb-7x29f"
Sep 16 19:11:17.693578 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:17.693534    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/ensemble-graph-016d3-58bf87cfb-7x29f" podStartSLOduration=1.693520029 podStartE2EDuration="1.693520029s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 19:11:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 19:11:17.692765134 +0000 UTC m=+1608.022267547" watchObservedRunningTime="2026-09-16 19:11:17.693520029 +0000 UTC m=+1608.023022443"
Sep 16 19:11:18.287801 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:18.287767    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-cb9a3-5ff6b444fd-5fq6s" podUID="abd27f76-1d33-41f7-ad59-bc0b0836de34" containerName="sequence-graph-cb9a3" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 16 19:11:23.288342 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:23.288262    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-cb9a3-5ff6b444fd-5fq6s" podUID="abd27f76-1d33-41f7-ad59-bc0b0836de34" containerName="sequence-graph-cb9a3" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 16 19:11:23.683536 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:23.683454    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/ensemble-graph-016d3-58bf87cfb-7x29f"
Sep 16 19:11:24.586671 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:24.586630    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-7ea69-predictor-6fbd78b44f-btvsd" podUID="3049e3b2-f9e2-475b-8602-6ba3e585511b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.40:8080: connect: connection refused"
Sep 16 19:11:25.888746 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:25.888720    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-cb9a3-5ff6b444fd-5fq6s"
Sep 16 19:11:25.960614 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:25.960582    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/abd27f76-1d33-41f7-ad59-bc0b0836de34-openshift-service-ca-bundle\") pod \"abd27f76-1d33-41f7-ad59-bc0b0836de34\" (UID: \"abd27f76-1d33-41f7-ad59-bc0b0836de34\") "
Sep 16 19:11:25.960614 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:25.960613    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/abd27f76-1d33-41f7-ad59-bc0b0836de34-proxy-tls\") pod \"abd27f76-1d33-41f7-ad59-bc0b0836de34\" (UID: \"abd27f76-1d33-41f7-ad59-bc0b0836de34\") "
Sep 16 19:11:25.960994 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:25.960967    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/abd27f76-1d33-41f7-ad59-bc0b0836de34-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "abd27f76-1d33-41f7-ad59-bc0b0836de34" (UID: "abd27f76-1d33-41f7-ad59-bc0b0836de34"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 19:11:25.962621 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:25.962604    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/abd27f76-1d33-41f7-ad59-bc0b0836de34-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "abd27f76-1d33-41f7-ad59-bc0b0836de34" (UID: "abd27f76-1d33-41f7-ad59-bc0b0836de34"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 19:11:26.061100 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:26.061069    2584 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/abd27f76-1d33-41f7-ad59-bc0b0836de34-openshift-service-ca-bundle\") on node \"ip-10-0-139-182.ec2.internal\" DevicePath \"\""
Sep 16 19:11:26.061100 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:26.061096    2584 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/abd27f76-1d33-41f7-ad59-bc0b0836de34-proxy-tls\") on node \"ip-10-0-139-182.ec2.internal\" DevicePath \"\""
Sep 16 19:11:26.247377 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:26.247336    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-016d3-58bf87cfb-7x29f"]
Sep 16 19:11:26.247558 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:26.247509    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/ensemble-graph-016d3-58bf87cfb-7x29f" podUID="86cf374c-6afd-472c-b160-cad83527d6ae" containerName="ensemble-graph-016d3" containerID="cri-o://d747621583b2c3fa148e3d0230a6030226fe5749ccd074d765f359c7192214a6" gracePeriod=30
Sep 16 19:11:26.287360 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:26.287323    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 19:11:26.386778 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:26.386697    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-016d3-predictor-698bb76d65-bphfw"]
Sep 16 19:11:26.386961 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:26.386934    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-016d3-predictor-698bb76d65-bphfw" podUID="8b84fe5f-226c-478a-bd7a-c5d815fba70f" containerName="kserve-container" containerID="cri-o://cd8056f266b6a4308e52cdf2f6a5f3cfad7d128ca77b0556efb4d2f4a9f58e12" gracePeriod=30
Sep 16 19:11:26.387030 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:26.386975    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-016d3-predictor-698bb76d65-bphfw" podUID="8b84fe5f-226c-478a-bd7a-c5d815fba70f" containerName="kube-rbac-proxy" containerID="cri-o://a840db21c3366603cf43b8c201c202c0ea001930a8e4b4a94d7801d5c7382159" gracePeriod=30
Sep 16 19:11:26.420978 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:26.420939    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 19:11:26.517541 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:26.517507    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 19:11:26.548768 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:26.548731    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-6b79a-predictor-544dbf6f99-vrqwv"]
Sep 16 19:11:26.549119 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:26.549106    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="abd27f76-1d33-41f7-ad59-bc0b0836de34" containerName="sequence-graph-cb9a3"
Sep 16 19:11:26.549165 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:26.549122    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="abd27f76-1d33-41f7-ad59-bc0b0836de34" containerName="sequence-graph-cb9a3"
Sep 16 19:11:26.549205 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:26.549193    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="abd27f76-1d33-41f7-ad59-bc0b0836de34" containerName="sequence-graph-cb9a3"
Sep 16 19:11:26.552250 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:26.552231    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-6b79a-predictor-544dbf6f99-vrqwv"
Sep 16 19:11:26.555128 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:26.555109    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-6b79a-predictor-serving-cert\""
Sep 16 19:11:26.555254 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:26.555133    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-6b79a-kube-rbac-proxy-sar-config\""
Sep 16 19:11:26.563472 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:26.563450    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-6b79a-predictor-544dbf6f99-vrqwv"]
Sep 16 19:11:26.564899 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:26.564878    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/f16ee8f1-b4a8-4e2d-9b6c-add6833c8c4b-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-6b79a-predictor-544dbf6f99-vrqwv\" (UID: \"f16ee8f1-b4a8-4e2d-9b6c-add6833c8c4b\") " pod="kserve-ci-e2e-test/success-200-isvc-6b79a-predictor-544dbf6f99-vrqwv"
Sep 16 19:11:26.564990 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:26.564937    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f16ee8f1-b4a8-4e2d-9b6c-add6833c8c4b-proxy-tls\") pod \"success-200-isvc-6b79a-predictor-544dbf6f99-vrqwv\" (UID: \"f16ee8f1-b4a8-4e2d-9b6c-add6833c8c4b\") " pod="kserve-ci-e2e-test/success-200-isvc-6b79a-predictor-544dbf6f99-vrqwv"
Sep 16 19:11:26.564990 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:26.564968    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7cwt8\" (UniqueName: \"kubernetes.io/projected/f16ee8f1-b4a8-4e2d-9b6c-add6833c8c4b-kube-api-access-7cwt8\") pod \"success-200-isvc-6b79a-predictor-544dbf6f99-vrqwv\" (UID: \"f16ee8f1-b4a8-4e2d-9b6c-add6833c8c4b\") " pod="kserve-ci-e2e-test/success-200-isvc-6b79a-predictor-544dbf6f99-vrqwv"
Sep 16 19:11:26.618313 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:26.618282    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 19:11:26.665560 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:26.665481    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7cwt8\" (UniqueName: \"kubernetes.io/projected/f16ee8f1-b4a8-4e2d-9b6c-add6833c8c4b-kube-api-access-7cwt8\") pod \"success-200-isvc-6b79a-predictor-544dbf6f99-vrqwv\" (UID: \"f16ee8f1-b4a8-4e2d-9b6c-add6833c8c4b\") " pod="kserve-ci-e2e-test/success-200-isvc-6b79a-predictor-544dbf6f99-vrqwv"
Sep 16 19:11:26.665560 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:26.665539    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/f16ee8f1-b4a8-4e2d-9b6c-add6833c8c4b-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-6b79a-predictor-544dbf6f99-vrqwv\" (UID: \"f16ee8f1-b4a8-4e2d-9b6c-add6833c8c4b\") " pod="kserve-ci-e2e-test/success-200-isvc-6b79a-predictor-544dbf6f99-vrqwv"
Sep 16 19:11:26.665750 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:26.665584    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f16ee8f1-b4a8-4e2d-9b6c-add6833c8c4b-proxy-tls\") pod \"success-200-isvc-6b79a-predictor-544dbf6f99-vrqwv\" (UID: \"f16ee8f1-b4a8-4e2d-9b6c-add6833c8c4b\") " pod="kserve-ci-e2e-test/success-200-isvc-6b79a-predictor-544dbf6f99-vrqwv"
Sep 16 19:11:26.665750 ip-10-0-139-182 kubenswrapper[2584]: E0916 19:11:26.665669    2584 secret.go:189] Couldn't get secret kserve-ci-e2e-test/success-200-isvc-6b79a-predictor-serving-cert: secret "success-200-isvc-6b79a-predictor-serving-cert" not found
Sep 16 19:11:26.665750 ip-10-0-139-182 kubenswrapper[2584]: E0916 19:11:26.665719    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f16ee8f1-b4a8-4e2d-9b6c-add6833c8c4b-proxy-tls podName:f16ee8f1-b4a8-4e2d-9b6c-add6833c8c4b nodeName:}" failed. No retries permitted until 2026-09-16 19:11:27.165701912 +0000 UTC m=+1617.495204320 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/f16ee8f1-b4a8-4e2d-9b6c-add6833c8c4b-proxy-tls") pod "success-200-isvc-6b79a-predictor-544dbf6f99-vrqwv" (UID: "f16ee8f1-b4a8-4e2d-9b6c-add6833c8c4b") : secret "success-200-isvc-6b79a-predictor-serving-cert" not found
Sep 16 19:11:26.666279 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:26.666259    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/f16ee8f1-b4a8-4e2d-9b6c-add6833c8c4b-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-6b79a-predictor-544dbf6f99-vrqwv\" (UID: \"f16ee8f1-b4a8-4e2d-9b6c-add6833c8c4b\") " pod="kserve-ci-e2e-test/success-200-isvc-6b79a-predictor-544dbf6f99-vrqwv"
Sep 16 19:11:26.678959 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:26.678937    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7cwt8\" (UniqueName: \"kubernetes.io/projected/f16ee8f1-b4a8-4e2d-9b6c-add6833c8c4b-kube-api-access-7cwt8\") pod \"success-200-isvc-6b79a-predictor-544dbf6f99-vrqwv\" (UID: \"f16ee8f1-b4a8-4e2d-9b6c-add6833c8c4b\") " pod="kserve-ci-e2e-test/success-200-isvc-6b79a-predictor-544dbf6f99-vrqwv"
Sep 16 19:11:26.714959 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:26.714929    2584 generic.go:356] "Generic (PLEG): container finished" podID="abd27f76-1d33-41f7-ad59-bc0b0836de34" containerID="b57e307975e20764563897d2bce7db0e54100389e2f95e683df55bc1d5f7130b" exitCode=0
Sep 16 19:11:26.715110 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:26.714973    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-cb9a3-5ff6b444fd-5fq6s"
Sep 16 19:11:26.715110 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:26.715017    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-cb9a3-5ff6b444fd-5fq6s" event={"ID":"abd27f76-1d33-41f7-ad59-bc0b0836de34","Type":"ContainerDied","Data":"b57e307975e20764563897d2bce7db0e54100389e2f95e683df55bc1d5f7130b"}
Sep 16 19:11:26.715228 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:26.715102    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-cb9a3-5ff6b444fd-5fq6s" event={"ID":"abd27f76-1d33-41f7-ad59-bc0b0836de34","Type":"ContainerDied","Data":"87d67eea490a5de9e78e0bb66d0ebcf0f62356cb337502798d3dfe3e752fdb12"}
Sep 16 19:11:26.715228 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:26.715161    2584 scope.go:117] "RemoveContainer" containerID="b57e307975e20764563897d2bce7db0e54100389e2f95e683df55bc1d5f7130b"
Sep 16 19:11:26.717490 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:26.717334    2584 generic.go:356] "Generic (PLEG): container finished" podID="8b84fe5f-226c-478a-bd7a-c5d815fba70f" containerID="a840db21c3366603cf43b8c201c202c0ea001930a8e4b4a94d7801d5c7382159" exitCode=2
Sep 16 19:11:26.717490 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:26.717421    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-016d3-predictor-698bb76d65-bphfw" event={"ID":"8b84fe5f-226c-478a-bd7a-c5d815fba70f","Type":"ContainerDied","Data":"a840db21c3366603cf43b8c201c202c0ea001930a8e4b4a94d7801d5c7382159"}
Sep 16 19:11:26.717794 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:26.717767    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 19:11:26.727321 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:26.727303    2584 scope.go:117] "RemoveContainer" containerID="b57e307975e20764563897d2bce7db0e54100389e2f95e683df55bc1d5f7130b"
Sep 16 19:11:26.727619 ip-10-0-139-182 kubenswrapper[2584]: E0916 19:11:26.727600    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b57e307975e20764563897d2bce7db0e54100389e2f95e683df55bc1d5f7130b\": container with ID starting with b57e307975e20764563897d2bce7db0e54100389e2f95e683df55bc1d5f7130b not found: ID does not exist" containerID="b57e307975e20764563897d2bce7db0e54100389e2f95e683df55bc1d5f7130b"
Sep 16 19:11:26.727666 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:26.727626    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b57e307975e20764563897d2bce7db0e54100389e2f95e683df55bc1d5f7130b"} err="failed to get container status \"b57e307975e20764563897d2bce7db0e54100389e2f95e683df55bc1d5f7130b\": rpc error: code = NotFound desc = could not find container \"b57e307975e20764563897d2bce7db0e54100389e2f95e683df55bc1d5f7130b\": container with ID starting with b57e307975e20764563897d2bce7db0e54100389e2f95e683df55bc1d5f7130b not found: ID does not exist"
Sep 16 19:11:26.734546 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:26.734523    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-cb9a3-5ff6b444fd-5fq6s"]
Sep 16 19:11:26.741034 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:26.741011    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-cb9a3-5ff6b444fd-5fq6s"]
Sep 16 19:11:26.953437 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:26.953346    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 19:11:27.085378 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:27.085344    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 19:11:27.170697 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:27.170665    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f16ee8f1-b4a8-4e2d-9b6c-add6833c8c4b-proxy-tls\") pod \"success-200-isvc-6b79a-predictor-544dbf6f99-vrqwv\" (UID: \"f16ee8f1-b4a8-4e2d-9b6c-add6833c8c4b\") " pod="kserve-ci-e2e-test/success-200-isvc-6b79a-predictor-544dbf6f99-vrqwv"
Sep 16 19:11:27.173166 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:27.173141    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f16ee8f1-b4a8-4e2d-9b6c-add6833c8c4b-proxy-tls\") pod \"success-200-isvc-6b79a-predictor-544dbf6f99-vrqwv\" (UID: \"f16ee8f1-b4a8-4e2d-9b6c-add6833c8c4b\") " pod="kserve-ci-e2e-test/success-200-isvc-6b79a-predictor-544dbf6f99-vrqwv"
Sep 16 19:11:27.463943 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:27.463906    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-6b79a-predictor-544dbf6f99-vrqwv"
Sep 16 19:11:27.737489 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:27.737463    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-6b79a-predictor-544dbf6f99-vrqwv"]
Sep 16 19:11:27.740344 ip-10-0-139-182 kubenswrapper[2584]: W0916 19:11:27.740320    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf16ee8f1_b4a8_4e2d_9b6c_add6833c8c4b.slice/crio-16cabab0329a68a325854110a78d2f0bedc17aae49dcbced028ec668b23de074 WatchSource:0}: Error finding container 16cabab0329a68a325854110a78d2f0bedc17aae49dcbced028ec668b23de074: Status 404 returned error can't find the container with id 16cabab0329a68a325854110a78d2f0bedc17aae49dcbced028ec668b23de074
Sep 16 19:11:28.116138 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:28.116109    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="abd27f76-1d33-41f7-ad59-bc0b0836de34" path="/var/lib/kubelet/pods/abd27f76-1d33-41f7-ad59-bc0b0836de34/volumes"
Sep 16 19:11:28.681595 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:28.681553    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-016d3-58bf87cfb-7x29f" podUID="86cf374c-6afd-472c-b160-cad83527d6ae" containerName="ensemble-graph-016d3" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 16 19:11:28.729814 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:28.729783    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-6b79a-predictor-544dbf6f99-vrqwv" event={"ID":"f16ee8f1-b4a8-4e2d-9b6c-add6833c8c4b","Type":"ContainerStarted","Data":"0f95730474301429a0498f2772bc8a68489ae158c1ab6ebc88962565886b635b"}
Sep 16 19:11:28.729983 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:28.729856    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-6b79a-predictor-544dbf6f99-vrqwv" event={"ID":"f16ee8f1-b4a8-4e2d-9b6c-add6833c8c4b","Type":"ContainerStarted","Data":"b19aa6eb96c5063d18efe2c6beaa3e8d7fe427ba06a0ca2973456ba3e4781722"}
Sep 16 19:11:28.729983 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:28.729883    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-6b79a-predictor-544dbf6f99-vrqwv" event={"ID":"f16ee8f1-b4a8-4e2d-9b6c-add6833c8c4b","Type":"ContainerStarted","Data":"16cabab0329a68a325854110a78d2f0bedc17aae49dcbced028ec668b23de074"}
Sep 16 19:11:28.729983 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:28.729959    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-6b79a-predictor-544dbf6f99-vrqwv"
Sep 16 19:11:28.750036 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:28.749971    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-6b79a-predictor-544dbf6f99-vrqwv" podStartSLOduration=2.749946578 podStartE2EDuration="2.749946578s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 19:11:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 19:11:28.748225618 +0000 UTC m=+1619.077728033" watchObservedRunningTime="2026-09-16 19:11:28.749946578 +0000 UTC m=+1619.079448992"
Sep 16 19:11:29.368514 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:29.368479    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-016d3-predictor-698bb76d65-bphfw" podUID="8b84fe5f-226c-478a-bd7a-c5d815fba70f" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.39:8643/healthz\": dial tcp 10.133.0.39:8643: connect: connection refused"
Sep 16 19:11:29.524956 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:29.524930    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-016d3-predictor-698bb76d65-bphfw"
Sep 16 19:11:29.592504 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:29.592475    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/8b84fe5f-226c-478a-bd7a-c5d815fba70f-kube-rbac-proxy-sar-config\") pod \"8b84fe5f-226c-478a-bd7a-c5d815fba70f\" (UID: \"8b84fe5f-226c-478a-bd7a-c5d815fba70f\") "
Sep 16 19:11:29.592692 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:29.592531    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6r268\" (UniqueName: \"kubernetes.io/projected/8b84fe5f-226c-478a-bd7a-c5d815fba70f-kube-api-access-6r268\") pod \"8b84fe5f-226c-478a-bd7a-c5d815fba70f\" (UID: \"8b84fe5f-226c-478a-bd7a-c5d815fba70f\") "
Sep 16 19:11:29.592692 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:29.592560    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8b84fe5f-226c-478a-bd7a-c5d815fba70f-proxy-tls\") pod \"8b84fe5f-226c-478a-bd7a-c5d815fba70f\" (UID: \"8b84fe5f-226c-478a-bd7a-c5d815fba70f\") "
Sep 16 19:11:29.592862 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:29.592816    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8b84fe5f-226c-478a-bd7a-c5d815fba70f-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "8b84fe5f-226c-478a-bd7a-c5d815fba70f" (UID: "8b84fe5f-226c-478a-bd7a-c5d815fba70f"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 19:11:29.594684 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:29.594659    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8b84fe5f-226c-478a-bd7a-c5d815fba70f-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "8b84fe5f-226c-478a-bd7a-c5d815fba70f" (UID: "8b84fe5f-226c-478a-bd7a-c5d815fba70f"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 19:11:29.594786 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:29.594726    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b84fe5f-226c-478a-bd7a-c5d815fba70f-kube-api-access-6r268" (OuterVolumeSpecName: "kube-api-access-6r268") pod "8b84fe5f-226c-478a-bd7a-c5d815fba70f" (UID: "8b84fe5f-226c-478a-bd7a-c5d815fba70f"). InnerVolumeSpecName "kube-api-access-6r268". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 19:11:29.693081 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:29.693046    2584 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/8b84fe5f-226c-478a-bd7a-c5d815fba70f-kube-rbac-proxy-sar-config\") on node \"ip-10-0-139-182.ec2.internal\" DevicePath \"\""
Sep 16 19:11:29.693081 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:29.693074    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6r268\" (UniqueName: \"kubernetes.io/projected/8b84fe5f-226c-478a-bd7a-c5d815fba70f-kube-api-access-6r268\") on node \"ip-10-0-139-182.ec2.internal\" DevicePath \"\""
Sep 16 19:11:29.693081 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:29.693084    2584 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8b84fe5f-226c-478a-bd7a-c5d815fba70f-proxy-tls\") on node \"ip-10-0-139-182.ec2.internal\" DevicePath \"\""
Sep 16 19:11:29.736496 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:29.736455    2584 generic.go:356] "Generic (PLEG): container finished" podID="8b84fe5f-226c-478a-bd7a-c5d815fba70f" containerID="cd8056f266b6a4308e52cdf2f6a5f3cfad7d128ca77b0556efb4d2f4a9f58e12" exitCode=0
Sep 16 19:11:29.736636 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:29.736508    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-016d3-predictor-698bb76d65-bphfw"
Sep 16 19:11:29.736636 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:29.736536    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-016d3-predictor-698bb76d65-bphfw" event={"ID":"8b84fe5f-226c-478a-bd7a-c5d815fba70f","Type":"ContainerDied","Data":"cd8056f266b6a4308e52cdf2f6a5f3cfad7d128ca77b0556efb4d2f4a9f58e12"}
Sep 16 19:11:29.736636 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:29.736590    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-016d3-predictor-698bb76d65-bphfw" event={"ID":"8b84fe5f-226c-478a-bd7a-c5d815fba70f","Type":"ContainerDied","Data":"945844f991192ff380e8ccc0ca7543ac7414f1e33db7ac8256705d91f4e31bab"}
Sep 16 19:11:29.736636 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:29.736617    2584 scope.go:117] "RemoveContainer" containerID="a840db21c3366603cf43b8c201c202c0ea001930a8e4b4a94d7801d5c7382159"
Sep 16 19:11:29.737025 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:29.736991    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-6b79a-predictor-544dbf6f99-vrqwv"
Sep 16 19:11:29.738907 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:29.738871    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-6b79a-predictor-544dbf6f99-vrqwv" podUID="f16ee8f1-b4a8-4e2d-9b6c-add6833c8c4b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.42:8080: connect: connection refused"
Sep 16 19:11:29.746991 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:29.746973    2584 scope.go:117] "RemoveContainer" containerID="cd8056f266b6a4308e52cdf2f6a5f3cfad7d128ca77b0556efb4d2f4a9f58e12"
Sep 16 19:11:29.754851 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:29.754808    2584 scope.go:117] "RemoveContainer" containerID="a840db21c3366603cf43b8c201c202c0ea001930a8e4b4a94d7801d5c7382159"
Sep 16 19:11:29.755102 ip-10-0-139-182 kubenswrapper[2584]: E0916 19:11:29.755083    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a840db21c3366603cf43b8c201c202c0ea001930a8e4b4a94d7801d5c7382159\": container with ID starting with a840db21c3366603cf43b8c201c202c0ea001930a8e4b4a94d7801d5c7382159 not found: ID does not exist" containerID="a840db21c3366603cf43b8c201c202c0ea001930a8e4b4a94d7801d5c7382159"
Sep 16 19:11:29.755149 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:29.755111    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a840db21c3366603cf43b8c201c202c0ea001930a8e4b4a94d7801d5c7382159"} err="failed to get container status \"a840db21c3366603cf43b8c201c202c0ea001930a8e4b4a94d7801d5c7382159\": rpc error: code = NotFound desc = could not find container \"a840db21c3366603cf43b8c201c202c0ea001930a8e4b4a94d7801d5c7382159\": container with ID starting with a840db21c3366603cf43b8c201c202c0ea001930a8e4b4a94d7801d5c7382159 not found: ID does not exist"
Sep 16 19:11:29.755149 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:29.755127    2584 scope.go:117] "RemoveContainer" containerID="cd8056f266b6a4308e52cdf2f6a5f3cfad7d128ca77b0556efb4d2f4a9f58e12"
Sep 16 19:11:29.755385 ip-10-0-139-182 kubenswrapper[2584]: E0916 19:11:29.755370    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cd8056f266b6a4308e52cdf2f6a5f3cfad7d128ca77b0556efb4d2f4a9f58e12\": container with ID starting with cd8056f266b6a4308e52cdf2f6a5f3cfad7d128ca77b0556efb4d2f4a9f58e12 not found: ID does not exist" containerID="cd8056f266b6a4308e52cdf2f6a5f3cfad7d128ca77b0556efb4d2f4a9f58e12"
Sep 16 19:11:29.755439 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:29.755389    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cd8056f266b6a4308e52cdf2f6a5f3cfad7d128ca77b0556efb4d2f4a9f58e12"} err="failed to get container status \"cd8056f266b6a4308e52cdf2f6a5f3cfad7d128ca77b0556efb4d2f4a9f58e12\": rpc error: code = NotFound desc = could not find container \"cd8056f266b6a4308e52cdf2f6a5f3cfad7d128ca77b0556efb4d2f4a9f58e12\": container with ID starting with cd8056f266b6a4308e52cdf2f6a5f3cfad7d128ca77b0556efb4d2f4a9f58e12 not found: ID does not exist"
Sep 16 19:11:29.759333 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:29.759311    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-016d3-predictor-698bb76d65-bphfw"]
Sep 16 19:11:29.765572 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:29.765553    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-016d3-predictor-698bb76d65-bphfw"]
Sep 16 19:11:30.117112 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:30.117080    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8b84fe5f-226c-478a-bd7a-c5d815fba70f" path="/var/lib/kubelet/pods/8b84fe5f-226c-478a-bd7a-c5d815fba70f/volumes"
Sep 16 19:11:30.740664 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:30.740622    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-6b79a-predictor-544dbf6f99-vrqwv" podUID="f16ee8f1-b4a8-4e2d-9b6c-add6833c8c4b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.42:8080: connect: connection refused"
Sep 16 19:11:33.681709 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:33.681668    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-016d3-58bf87cfb-7x29f" podUID="86cf374c-6afd-472c-b160-cad83527d6ae" containerName="ensemble-graph-016d3" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 16 19:11:34.586328 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:34.586286    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-7ea69-predictor-6fbd78b44f-btvsd" podUID="3049e3b2-f9e2-475b-8602-6ba3e585511b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.40:8080: connect: connection refused"
Sep 16 19:11:35.746628 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:35.746595    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-6b79a-predictor-544dbf6f99-vrqwv"
Sep 16 19:11:35.747143 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:35.747100    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-6b79a-predictor-544dbf6f99-vrqwv" podUID="f16ee8f1-b4a8-4e2d-9b6c-add6833c8c4b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.42:8080: connect: connection refused"
Sep 16 19:11:38.681755 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:38.681714    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-016d3-58bf87cfb-7x29f" podUID="86cf374c-6afd-472c-b160-cad83527d6ae" containerName="ensemble-graph-016d3" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 16 19:11:38.682161 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:38.681817    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/ensemble-graph-016d3-58bf87cfb-7x29f"
Sep 16 19:11:43.682160 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:43.682114    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-016d3-58bf87cfb-7x29f" podUID="86cf374c-6afd-472c-b160-cad83527d6ae" containerName="ensemble-graph-016d3" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 16 19:11:44.587532 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:44.587500    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-7ea69-predictor-6fbd78b44f-btvsd"
Sep 16 19:11:45.747565 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:45.747526    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-6b79a-predictor-544dbf6f99-vrqwv" podUID="f16ee8f1-b4a8-4e2d-9b6c-add6833c8c4b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.42:8080: connect: connection refused"
Sep 16 19:11:48.682544 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:48.682503    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-016d3-58bf87cfb-7x29f" podUID="86cf374c-6afd-472c-b160-cad83527d6ae" containerName="ensemble-graph-016d3" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 16 19:11:53.682121 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:53.682080    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-016d3-58bf87cfb-7x29f" podUID="86cf374c-6afd-472c-b160-cad83527d6ae" containerName="ensemble-graph-016d3" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 16 19:11:55.748029 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:55.747980    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-6b79a-predictor-544dbf6f99-vrqwv" podUID="f16ee8f1-b4a8-4e2d-9b6c-add6833c8c4b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.42:8080: connect: connection refused"
Sep 16 19:11:55.967769 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:55.967733    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/sequence-graph-7ea69-7f5dd56756-knxkh"]
Sep 16 19:11:55.968099 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:55.968086    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b84fe5f-226c-478a-bd7a-c5d815fba70f" containerName="kserve-container"
Sep 16 19:11:55.968146 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:55.968100    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b84fe5f-226c-478a-bd7a-c5d815fba70f" containerName="kserve-container"
Sep 16 19:11:55.968146 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:55.968121    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b84fe5f-226c-478a-bd7a-c5d815fba70f" containerName="kube-rbac-proxy"
Sep 16 19:11:55.968146 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:55.968126    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b84fe5f-226c-478a-bd7a-c5d815fba70f" containerName="kube-rbac-proxy"
Sep 16 19:11:55.968240 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:55.968173    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="8b84fe5f-226c-478a-bd7a-c5d815fba70f" containerName="kube-rbac-proxy"
Sep 16 19:11:55.968240 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:55.968184    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="8b84fe5f-226c-478a-bd7a-c5d815fba70f" containerName="kserve-container"
Sep 16 19:11:55.971141 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:55.971120    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-7ea69-7f5dd56756-knxkh"
Sep 16 19:11:55.973704 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:55.973683    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"sequence-graph-7ea69-kube-rbac-proxy-sar-config\""
Sep 16 19:11:55.973820 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:55.973703    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"sequence-graph-7ea69-serving-cert\""
Sep 16 19:11:55.974765 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:55.974731    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 19:11:55.979970 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:55.979949    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-7ea69-7f5dd56756-knxkh"]
Sep 16 19:11:56.113106 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:56.113067    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/4984e2cc-5589-4bad-b0de-5e5c5986c089-proxy-tls\") pod \"sequence-graph-7ea69-7f5dd56756-knxkh\" (UID: \"4984e2cc-5589-4bad-b0de-5e5c5986c089\") " pod="kserve-ci-e2e-test/sequence-graph-7ea69-7f5dd56756-knxkh"
Sep 16 19:11:56.113268 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:56.113223    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4984e2cc-5589-4bad-b0de-5e5c5986c089-openshift-service-ca-bundle\") pod \"sequence-graph-7ea69-7f5dd56756-knxkh\" (UID: \"4984e2cc-5589-4bad-b0de-5e5c5986c089\") " pod="kserve-ci-e2e-test/sequence-graph-7ea69-7f5dd56756-knxkh"
Sep 16 19:11:56.214301 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:56.214255    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4984e2cc-5589-4bad-b0de-5e5c5986c089-openshift-service-ca-bundle\") pod \"sequence-graph-7ea69-7f5dd56756-knxkh\" (UID: \"4984e2cc-5589-4bad-b0de-5e5c5986c089\") " pod="kserve-ci-e2e-test/sequence-graph-7ea69-7f5dd56756-knxkh"
Sep 16 19:11:56.214301 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:56.214309    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/4984e2cc-5589-4bad-b0de-5e5c5986c089-proxy-tls\") pod \"sequence-graph-7ea69-7f5dd56756-knxkh\" (UID: \"4984e2cc-5589-4bad-b0de-5e5c5986c089\") " pod="kserve-ci-e2e-test/sequence-graph-7ea69-7f5dd56756-knxkh"
Sep 16 19:11:56.214525 ip-10-0-139-182 kubenswrapper[2584]: E0916 19:11:56.214446    2584 secret.go:189] Couldn't get secret kserve-ci-e2e-test/sequence-graph-7ea69-serving-cert: secret "sequence-graph-7ea69-serving-cert" not found
Sep 16 19:11:56.214525 ip-10-0-139-182 kubenswrapper[2584]: E0916 19:11:56.214514    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4984e2cc-5589-4bad-b0de-5e5c5986c089-proxy-tls podName:4984e2cc-5589-4bad-b0de-5e5c5986c089 nodeName:}" failed. No retries permitted until 2026-09-16 19:11:56.714493804 +0000 UTC m=+1647.043996195 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/4984e2cc-5589-4bad-b0de-5e5c5986c089-proxy-tls") pod "sequence-graph-7ea69-7f5dd56756-knxkh" (UID: "4984e2cc-5589-4bad-b0de-5e5c5986c089") : secret "sequence-graph-7ea69-serving-cert" not found
Sep 16 19:11:56.214902 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:56.214883    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4984e2cc-5589-4bad-b0de-5e5c5986c089-openshift-service-ca-bundle\") pod \"sequence-graph-7ea69-7f5dd56756-knxkh\" (UID: \"4984e2cc-5589-4bad-b0de-5e5c5986c089\") " pod="kserve-ci-e2e-test/sequence-graph-7ea69-7f5dd56756-knxkh"
Sep 16 19:11:56.326132 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:56.326096    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 19:11:56.424687 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:56.424664    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-016d3-58bf87cfb-7x29f"
Sep 16 19:11:56.515915 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:56.515883    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/86cf374c-6afd-472c-b160-cad83527d6ae-openshift-service-ca-bundle\") pod \"86cf374c-6afd-472c-b160-cad83527d6ae\" (UID: \"86cf374c-6afd-472c-b160-cad83527d6ae\") "
Sep 16 19:11:56.516093 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:56.515950    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/86cf374c-6afd-472c-b160-cad83527d6ae-proxy-tls\") pod \"86cf374c-6afd-472c-b160-cad83527d6ae\" (UID: \"86cf374c-6afd-472c-b160-cad83527d6ae\") "
Sep 16 19:11:56.516254 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:56.516231    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/86cf374c-6afd-472c-b160-cad83527d6ae-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "86cf374c-6afd-472c-b160-cad83527d6ae" (UID: "86cf374c-6afd-472c-b160-cad83527d6ae"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 19:11:56.518104 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:56.518065    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/86cf374c-6afd-472c-b160-cad83527d6ae-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "86cf374c-6afd-472c-b160-cad83527d6ae" (UID: "86cf374c-6afd-472c-b160-cad83527d6ae"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 19:11:56.616933 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:56.616904    2584 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/86cf374c-6afd-472c-b160-cad83527d6ae-openshift-service-ca-bundle\") on node \"ip-10-0-139-182.ec2.internal\" DevicePath \"\""
Sep 16 19:11:56.616933 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:56.616930    2584 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/86cf374c-6afd-472c-b160-cad83527d6ae-proxy-tls\") on node \"ip-10-0-139-182.ec2.internal\" DevicePath \"\""
Sep 16 19:11:56.718099 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:56.718010    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/4984e2cc-5589-4bad-b0de-5e5c5986c089-proxy-tls\") pod \"sequence-graph-7ea69-7f5dd56756-knxkh\" (UID: \"4984e2cc-5589-4bad-b0de-5e5c5986c089\") " pod="kserve-ci-e2e-test/sequence-graph-7ea69-7f5dd56756-knxkh"
Sep 16 19:11:56.720396 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:56.720369    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/4984e2cc-5589-4bad-b0de-5e5c5986c089-proxy-tls\") pod \"sequence-graph-7ea69-7f5dd56756-knxkh\" (UID: \"4984e2cc-5589-4bad-b0de-5e5c5986c089\") " pod="kserve-ci-e2e-test/sequence-graph-7ea69-7f5dd56756-knxkh"
Sep 16 19:11:56.843883 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:56.843848    2584 generic.go:356] "Generic (PLEG): container finished" podID="86cf374c-6afd-472c-b160-cad83527d6ae" containerID="d747621583b2c3fa148e3d0230a6030226fe5749ccd074d765f359c7192214a6" exitCode=137
Sep 16 19:11:56.844364 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:56.843893    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-016d3-58bf87cfb-7x29f"
Sep 16 19:11:56.844364 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:56.843945    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-016d3-58bf87cfb-7x29f" event={"ID":"86cf374c-6afd-472c-b160-cad83527d6ae","Type":"ContainerDied","Data":"d747621583b2c3fa148e3d0230a6030226fe5749ccd074d765f359c7192214a6"}
Sep 16 19:11:56.844364 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:56.844000    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-016d3-58bf87cfb-7x29f" event={"ID":"86cf374c-6afd-472c-b160-cad83527d6ae","Type":"ContainerDied","Data":"4866f42727824a1be2ffdc4d15279f67312b6ea763360d3d2cf2b978b104a26d"}
Sep 16 19:11:56.844364 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:56.844026    2584 scope.go:117] "RemoveContainer" containerID="d747621583b2c3fa148e3d0230a6030226fe5749ccd074d765f359c7192214a6"
Sep 16 19:11:56.853690 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:56.853671    2584 scope.go:117] "RemoveContainer" containerID="d747621583b2c3fa148e3d0230a6030226fe5749ccd074d765f359c7192214a6"
Sep 16 19:11:56.854050 ip-10-0-139-182 kubenswrapper[2584]: E0916 19:11:56.854022    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d747621583b2c3fa148e3d0230a6030226fe5749ccd074d765f359c7192214a6\": container with ID starting with d747621583b2c3fa148e3d0230a6030226fe5749ccd074d765f359c7192214a6 not found: ID does not exist" containerID="d747621583b2c3fa148e3d0230a6030226fe5749ccd074d765f359c7192214a6"
Sep 16 19:11:56.854145 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:56.854054    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d747621583b2c3fa148e3d0230a6030226fe5749ccd074d765f359c7192214a6"} err="failed to get container status \"d747621583b2c3fa148e3d0230a6030226fe5749ccd074d765f359c7192214a6\": rpc error: code = NotFound desc = could not find container \"d747621583b2c3fa148e3d0230a6030226fe5749ccd074d765f359c7192214a6\": container with ID starting with d747621583b2c3fa148e3d0230a6030226fe5749ccd074d765f359c7192214a6 not found: ID does not exist"
Sep 16 19:11:56.867825 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:56.867799    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-016d3-58bf87cfb-7x29f"]
Sep 16 19:11:56.871189 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:56.871168    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-016d3-58bf87cfb-7x29f"]
Sep 16 19:11:56.881636 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:56.881608    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-7ea69-7f5dd56756-knxkh"
Sep 16 19:11:57.150517 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:57.150487    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-7ea69-7f5dd56756-knxkh"]
Sep 16 19:11:57.154761 ip-10-0-139-182 kubenswrapper[2584]: W0916 19:11:57.154728    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4984e2cc_5589_4bad_b0de_5e5c5986c089.slice/crio-85e59cb6977e68ab4e15900768a3e9add9ac72a4cdf9332432b338b62ed2fbb8 WatchSource:0}: Error finding container 85e59cb6977e68ab4e15900768a3e9add9ac72a4cdf9332432b338b62ed2fbb8: Status 404 returned error can't find the container with id 85e59cb6977e68ab4e15900768a3e9add9ac72a4cdf9332432b338b62ed2fbb8
Sep 16 19:11:57.850703 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:57.850668    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-7ea69-7f5dd56756-knxkh" event={"ID":"4984e2cc-5589-4bad-b0de-5e5c5986c089","Type":"ContainerStarted","Data":"d05417c6b9176ed931b5a1db6612078c454aef1dc44b6e34efac6c3d58010696"}
Sep 16 19:11:57.851099 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:57.850750    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-7ea69-7f5dd56756-knxkh" event={"ID":"4984e2cc-5589-4bad-b0de-5e5c5986c089","Type":"ContainerStarted","Data":"85e59cb6977e68ab4e15900768a3e9add9ac72a4cdf9332432b338b62ed2fbb8"}
Sep 16 19:11:57.851099 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:57.850776    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/sequence-graph-7ea69-7f5dd56756-knxkh"
Sep 16 19:11:57.867284 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:57.867230    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/sequence-graph-7ea69-7f5dd56756-knxkh" podStartSLOduration=2.867213618 podStartE2EDuration="2.867213618s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 19:11:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 19:11:57.866913411 +0000 UTC m=+1648.196415824" watchObservedRunningTime="2026-09-16 19:11:57.867213618 +0000 UTC m=+1648.196716050"
Sep 16 19:11:58.116255 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:11:58.116177    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="86cf374c-6afd-472c-b160-cad83527d6ae" path="/var/lib/kubelet/pods/86cf374c-6afd-472c-b160-cad83527d6ae/volumes"
Sep 16 19:12:03.860914 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:12:03.860879    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/sequence-graph-7ea69-7f5dd56756-knxkh"
Sep 16 19:12:05.747164 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:12:05.747120    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-6b79a-predictor-544dbf6f99-vrqwv" podUID="f16ee8f1-b4a8-4e2d-9b6c-add6833c8c4b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.42:8080: connect: connection refused"
Sep 16 19:12:06.047550 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:12:06.047512    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-7ea69-7f5dd56756-knxkh"]
Sep 16 19:12:06.047766 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:12:06.047724    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/sequence-graph-7ea69-7f5dd56756-knxkh" podUID="4984e2cc-5589-4bad-b0de-5e5c5986c089" containerName="sequence-graph-7ea69" containerID="cri-o://d05417c6b9176ed931b5a1db6612078c454aef1dc44b6e34efac6c3d58010696" gracePeriod=30
Sep 16 19:12:06.072464 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:12:06.072433    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 19:12:06.159271 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:12:06.159230    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-7ea69-predictor-6fbd78b44f-btvsd"]
Sep 16 19:12:06.159602 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:12:06.159569    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-7ea69-predictor-6fbd78b44f-btvsd" podUID="3049e3b2-f9e2-475b-8602-6ba3e585511b" containerName="kserve-container" containerID="cri-o://31afc00aad77e3dbee3f68c435d01b81b555cf81cc03ec65967c83d8482a1c4a" gracePeriod=30
Sep 16 19:12:06.159745 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:12:06.159599    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-7ea69-predictor-6fbd78b44f-btvsd" podUID="3049e3b2-f9e2-475b-8602-6ba3e585511b" containerName="kube-rbac-proxy" containerID="cri-o://a0c57ec2e17878d6f95afab0d815c0111897088b71df3b06723330ad62d7fa98" gracePeriod=30
Sep 16 19:12:06.191129 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:12:06.191071    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 19:12:06.290169 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:12:06.290136    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 19:12:06.293898 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:12:06.293874    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-f025b-predictor-68bc64675f-hthl4"]
Sep 16 19:12:06.294247 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:12:06.294233    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="86cf374c-6afd-472c-b160-cad83527d6ae" containerName="ensemble-graph-016d3"
Sep 16 19:12:06.294291 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:12:06.294248    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="86cf374c-6afd-472c-b160-cad83527d6ae" containerName="ensemble-graph-016d3"
Sep 16 19:12:06.294326 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:12:06.294319    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="86cf374c-6afd-472c-b160-cad83527d6ae" containerName="ensemble-graph-016d3"
Sep 16 19:12:06.297209 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:12:06.297191    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-f025b-predictor-68bc64675f-hthl4"
Sep 16 19:12:06.299927 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:12:06.299864    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-f025b-predictor-serving-cert\""
Sep 16 19:12:06.299927 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:12:06.299868    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-f025b-kube-rbac-proxy-sar-config\""
Sep 16 19:12:06.308193 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:12:06.308165    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-f025b-predictor-68bc64675f-hthl4"]
Sep 16 19:12:06.356594 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:12:06.356546    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 19:12:06.407680 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:12:06.407646    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tqpg8\" (UniqueName: \"kubernetes.io/projected/fd98557c-48e9-4481-a395-afffbf0448a5-kube-api-access-tqpg8\") pod \"success-200-isvc-f025b-predictor-68bc64675f-hthl4\" (UID: \"fd98557c-48e9-4481-a395-afffbf0448a5\") " pod="kserve-ci-e2e-test/success-200-isvc-f025b-predictor-68bc64675f-hthl4"
Sep 16 19:12:06.407680 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:12:06.407685    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/fd98557c-48e9-4481-a395-afffbf0448a5-proxy-tls\") pod \"success-200-isvc-f025b-predictor-68bc64675f-hthl4\" (UID: \"fd98557c-48e9-4481-a395-afffbf0448a5\") " pod="kserve-ci-e2e-test/success-200-isvc-f025b-predictor-68bc64675f-hthl4"
Sep 16 19:12:06.407926 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:12:06.407721    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/fd98557c-48e9-4481-a395-afffbf0448a5-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-f025b-predictor-68bc64675f-hthl4\" (UID: \"fd98557c-48e9-4481-a395-afffbf0448a5\") " pod="kserve-ci-e2e-test/success-200-isvc-f025b-predictor-68bc64675f-hthl4"
Sep 16 19:12:06.496561 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:12:06.496525    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 19:12:06.508822 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:12:06.508792    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tqpg8\" (UniqueName: \"kubernetes.io/projected/fd98557c-48e9-4481-a395-afffbf0448a5-kube-api-access-tqpg8\") pod \"success-200-isvc-f025b-predictor-68bc64675f-hthl4\" (UID: \"fd98557c-48e9-4481-a395-afffbf0448a5\") " pod="kserve-ci-e2e-test/success-200-isvc-f025b-predictor-68bc64675f-hthl4"
Sep 16 19:12:06.508991 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:12:06.508861    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/fd98557c-48e9-4481-a395-afffbf0448a5-proxy-tls\") pod \"success-200-isvc-f025b-predictor-68bc64675f-hthl4\" (UID: \"fd98557c-48e9-4481-a395-afffbf0448a5\") " pod="kserve-ci-e2e-test/success-200-isvc-f025b-predictor-68bc64675f-hthl4"
Sep 16 19:12:06.508991 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:12:06.508895    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/fd98557c-48e9-4481-a395-afffbf0448a5-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-f025b-predictor-68bc64675f-hthl4\" (UID: \"fd98557c-48e9-4481-a395-afffbf0448a5\") " pod="kserve-ci-e2e-test/success-200-isvc-f025b-predictor-68bc64675f-hthl4"
Sep 16 19:12:06.509571 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:12:06.509550    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/fd98557c-48e9-4481-a395-afffbf0448a5-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-f025b-predictor-68bc64675f-hthl4\" (UID: \"fd98557c-48e9-4481-a395-afffbf0448a5\") " pod="kserve-ci-e2e-test/success-200-isvc-f025b-predictor-68bc64675f-hthl4"
Sep 16 19:12:06.511446 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:12:06.511424    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/fd98557c-48e9-4481-a395-afffbf0448a5-proxy-tls\") pod \"success-200-isvc-f025b-predictor-68bc64675f-hthl4\" (UID: \"fd98557c-48e9-4481-a395-afffbf0448a5\") " pod="kserve-ci-e2e-test/success-200-isvc-f025b-predictor-68bc64675f-hthl4"
Sep 16 19:12:06.519397 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:12:06.519376    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tqpg8\" (UniqueName: \"kubernetes.io/projected/fd98557c-48e9-4481-a395-afffbf0448a5-kube-api-access-tqpg8\") pod \"success-200-isvc-f025b-predictor-68bc64675f-hthl4\" (UID: \"fd98557c-48e9-4481-a395-afffbf0448a5\") " pod="kserve-ci-e2e-test/success-200-isvc-f025b-predictor-68bc64675f-hthl4"
Sep 16 19:12:06.591147 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:12:06.591064    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 19:12:06.609351 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:12:06.609321    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-f025b-predictor-68bc64675f-hthl4"
Sep 16 19:12:06.882385 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:12:06.882358    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-f025b-predictor-68bc64675f-hthl4"]
Sep 16 19:12:06.884542 ip-10-0-139-182 kubenswrapper[2584]: W0916 19:12:06.884514    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd98557c_48e9_4481_a395_afffbf0448a5.slice/crio-e2eb79bf3e2477101a1b77500738cee46e02e0d6272de26465a3ee04f9f3e942 WatchSource:0}: Error finding container e2eb79bf3e2477101a1b77500738cee46e02e0d6272de26465a3ee04f9f3e942: Status 404 returned error can't find the container with id e2eb79bf3e2477101a1b77500738cee46e02e0d6272de26465a3ee04f9f3e942
Sep 16 19:12:06.893476 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:12:06.893449    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-f025b-predictor-68bc64675f-hthl4" event={"ID":"fd98557c-48e9-4481-a395-afffbf0448a5","Type":"ContainerStarted","Data":"e2eb79bf3e2477101a1b77500738cee46e02e0d6272de26465a3ee04f9f3e942"}
Sep 16 19:12:06.895440 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:12:06.895418    2584 generic.go:356] "Generic (PLEG): container finished" podID="3049e3b2-f9e2-475b-8602-6ba3e585511b" containerID="a0c57ec2e17878d6f95afab0d815c0111897088b71df3b06723330ad62d7fa98" exitCode=2
Sep 16 19:12:06.895522 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:12:06.895456    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-7ea69-predictor-6fbd78b44f-btvsd" event={"ID":"3049e3b2-f9e2-475b-8602-6ba3e585511b","Type":"ContainerDied","Data":"a0c57ec2e17878d6f95afab0d815c0111897088b71df3b06723330ad62d7fa98"}
Sep 16 19:12:07.902269 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:12:07.902232    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-f025b-predictor-68bc64675f-hthl4" event={"ID":"fd98557c-48e9-4481-a395-afffbf0448a5","Type":"ContainerStarted","Data":"1fae9a3227bcb4101df39ec54e2e5fc7bb29981f9477e5bd6e198acf9fbdaa9f"}
Sep 16 19:12:07.902692 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:12:07.902297    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-f025b-predictor-68bc64675f-hthl4" event={"ID":"fd98557c-48e9-4481-a395-afffbf0448a5","Type":"ContainerStarted","Data":"73bc102d545f611b800f1935e904bb3208ff55e61da508d74239fdcbca7818b0"}
Sep 16 19:12:07.902692 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:12:07.902403    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-f025b-predictor-68bc64675f-hthl4"
Sep 16 19:12:07.902692 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:12:07.902569    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-f025b-predictor-68bc64675f-hthl4"
Sep 16 19:12:07.904126 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:12:07.904099    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-f025b-predictor-68bc64675f-hthl4" podUID="fd98557c-48e9-4481-a395-afffbf0448a5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.44:8080: connect: connection refused"
Sep 16 19:12:07.920046 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:12:07.920007    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-f025b-predictor-68bc64675f-hthl4" podStartSLOduration=1.919994028 podStartE2EDuration="1.919994028s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 19:12:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 19:12:07.918967936 +0000 UTC m=+1658.248470348" watchObservedRunningTime="2026-09-16 19:12:07.919994028 +0000 UTC m=+1658.249496442"
Sep 16 19:12:08.859432 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:12:08.859395    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-7ea69-7f5dd56756-knxkh" podUID="4984e2cc-5589-4bad-b0de-5e5c5986c089" containerName="sequence-graph-7ea69" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 16 19:12:08.915088 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:12:08.915040    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-f025b-predictor-68bc64675f-hthl4" podUID="fd98557c-48e9-4481-a395-afffbf0448a5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.44:8080: connect: connection refused"
Sep 16 19:12:09.307755 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:12:09.307730    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-7ea69-predictor-6fbd78b44f-btvsd"
Sep 16 19:12:09.433557 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:12:09.433459    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zswd8\" (UniqueName: \"kubernetes.io/projected/3049e3b2-f9e2-475b-8602-6ba3e585511b-kube-api-access-zswd8\") pod \"3049e3b2-f9e2-475b-8602-6ba3e585511b\" (UID: \"3049e3b2-f9e2-475b-8602-6ba3e585511b\") "
Sep 16 19:12:09.433557 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:12:09.433538    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/3049e3b2-f9e2-475b-8602-6ba3e585511b-kube-rbac-proxy-sar-config\") pod \"3049e3b2-f9e2-475b-8602-6ba3e585511b\" (UID: \"3049e3b2-f9e2-475b-8602-6ba3e585511b\") "
Sep 16 19:12:09.433750 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:12:09.433575    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3049e3b2-f9e2-475b-8602-6ba3e585511b-proxy-tls\") pod \"3049e3b2-f9e2-475b-8602-6ba3e585511b\" (UID: \"3049e3b2-f9e2-475b-8602-6ba3e585511b\") "
Sep 16 19:12:09.433930 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:12:09.433904    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3049e3b2-f9e2-475b-8602-6ba3e585511b-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "3049e3b2-f9e2-475b-8602-6ba3e585511b" (UID: "3049e3b2-f9e2-475b-8602-6ba3e585511b"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 19:12:09.435663 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:12:09.435632    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3049e3b2-f9e2-475b-8602-6ba3e585511b-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "3049e3b2-f9e2-475b-8602-6ba3e585511b" (UID: "3049e3b2-f9e2-475b-8602-6ba3e585511b"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 19:12:09.435663 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:12:09.435638    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3049e3b2-f9e2-475b-8602-6ba3e585511b-kube-api-access-zswd8" (OuterVolumeSpecName: "kube-api-access-zswd8") pod "3049e3b2-f9e2-475b-8602-6ba3e585511b" (UID: "3049e3b2-f9e2-475b-8602-6ba3e585511b"). InnerVolumeSpecName "kube-api-access-zswd8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 19:12:09.535230 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:12:09.535188    2584 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/3049e3b2-f9e2-475b-8602-6ba3e585511b-kube-rbac-proxy-sar-config\") on node \"ip-10-0-139-182.ec2.internal\" DevicePath \"\""
Sep 16 19:12:09.535230 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:12:09.535219    2584 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3049e3b2-f9e2-475b-8602-6ba3e585511b-proxy-tls\") on node \"ip-10-0-139-182.ec2.internal\" DevicePath \"\""
Sep 16 19:12:09.535230 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:12:09.535230    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zswd8\" (UniqueName: \"kubernetes.io/projected/3049e3b2-f9e2-475b-8602-6ba3e585511b-kube-api-access-zswd8\") on node \"ip-10-0-139-182.ec2.internal\" DevicePath \"\""
Sep 16 19:12:09.921722 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:12:09.921690    2584 generic.go:356] "Generic (PLEG): container finished" podID="3049e3b2-f9e2-475b-8602-6ba3e585511b" containerID="31afc00aad77e3dbee3f68c435d01b81b555cf81cc03ec65967c83d8482a1c4a" exitCode=0
Sep 16 19:12:09.922222 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:12:09.921737    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-7ea69-predictor-6fbd78b44f-btvsd"
Sep 16 19:12:09.922222 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:12:09.921772    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-7ea69-predictor-6fbd78b44f-btvsd" event={"ID":"3049e3b2-f9e2-475b-8602-6ba3e585511b","Type":"ContainerDied","Data":"31afc00aad77e3dbee3f68c435d01b81b555cf81cc03ec65967c83d8482a1c4a"}
Sep 16 19:12:09.922222 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:12:09.921896    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-7ea69-predictor-6fbd78b44f-btvsd" event={"ID":"3049e3b2-f9e2-475b-8602-6ba3e585511b","Type":"ContainerDied","Data":"608230e140c46d3c99027a4ac6be225405fd852ecc3054a096f1e27ed91b76c2"}
Sep 16 19:12:09.922222 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:12:09.921939    2584 scope.go:117] "RemoveContainer" containerID="a0c57ec2e17878d6f95afab0d815c0111897088b71df3b06723330ad62d7fa98"
Sep 16 19:12:09.930127 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:12:09.930107    2584 scope.go:117] "RemoveContainer" containerID="31afc00aad77e3dbee3f68c435d01b81b555cf81cc03ec65967c83d8482a1c4a"
Sep 16 19:12:09.937526 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:12:09.937511    2584 scope.go:117] "RemoveContainer" containerID="a0c57ec2e17878d6f95afab0d815c0111897088b71df3b06723330ad62d7fa98"
Sep 16 19:12:09.937788 ip-10-0-139-182 kubenswrapper[2584]: E0916 19:12:09.937768    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a0c57ec2e17878d6f95afab0d815c0111897088b71df3b06723330ad62d7fa98\": container with ID starting with a0c57ec2e17878d6f95afab0d815c0111897088b71df3b06723330ad62d7fa98 not found: ID does not exist" containerID="a0c57ec2e17878d6f95afab0d815c0111897088b71df3b06723330ad62d7fa98"
Sep 16 19:12:09.937889 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:12:09.937798    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a0c57ec2e17878d6f95afab0d815c0111897088b71df3b06723330ad62d7fa98"} err="failed to get container status \"a0c57ec2e17878d6f95afab0d815c0111897088b71df3b06723330ad62d7fa98\": rpc error: code = NotFound desc = could not find container \"a0c57ec2e17878d6f95afab0d815c0111897088b71df3b06723330ad62d7fa98\": container with ID starting with a0c57ec2e17878d6f95afab0d815c0111897088b71df3b06723330ad62d7fa98 not found: ID does not exist"
Sep 16 19:12:09.937889 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:12:09.937820    2584 scope.go:117] "RemoveContainer" containerID="31afc00aad77e3dbee3f68c435d01b81b555cf81cc03ec65967c83d8482a1c4a"
Sep 16 19:12:09.938203 ip-10-0-139-182 kubenswrapper[2584]: E0916 19:12:09.938179    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"31afc00aad77e3dbee3f68c435d01b81b555cf81cc03ec65967c83d8482a1c4a\": container with ID starting with 31afc00aad77e3dbee3f68c435d01b81b555cf81cc03ec65967c83d8482a1c4a not found: ID does not exist" containerID="31afc00aad77e3dbee3f68c435d01b81b555cf81cc03ec65967c83d8482a1c4a"
Sep 16 19:12:09.938258 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:12:09.938210    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"31afc00aad77e3dbee3f68c435d01b81b555cf81cc03ec65967c83d8482a1c4a"} err="failed to get container status \"31afc00aad77e3dbee3f68c435d01b81b555cf81cc03ec65967c83d8482a1c4a\": rpc error: code = NotFound desc = could not find container \"31afc00aad77e3dbee3f68c435d01b81b555cf81cc03ec65967c83d8482a1c4a\": container with ID starting with 31afc00aad77e3dbee3f68c435d01b81b555cf81cc03ec65967c83d8482a1c4a not found: ID does not exist"
Sep 16 19:12:09.945619 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:12:09.945595    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-7ea69-predictor-6fbd78b44f-btvsd"]
Sep 16 19:12:09.950581 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:12:09.950555    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-7ea69-predictor-6fbd78b44f-btvsd"]
Sep 16 19:12:10.116553 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:12:10.116520    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3049e3b2-f9e2-475b-8602-6ba3e585511b" path="/var/lib/kubelet/pods/3049e3b2-f9e2-475b-8602-6ba3e585511b/volumes"
Sep 16 19:12:13.858594 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:12:13.858557    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-7ea69-7f5dd56756-knxkh" podUID="4984e2cc-5589-4bad-b0de-5e5c5986c089" containerName="sequence-graph-7ea69" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 16 19:12:13.919615 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:12:13.919586    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-f025b-predictor-68bc64675f-hthl4"
Sep 16 19:12:13.920083 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:12:13.920055    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-f025b-predictor-68bc64675f-hthl4" podUID="fd98557c-48e9-4481-a395-afffbf0448a5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.44:8080: connect: connection refused"
Sep 16 19:12:15.747516 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:12:15.747479    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-6b79a-predictor-544dbf6f99-vrqwv"
Sep 16 19:12:18.859370 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:12:18.859325    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-7ea69-7f5dd56756-knxkh" podUID="4984e2cc-5589-4bad-b0de-5e5c5986c089" containerName="sequence-graph-7ea69" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 16 19:12:18.859740 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:12:18.859428    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/sequence-graph-7ea69-7f5dd56756-knxkh"
Sep 16 19:12:23.858804 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:12:23.858765    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-7ea69-7f5dd56756-knxkh" podUID="4984e2cc-5589-4bad-b0de-5e5c5986c089" containerName="sequence-graph-7ea69" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 16 19:12:23.920236 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:12:23.920195    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-f025b-predictor-68bc64675f-hthl4" podUID="fd98557c-48e9-4481-a395-afffbf0448a5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.44:8080: connect: connection refused"
Sep 16 19:12:26.536729 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:12:26.536689    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-6b79a-556b7955dc-qxqq4"]
Sep 16 19:12:26.537213 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:12:26.537133    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3049e3b2-f9e2-475b-8602-6ba3e585511b" containerName="kube-rbac-proxy"
Sep 16 19:12:26.537213 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:12:26.537151    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="3049e3b2-f9e2-475b-8602-6ba3e585511b" containerName="kube-rbac-proxy"
Sep 16 19:12:26.537213 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:12:26.537170    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3049e3b2-f9e2-475b-8602-6ba3e585511b" containerName="kserve-container"
Sep 16 19:12:26.537213 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:12:26.537176    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="3049e3b2-f9e2-475b-8602-6ba3e585511b" containerName="kserve-container"
Sep 16 19:12:26.537419 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:12:26.537236    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="3049e3b2-f9e2-475b-8602-6ba3e585511b" containerName="kube-rbac-proxy"
Sep 16 19:12:26.537419 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:12:26.537247    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="3049e3b2-f9e2-475b-8602-6ba3e585511b" containerName="kserve-container"
Sep 16 19:12:26.541917 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:12:26.541895    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-6b79a-556b7955dc-qxqq4"
Sep 16 19:12:26.545397 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:12:26.545369    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"ensemble-graph-6b79a-serving-cert\""
Sep 16 19:12:26.545536 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:12:26.545404    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"ensemble-graph-6b79a-kube-rbac-proxy-sar-config\""
Sep 16 19:12:26.549722 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:12:26.549590    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 19:12:26.552920 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:12:26.552892    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-6b79a-556b7955dc-qxqq4"]
Sep 16 19:12:26.576282 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:12:26.576248    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/caff3c6c-3daf-4931-8da5-40aea8ef097e-proxy-tls\") pod \"ensemble-graph-6b79a-556b7955dc-qxqq4\" (UID: \"caff3c6c-3daf-4931-8da5-40aea8ef097e\") " pod="kserve-ci-e2e-test/ensemble-graph-6b79a-556b7955dc-qxqq4"
Sep 16 19:12:26.576436 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:12:26.576300    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/caff3c6c-3daf-4931-8da5-40aea8ef097e-openshift-service-ca-bundle\") pod \"ensemble-graph-6b79a-556b7955dc-qxqq4\" (UID: \"caff3c6c-3daf-4931-8da5-40aea8ef097e\") " pod="kserve-ci-e2e-test/ensemble-graph-6b79a-556b7955dc-qxqq4"
Sep 16 19:12:26.676781 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:12:26.676740    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/caff3c6c-3daf-4931-8da5-40aea8ef097e-proxy-tls\") pod \"ensemble-graph-6b79a-556b7955dc-qxqq4\" (UID: \"caff3c6c-3daf-4931-8da5-40aea8ef097e\") " pod="kserve-ci-e2e-test/ensemble-graph-6b79a-556b7955dc-qxqq4"
Sep 16 19:12:26.677282 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:12:26.676807    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/caff3c6c-3daf-4931-8da5-40aea8ef097e-openshift-service-ca-bundle\") pod \"ensemble-graph-6b79a-556b7955dc-qxqq4\" (UID: \"caff3c6c-3daf-4931-8da5-40aea8ef097e\") " pod="kserve-ci-e2e-test/ensemble-graph-6b79a-556b7955dc-qxqq4"
Sep 16 19:12:26.677282 ip-10-0-139-182 kubenswrapper[2584]: E0916 19:12:26.676923    2584 secret.go:189] Couldn't get secret kserve-ci-e2e-test/ensemble-graph-6b79a-serving-cert: secret "ensemble-graph-6b79a-serving-cert" not found
Sep 16 19:12:26.677282 ip-10-0-139-182 kubenswrapper[2584]: E0916 19:12:26.676999    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/caff3c6c-3daf-4931-8da5-40aea8ef097e-proxy-tls podName:caff3c6c-3daf-4931-8da5-40aea8ef097e nodeName:}" failed. No retries permitted until 2026-09-16 19:12:27.176979177 +0000 UTC m=+1677.506481583 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/caff3c6c-3daf-4931-8da5-40aea8ef097e-proxy-tls") pod "ensemble-graph-6b79a-556b7955dc-qxqq4" (UID: "caff3c6c-3daf-4931-8da5-40aea8ef097e") : secret "ensemble-graph-6b79a-serving-cert" not found
Sep 16 19:12:26.677464 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:12:26.677446    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/caff3c6c-3daf-4931-8da5-40aea8ef097e-openshift-service-ca-bundle\") pod \"ensemble-graph-6b79a-556b7955dc-qxqq4\" (UID: \"caff3c6c-3daf-4931-8da5-40aea8ef097e\") " pod="kserve-ci-e2e-test/ensemble-graph-6b79a-556b7955dc-qxqq4"
Sep 16 19:12:27.005256 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:12:27.005159    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 19:12:27.181133 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:12:27.181095    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/caff3c6c-3daf-4931-8da5-40aea8ef097e-proxy-tls\") pod \"ensemble-graph-6b79a-556b7955dc-qxqq4\" (UID: \"caff3c6c-3daf-4931-8da5-40aea8ef097e\") " pod="kserve-ci-e2e-test/ensemble-graph-6b79a-556b7955dc-qxqq4"
Sep 16 19:12:27.183598 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:12:27.183570    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/caff3c6c-3daf-4931-8da5-40aea8ef097e-proxy-tls\") pod \"ensemble-graph-6b79a-556b7955dc-qxqq4\" (UID: \"caff3c6c-3daf-4931-8da5-40aea8ef097e\") " pod="kserve-ci-e2e-test/ensemble-graph-6b79a-556b7955dc-qxqq4"
Sep 16 19:12:27.453394 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:12:27.453356    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-6b79a-556b7955dc-qxqq4"
Sep 16 19:12:27.721654 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:12:27.721612    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-6b79a-556b7955dc-qxqq4"]
Sep 16 19:12:27.723972 ip-10-0-139-182 kubenswrapper[2584]: W0916 19:12:27.723942    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcaff3c6c_3daf_4931_8da5_40aea8ef097e.slice/crio-b63dc398f34b54292445aac6911ab7a5b500223e7144073968e2b777ce562366 WatchSource:0}: Error finding container b63dc398f34b54292445aac6911ab7a5b500223e7144073968e2b777ce562366: Status 404 returned error can't find the container with id b63dc398f34b54292445aac6911ab7a5b500223e7144073968e2b777ce562366
Sep 16 19:12:28.010623 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:12:28.010589    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-6b79a-556b7955dc-qxqq4" event={"ID":"caff3c6c-3daf-4931-8da5-40aea8ef097e","Type":"ContainerStarted","Data":"c378ca7e393d558c6b2819fa1048e960ebd9f3911a45f456872cbf3f6a6ec0f7"}
Sep 16 19:12:28.010778 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:12:28.010641    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-6b79a-556b7955dc-qxqq4" event={"ID":"caff3c6c-3daf-4931-8da5-40aea8ef097e","Type":"ContainerStarted","Data":"b63dc398f34b54292445aac6911ab7a5b500223e7144073968e2b777ce562366"}
Sep 16 19:12:28.010823 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:12:28.010777    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/ensemble-graph-6b79a-556b7955dc-qxqq4"
Sep 16 19:12:28.027637 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:12:28.027590    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/ensemble-graph-6b79a-556b7955dc-qxqq4" podStartSLOduration=2.027575763 podStartE2EDuration="2.027575763s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 19:12:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 19:12:28.02709827 +0000 UTC m=+1678.356600687" watchObservedRunningTime="2026-09-16 19:12:28.027575763 +0000 UTC m=+1678.357078176"
Sep 16 19:12:28.859690 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:12:28.859646    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-7ea69-7f5dd56756-knxkh" podUID="4984e2cc-5589-4bad-b0de-5e5c5986c089" containerName="sequence-graph-7ea69" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 16 19:12:33.859364 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:12:33.859322    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-7ea69-7f5dd56756-knxkh" podUID="4984e2cc-5589-4bad-b0de-5e5c5986c089" containerName="sequence-graph-7ea69" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 16 19:12:33.920971 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:12:33.920931    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-f025b-predictor-68bc64675f-hthl4" podUID="fd98557c-48e9-4481-a395-afffbf0448a5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.44:8080: connect: connection refused"
Sep 16 19:12:34.021564 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:12:34.021535    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/ensemble-graph-6b79a-556b7955dc-qxqq4"
Sep 16 19:12:36.190019 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:12:36.189989    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-7ea69-7f5dd56756-knxkh"
Sep 16 19:12:36.259397 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:12:36.259363    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4984e2cc-5589-4bad-b0de-5e5c5986c089-openshift-service-ca-bundle\") pod \"4984e2cc-5589-4bad-b0de-5e5c5986c089\" (UID: \"4984e2cc-5589-4bad-b0de-5e5c5986c089\") "
Sep 16 19:12:36.259579 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:12:36.259437    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/4984e2cc-5589-4bad-b0de-5e5c5986c089-proxy-tls\") pod \"4984e2cc-5589-4bad-b0de-5e5c5986c089\" (UID: \"4984e2cc-5589-4bad-b0de-5e5c5986c089\") "
Sep 16 19:12:36.259775 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:12:36.259747    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4984e2cc-5589-4bad-b0de-5e5c5986c089-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "4984e2cc-5589-4bad-b0de-5e5c5986c089" (UID: "4984e2cc-5589-4bad-b0de-5e5c5986c089"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 19:12:36.261473 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:12:36.261444    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4984e2cc-5589-4bad-b0de-5e5c5986c089-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "4984e2cc-5589-4bad-b0de-5e5c5986c089" (UID: "4984e2cc-5589-4bad-b0de-5e5c5986c089"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 19:12:36.360784 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:12:36.360707    2584 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/4984e2cc-5589-4bad-b0de-5e5c5986c089-proxy-tls\") on node \"ip-10-0-139-182.ec2.internal\" DevicePath \"\""
Sep 16 19:12:36.360784 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:12:36.360737    2584 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4984e2cc-5589-4bad-b0de-5e5c5986c089-openshift-service-ca-bundle\") on node \"ip-10-0-139-182.ec2.internal\" DevicePath \"\""
Sep 16 19:12:37.054275 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:12:37.054233    2584 generic.go:356] "Generic (PLEG): container finished" podID="4984e2cc-5589-4bad-b0de-5e5c5986c089" containerID="d05417c6b9176ed931b5a1db6612078c454aef1dc44b6e34efac6c3d58010696" exitCode=0
Sep 16 19:12:37.054438 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:12:37.054282    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-7ea69-7f5dd56756-knxkh"
Sep 16 19:12:37.054438 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:12:37.054312    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-7ea69-7f5dd56756-knxkh" event={"ID":"4984e2cc-5589-4bad-b0de-5e5c5986c089","Type":"ContainerDied","Data":"d05417c6b9176ed931b5a1db6612078c454aef1dc44b6e34efac6c3d58010696"}
Sep 16 19:12:37.054438 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:12:37.054374    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-7ea69-7f5dd56756-knxkh" event={"ID":"4984e2cc-5589-4bad-b0de-5e5c5986c089","Type":"ContainerDied","Data":"85e59cb6977e68ab4e15900768a3e9add9ac72a4cdf9332432b338b62ed2fbb8"}
Sep 16 19:12:37.054438 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:12:37.054397    2584 scope.go:117] "RemoveContainer" containerID="d05417c6b9176ed931b5a1db6612078c454aef1dc44b6e34efac6c3d58010696"
Sep 16 19:12:37.062953 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:12:37.062934    2584 scope.go:117] "RemoveContainer" containerID="d05417c6b9176ed931b5a1db6612078c454aef1dc44b6e34efac6c3d58010696"
Sep 16 19:12:37.063225 ip-10-0-139-182 kubenswrapper[2584]: E0916 19:12:37.063209    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d05417c6b9176ed931b5a1db6612078c454aef1dc44b6e34efac6c3d58010696\": container with ID starting with d05417c6b9176ed931b5a1db6612078c454aef1dc44b6e34efac6c3d58010696 not found: ID does not exist" containerID="d05417c6b9176ed931b5a1db6612078c454aef1dc44b6e34efac6c3d58010696"
Sep 16 19:12:37.063273 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:12:37.063231    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d05417c6b9176ed931b5a1db6612078c454aef1dc44b6e34efac6c3d58010696"} err="failed to get container status \"d05417c6b9176ed931b5a1db6612078c454aef1dc44b6e34efac6c3d58010696\": rpc error: code = NotFound desc = could not find container \"d05417c6b9176ed931b5a1db6612078c454aef1dc44b6e34efac6c3d58010696\": container with ID starting with d05417c6b9176ed931b5a1db6612078c454aef1dc44b6e34efac6c3d58010696 not found: ID does not exist"
Sep 16 19:12:37.076987 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:12:37.076959    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-7ea69-7f5dd56756-knxkh"]
Sep 16 19:12:37.080798 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:12:37.080772    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-7ea69-7f5dd56756-knxkh"]
Sep 16 19:12:38.116001 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:12:38.115967    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4984e2cc-5589-4bad-b0de-5e5c5986c089" path="/var/lib/kubelet/pods/4984e2cc-5589-4bad-b0de-5e5c5986c089/volumes"
Sep 16 19:12:43.921018 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:12:43.920976    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-f025b-predictor-68bc64675f-hthl4" podUID="fd98557c-48e9-4481-a395-afffbf0448a5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.44:8080: connect: connection refused"
Sep 16 19:12:53.921795 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:12:53.921702    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-f025b-predictor-68bc64675f-hthl4"
Sep 16 19:13:06.303289 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:13:06.303254    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/sequence-graph-f025b-545b74b6c4-lcc6q"]
Sep 16 19:13:06.303759 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:13:06.303618    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4984e2cc-5589-4bad-b0de-5e5c5986c089" containerName="sequence-graph-7ea69"
Sep 16 19:13:06.303759 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:13:06.303629    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="4984e2cc-5589-4bad-b0de-5e5c5986c089" containerName="sequence-graph-7ea69"
Sep 16 19:13:06.303759 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:13:06.303685    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="4984e2cc-5589-4bad-b0de-5e5c5986c089" containerName="sequence-graph-7ea69"
Sep 16 19:13:06.306498 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:13:06.306464    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-f025b-545b74b6c4-lcc6q"
Sep 16 19:13:06.306698 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:13:06.306670    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 19:13:06.309603 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:13:06.309581    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"sequence-graph-f025b-serving-cert\""
Sep 16 19:13:06.310269 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:13:06.310254    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"sequence-graph-f025b-kube-rbac-proxy-sar-config\""
Sep 16 19:13:06.323738 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:13:06.323705    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/109a6ec8-78d3-45bf-a316-fce6791c6399-proxy-tls\") pod \"sequence-graph-f025b-545b74b6c4-lcc6q\" (UID: \"109a6ec8-78d3-45bf-a316-fce6791c6399\") " pod="kserve-ci-e2e-test/sequence-graph-f025b-545b74b6c4-lcc6q"
Sep 16 19:13:06.323899 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:13:06.323816    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/109a6ec8-78d3-45bf-a316-fce6791c6399-openshift-service-ca-bundle\") pod \"sequence-graph-f025b-545b74b6c4-lcc6q\" (UID: \"109a6ec8-78d3-45bf-a316-fce6791c6399\") " pod="kserve-ci-e2e-test/sequence-graph-f025b-545b74b6c4-lcc6q"
Sep 16 19:13:06.327526 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:13:06.327499    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-f025b-545b74b6c4-lcc6q"]
Sep 16 19:13:06.424598 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:13:06.424557    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/109a6ec8-78d3-45bf-a316-fce6791c6399-openshift-service-ca-bundle\") pod \"sequence-graph-f025b-545b74b6c4-lcc6q\" (UID: \"109a6ec8-78d3-45bf-a316-fce6791c6399\") " pod="kserve-ci-e2e-test/sequence-graph-f025b-545b74b6c4-lcc6q"
Sep 16 19:13:06.424802 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:13:06.424641    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/109a6ec8-78d3-45bf-a316-fce6791c6399-proxy-tls\") pod \"sequence-graph-f025b-545b74b6c4-lcc6q\" (UID: \"109a6ec8-78d3-45bf-a316-fce6791c6399\") " pod="kserve-ci-e2e-test/sequence-graph-f025b-545b74b6c4-lcc6q"
Sep 16 19:13:06.425299 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:13:06.425269    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/109a6ec8-78d3-45bf-a316-fce6791c6399-openshift-service-ca-bundle\") pod \"sequence-graph-f025b-545b74b6c4-lcc6q\" (UID: \"109a6ec8-78d3-45bf-a316-fce6791c6399\") " pod="kserve-ci-e2e-test/sequence-graph-f025b-545b74b6c4-lcc6q"
Sep 16 19:13:06.427089 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:13:06.427067    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/109a6ec8-78d3-45bf-a316-fce6791c6399-proxy-tls\") pod \"sequence-graph-f025b-545b74b6c4-lcc6q\" (UID: \"109a6ec8-78d3-45bf-a316-fce6791c6399\") " pod="kserve-ci-e2e-test/sequence-graph-f025b-545b74b6c4-lcc6q"
Sep 16 19:13:06.441190 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:13:06.441153    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 19:13:06.616923 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:13:06.616795    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-f025b-545b74b6c4-lcc6q"
Sep 16 19:13:06.891671 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:13:06.891645    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-f025b-545b74b6c4-lcc6q"]
Sep 16 19:13:06.894453 ip-10-0-139-182 kubenswrapper[2584]: W0916 19:13:06.894421    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod109a6ec8_78d3_45bf_a316_fce6791c6399.slice/crio-f5ea4b28107df951d25f5ab1c49d141e96a5a5d6dc43e6db56296159255af49c WatchSource:0}: Error finding container f5ea4b28107df951d25f5ab1c49d141e96a5a5d6dc43e6db56296159255af49c: Status 404 returned error can't find the container with id f5ea4b28107df951d25f5ab1c49d141e96a5a5d6dc43e6db56296159255af49c
Sep 16 19:13:07.192588 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:13:07.192494    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-f025b-545b74b6c4-lcc6q" event={"ID":"109a6ec8-78d3-45bf-a316-fce6791c6399","Type":"ContainerStarted","Data":"06bd1081638e02ff81f947b090944079bbc73ea4f25b664d8850b396241a7143"}
Sep 16 19:13:07.192588 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:13:07.192548    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-f025b-545b74b6c4-lcc6q" event={"ID":"109a6ec8-78d3-45bf-a316-fce6791c6399","Type":"ContainerStarted","Data":"f5ea4b28107df951d25f5ab1c49d141e96a5a5d6dc43e6db56296159255af49c"}
Sep 16 19:13:07.192787 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:13:07.192605    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/sequence-graph-f025b-545b74b6c4-lcc6q"
Sep 16 19:13:07.209622 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:13:07.209565    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/sequence-graph-f025b-545b74b6c4-lcc6q" podStartSLOduration=1.209545692 podStartE2EDuration="1.209545692s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 19:13:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 19:13:07.20853738 +0000 UTC m=+1717.538039794" watchObservedRunningTime="2026-09-16 19:13:07.209545692 +0000 UTC m=+1717.539048105"
Sep 16 19:13:13.203229 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:13:13.203196    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/sequence-graph-f025b-545b74b6c4-lcc6q"
Sep 16 19:14:36.156200 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:14:36.156081    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-crjw7_ecf273b5-b465-4008-b035-f7ac0e740d58/console-operator/1.log"
Sep 16 19:14:36.159237 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:14:36.159203    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-crjw7_ecf273b5-b465-4008-b035-f7ac0e740d58/console-operator/1.log"
Sep 16 19:14:56.076922 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:14:56.076883    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 19:16:18.362485 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:16:18.362398    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 19:18:33.499320 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:18:33.499281    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 19:19:36.186363 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:19:36.186221    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-crjw7_ecf273b5-b465-4008-b035-f7ac0e740d58/console-operator/1.log"
Sep 16 19:19:36.190250 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:19:36.189759    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-crjw7_ecf273b5-b465-4008-b035-f7ac0e740d58/console-operator/1.log"
Sep 16 19:19:51.301084 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:19:51.301036    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 19:20:41.267670 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:20:41.267635    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-6b79a-556b7955dc-qxqq4"]
Sep 16 19:20:41.268354 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:20:41.267824    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/ensemble-graph-6b79a-556b7955dc-qxqq4" podUID="caff3c6c-3daf-4931-8da5-40aea8ef097e" containerName="ensemble-graph-6b79a" containerID="cri-o://c378ca7e393d558c6b2819fa1048e960ebd9f3911a45f456872cbf3f6a6ec0f7" gracePeriod=30
Sep 16 19:20:41.321484 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:20:41.321451    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 19:20:41.457391 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:20:41.457356    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-6b79a-predictor-544dbf6f99-vrqwv"]
Sep 16 19:20:41.457621 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:20:41.457582    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-6b79a-predictor-544dbf6f99-vrqwv" podUID="f16ee8f1-b4a8-4e2d-9b6c-add6833c8c4b" containerName="kserve-container" containerID="cri-o://b19aa6eb96c5063d18efe2c6beaa3e8d7fe427ba06a0ca2973456ba3e4781722" gracePeriod=30
Sep 16 19:20:41.457707 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:20:41.457629    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-6b79a-predictor-544dbf6f99-vrqwv" podUID="f16ee8f1-b4a8-4e2d-9b6c-add6833c8c4b" containerName="kube-rbac-proxy" containerID="cri-o://0f95730474301429a0498f2772bc8a68489ae158c1ab6ebc88962565886b635b" gracePeriod=30
Sep 16 19:20:41.587035 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:20:41.587002    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-d799f-predictor-6f68cb5f6c-dhrft"]
Sep 16 19:20:41.590808 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:20:41.590784    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-d799f-predictor-6f68cb5f6c-dhrft"
Sep 16 19:20:41.595380 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:20:41.595347    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-d799f-kube-rbac-proxy-sar-config\""
Sep 16 19:20:41.595380 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:20:41.595347    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-d799f-predictor-serving-cert\""
Sep 16 19:20:41.607945 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:20:41.607916    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-d799f-predictor-6f68cb5f6c-dhrft"]
Sep 16 19:20:41.673570 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:20:41.673539    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jcdrm\" (UniqueName: \"kubernetes.io/projected/6785a5af-ba5e-4e89-b9ca-0fdaecfc54bd-kube-api-access-jcdrm\") pod \"success-200-isvc-d799f-predictor-6f68cb5f6c-dhrft\" (UID: \"6785a5af-ba5e-4e89-b9ca-0fdaecfc54bd\") " pod="kserve-ci-e2e-test/success-200-isvc-d799f-predictor-6f68cb5f6c-dhrft"
Sep 16 19:20:41.673570 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:20:41.673579    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/6785a5af-ba5e-4e89-b9ca-0fdaecfc54bd-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-d799f-predictor-6f68cb5f6c-dhrft\" (UID: \"6785a5af-ba5e-4e89-b9ca-0fdaecfc54bd\") " pod="kserve-ci-e2e-test/success-200-isvc-d799f-predictor-6f68cb5f6c-dhrft"
Sep 16 19:20:41.673786 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:20:41.673650    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/6785a5af-ba5e-4e89-b9ca-0fdaecfc54bd-proxy-tls\") pod \"success-200-isvc-d799f-predictor-6f68cb5f6c-dhrft\" (UID: \"6785a5af-ba5e-4e89-b9ca-0fdaecfc54bd\") " pod="kserve-ci-e2e-test/success-200-isvc-d799f-predictor-6f68cb5f6c-dhrft"
Sep 16 19:20:41.774876 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:20:41.774822    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jcdrm\" (UniqueName: \"kubernetes.io/projected/6785a5af-ba5e-4e89-b9ca-0fdaecfc54bd-kube-api-access-jcdrm\") pod \"success-200-isvc-d799f-predictor-6f68cb5f6c-dhrft\" (UID: \"6785a5af-ba5e-4e89-b9ca-0fdaecfc54bd\") " pod="kserve-ci-e2e-test/success-200-isvc-d799f-predictor-6f68cb5f6c-dhrft"
Sep 16 19:20:41.775058 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:20:41.774896    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/6785a5af-ba5e-4e89-b9ca-0fdaecfc54bd-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-d799f-predictor-6f68cb5f6c-dhrft\" (UID: \"6785a5af-ba5e-4e89-b9ca-0fdaecfc54bd\") " pod="kserve-ci-e2e-test/success-200-isvc-d799f-predictor-6f68cb5f6c-dhrft"
Sep 16 19:20:41.775058 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:20:41.774952    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/6785a5af-ba5e-4e89-b9ca-0fdaecfc54bd-proxy-tls\") pod \"success-200-isvc-d799f-predictor-6f68cb5f6c-dhrft\" (UID: \"6785a5af-ba5e-4e89-b9ca-0fdaecfc54bd\") " pod="kserve-ci-e2e-test/success-200-isvc-d799f-predictor-6f68cb5f6c-dhrft"
Sep 16 19:20:41.775144 ip-10-0-139-182 kubenswrapper[2584]: E0916 19:20:41.775063    2584 secret.go:189] Couldn't get secret kserve-ci-e2e-test/success-200-isvc-d799f-predictor-serving-cert: secret "success-200-isvc-d799f-predictor-serving-cert" not found
Sep 16 19:20:41.775144 ip-10-0-139-182 kubenswrapper[2584]: E0916 19:20:41.775141    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6785a5af-ba5e-4e89-b9ca-0fdaecfc54bd-proxy-tls podName:6785a5af-ba5e-4e89-b9ca-0fdaecfc54bd nodeName:}" failed. No retries permitted until 2026-09-16 19:20:42.275119679 +0000 UTC m=+2172.604622086 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/6785a5af-ba5e-4e89-b9ca-0fdaecfc54bd-proxy-tls") pod "success-200-isvc-d799f-predictor-6f68cb5f6c-dhrft" (UID: "6785a5af-ba5e-4e89-b9ca-0fdaecfc54bd") : secret "success-200-isvc-d799f-predictor-serving-cert" not found
Sep 16 19:20:41.775525 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:20:41.775504    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/6785a5af-ba5e-4e89-b9ca-0fdaecfc54bd-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-d799f-predictor-6f68cb5f6c-dhrft\" (UID: \"6785a5af-ba5e-4e89-b9ca-0fdaecfc54bd\") " pod="kserve-ci-e2e-test/success-200-isvc-d799f-predictor-6f68cb5f6c-dhrft"
Sep 16 19:20:41.794951 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:20:41.794905    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jcdrm\" (UniqueName: \"kubernetes.io/projected/6785a5af-ba5e-4e89-b9ca-0fdaecfc54bd-kube-api-access-jcdrm\") pod \"success-200-isvc-d799f-predictor-6f68cb5f6c-dhrft\" (UID: \"6785a5af-ba5e-4e89-b9ca-0fdaecfc54bd\") " pod="kserve-ci-e2e-test/success-200-isvc-d799f-predictor-6f68cb5f6c-dhrft"
Sep 16 19:20:42.261055 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:20:42.261019    2584 generic.go:356] "Generic (PLEG): container finished" podID="f16ee8f1-b4a8-4e2d-9b6c-add6833c8c4b" containerID="0f95730474301429a0498f2772bc8a68489ae158c1ab6ebc88962565886b635b" exitCode=2
Sep 16 19:20:42.261223 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:20:42.261092    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-6b79a-predictor-544dbf6f99-vrqwv" event={"ID":"f16ee8f1-b4a8-4e2d-9b6c-add6833c8c4b","Type":"ContainerDied","Data":"0f95730474301429a0498f2772bc8a68489ae158c1ab6ebc88962565886b635b"}
Sep 16 19:20:42.279618 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:20:42.279587    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/6785a5af-ba5e-4e89-b9ca-0fdaecfc54bd-proxy-tls\") pod \"success-200-isvc-d799f-predictor-6f68cb5f6c-dhrft\" (UID: \"6785a5af-ba5e-4e89-b9ca-0fdaecfc54bd\") " pod="kserve-ci-e2e-test/success-200-isvc-d799f-predictor-6f68cb5f6c-dhrft"
Sep 16 19:20:42.282016 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:20:42.281984    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/6785a5af-ba5e-4e89-b9ca-0fdaecfc54bd-proxy-tls\") pod \"success-200-isvc-d799f-predictor-6f68cb5f6c-dhrft\" (UID: \"6785a5af-ba5e-4e89-b9ca-0fdaecfc54bd\") " pod="kserve-ci-e2e-test/success-200-isvc-d799f-predictor-6f68cb5f6c-dhrft"
Sep 16 19:20:42.505781 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:20:42.505726    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-d799f-predictor-6f68cb5f6c-dhrft"
Sep 16 19:20:42.617356 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:20:42.617303    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 19:20:42.782024 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:20:42.781939    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-d799f-predictor-6f68cb5f6c-dhrft"]
Sep 16 19:20:42.784410 ip-10-0-139-182 kubenswrapper[2584]: W0916 19:20:42.784383    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6785a5af_ba5e_4e89_b9ca_0fdaecfc54bd.slice/crio-76db3b4b9e304718e090dcb8866cf5c02632dcc2f515a47cd8f40272e79c9ebf WatchSource:0}: Error finding container 76db3b4b9e304718e090dcb8866cf5c02632dcc2f515a47cd8f40272e79c9ebf: Status 404 returned error can't find the container with id 76db3b4b9e304718e090dcb8866cf5c02632dcc2f515a47cd8f40272e79c9ebf
Sep 16 19:20:43.267868 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:20:43.267813    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-d799f-predictor-6f68cb5f6c-dhrft" event={"ID":"6785a5af-ba5e-4e89-b9ca-0fdaecfc54bd","Type":"ContainerStarted","Data":"df997c1dcb2ce9c839d8e9b43c63a6aeb8edbd2bec2549204df63f70e1f09cc6"}
Sep 16 19:20:43.268061 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:20:43.267894    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-d799f-predictor-6f68cb5f6c-dhrft" event={"ID":"6785a5af-ba5e-4e89-b9ca-0fdaecfc54bd","Type":"ContainerStarted","Data":"8fecde307df998cc6dbb5dec102a5afcddba2bdad66844b9d24e56b92b238624"}
Sep 16 19:20:43.268061 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:20:43.267915    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-d799f-predictor-6f68cb5f6c-dhrft" event={"ID":"6785a5af-ba5e-4e89-b9ca-0fdaecfc54bd","Type":"ContainerStarted","Data":"76db3b4b9e304718e090dcb8866cf5c02632dcc2f515a47cd8f40272e79c9ebf"}
Sep 16 19:20:43.268061 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:20:43.267937    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-d799f-predictor-6f68cb5f6c-dhrft"
Sep 16 19:20:43.295775 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:20:43.292567    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-d799f-predictor-6f68cb5f6c-dhrft" podStartSLOduration=2.292547922 podStartE2EDuration="2.292547922s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 19:20:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 19:20:43.289979401 +0000 UTC m=+2173.619481815" watchObservedRunningTime="2026-09-16 19:20:43.292547922 +0000 UTC m=+2173.622050339"
Sep 16 19:20:44.019442 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:20:44.019401    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-6b79a-556b7955dc-qxqq4" podUID="caff3c6c-3daf-4931-8da5-40aea8ef097e" containerName="ensemble-graph-6b79a" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 16 19:20:44.272958 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:20:44.272869    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-d799f-predictor-6f68cb5f6c-dhrft"
Sep 16 19:20:44.274366 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:20:44.274335    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-d799f-predictor-6f68cb5f6c-dhrft" podUID="6785a5af-ba5e-4e89-b9ca-0fdaecfc54bd" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.47:8080: connect: connection refused"
Sep 16 19:20:44.710734 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:20:44.710707    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-6b79a-predictor-544dbf6f99-vrqwv"
Sep 16 19:20:44.799661 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:20:44.799560    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f16ee8f1-b4a8-4e2d-9b6c-add6833c8c4b-proxy-tls\") pod \"f16ee8f1-b4a8-4e2d-9b6c-add6833c8c4b\" (UID: \"f16ee8f1-b4a8-4e2d-9b6c-add6833c8c4b\") "
Sep 16 19:20:44.799661 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:20:44.799645    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7cwt8\" (UniqueName: \"kubernetes.io/projected/f16ee8f1-b4a8-4e2d-9b6c-add6833c8c4b-kube-api-access-7cwt8\") pod \"f16ee8f1-b4a8-4e2d-9b6c-add6833c8c4b\" (UID: \"f16ee8f1-b4a8-4e2d-9b6c-add6833c8c4b\") "
Sep 16 19:20:44.799930 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:20:44.799681    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/f16ee8f1-b4a8-4e2d-9b6c-add6833c8c4b-kube-rbac-proxy-sar-config\") pod \"f16ee8f1-b4a8-4e2d-9b6c-add6833c8c4b\" (UID: \"f16ee8f1-b4a8-4e2d-9b6c-add6833c8c4b\") "
Sep 16 19:20:44.800142 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:20:44.800110    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f16ee8f1-b4a8-4e2d-9b6c-add6833c8c4b-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "f16ee8f1-b4a8-4e2d-9b6c-add6833c8c4b" (UID: "f16ee8f1-b4a8-4e2d-9b6c-add6833c8c4b"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 19:20:44.801749 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:20:44.801719    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f16ee8f1-b4a8-4e2d-9b6c-add6833c8c4b-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "f16ee8f1-b4a8-4e2d-9b6c-add6833c8c4b" (UID: "f16ee8f1-b4a8-4e2d-9b6c-add6833c8c4b"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 19:20:44.801855 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:20:44.801759    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f16ee8f1-b4a8-4e2d-9b6c-add6833c8c4b-kube-api-access-7cwt8" (OuterVolumeSpecName: "kube-api-access-7cwt8") pod "f16ee8f1-b4a8-4e2d-9b6c-add6833c8c4b" (UID: "f16ee8f1-b4a8-4e2d-9b6c-add6833c8c4b"). InnerVolumeSpecName "kube-api-access-7cwt8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 19:20:44.900618 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:20:44.900576    2584 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f16ee8f1-b4a8-4e2d-9b6c-add6833c8c4b-proxy-tls\") on node \"ip-10-0-139-182.ec2.internal\" DevicePath \"\""
Sep 16 19:20:44.900618 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:20:44.900612    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7cwt8\" (UniqueName: \"kubernetes.io/projected/f16ee8f1-b4a8-4e2d-9b6c-add6833c8c4b-kube-api-access-7cwt8\") on node \"ip-10-0-139-182.ec2.internal\" DevicePath \"\""
Sep 16 19:20:44.900618 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:20:44.900622    2584 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/f16ee8f1-b4a8-4e2d-9b6c-add6833c8c4b-kube-rbac-proxy-sar-config\") on node \"ip-10-0-139-182.ec2.internal\" DevicePath \"\""
Sep 16 19:20:45.282195 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:20:45.282157    2584 generic.go:356] "Generic (PLEG): container finished" podID="f16ee8f1-b4a8-4e2d-9b6c-add6833c8c4b" containerID="b19aa6eb96c5063d18efe2c6beaa3e8d7fe427ba06a0ca2973456ba3e4781722" exitCode=0
Sep 16 19:20:45.282404 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:20:45.282216    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-6b79a-predictor-544dbf6f99-vrqwv"
Sep 16 19:20:45.282404 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:20:45.282235    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-6b79a-predictor-544dbf6f99-vrqwv" event={"ID":"f16ee8f1-b4a8-4e2d-9b6c-add6833c8c4b","Type":"ContainerDied","Data":"b19aa6eb96c5063d18efe2c6beaa3e8d7fe427ba06a0ca2973456ba3e4781722"}
Sep 16 19:20:45.282404 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:20:45.282288    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-6b79a-predictor-544dbf6f99-vrqwv" event={"ID":"f16ee8f1-b4a8-4e2d-9b6c-add6833c8c4b","Type":"ContainerDied","Data":"16cabab0329a68a325854110a78d2f0bedc17aae49dcbced028ec668b23de074"}
Sep 16 19:20:45.282404 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:20:45.282314    2584 scope.go:117] "RemoveContainer" containerID="0f95730474301429a0498f2772bc8a68489ae158c1ab6ebc88962565886b635b"
Sep 16 19:20:45.283019 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:20:45.282979    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-d799f-predictor-6f68cb5f6c-dhrft" podUID="6785a5af-ba5e-4e89-b9ca-0fdaecfc54bd" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.47:8080: connect: connection refused"
Sep 16 19:20:45.291339 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:20:45.291317    2584 scope.go:117] "RemoveContainer" containerID="b19aa6eb96c5063d18efe2c6beaa3e8d7fe427ba06a0ca2973456ba3e4781722"
Sep 16 19:20:45.298914 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:20:45.298894    2584 scope.go:117] "RemoveContainer" containerID="0f95730474301429a0498f2772bc8a68489ae158c1ab6ebc88962565886b635b"
Sep 16 19:20:45.299206 ip-10-0-139-182 kubenswrapper[2584]: E0916 19:20:45.299188    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0f95730474301429a0498f2772bc8a68489ae158c1ab6ebc88962565886b635b\": container with ID starting with 0f95730474301429a0498f2772bc8a68489ae158c1ab6ebc88962565886b635b not found: ID does not exist" containerID="0f95730474301429a0498f2772bc8a68489ae158c1ab6ebc88962565886b635b"
Sep 16 19:20:45.299247 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:20:45.299216    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0f95730474301429a0498f2772bc8a68489ae158c1ab6ebc88962565886b635b"} err="failed to get container status \"0f95730474301429a0498f2772bc8a68489ae158c1ab6ebc88962565886b635b\": rpc error: code = NotFound desc = could not find container \"0f95730474301429a0498f2772bc8a68489ae158c1ab6ebc88962565886b635b\": container with ID starting with 0f95730474301429a0498f2772bc8a68489ae158c1ab6ebc88962565886b635b not found: ID does not exist"
Sep 16 19:20:45.299247 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:20:45.299234    2584 scope.go:117] "RemoveContainer" containerID="b19aa6eb96c5063d18efe2c6beaa3e8d7fe427ba06a0ca2973456ba3e4781722"
Sep 16 19:20:45.299508 ip-10-0-139-182 kubenswrapper[2584]: E0916 19:20:45.299491    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b19aa6eb96c5063d18efe2c6beaa3e8d7fe427ba06a0ca2973456ba3e4781722\": container with ID starting with b19aa6eb96c5063d18efe2c6beaa3e8d7fe427ba06a0ca2973456ba3e4781722 not found: ID does not exist" containerID="b19aa6eb96c5063d18efe2c6beaa3e8d7fe427ba06a0ca2973456ba3e4781722"
Sep 16 19:20:45.299546 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:20:45.299515    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b19aa6eb96c5063d18efe2c6beaa3e8d7fe427ba06a0ca2973456ba3e4781722"} err="failed to get container status \"b19aa6eb96c5063d18efe2c6beaa3e8d7fe427ba06a0ca2973456ba3e4781722\": rpc error: code = NotFound desc = could not find container \"b19aa6eb96c5063d18efe2c6beaa3e8d7fe427ba06a0ca2973456ba3e4781722\": container with ID starting with b19aa6eb96c5063d18efe2c6beaa3e8d7fe427ba06a0ca2973456ba3e4781722 not found: ID does not exist"
Sep 16 19:20:45.304493 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:20:45.304467    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-6b79a-predictor-544dbf6f99-vrqwv"]
Sep 16 19:20:45.308464 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:20:45.308440    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-6b79a-predictor-544dbf6f99-vrqwv"]
Sep 16 19:20:46.115789 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:20:46.115760    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f16ee8f1-b4a8-4e2d-9b6c-add6833c8c4b" path="/var/lib/kubelet/pods/f16ee8f1-b4a8-4e2d-9b6c-add6833c8c4b/volumes"
Sep 16 19:20:49.020075 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:20:49.020030    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-6b79a-556b7955dc-qxqq4" podUID="caff3c6c-3daf-4931-8da5-40aea8ef097e" containerName="ensemble-graph-6b79a" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 16 19:20:50.286856 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:20:50.286816    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-d799f-predictor-6f68cb5f6c-dhrft"
Sep 16 19:20:50.287468 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:20:50.287443    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-d799f-predictor-6f68cb5f6c-dhrft" podUID="6785a5af-ba5e-4e89-b9ca-0fdaecfc54bd" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.47:8080: connect: connection refused"
Sep 16 19:20:51.128537 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:20:51.128474    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 19:20:54.019725 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:20:54.019684    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-6b79a-556b7955dc-qxqq4" podUID="caff3c6c-3daf-4931-8da5-40aea8ef097e" containerName="ensemble-graph-6b79a" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 16 19:20:54.020236 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:20:54.019781    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/ensemble-graph-6b79a-556b7955dc-qxqq4"
Sep 16 19:20:59.019443 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:20:59.019400    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-6b79a-556b7955dc-qxqq4" podUID="caff3c6c-3daf-4931-8da5-40aea8ef097e" containerName="ensemble-graph-6b79a" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 16 19:21:00.287443 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:00.287398    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-d799f-predictor-6f68cb5f6c-dhrft" podUID="6785a5af-ba5e-4e89-b9ca-0fdaecfc54bd" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.47:8080: connect: connection refused"
Sep 16 19:21:04.019861 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:04.019801    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-6b79a-556b7955dc-qxqq4" podUID="caff3c6c-3daf-4931-8da5-40aea8ef097e" containerName="ensemble-graph-6b79a" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 16 19:21:09.020257 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:09.020215    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-6b79a-556b7955dc-qxqq4" podUID="caff3c6c-3daf-4931-8da5-40aea8ef097e" containerName="ensemble-graph-6b79a" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 16 19:21:10.287644 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:10.287603    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-d799f-predictor-6f68cb5f6c-dhrft" podUID="6785a5af-ba5e-4e89-b9ca-0fdaecfc54bd" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.47:8080: connect: connection refused"
Sep 16 19:21:11.404163 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:11.404122    2584 generic.go:356] "Generic (PLEG): container finished" podID="caff3c6c-3daf-4931-8da5-40aea8ef097e" containerID="c378ca7e393d558c6b2819fa1048e960ebd9f3911a45f456872cbf3f6a6ec0f7" exitCode=0
Sep 16 19:21:11.404163 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:11.404161    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-6b79a-556b7955dc-qxqq4" event={"ID":"caff3c6c-3daf-4931-8da5-40aea8ef097e","Type":"ContainerDied","Data":"c378ca7e393d558c6b2819fa1048e960ebd9f3911a45f456872cbf3f6a6ec0f7"}
Sep 16 19:21:11.913383 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:11.913356    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-6b79a-556b7955dc-qxqq4"
Sep 16 19:21:12.039784 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:12.039695    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/caff3c6c-3daf-4931-8da5-40aea8ef097e-openshift-service-ca-bundle\") pod \"caff3c6c-3daf-4931-8da5-40aea8ef097e\" (UID: \"caff3c6c-3daf-4931-8da5-40aea8ef097e\") "
Sep 16 19:21:12.040063 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:12.039851    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/caff3c6c-3daf-4931-8da5-40aea8ef097e-proxy-tls\") pod \"caff3c6c-3daf-4931-8da5-40aea8ef097e\" (UID: \"caff3c6c-3daf-4931-8da5-40aea8ef097e\") "
Sep 16 19:21:12.040133 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:12.040078    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/caff3c6c-3daf-4931-8da5-40aea8ef097e-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "caff3c6c-3daf-4931-8da5-40aea8ef097e" (UID: "caff3c6c-3daf-4931-8da5-40aea8ef097e"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 19:21:12.041817 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:12.041791    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/caff3c6c-3daf-4931-8da5-40aea8ef097e-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "caff3c6c-3daf-4931-8da5-40aea8ef097e" (UID: "caff3c6c-3daf-4931-8da5-40aea8ef097e"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 19:21:12.141282 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:12.141254    2584 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/caff3c6c-3daf-4931-8da5-40aea8ef097e-proxy-tls\") on node \"ip-10-0-139-182.ec2.internal\" DevicePath \"\""
Sep 16 19:21:12.141282 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:12.141281    2584 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/caff3c6c-3daf-4931-8da5-40aea8ef097e-openshift-service-ca-bundle\") on node \"ip-10-0-139-182.ec2.internal\" DevicePath \"\""
Sep 16 19:21:12.410555 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:12.410473    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-6b79a-556b7955dc-qxqq4"
Sep 16 19:21:12.410555 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:12.410500    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-6b79a-556b7955dc-qxqq4" event={"ID":"caff3c6c-3daf-4931-8da5-40aea8ef097e","Type":"ContainerDied","Data":"b63dc398f34b54292445aac6911ab7a5b500223e7144073968e2b777ce562366"}
Sep 16 19:21:12.411040 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:12.410563    2584 scope.go:117] "RemoveContainer" containerID="c378ca7e393d558c6b2819fa1048e960ebd9f3911a45f456872cbf3f6a6ec0f7"
Sep 16 19:21:12.428992 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:12.428963    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-6b79a-556b7955dc-qxqq4"]
Sep 16 19:21:12.433315 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:12.433294    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-6b79a-556b7955dc-qxqq4"]
Sep 16 19:21:14.115774 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:14.115738    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="caff3c6c-3daf-4931-8da5-40aea8ef097e" path="/var/lib/kubelet/pods/caff3c6c-3daf-4931-8da5-40aea8ef097e/volumes"
Sep 16 19:21:20.287574 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:20.287535    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-d799f-predictor-6f68cb5f6c-dhrft" podUID="6785a5af-ba5e-4e89-b9ca-0fdaecfc54bd" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.47:8080: connect: connection refused"
Sep 16 19:21:20.959794 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:20.959759    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-f025b-545b74b6c4-lcc6q"]
Sep 16 19:21:20.960009 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:20.959987    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/sequence-graph-f025b-545b74b6c4-lcc6q" podUID="109a6ec8-78d3-45bf-a316-fce6791c6399" containerName="sequence-graph-f025b" containerID="cri-o://06bd1081638e02ff81f947b090944079bbc73ea4f25b664d8850b396241a7143" gracePeriod=30
Sep 16 19:21:21.023228 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:21.023194    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 19:21:21.164496 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:21.164419    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 19:21:21.173355 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:21.173315    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-f025b-predictor-68bc64675f-hthl4"]
Sep 16 19:21:21.173618 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:21.173592    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-f025b-predictor-68bc64675f-hthl4" podUID="fd98557c-48e9-4481-a395-afffbf0448a5" containerName="kserve-container" containerID="cri-o://73bc102d545f611b800f1935e904bb3208ff55e61da508d74239fdcbca7818b0" gracePeriod=30
Sep 16 19:21:21.173684 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:21.173601    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-f025b-predictor-68bc64675f-hthl4" podUID="fd98557c-48e9-4481-a395-afffbf0448a5" containerName="kube-rbac-proxy" containerID="cri-o://1fae9a3227bcb4101df39ec54e2e5fc7bb29981f9477e5bd6e198acf9fbdaa9f" gracePeriod=30
Sep 16 19:21:21.273454 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:21.273418    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 19:21:21.306967 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:21.306931    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-0263c-predictor-7bfb46579c-n6fgr"]
Sep 16 19:21:21.307497 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:21.307477    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f16ee8f1-b4a8-4e2d-9b6c-add6833c8c4b" containerName="kube-rbac-proxy"
Sep 16 19:21:21.307541 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:21.307501    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="f16ee8f1-b4a8-4e2d-9b6c-add6833c8c4b" containerName="kube-rbac-proxy"
Sep 16 19:21:21.307541 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:21.307532    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f16ee8f1-b4a8-4e2d-9b6c-add6833c8c4b" containerName="kserve-container"
Sep 16 19:21:21.307604 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:21.307542    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="f16ee8f1-b4a8-4e2d-9b6c-add6833c8c4b" containerName="kserve-container"
Sep 16 19:21:21.307604 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:21.307555    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="caff3c6c-3daf-4931-8da5-40aea8ef097e" containerName="ensemble-graph-6b79a"
Sep 16 19:21:21.307604 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:21.307564    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="caff3c6c-3daf-4931-8da5-40aea8ef097e" containerName="ensemble-graph-6b79a"
Sep 16 19:21:21.307697 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:21.307657    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="f16ee8f1-b4a8-4e2d-9b6c-add6833c8c4b" containerName="kserve-container"
Sep 16 19:21:21.307697 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:21.307670    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="caff3c6c-3daf-4931-8da5-40aea8ef097e" containerName="ensemble-graph-6b79a"
Sep 16 19:21:21.307697 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:21.307681    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="f16ee8f1-b4a8-4e2d-9b6c-add6833c8c4b" containerName="kube-rbac-proxy"
Sep 16 19:21:21.311244 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:21.311225    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-0263c-predictor-7bfb46579c-n6fgr"
Sep 16 19:21:21.315803 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:21.315781    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-0263c-predictor-serving-cert\""
Sep 16 19:21:21.315951 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:21.315933    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-0263c-kube-rbac-proxy-sar-config\""
Sep 16 19:21:21.336319 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:21.336292    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-0263c-predictor-7bfb46579c-n6fgr"]
Sep 16 19:21:21.403421 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:21.403374    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 19:21:21.418310 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:21.418281    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/cb9ea1f8-55c4-49d4-8699-a23e889dd5fb-proxy-tls\") pod \"success-200-isvc-0263c-predictor-7bfb46579c-n6fgr\" (UID: \"cb9ea1f8-55c4-49d4-8699-a23e889dd5fb\") " pod="kserve-ci-e2e-test/success-200-isvc-0263c-predictor-7bfb46579c-n6fgr"
Sep 16 19:21:21.418466 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:21.418327    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/cb9ea1f8-55c4-49d4-8699-a23e889dd5fb-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-0263c-predictor-7bfb46579c-n6fgr\" (UID: \"cb9ea1f8-55c4-49d4-8699-a23e889dd5fb\") " pod="kserve-ci-e2e-test/success-200-isvc-0263c-predictor-7bfb46579c-n6fgr"
Sep 16 19:21:21.418466 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:21.418407    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7hzbl\" (UniqueName: \"kubernetes.io/projected/cb9ea1f8-55c4-49d4-8699-a23e889dd5fb-kube-api-access-7hzbl\") pod \"success-200-isvc-0263c-predictor-7bfb46579c-n6fgr\" (UID: \"cb9ea1f8-55c4-49d4-8699-a23e889dd5fb\") " pod="kserve-ci-e2e-test/success-200-isvc-0263c-predictor-7bfb46579c-n6fgr"
Sep 16 19:21:21.455182 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:21.455144    2584 generic.go:356] "Generic (PLEG): container finished" podID="fd98557c-48e9-4481-a395-afffbf0448a5" containerID="1fae9a3227bcb4101df39ec54e2e5fc7bb29981f9477e5bd6e198acf9fbdaa9f" exitCode=2
Sep 16 19:21:21.455345 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:21.455219    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-f025b-predictor-68bc64675f-hthl4" event={"ID":"fd98557c-48e9-4481-a395-afffbf0448a5","Type":"ContainerDied","Data":"1fae9a3227bcb4101df39ec54e2e5fc7bb29981f9477e5bd6e198acf9fbdaa9f"}
Sep 16 19:21:21.518342 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:21.518306    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 19:21:21.519244 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:21.519223    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/cb9ea1f8-55c4-49d4-8699-a23e889dd5fb-proxy-tls\") pod \"success-200-isvc-0263c-predictor-7bfb46579c-n6fgr\" (UID: \"cb9ea1f8-55c4-49d4-8699-a23e889dd5fb\") " pod="kserve-ci-e2e-test/success-200-isvc-0263c-predictor-7bfb46579c-n6fgr"
Sep 16 19:21:21.519304 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:21.519262    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/cb9ea1f8-55c4-49d4-8699-a23e889dd5fb-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-0263c-predictor-7bfb46579c-n6fgr\" (UID: \"cb9ea1f8-55c4-49d4-8699-a23e889dd5fb\") " pod="kserve-ci-e2e-test/success-200-isvc-0263c-predictor-7bfb46579c-n6fgr"
Sep 16 19:21:21.519304 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:21.519294    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7hzbl\" (UniqueName: \"kubernetes.io/projected/cb9ea1f8-55c4-49d4-8699-a23e889dd5fb-kube-api-access-7hzbl\") pod \"success-200-isvc-0263c-predictor-7bfb46579c-n6fgr\" (UID: \"cb9ea1f8-55c4-49d4-8699-a23e889dd5fb\") " pod="kserve-ci-e2e-test/success-200-isvc-0263c-predictor-7bfb46579c-n6fgr"
Sep 16 19:21:21.519986 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:21.519964    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/cb9ea1f8-55c4-49d4-8699-a23e889dd5fb-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-0263c-predictor-7bfb46579c-n6fgr\" (UID: \"cb9ea1f8-55c4-49d4-8699-a23e889dd5fb\") " pod="kserve-ci-e2e-test/success-200-isvc-0263c-predictor-7bfb46579c-n6fgr"
Sep 16 19:21:21.521713 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:21.521694    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/cb9ea1f8-55c4-49d4-8699-a23e889dd5fb-proxy-tls\") pod \"success-200-isvc-0263c-predictor-7bfb46579c-n6fgr\" (UID: \"cb9ea1f8-55c4-49d4-8699-a23e889dd5fb\") " pod="kserve-ci-e2e-test/success-200-isvc-0263c-predictor-7bfb46579c-n6fgr"
Sep 16 19:21:21.528999 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:21.528943    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7hzbl\" (UniqueName: \"kubernetes.io/projected/cb9ea1f8-55c4-49d4-8699-a23e889dd5fb-kube-api-access-7hzbl\") pod \"success-200-isvc-0263c-predictor-7bfb46579c-n6fgr\" (UID: \"cb9ea1f8-55c4-49d4-8699-a23e889dd5fb\") " pod="kserve-ci-e2e-test/success-200-isvc-0263c-predictor-7bfb46579c-n6fgr"
Sep 16 19:21:21.621336 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:21.621304    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-0263c-predictor-7bfb46579c-n6fgr"
Sep 16 19:21:21.691574 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:21.691534    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 19:21:21.896478 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:21.896453    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-0263c-predictor-7bfb46579c-n6fgr"]
Sep 16 19:21:21.898588 ip-10-0-139-182 kubenswrapper[2584]: W0916 19:21:21.898559    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb9ea1f8_55c4_49d4_8699_a23e889dd5fb.slice/crio-2ea1ae2218ae51208cd9336d66fee719da4c1b5bc80e8806a4243080e7f47094 WatchSource:0}: Error finding container 2ea1ae2218ae51208cd9336d66fee719da4c1b5bc80e8806a4243080e7f47094: Status 404 returned error can't find the container with id 2ea1ae2218ae51208cd9336d66fee719da4c1b5bc80e8806a4243080e7f47094
Sep 16 19:21:22.462804 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:22.462760    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-0263c-predictor-7bfb46579c-n6fgr" event={"ID":"cb9ea1f8-55c4-49d4-8699-a23e889dd5fb","Type":"ContainerStarted","Data":"a269bc4b2c6503b0fe3992b908f6320f268bfb87a77f8ed504534cb88f6164d8"}
Sep 16 19:21:22.463316 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:22.462974    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-0263c-predictor-7bfb46579c-n6fgr"
Sep 16 19:21:22.463316 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:22.463038    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-0263c-predictor-7bfb46579c-n6fgr" event={"ID":"cb9ea1f8-55c4-49d4-8699-a23e889dd5fb","Type":"ContainerStarted","Data":"790d21a48ba7fd6a9b17013a6163a78f7d8179ca1ab6e39136c0187f4bfda729"}
Sep 16 19:21:22.463316 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:22.463064    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-0263c-predictor-7bfb46579c-n6fgr" event={"ID":"cb9ea1f8-55c4-49d4-8699-a23e889dd5fb","Type":"ContainerStarted","Data":"2ea1ae2218ae51208cd9336d66fee719da4c1b5bc80e8806a4243080e7f47094"}
Sep 16 19:21:22.480664 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:22.480613    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-0263c-predictor-7bfb46579c-n6fgr" podStartSLOduration=1.480598033 podStartE2EDuration="1.480598033s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 19:21:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 19:21:22.480246568 +0000 UTC m=+2212.809748982" watchObservedRunningTime="2026-09-16 19:21:22.480598033 +0000 UTC m=+2212.810100447"
Sep 16 19:21:23.201794 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:23.201755    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-f025b-545b74b6c4-lcc6q" podUID="109a6ec8-78d3-45bf-a316-fce6791c6399" containerName="sequence-graph-f025b" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 16 19:21:23.468888 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:23.468794    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-0263c-predictor-7bfb46579c-n6fgr"
Sep 16 19:21:23.470373 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:23.470343    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-0263c-predictor-7bfb46579c-n6fgr" podUID="cb9ea1f8-55c4-49d4-8699-a23e889dd5fb" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.48:8080: connect: connection refused"
Sep 16 19:21:23.915686 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:23.915642    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-f025b-predictor-68bc64675f-hthl4" podUID="fd98557c-48e9-4481-a395-afffbf0448a5" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.44:8643/healthz\": dial tcp 10.133.0.44:8643: connect: connection refused"
Sep 16 19:21:23.920039 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:23.920016    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-f025b-predictor-68bc64675f-hthl4" podUID="fd98557c-48e9-4481-a395-afffbf0448a5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.44:8080: connect: connection refused"
Sep 16 19:21:24.328562 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:24.328531    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-f025b-predictor-68bc64675f-hthl4"
Sep 16 19:21:24.343424 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:24.343392    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/fd98557c-48e9-4481-a395-afffbf0448a5-kube-rbac-proxy-sar-config\") pod \"fd98557c-48e9-4481-a395-afffbf0448a5\" (UID: \"fd98557c-48e9-4481-a395-afffbf0448a5\") "
Sep 16 19:21:24.343569 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:24.343435    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/fd98557c-48e9-4481-a395-afffbf0448a5-proxy-tls\") pod \"fd98557c-48e9-4481-a395-afffbf0448a5\" (UID: \"fd98557c-48e9-4481-a395-afffbf0448a5\") "
Sep 16 19:21:24.343569 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:24.343485    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tqpg8\" (UniqueName: \"kubernetes.io/projected/fd98557c-48e9-4481-a395-afffbf0448a5-kube-api-access-tqpg8\") pod \"fd98557c-48e9-4481-a395-afffbf0448a5\" (UID: \"fd98557c-48e9-4481-a395-afffbf0448a5\") "
Sep 16 19:21:24.343809 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:24.343780    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fd98557c-48e9-4481-a395-afffbf0448a5-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "fd98557c-48e9-4481-a395-afffbf0448a5" (UID: "fd98557c-48e9-4481-a395-afffbf0448a5"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 19:21:24.345641 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:24.345615    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd98557c-48e9-4481-a395-afffbf0448a5-kube-api-access-tqpg8" (OuterVolumeSpecName: "kube-api-access-tqpg8") pod "fd98557c-48e9-4481-a395-afffbf0448a5" (UID: "fd98557c-48e9-4481-a395-afffbf0448a5"). InnerVolumeSpecName "kube-api-access-tqpg8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 19:21:24.345972 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:24.345956    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fd98557c-48e9-4481-a395-afffbf0448a5-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "fd98557c-48e9-4481-a395-afffbf0448a5" (UID: "fd98557c-48e9-4481-a395-afffbf0448a5"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 19:21:24.444717 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:24.444626    2584 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/fd98557c-48e9-4481-a395-afffbf0448a5-kube-rbac-proxy-sar-config\") on node \"ip-10-0-139-182.ec2.internal\" DevicePath \"\""
Sep 16 19:21:24.444717 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:24.444668    2584 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/fd98557c-48e9-4481-a395-afffbf0448a5-proxy-tls\") on node \"ip-10-0-139-182.ec2.internal\" DevicePath \"\""
Sep 16 19:21:24.444717 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:24.444683    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tqpg8\" (UniqueName: \"kubernetes.io/projected/fd98557c-48e9-4481-a395-afffbf0448a5-kube-api-access-tqpg8\") on node \"ip-10-0-139-182.ec2.internal\" DevicePath \"\""
Sep 16 19:21:24.475133 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:24.475097    2584 generic.go:356] "Generic (PLEG): container finished" podID="fd98557c-48e9-4481-a395-afffbf0448a5" containerID="73bc102d545f611b800f1935e904bb3208ff55e61da508d74239fdcbca7818b0" exitCode=0
Sep 16 19:21:24.475580 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:24.475151    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-f025b-predictor-68bc64675f-hthl4"
Sep 16 19:21:24.475580 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:24.475159    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-f025b-predictor-68bc64675f-hthl4" event={"ID":"fd98557c-48e9-4481-a395-afffbf0448a5","Type":"ContainerDied","Data":"73bc102d545f611b800f1935e904bb3208ff55e61da508d74239fdcbca7818b0"}
Sep 16 19:21:24.475580 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:24.475216    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-f025b-predictor-68bc64675f-hthl4" event={"ID":"fd98557c-48e9-4481-a395-afffbf0448a5","Type":"ContainerDied","Data":"e2eb79bf3e2477101a1b77500738cee46e02e0d6272de26465a3ee04f9f3e942"}
Sep 16 19:21:24.475580 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:24.475244    2584 scope.go:117] "RemoveContainer" containerID="1fae9a3227bcb4101df39ec54e2e5fc7bb29981f9477e5bd6e198acf9fbdaa9f"
Sep 16 19:21:24.475808 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:24.475742    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-0263c-predictor-7bfb46579c-n6fgr" podUID="cb9ea1f8-55c4-49d4-8699-a23e889dd5fb" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.48:8080: connect: connection refused"
Sep 16 19:21:24.486724 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:24.486705    2584 scope.go:117] "RemoveContainer" containerID="73bc102d545f611b800f1935e904bb3208ff55e61da508d74239fdcbca7818b0"
Sep 16 19:21:24.494381 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:24.494363    2584 scope.go:117] "RemoveContainer" containerID="1fae9a3227bcb4101df39ec54e2e5fc7bb29981f9477e5bd6e198acf9fbdaa9f"
Sep 16 19:21:24.494647 ip-10-0-139-182 kubenswrapper[2584]: E0916 19:21:24.494628    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1fae9a3227bcb4101df39ec54e2e5fc7bb29981f9477e5bd6e198acf9fbdaa9f\": container with ID starting with 1fae9a3227bcb4101df39ec54e2e5fc7bb29981f9477e5bd6e198acf9fbdaa9f not found: ID does not exist" containerID="1fae9a3227bcb4101df39ec54e2e5fc7bb29981f9477e5bd6e198acf9fbdaa9f"
Sep 16 19:21:24.494726 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:24.494659    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1fae9a3227bcb4101df39ec54e2e5fc7bb29981f9477e5bd6e198acf9fbdaa9f"} err="failed to get container status \"1fae9a3227bcb4101df39ec54e2e5fc7bb29981f9477e5bd6e198acf9fbdaa9f\": rpc error: code = NotFound desc = could not find container \"1fae9a3227bcb4101df39ec54e2e5fc7bb29981f9477e5bd6e198acf9fbdaa9f\": container with ID starting with 1fae9a3227bcb4101df39ec54e2e5fc7bb29981f9477e5bd6e198acf9fbdaa9f not found: ID does not exist"
Sep 16 19:21:24.494726 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:24.494684    2584 scope.go:117] "RemoveContainer" containerID="73bc102d545f611b800f1935e904bb3208ff55e61da508d74239fdcbca7818b0"
Sep 16 19:21:24.494998 ip-10-0-139-182 kubenswrapper[2584]: E0916 19:21:24.494980    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"73bc102d545f611b800f1935e904bb3208ff55e61da508d74239fdcbca7818b0\": container with ID starting with 73bc102d545f611b800f1935e904bb3208ff55e61da508d74239fdcbca7818b0 not found: ID does not exist" containerID="73bc102d545f611b800f1935e904bb3208ff55e61da508d74239fdcbca7818b0"
Sep 16 19:21:24.495036 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:24.495004    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"73bc102d545f611b800f1935e904bb3208ff55e61da508d74239fdcbca7818b0"} err="failed to get container status \"73bc102d545f611b800f1935e904bb3208ff55e61da508d74239fdcbca7818b0\": rpc error: code = NotFound desc = could not find container \"73bc102d545f611b800f1935e904bb3208ff55e61da508d74239fdcbca7818b0\": container with ID starting with 73bc102d545f611b800f1935e904bb3208ff55e61da508d74239fdcbca7818b0 not found: ID does not exist"
Sep 16 19:21:24.500233 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:24.500207    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-f025b-predictor-68bc64675f-hthl4"]
Sep 16 19:21:24.502553 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:24.502526    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-f025b-predictor-68bc64675f-hthl4"]
Sep 16 19:21:26.116406 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:26.116372    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fd98557c-48e9-4481-a395-afffbf0448a5" path="/var/lib/kubelet/pods/fd98557c-48e9-4481-a395-afffbf0448a5/volumes"
Sep 16 19:21:28.201469 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:28.201429    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-f025b-545b74b6c4-lcc6q" podUID="109a6ec8-78d3-45bf-a316-fce6791c6399" containerName="sequence-graph-f025b" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 16 19:21:29.480957 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:29.480923    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-0263c-predictor-7bfb46579c-n6fgr"
Sep 16 19:21:29.481931 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:29.481902    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-0263c-predictor-7bfb46579c-n6fgr" podUID="cb9ea1f8-55c4-49d4-8699-a23e889dd5fb" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.48:8080: connect: connection refused"
Sep 16 19:21:30.288462 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:30.288434    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-d799f-predictor-6f68cb5f6c-dhrft"
Sep 16 19:21:33.201956 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:33.201919    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-f025b-545b74b6c4-lcc6q" podUID="109a6ec8-78d3-45bf-a316-fce6791c6399" containerName="sequence-graph-f025b" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 16 19:21:33.202423 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:33.202020    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/sequence-graph-f025b-545b74b6c4-lcc6q"
Sep 16 19:21:35.736769 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:35.736733    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 19:21:38.202785 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:38.202741    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-f025b-545b74b6c4-lcc6q" podUID="109a6ec8-78d3-45bf-a316-fce6791c6399" containerName="sequence-graph-f025b" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 16 19:21:39.482330 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:39.482290    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-0263c-predictor-7bfb46579c-n6fgr" podUID="cb9ea1f8-55c4-49d4-8699-a23e889dd5fb" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.48:8080: connect: connection refused"
Sep 16 19:21:41.629325 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:41.629295    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/splitter-graph-d799f-778646d678-zw694"]
Sep 16 19:21:41.629737 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:41.629635    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fd98557c-48e9-4481-a395-afffbf0448a5" containerName="kserve-container"
Sep 16 19:21:41.629737 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:41.629646    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="fd98557c-48e9-4481-a395-afffbf0448a5" containerName="kserve-container"
Sep 16 19:21:41.629737 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:41.629653    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fd98557c-48e9-4481-a395-afffbf0448a5" containerName="kube-rbac-proxy"
Sep 16 19:21:41.629737 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:41.629659    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="fd98557c-48e9-4481-a395-afffbf0448a5" containerName="kube-rbac-proxy"
Sep 16 19:21:41.629737 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:41.629713    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="fd98557c-48e9-4481-a395-afffbf0448a5" containerName="kserve-container"
Sep 16 19:21:41.629737 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:41.629722    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="fd98557c-48e9-4481-a395-afffbf0448a5" containerName="kube-rbac-proxy"
Sep 16 19:21:41.632757 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:41.632735    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-d799f-778646d678-zw694"
Sep 16 19:21:41.635656 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:41.635631    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"splitter-graph-d799f-serving-cert\""
Sep 16 19:21:41.635656 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:41.635644    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"splitter-graph-d799f-kube-rbac-proxy-sar-config\""
Sep 16 19:21:41.645608 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:41.645584    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-d799f-778646d678-zw694"]
Sep 16 19:21:41.649957 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:41.649909    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 19:21:41.697816 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:41.697783    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/60354ac8-d364-4367-accd-e65f85cf2bcf-proxy-tls\") pod \"splitter-graph-d799f-778646d678-zw694\" (UID: \"60354ac8-d364-4367-accd-e65f85cf2bcf\") " pod="kserve-ci-e2e-test/splitter-graph-d799f-778646d678-zw694"
Sep 16 19:21:41.697971 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:41.697892    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/60354ac8-d364-4367-accd-e65f85cf2bcf-openshift-service-ca-bundle\") pod \"splitter-graph-d799f-778646d678-zw694\" (UID: \"60354ac8-d364-4367-accd-e65f85cf2bcf\") " pod="kserve-ci-e2e-test/splitter-graph-d799f-778646d678-zw694"
Sep 16 19:21:41.798739 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:41.798711    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/60354ac8-d364-4367-accd-e65f85cf2bcf-openshift-service-ca-bundle\") pod \"splitter-graph-d799f-778646d678-zw694\" (UID: \"60354ac8-d364-4367-accd-e65f85cf2bcf\") " pod="kserve-ci-e2e-test/splitter-graph-d799f-778646d678-zw694"
Sep 16 19:21:41.798960 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:41.798775    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/60354ac8-d364-4367-accd-e65f85cf2bcf-proxy-tls\") pod \"splitter-graph-d799f-778646d678-zw694\" (UID: \"60354ac8-d364-4367-accd-e65f85cf2bcf\") " pod="kserve-ci-e2e-test/splitter-graph-d799f-778646d678-zw694"
Sep 16 19:21:41.798960 ip-10-0-139-182 kubenswrapper[2584]: E0916 19:21:41.798899    2584 secret.go:189] Couldn't get secret kserve-ci-e2e-test/splitter-graph-d799f-serving-cert: secret "splitter-graph-d799f-serving-cert" not found
Sep 16 19:21:41.798960 ip-10-0-139-182 kubenswrapper[2584]: E0916 19:21:41.798956    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60354ac8-d364-4367-accd-e65f85cf2bcf-proxy-tls podName:60354ac8-d364-4367-accd-e65f85cf2bcf nodeName:}" failed. No retries permitted until 2026-09-16 19:21:42.298940156 +0000 UTC m=+2232.628442547 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/60354ac8-d364-4367-accd-e65f85cf2bcf-proxy-tls") pod "splitter-graph-d799f-778646d678-zw694" (UID: "60354ac8-d364-4367-accd-e65f85cf2bcf") : secret "splitter-graph-d799f-serving-cert" not found
Sep 16 19:21:41.799448 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:41.799421    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/60354ac8-d364-4367-accd-e65f85cf2bcf-openshift-service-ca-bundle\") pod \"splitter-graph-d799f-778646d678-zw694\" (UID: \"60354ac8-d364-4367-accd-e65f85cf2bcf\") " pod="kserve-ci-e2e-test/splitter-graph-d799f-778646d678-zw694"
Sep 16 19:21:42.041876 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:42.041817    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 19:21:42.303179 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:42.303085    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/60354ac8-d364-4367-accd-e65f85cf2bcf-proxy-tls\") pod \"splitter-graph-d799f-778646d678-zw694\" (UID: \"60354ac8-d364-4367-accd-e65f85cf2bcf\") " pod="kserve-ci-e2e-test/splitter-graph-d799f-778646d678-zw694"
Sep 16 19:21:42.305507 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:42.305480    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/60354ac8-d364-4367-accd-e65f85cf2bcf-proxy-tls\") pod \"splitter-graph-d799f-778646d678-zw694\" (UID: \"60354ac8-d364-4367-accd-e65f85cf2bcf\") " pod="kserve-ci-e2e-test/splitter-graph-d799f-778646d678-zw694"
Sep 16 19:21:42.542930 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:42.542892    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-d799f-778646d678-zw694"
Sep 16 19:21:42.814507 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:42.814480    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-d799f-778646d678-zw694"]
Sep 16 19:21:42.816577 ip-10-0-139-182 kubenswrapper[2584]: W0916 19:21:42.816547    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60354ac8_d364_4367_accd_e65f85cf2bcf.slice/crio-99bc5da75e60b218ff2cca104b95776d307c03658d0033a725c903aabe7d9244 WatchSource:0}: Error finding container 99bc5da75e60b218ff2cca104b95776d307c03658d0033a725c903aabe7d9244: Status 404 returned error can't find the container with id 99bc5da75e60b218ff2cca104b95776d307c03658d0033a725c903aabe7d9244
Sep 16 19:21:43.202278 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:43.202188    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-f025b-545b74b6c4-lcc6q" podUID="109a6ec8-78d3-45bf-a316-fce6791c6399" containerName="sequence-graph-f025b" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 16 19:21:43.559477 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:43.559438    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-d799f-778646d678-zw694" event={"ID":"60354ac8-d364-4367-accd-e65f85cf2bcf","Type":"ContainerStarted","Data":"ec904d6c6cf3c44bdfb2dd4013b8188eabd8fb63e0267beebb6956ca9e8e83eb"}
Sep 16 19:21:43.559686 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:43.559527    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-d799f-778646d678-zw694" event={"ID":"60354ac8-d364-4367-accd-e65f85cf2bcf","Type":"ContainerStarted","Data":"99bc5da75e60b218ff2cca104b95776d307c03658d0033a725c903aabe7d9244"}
Sep 16 19:21:43.559686 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:43.559582    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/splitter-graph-d799f-778646d678-zw694"
Sep 16 19:21:43.578544 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:43.578487    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/splitter-graph-d799f-778646d678-zw694" podStartSLOduration=2.578474014 podStartE2EDuration="2.578474014s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 19:21:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 19:21:43.577101356 +0000 UTC m=+2233.906603779" watchObservedRunningTime="2026-09-16 19:21:43.578474014 +0000 UTC m=+2233.907976427"
Sep 16 19:21:48.201650 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:48.201613    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-f025b-545b74b6c4-lcc6q" podUID="109a6ec8-78d3-45bf-a316-fce6791c6399" containerName="sequence-graph-f025b" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 16 19:21:49.482484 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:49.482439    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-0263c-predictor-7bfb46579c-n6fgr" podUID="cb9ea1f8-55c4-49d4-8699-a23e889dd5fb" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.48:8080: connect: connection refused"
Sep 16 19:21:49.569877 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:49.569825    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/splitter-graph-d799f-778646d678-zw694"
Sep 16 19:21:51.104427 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:51.104401    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-f025b-545b74b6c4-lcc6q"
Sep 16 19:21:51.174660 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:51.174629    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/109a6ec8-78d3-45bf-a316-fce6791c6399-proxy-tls\") pod \"109a6ec8-78d3-45bf-a316-fce6791c6399\" (UID: \"109a6ec8-78d3-45bf-a316-fce6791c6399\") "
Sep 16 19:21:51.174660 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:51.174666    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/109a6ec8-78d3-45bf-a316-fce6791c6399-openshift-service-ca-bundle\") pod \"109a6ec8-78d3-45bf-a316-fce6791c6399\" (UID: \"109a6ec8-78d3-45bf-a316-fce6791c6399\") "
Sep 16 19:21:51.175099 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:51.175067    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/109a6ec8-78d3-45bf-a316-fce6791c6399-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "109a6ec8-78d3-45bf-a316-fce6791c6399" (UID: "109a6ec8-78d3-45bf-a316-fce6791c6399"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 19:21:51.176700 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:51.176681    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/109a6ec8-78d3-45bf-a316-fce6791c6399-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "109a6ec8-78d3-45bf-a316-fce6791c6399" (UID: "109a6ec8-78d3-45bf-a316-fce6791c6399"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 19:21:51.275620 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:51.275589    2584 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/109a6ec8-78d3-45bf-a316-fce6791c6399-proxy-tls\") on node \"ip-10-0-139-182.ec2.internal\" DevicePath \"\""
Sep 16 19:21:51.275620 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:51.275617    2584 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/109a6ec8-78d3-45bf-a316-fce6791c6399-openshift-service-ca-bundle\") on node \"ip-10-0-139-182.ec2.internal\" DevicePath \"\""
Sep 16 19:21:51.596005 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:51.595973    2584 generic.go:356] "Generic (PLEG): container finished" podID="109a6ec8-78d3-45bf-a316-fce6791c6399" containerID="06bd1081638e02ff81f947b090944079bbc73ea4f25b664d8850b396241a7143" exitCode=0
Sep 16 19:21:51.596177 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:51.596019    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-f025b-545b74b6c4-lcc6q"
Sep 16 19:21:51.596177 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:51.596064    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-f025b-545b74b6c4-lcc6q" event={"ID":"109a6ec8-78d3-45bf-a316-fce6791c6399","Type":"ContainerDied","Data":"06bd1081638e02ff81f947b090944079bbc73ea4f25b664d8850b396241a7143"}
Sep 16 19:21:51.596177 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:51.596122    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-f025b-545b74b6c4-lcc6q" event={"ID":"109a6ec8-78d3-45bf-a316-fce6791c6399","Type":"ContainerDied","Data":"f5ea4b28107df951d25f5ab1c49d141e96a5a5d6dc43e6db56296159255af49c"}
Sep 16 19:21:51.596177 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:51.596151    2584 scope.go:117] "RemoveContainer" containerID="06bd1081638e02ff81f947b090944079bbc73ea4f25b664d8850b396241a7143"
Sep 16 19:21:51.605411 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:51.605393    2584 scope.go:117] "RemoveContainer" containerID="06bd1081638e02ff81f947b090944079bbc73ea4f25b664d8850b396241a7143"
Sep 16 19:21:51.605691 ip-10-0-139-182 kubenswrapper[2584]: E0916 19:21:51.605672    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"06bd1081638e02ff81f947b090944079bbc73ea4f25b664d8850b396241a7143\": container with ID starting with 06bd1081638e02ff81f947b090944079bbc73ea4f25b664d8850b396241a7143 not found: ID does not exist" containerID="06bd1081638e02ff81f947b090944079bbc73ea4f25b664d8850b396241a7143"
Sep 16 19:21:51.605743 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:51.605700    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"06bd1081638e02ff81f947b090944079bbc73ea4f25b664d8850b396241a7143"} err="failed to get container status \"06bd1081638e02ff81f947b090944079bbc73ea4f25b664d8850b396241a7143\": rpc error: code = NotFound desc = could not find container \"06bd1081638e02ff81f947b090944079bbc73ea4f25b664d8850b396241a7143\": container with ID starting with 06bd1081638e02ff81f947b090944079bbc73ea4f25b664d8850b396241a7143 not found: ID does not exist"
Sep 16 19:21:51.619185 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:51.619162    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-f025b-545b74b6c4-lcc6q"]
Sep 16 19:21:51.623117 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:51.623092    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-f025b-545b74b6c4-lcc6q"]
Sep 16 19:21:51.692555 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:51.692522    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-d799f-778646d678-zw694"]
Sep 16 19:21:51.692771 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:51.692746    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/splitter-graph-d799f-778646d678-zw694" podUID="60354ac8-d364-4367-accd-e65f85cf2bcf" containerName="splitter-graph-d799f" containerID="cri-o://ec904d6c6cf3c44bdfb2dd4013b8188eabd8fb63e0267beebb6956ca9e8e83eb" gracePeriod=30
Sep 16 19:21:51.720376 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:51.720348    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 19:21:51.852998 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:51.852909    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-d799f-predictor-6f68cb5f6c-dhrft"]
Sep 16 19:21:51.853169 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:51.853143    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-d799f-predictor-6f68cb5f6c-dhrft" podUID="6785a5af-ba5e-4e89-b9ca-0fdaecfc54bd" containerName="kserve-container" containerID="cri-o://8fecde307df998cc6dbb5dec102a5afcddba2bdad66844b9d24e56b92b238624" gracePeriod=30
Sep 16 19:21:51.853237 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:51.853204    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-d799f-predictor-6f68cb5f6c-dhrft" podUID="6785a5af-ba5e-4e89-b9ca-0fdaecfc54bd" containerName="kube-rbac-proxy" containerID="cri-o://df997c1dcb2ce9c839d8e9b43c63a6aeb8edbd2bec2549204df63f70e1f09cc6" gracePeriod=30
Sep 16 19:21:51.886402 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:51.886364    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 19:21:51.985483 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:51.985449    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 19:21:52.015839 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:52.015789    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-c3e3d-predictor-866dc5fdb5-sp6mk"]
Sep 16 19:21:52.016229 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:52.016208    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="109a6ec8-78d3-45bf-a316-fce6791c6399" containerName="sequence-graph-f025b"
Sep 16 19:21:52.016318 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:52.016231    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="109a6ec8-78d3-45bf-a316-fce6791c6399" containerName="sequence-graph-f025b"
Sep 16 19:21:52.016373 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:52.016330    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="109a6ec8-78d3-45bf-a316-fce6791c6399" containerName="sequence-graph-f025b"
Sep 16 19:21:52.019409 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:52.019388    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-c3e3d-predictor-866dc5fdb5-sp6mk"
Sep 16 19:21:52.022490 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:52.022471    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-c3e3d-predictor-serving-cert\""
Sep 16 19:21:52.022600 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:52.022479    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-c3e3d-kube-rbac-proxy-sar-config\""
Sep 16 19:21:52.030119 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:52.030097    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-c3e3d-predictor-866dc5fdb5-sp6mk"]
Sep 16 19:21:52.081947 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:52.081915    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 19:21:52.082145 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:52.082081    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xnr6t\" (UniqueName: \"kubernetes.io/projected/ba5c9b0b-507d-4f15-a924-699f630663ab-kube-api-access-xnr6t\") pod \"success-200-isvc-c3e3d-predictor-866dc5fdb5-sp6mk\" (UID: \"ba5c9b0b-507d-4f15-a924-699f630663ab\") " pod="kserve-ci-e2e-test/success-200-isvc-c3e3d-predictor-866dc5fdb5-sp6mk"
Sep 16 19:21:52.082287 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:52.082145    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ba5c9b0b-507d-4f15-a924-699f630663ab-proxy-tls\") pod \"success-200-isvc-c3e3d-predictor-866dc5fdb5-sp6mk\" (UID: \"ba5c9b0b-507d-4f15-a924-699f630663ab\") " pod="kserve-ci-e2e-test/success-200-isvc-c3e3d-predictor-866dc5fdb5-sp6mk"
Sep 16 19:21:52.082287 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:52.082231    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ba5c9b0b-507d-4f15-a924-699f630663ab-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-c3e3d-predictor-866dc5fdb5-sp6mk\" (UID: \"ba5c9b0b-507d-4f15-a924-699f630663ab\") " pod="kserve-ci-e2e-test/success-200-isvc-c3e3d-predictor-866dc5fdb5-sp6mk"
Sep 16 19:21:52.116288 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:52.116211    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="109a6ec8-78d3-45bf-a316-fce6791c6399" path="/var/lib/kubelet/pods/109a6ec8-78d3-45bf-a316-fce6791c6399/volumes"
Sep 16 19:21:52.173357 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:52.173328    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 19:21:52.183422 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:52.183393    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ba5c9b0b-507d-4f15-a924-699f630663ab-proxy-tls\") pod \"success-200-isvc-c3e3d-predictor-866dc5fdb5-sp6mk\" (UID: \"ba5c9b0b-507d-4f15-a924-699f630663ab\") " pod="kserve-ci-e2e-test/success-200-isvc-c3e3d-predictor-866dc5fdb5-sp6mk"
Sep 16 19:21:52.183580 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:52.183458    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ba5c9b0b-507d-4f15-a924-699f630663ab-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-c3e3d-predictor-866dc5fdb5-sp6mk\" (UID: \"ba5c9b0b-507d-4f15-a924-699f630663ab\") " pod="kserve-ci-e2e-test/success-200-isvc-c3e3d-predictor-866dc5fdb5-sp6mk"
Sep 16 19:21:52.183580 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:52.183504    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xnr6t\" (UniqueName: \"kubernetes.io/projected/ba5c9b0b-507d-4f15-a924-699f630663ab-kube-api-access-xnr6t\") pod \"success-200-isvc-c3e3d-predictor-866dc5fdb5-sp6mk\" (UID: \"ba5c9b0b-507d-4f15-a924-699f630663ab\") " pod="kserve-ci-e2e-test/success-200-isvc-c3e3d-predictor-866dc5fdb5-sp6mk"
Sep 16 19:21:52.183580 ip-10-0-139-182 kubenswrapper[2584]: E0916 19:21:52.183553    2584 secret.go:189] Couldn't get secret kserve-ci-e2e-test/success-200-isvc-c3e3d-predictor-serving-cert: secret "success-200-isvc-c3e3d-predictor-serving-cert" not found
Sep 16 19:21:52.183734 ip-10-0-139-182 kubenswrapper[2584]: E0916 19:21:52.183623    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ba5c9b0b-507d-4f15-a924-699f630663ab-proxy-tls podName:ba5c9b0b-507d-4f15-a924-699f630663ab nodeName:}" failed. No retries permitted until 2026-09-16 19:21:52.683604145 +0000 UTC m=+2243.013106551 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/ba5c9b0b-507d-4f15-a924-699f630663ab-proxy-tls") pod "success-200-isvc-c3e3d-predictor-866dc5fdb5-sp6mk" (UID: "ba5c9b0b-507d-4f15-a924-699f630663ab") : secret "success-200-isvc-c3e3d-predictor-serving-cert" not found
Sep 16 19:21:52.184194 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:52.184173    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ba5c9b0b-507d-4f15-a924-699f630663ab-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-c3e3d-predictor-866dc5fdb5-sp6mk\" (UID: \"ba5c9b0b-507d-4f15-a924-699f630663ab\") " pod="kserve-ci-e2e-test/success-200-isvc-c3e3d-predictor-866dc5fdb5-sp6mk"
Sep 16 19:21:52.192351 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:52.192322    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xnr6t\" (UniqueName: \"kubernetes.io/projected/ba5c9b0b-507d-4f15-a924-699f630663ab-kube-api-access-xnr6t\") pod \"success-200-isvc-c3e3d-predictor-866dc5fdb5-sp6mk\" (UID: \"ba5c9b0b-507d-4f15-a924-699f630663ab\") " pod="kserve-ci-e2e-test/success-200-isvc-c3e3d-predictor-866dc5fdb5-sp6mk"
Sep 16 19:21:52.503578 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:52.503499    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 19:21:52.603528 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:52.603496    2584 generic.go:356] "Generic (PLEG): container finished" podID="6785a5af-ba5e-4e89-b9ca-0fdaecfc54bd" containerID="df997c1dcb2ce9c839d8e9b43c63a6aeb8edbd2bec2549204df63f70e1f09cc6" exitCode=2
Sep 16 19:21:52.603688 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:52.603553    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-d799f-predictor-6f68cb5f6c-dhrft" event={"ID":"6785a5af-ba5e-4e89-b9ca-0fdaecfc54bd","Type":"ContainerDied","Data":"df997c1dcb2ce9c839d8e9b43c63a6aeb8edbd2bec2549204df63f70e1f09cc6"}
Sep 16 19:21:52.687520 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:52.687436    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ba5c9b0b-507d-4f15-a924-699f630663ab-proxy-tls\") pod \"success-200-isvc-c3e3d-predictor-866dc5fdb5-sp6mk\" (UID: \"ba5c9b0b-507d-4f15-a924-699f630663ab\") " pod="kserve-ci-e2e-test/success-200-isvc-c3e3d-predictor-866dc5fdb5-sp6mk"
Sep 16 19:21:52.687679 ip-10-0-139-182 kubenswrapper[2584]: E0916 19:21:52.687585    2584 secret.go:189] Couldn't get secret kserve-ci-e2e-test/success-200-isvc-c3e3d-predictor-serving-cert: secret "success-200-isvc-c3e3d-predictor-serving-cert" not found
Sep 16 19:21:52.687679 ip-10-0-139-182 kubenswrapper[2584]: E0916 19:21:52.687659    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ba5c9b0b-507d-4f15-a924-699f630663ab-proxy-tls podName:ba5c9b0b-507d-4f15-a924-699f630663ab nodeName:}" failed. No retries permitted until 2026-09-16 19:21:53.687641512 +0000 UTC m=+2244.017143914 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/ba5c9b0b-507d-4f15-a924-699f630663ab-proxy-tls") pod "success-200-isvc-c3e3d-predictor-866dc5fdb5-sp6mk" (UID: "ba5c9b0b-507d-4f15-a924-699f630663ab") : secret "success-200-isvc-c3e3d-predictor-serving-cert" not found
Sep 16 19:21:53.067538 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:53.067505    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 19:21:53.695807 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:53.695769    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ba5c9b0b-507d-4f15-a924-699f630663ab-proxy-tls\") pod \"success-200-isvc-c3e3d-predictor-866dc5fdb5-sp6mk\" (UID: \"ba5c9b0b-507d-4f15-a924-699f630663ab\") " pod="kserve-ci-e2e-test/success-200-isvc-c3e3d-predictor-866dc5fdb5-sp6mk"
Sep 16 19:21:53.698242 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:53.698219    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ba5c9b0b-507d-4f15-a924-699f630663ab-proxy-tls\") pod \"success-200-isvc-c3e3d-predictor-866dc5fdb5-sp6mk\" (UID: \"ba5c9b0b-507d-4f15-a924-699f630663ab\") " pod="kserve-ci-e2e-test/success-200-isvc-c3e3d-predictor-866dc5fdb5-sp6mk"
Sep 16 19:21:53.830498 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:53.830446    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-c3e3d-predictor-866dc5fdb5-sp6mk"
Sep 16 19:21:54.109963 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:54.109935    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-c3e3d-predictor-866dc5fdb5-sp6mk"]
Sep 16 19:21:54.111495 ip-10-0-139-182 kubenswrapper[2584]: W0916 19:21:54.111469    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba5c9b0b_507d_4f15_a924_699f630663ab.slice/crio-8a32ce810e50df0287c5388d65c039614e2242f230f0ae2e7b08e193c97c1f92 WatchSource:0}: Error finding container 8a32ce810e50df0287c5388d65c039614e2242f230f0ae2e7b08e193c97c1f92: Status 404 returned error can't find the container with id 8a32ce810e50df0287c5388d65c039614e2242f230f0ae2e7b08e193c97c1f92
Sep 16 19:21:54.568317 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:54.568276    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-d799f-778646d678-zw694" podUID="60354ac8-d364-4367-accd-e65f85cf2bcf" containerName="splitter-graph-d799f" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 16 19:21:54.613882 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:54.613824    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-c3e3d-predictor-866dc5fdb5-sp6mk" event={"ID":"ba5c9b0b-507d-4f15-a924-699f630663ab","Type":"ContainerStarted","Data":"4b675cfff62e93427f441be8062bd49d1ff0f5d723d2f8a13fe1390645773a5d"}
Sep 16 19:21:54.614066 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:54.613921    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-c3e3d-predictor-866dc5fdb5-sp6mk" event={"ID":"ba5c9b0b-507d-4f15-a924-699f630663ab","Type":"ContainerStarted","Data":"d419ad234dff341b449769dbbe03fbac773ccea93bafa209c84899e861147735"}
Sep 16 19:21:54.614066 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:54.613958    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-c3e3d-predictor-866dc5fdb5-sp6mk" event={"ID":"ba5c9b0b-507d-4f15-a924-699f630663ab","Type":"ContainerStarted","Data":"8a32ce810e50df0287c5388d65c039614e2242f230f0ae2e7b08e193c97c1f92"}
Sep 16 19:21:54.614066 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:54.613992    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-c3e3d-predictor-866dc5fdb5-sp6mk"
Sep 16 19:21:54.635641 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:54.635572    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-c3e3d-predictor-866dc5fdb5-sp6mk" podStartSLOduration=3.63555085 podStartE2EDuration="3.63555085s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 19:21:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 19:21:54.634515094 +0000 UTC m=+2244.964017508" watchObservedRunningTime="2026-09-16 19:21:54.63555085 +0000 UTC m=+2244.965053275"
Sep 16 19:21:55.000090 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:55.000065    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-d799f-predictor-6f68cb5f6c-dhrft"
Sep 16 19:21:55.107702 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:55.107617    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jcdrm\" (UniqueName: \"kubernetes.io/projected/6785a5af-ba5e-4e89-b9ca-0fdaecfc54bd-kube-api-access-jcdrm\") pod \"6785a5af-ba5e-4e89-b9ca-0fdaecfc54bd\" (UID: \"6785a5af-ba5e-4e89-b9ca-0fdaecfc54bd\") "
Sep 16 19:21:55.107702 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:55.107676    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/6785a5af-ba5e-4e89-b9ca-0fdaecfc54bd-proxy-tls\") pod \"6785a5af-ba5e-4e89-b9ca-0fdaecfc54bd\" (UID: \"6785a5af-ba5e-4e89-b9ca-0fdaecfc54bd\") "
Sep 16 19:21:55.107942 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:55.107720    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/6785a5af-ba5e-4e89-b9ca-0fdaecfc54bd-kube-rbac-proxy-sar-config\") pod \"6785a5af-ba5e-4e89-b9ca-0fdaecfc54bd\" (UID: \"6785a5af-ba5e-4e89-b9ca-0fdaecfc54bd\") "
Sep 16 19:21:55.108162 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:55.108139    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6785a5af-ba5e-4e89-b9ca-0fdaecfc54bd-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "6785a5af-ba5e-4e89-b9ca-0fdaecfc54bd" (UID: "6785a5af-ba5e-4e89-b9ca-0fdaecfc54bd"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 19:21:55.109765 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:55.109735    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6785a5af-ba5e-4e89-b9ca-0fdaecfc54bd-kube-api-access-jcdrm" (OuterVolumeSpecName: "kube-api-access-jcdrm") pod "6785a5af-ba5e-4e89-b9ca-0fdaecfc54bd" (UID: "6785a5af-ba5e-4e89-b9ca-0fdaecfc54bd"). InnerVolumeSpecName "kube-api-access-jcdrm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 19:21:55.109765 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:55.109737    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6785a5af-ba5e-4e89-b9ca-0fdaecfc54bd-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "6785a5af-ba5e-4e89-b9ca-0fdaecfc54bd" (UID: "6785a5af-ba5e-4e89-b9ca-0fdaecfc54bd"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 19:21:55.209289 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:55.209259    2584 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/6785a5af-ba5e-4e89-b9ca-0fdaecfc54bd-kube-rbac-proxy-sar-config\") on node \"ip-10-0-139-182.ec2.internal\" DevicePath \"\""
Sep 16 19:21:55.209289 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:55.209286    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jcdrm\" (UniqueName: \"kubernetes.io/projected/6785a5af-ba5e-4e89-b9ca-0fdaecfc54bd-kube-api-access-jcdrm\") on node \"ip-10-0-139-182.ec2.internal\" DevicePath \"\""
Sep 16 19:21:55.209289 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:55.209296    2584 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/6785a5af-ba5e-4e89-b9ca-0fdaecfc54bd-proxy-tls\") on node \"ip-10-0-139-182.ec2.internal\" DevicePath \"\""
Sep 16 19:21:55.619634 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:55.619597    2584 generic.go:356] "Generic (PLEG): container finished" podID="6785a5af-ba5e-4e89-b9ca-0fdaecfc54bd" containerID="8fecde307df998cc6dbb5dec102a5afcddba2bdad66844b9d24e56b92b238624" exitCode=0
Sep 16 19:21:55.619808 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:55.619646    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-d799f-predictor-6f68cb5f6c-dhrft"
Sep 16 19:21:55.619808 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:55.619688    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-d799f-predictor-6f68cb5f6c-dhrft" event={"ID":"6785a5af-ba5e-4e89-b9ca-0fdaecfc54bd","Type":"ContainerDied","Data":"8fecde307df998cc6dbb5dec102a5afcddba2bdad66844b9d24e56b92b238624"}
Sep 16 19:21:55.619808 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:55.619754    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-d799f-predictor-6f68cb5f6c-dhrft" event={"ID":"6785a5af-ba5e-4e89-b9ca-0fdaecfc54bd","Type":"ContainerDied","Data":"76db3b4b9e304718e090dcb8866cf5c02632dcc2f515a47cd8f40272e79c9ebf"}
Sep 16 19:21:55.619808 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:55.619782    2584 scope.go:117] "RemoveContainer" containerID="df997c1dcb2ce9c839d8e9b43c63a6aeb8edbd2bec2549204df63f70e1f09cc6"
Sep 16 19:21:55.620222 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:55.620205    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-c3e3d-predictor-866dc5fdb5-sp6mk"
Sep 16 19:21:55.621874 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:55.621843    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-c3e3d-predictor-866dc5fdb5-sp6mk" podUID="ba5c9b0b-507d-4f15-a924-699f630663ab" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.50:8080: connect: connection refused"
Sep 16 19:21:55.629382 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:55.629364    2584 scope.go:117] "RemoveContainer" containerID="8fecde307df998cc6dbb5dec102a5afcddba2bdad66844b9d24e56b92b238624"
Sep 16 19:21:55.639916 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:55.639895    2584 scope.go:117] "RemoveContainer" containerID="df997c1dcb2ce9c839d8e9b43c63a6aeb8edbd2bec2549204df63f70e1f09cc6"
Sep 16 19:21:55.640231 ip-10-0-139-182 kubenswrapper[2584]: E0916 19:21:55.640207    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"df997c1dcb2ce9c839d8e9b43c63a6aeb8edbd2bec2549204df63f70e1f09cc6\": container with ID starting with df997c1dcb2ce9c839d8e9b43c63a6aeb8edbd2bec2549204df63f70e1f09cc6 not found: ID does not exist" containerID="df997c1dcb2ce9c839d8e9b43c63a6aeb8edbd2bec2549204df63f70e1f09cc6"
Sep 16 19:21:55.640316 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:55.640239    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"df997c1dcb2ce9c839d8e9b43c63a6aeb8edbd2bec2549204df63f70e1f09cc6"} err="failed to get container status \"df997c1dcb2ce9c839d8e9b43c63a6aeb8edbd2bec2549204df63f70e1f09cc6\": rpc error: code = NotFound desc = could not find container \"df997c1dcb2ce9c839d8e9b43c63a6aeb8edbd2bec2549204df63f70e1f09cc6\": container with ID starting with df997c1dcb2ce9c839d8e9b43c63a6aeb8edbd2bec2549204df63f70e1f09cc6 not found: ID does not exist"
Sep 16 19:21:55.640316 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:55.640257    2584 scope.go:117] "RemoveContainer" containerID="8fecde307df998cc6dbb5dec102a5afcddba2bdad66844b9d24e56b92b238624"
Sep 16 19:21:55.640550 ip-10-0-139-182 kubenswrapper[2584]: E0916 19:21:55.640528    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8fecde307df998cc6dbb5dec102a5afcddba2bdad66844b9d24e56b92b238624\": container with ID starting with 8fecde307df998cc6dbb5dec102a5afcddba2bdad66844b9d24e56b92b238624 not found: ID does not exist" containerID="8fecde307df998cc6dbb5dec102a5afcddba2bdad66844b9d24e56b92b238624"
Sep 16 19:21:55.640608 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:55.640562    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8fecde307df998cc6dbb5dec102a5afcddba2bdad66844b9d24e56b92b238624"} err="failed to get container status \"8fecde307df998cc6dbb5dec102a5afcddba2bdad66844b9d24e56b92b238624\": rpc error: code = NotFound desc = could not find container \"8fecde307df998cc6dbb5dec102a5afcddba2bdad66844b9d24e56b92b238624\": container with ID starting with 8fecde307df998cc6dbb5dec102a5afcddba2bdad66844b9d24e56b92b238624 not found: ID does not exist"
Sep 16 19:21:55.651675 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:55.651643    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-d799f-predictor-6f68cb5f6c-dhrft"]
Sep 16 19:21:55.657929 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:55.657901    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-d799f-predictor-6f68cb5f6c-dhrft"]
Sep 16 19:21:56.115644 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:56.115608    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6785a5af-ba5e-4e89-b9ca-0fdaecfc54bd" path="/var/lib/kubelet/pods/6785a5af-ba5e-4e89-b9ca-0fdaecfc54bd/volumes"
Sep 16 19:21:56.625615 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:56.625575    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-c3e3d-predictor-866dc5fdb5-sp6mk" podUID="ba5c9b0b-507d-4f15-a924-699f630663ab" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.50:8080: connect: connection refused"
Sep 16 19:21:59.482068 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:59.482028    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-0263c-predictor-7bfb46579c-n6fgr" podUID="cb9ea1f8-55c4-49d4-8699-a23e889dd5fb" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.48:8080: connect: connection refused"
Sep 16 19:21:59.568033 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:21:59.567987    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-d799f-778646d678-zw694" podUID="60354ac8-d364-4367-accd-e65f85cf2bcf" containerName="splitter-graph-d799f" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 16 19:22:01.629952 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:22:01.629923    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-c3e3d-predictor-866dc5fdb5-sp6mk"
Sep 16 19:22:01.630533 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:22:01.630507    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-c3e3d-predictor-866dc5fdb5-sp6mk" podUID="ba5c9b0b-507d-4f15-a924-699f630663ab" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.50:8080: connect: connection refused"
Sep 16 19:22:04.567867 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:22:04.567809    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-d799f-778646d678-zw694" podUID="60354ac8-d364-4367-accd-e65f85cf2bcf" containerName="splitter-graph-d799f" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 16 19:22:04.568315 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:22:04.567937    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/splitter-graph-d799f-778646d678-zw694"
Sep 16 19:22:09.482537 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:22:09.482506    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-0263c-predictor-7bfb46579c-n6fgr"
Sep 16 19:22:09.568081 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:22:09.568040    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-d799f-778646d678-zw694" podUID="60354ac8-d364-4367-accd-e65f85cf2bcf" containerName="splitter-graph-d799f" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 16 19:22:11.631482 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:22:11.631445    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-c3e3d-predictor-866dc5fdb5-sp6mk" podUID="ba5c9b0b-507d-4f15-a924-699f630663ab" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.50:8080: connect: connection refused"
Sep 16 19:22:14.568027 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:22:14.567987    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-d799f-778646d678-zw694" podUID="60354ac8-d364-4367-accd-e65f85cf2bcf" containerName="splitter-graph-d799f" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 16 19:22:19.568513 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:22:19.568473    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-d799f-778646d678-zw694" podUID="60354ac8-d364-4367-accd-e65f85cf2bcf" containerName="splitter-graph-d799f" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 16 19:22:21.266369 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:22:21.266336    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/switch-graph-0263c-6d797d88cf-8bs4n"]
Sep 16 19:22:21.266748 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:22:21.266706    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6785a5af-ba5e-4e89-b9ca-0fdaecfc54bd" containerName="kserve-container"
Sep 16 19:22:21.266748 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:22:21.266724    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="6785a5af-ba5e-4e89-b9ca-0fdaecfc54bd" containerName="kserve-container"
Sep 16 19:22:21.266748 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:22:21.266748    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6785a5af-ba5e-4e89-b9ca-0fdaecfc54bd" containerName="kube-rbac-proxy"
Sep 16 19:22:21.266930 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:22:21.266756    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="6785a5af-ba5e-4e89-b9ca-0fdaecfc54bd" containerName="kube-rbac-proxy"
Sep 16 19:22:21.266930 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:22:21.266851    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="6785a5af-ba5e-4e89-b9ca-0fdaecfc54bd" containerName="kube-rbac-proxy"
Sep 16 19:22:21.266930 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:22:21.266866    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="6785a5af-ba5e-4e89-b9ca-0fdaecfc54bd" containerName="kserve-container"
Sep 16 19:22:21.270037 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:22:21.270018    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-0263c-6d797d88cf-8bs4n"
Sep 16 19:22:21.272987 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:22:21.272969    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"switch-graph-0263c-kube-rbac-proxy-sar-config\""
Sep 16 19:22:21.273336 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:22:21.273320    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"switch-graph-0263c-serving-cert\""
Sep 16 19:22:21.282261 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:22:21.282236    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/switch-graph-0263c-6d797d88cf-8bs4n"]
Sep 16 19:22:21.301255 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:22:21.301230    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 19:22:21.320380 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:22:21.320349    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/50df7308-89a9-4106-854e-f259be58ad45-openshift-service-ca-bundle\") pod \"switch-graph-0263c-6d797d88cf-8bs4n\" (UID: \"50df7308-89a9-4106-854e-f259be58ad45\") " pod="kserve-ci-e2e-test/switch-graph-0263c-6d797d88cf-8bs4n"
Sep 16 19:22:21.320490 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:22:21.320386    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/50df7308-89a9-4106-854e-f259be58ad45-proxy-tls\") pod \"switch-graph-0263c-6d797d88cf-8bs4n\" (UID: \"50df7308-89a9-4106-854e-f259be58ad45\") " pod="kserve-ci-e2e-test/switch-graph-0263c-6d797d88cf-8bs4n"
Sep 16 19:22:21.421023 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:22:21.420987    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/50df7308-89a9-4106-854e-f259be58ad45-openshift-service-ca-bundle\") pod \"switch-graph-0263c-6d797d88cf-8bs4n\" (UID: \"50df7308-89a9-4106-854e-f259be58ad45\") " pod="kserve-ci-e2e-test/switch-graph-0263c-6d797d88cf-8bs4n"
Sep 16 19:22:21.421023 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:22:21.421024    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/50df7308-89a9-4106-854e-f259be58ad45-proxy-tls\") pod \"switch-graph-0263c-6d797d88cf-8bs4n\" (UID: \"50df7308-89a9-4106-854e-f259be58ad45\") " pod="kserve-ci-e2e-test/switch-graph-0263c-6d797d88cf-8bs4n"
Sep 16 19:22:21.421278 ip-10-0-139-182 kubenswrapper[2584]: E0916 19:22:21.421121    2584 secret.go:189] Couldn't get secret kserve-ci-e2e-test/switch-graph-0263c-serving-cert: secret "switch-graph-0263c-serving-cert" not found
Sep 16 19:22:21.421278 ip-10-0-139-182 kubenswrapper[2584]: E0916 19:22:21.421184    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/50df7308-89a9-4106-854e-f259be58ad45-proxy-tls podName:50df7308-89a9-4106-854e-f259be58ad45 nodeName:}" failed. No retries permitted until 2026-09-16 19:22:21.921169348 +0000 UTC m=+2272.250671739 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/50df7308-89a9-4106-854e-f259be58ad45-proxy-tls") pod "switch-graph-0263c-6d797d88cf-8bs4n" (UID: "50df7308-89a9-4106-854e-f259be58ad45") : secret "switch-graph-0263c-serving-cert" not found
Sep 16 19:22:21.421756 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:22:21.421729    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/50df7308-89a9-4106-854e-f259be58ad45-openshift-service-ca-bundle\") pod \"switch-graph-0263c-6d797d88cf-8bs4n\" (UID: \"50df7308-89a9-4106-854e-f259be58ad45\") " pod="kserve-ci-e2e-test/switch-graph-0263c-6d797d88cf-8bs4n"
Sep 16 19:22:21.631270 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:22:21.631183    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-c3e3d-predictor-866dc5fdb5-sp6mk" podUID="ba5c9b0b-507d-4f15-a924-699f630663ab" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.50:8080: connect: connection refused"
Sep 16 19:22:21.705362 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:22:21.705332    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 19:22:21.739662 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:22:21.739635    2584 generic.go:356] "Generic (PLEG): container finished" podID="60354ac8-d364-4367-accd-e65f85cf2bcf" containerID="ec904d6c6cf3c44bdfb2dd4013b8188eabd8fb63e0267beebb6956ca9e8e83eb" exitCode=0
Sep 16 19:22:21.739763 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:22:21.739707    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-d799f-778646d678-zw694" event={"ID":"60354ac8-d364-4367-accd-e65f85cf2bcf","Type":"ContainerDied","Data":"ec904d6c6cf3c44bdfb2dd4013b8188eabd8fb63e0267beebb6956ca9e8e83eb"}
Sep 16 19:22:21.868462 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:22:21.868438    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-d799f-778646d678-zw694"
Sep 16 19:22:21.923475 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:22:21.923399    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/60354ac8-d364-4367-accd-e65f85cf2bcf-proxy-tls\") pod \"60354ac8-d364-4367-accd-e65f85cf2bcf\" (UID: \"60354ac8-d364-4367-accd-e65f85cf2bcf\") "
Sep 16 19:22:21.923475 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:22:21.923463    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/60354ac8-d364-4367-accd-e65f85cf2bcf-openshift-service-ca-bundle\") pod \"60354ac8-d364-4367-accd-e65f85cf2bcf\" (UID: \"60354ac8-d364-4367-accd-e65f85cf2bcf\") "
Sep 16 19:22:21.923713 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:22:21.923593    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/50df7308-89a9-4106-854e-f259be58ad45-proxy-tls\") pod \"switch-graph-0263c-6d797d88cf-8bs4n\" (UID: \"50df7308-89a9-4106-854e-f259be58ad45\") " pod="kserve-ci-e2e-test/switch-graph-0263c-6d797d88cf-8bs4n"
Sep 16 19:22:21.923821 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:22:21.923798    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/60354ac8-d364-4367-accd-e65f85cf2bcf-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "60354ac8-d364-4367-accd-e65f85cf2bcf" (UID: "60354ac8-d364-4367-accd-e65f85cf2bcf"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 19:22:21.925415 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:22:21.925394    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/60354ac8-d364-4367-accd-e65f85cf2bcf-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "60354ac8-d364-4367-accd-e65f85cf2bcf" (UID: "60354ac8-d364-4367-accd-e65f85cf2bcf"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 19:22:21.925963 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:22:21.925938    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/50df7308-89a9-4106-854e-f259be58ad45-proxy-tls\") pod \"switch-graph-0263c-6d797d88cf-8bs4n\" (UID: \"50df7308-89a9-4106-854e-f259be58ad45\") " pod="kserve-ci-e2e-test/switch-graph-0263c-6d797d88cf-8bs4n"
Sep 16 19:22:22.024321 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:22:22.024279    2584 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/60354ac8-d364-4367-accd-e65f85cf2bcf-openshift-service-ca-bundle\") on node \"ip-10-0-139-182.ec2.internal\" DevicePath \"\""
Sep 16 19:22:22.024321 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:22:22.024316    2584 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/60354ac8-d364-4367-accd-e65f85cf2bcf-proxy-tls\") on node \"ip-10-0-139-182.ec2.internal\" DevicePath \"\""
Sep 16 19:22:22.179731 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:22:22.179650    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-0263c-6d797d88cf-8bs4n"
Sep 16 19:22:22.459709 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:22:22.459670    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/switch-graph-0263c-6d797d88cf-8bs4n"]
Sep 16 19:22:22.462437 ip-10-0-139-182 kubenswrapper[2584]: W0916 19:22:22.462409    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50df7308_89a9_4106_854e_f259be58ad45.slice/crio-d926b46e3a287c94e13d29c5df8fb4b829d8ea61ca721931002b3dac6210d5ff WatchSource:0}: Error finding container d926b46e3a287c94e13d29c5df8fb4b829d8ea61ca721931002b3dac6210d5ff: Status 404 returned error can't find the container with id d926b46e3a287c94e13d29c5df8fb4b829d8ea61ca721931002b3dac6210d5ff
Sep 16 19:22:22.746692 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:22:22.746594    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-0263c-6d797d88cf-8bs4n" event={"ID":"50df7308-89a9-4106-854e-f259be58ad45","Type":"ContainerStarted","Data":"0e33648777ae79675e0b2ce490e66cc62d9569906c0a90e82bf6154caa0f45e3"}
Sep 16 19:22:22.746889 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:22:22.746697    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/switch-graph-0263c-6d797d88cf-8bs4n"
Sep 16 19:22:22.746889 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:22:22.746725    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-0263c-6d797d88cf-8bs4n" event={"ID":"50df7308-89a9-4106-854e-f259be58ad45","Type":"ContainerStarted","Data":"d926b46e3a287c94e13d29c5df8fb4b829d8ea61ca721931002b3dac6210d5ff"}
Sep 16 19:22:22.748134 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:22:22.748106    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-d799f-778646d678-zw694" event={"ID":"60354ac8-d364-4367-accd-e65f85cf2bcf","Type":"ContainerDied","Data":"99bc5da75e60b218ff2cca104b95776d307c03658d0033a725c903aabe7d9244"}
Sep 16 19:22:22.748230 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:22:22.748148    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-d799f-778646d678-zw694"
Sep 16 19:22:22.748230 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:22:22.748167    2584 scope.go:117] "RemoveContainer" containerID="ec904d6c6cf3c44bdfb2dd4013b8188eabd8fb63e0267beebb6956ca9e8e83eb"
Sep 16 19:22:22.765725 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:22:22.765678    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/switch-graph-0263c-6d797d88cf-8bs4n" podStartSLOduration=1.765663304 podStartE2EDuration="1.765663304s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 19:22:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 19:22:22.765133089 +0000 UTC m=+2273.094635515" watchObservedRunningTime="2026-09-16 19:22:22.765663304 +0000 UTC m=+2273.095165716"
Sep 16 19:22:22.778873 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:22:22.778846    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-d799f-778646d678-zw694"]
Sep 16 19:22:22.781171 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:22:22.781148    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-d799f-778646d678-zw694"]
Sep 16 19:22:24.116299 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:22:24.116261    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="60354ac8-d364-4367-accd-e65f85cf2bcf" path="/var/lib/kubelet/pods/60354ac8-d364-4367-accd-e65f85cf2bcf/volumes"
Sep 16 19:22:28.759682 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:22:28.759654    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/switch-graph-0263c-6d797d88cf-8bs4n"
Sep 16 19:22:31.630678 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:22:31.630637    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-c3e3d-predictor-866dc5fdb5-sp6mk" podUID="ba5c9b0b-507d-4f15-a924-699f630663ab" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.50:8080: connect: connection refused"
Sep 16 19:22:41.631824 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:22:41.631793    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-c3e3d-predictor-866dc5fdb5-sp6mk"
Sep 16 19:22:51.957842 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:22:51.957794    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 19:22:51.962668 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:22:51.962641    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/splitter-graph-c3e3d-695cdcbd4-kchln"]
Sep 16 19:22:51.963001 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:22:51.962988    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="60354ac8-d364-4367-accd-e65f85cf2bcf" containerName="splitter-graph-d799f"
Sep 16 19:22:51.963050 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:22:51.963003    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="60354ac8-d364-4367-accd-e65f85cf2bcf" containerName="splitter-graph-d799f"
Sep 16 19:22:51.963085 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:22:51.963073    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="60354ac8-d364-4367-accd-e65f85cf2bcf" containerName="splitter-graph-d799f"
Sep 16 19:22:51.966032 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:22:51.966013    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-c3e3d-695cdcbd4-kchln"
Sep 16 19:22:51.969511 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:22:51.969488    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"splitter-graph-c3e3d-serving-cert\""
Sep 16 19:22:51.969974 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:22:51.969958    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"splitter-graph-c3e3d-kube-rbac-proxy-sar-config\""
Sep 16 19:22:51.979106 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:22:51.979086    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-c3e3d-695cdcbd4-kchln"]
Sep 16 19:22:52.084578 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:22:52.084542    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ebb65cb2-c696-42ff-b2b9-7e5407f6cbbd-proxy-tls\") pod \"splitter-graph-c3e3d-695cdcbd4-kchln\" (UID: \"ebb65cb2-c696-42ff-b2b9-7e5407f6cbbd\") " pod="kserve-ci-e2e-test/splitter-graph-c3e3d-695cdcbd4-kchln"
Sep 16 19:22:52.084786 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:22:52.084591    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ebb65cb2-c696-42ff-b2b9-7e5407f6cbbd-openshift-service-ca-bundle\") pod \"splitter-graph-c3e3d-695cdcbd4-kchln\" (UID: \"ebb65cb2-c696-42ff-b2b9-7e5407f6cbbd\") " pod="kserve-ci-e2e-test/splitter-graph-c3e3d-695cdcbd4-kchln"
Sep 16 19:22:52.185555 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:22:52.185522    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ebb65cb2-c696-42ff-b2b9-7e5407f6cbbd-proxy-tls\") pod \"splitter-graph-c3e3d-695cdcbd4-kchln\" (UID: \"ebb65cb2-c696-42ff-b2b9-7e5407f6cbbd\") " pod="kserve-ci-e2e-test/splitter-graph-c3e3d-695cdcbd4-kchln"
Sep 16 19:22:52.185725 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:22:52.185568    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ebb65cb2-c696-42ff-b2b9-7e5407f6cbbd-openshift-service-ca-bundle\") pod \"splitter-graph-c3e3d-695cdcbd4-kchln\" (UID: \"ebb65cb2-c696-42ff-b2b9-7e5407f6cbbd\") " pod="kserve-ci-e2e-test/splitter-graph-c3e3d-695cdcbd4-kchln"
Sep 16 19:22:52.186229 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:22:52.186205    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ebb65cb2-c696-42ff-b2b9-7e5407f6cbbd-openshift-service-ca-bundle\") pod \"splitter-graph-c3e3d-695cdcbd4-kchln\" (UID: \"ebb65cb2-c696-42ff-b2b9-7e5407f6cbbd\") " pod="kserve-ci-e2e-test/splitter-graph-c3e3d-695cdcbd4-kchln"
Sep 16 19:22:52.187985 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:22:52.187962    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ebb65cb2-c696-42ff-b2b9-7e5407f6cbbd-proxy-tls\") pod \"splitter-graph-c3e3d-695cdcbd4-kchln\" (UID: \"ebb65cb2-c696-42ff-b2b9-7e5407f6cbbd\") " pod="kserve-ci-e2e-test/splitter-graph-c3e3d-695cdcbd4-kchln"
Sep 16 19:22:52.210206 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:22:52.210131    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 19:22:52.275957 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:22:52.275929    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-c3e3d-695cdcbd4-kchln"
Sep 16 19:22:52.550399 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:22:52.550361    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-c3e3d-695cdcbd4-kchln"]
Sep 16 19:22:52.553274 ip-10-0-139-182 kubenswrapper[2584]: W0916 19:22:52.553248    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podebb65cb2_c696_42ff_b2b9_7e5407f6cbbd.slice/crio-2e0abdf1fd42c022368ca97b6285c3720fdc6205246061cc7717184d570adef0 WatchSource:0}: Error finding container 2e0abdf1fd42c022368ca97b6285c3720fdc6205246061cc7717184d570adef0: Status 404 returned error can't find the container with id 2e0abdf1fd42c022368ca97b6285c3720fdc6205246061cc7717184d570adef0
Sep 16 19:22:52.873658 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:22:52.873566    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-c3e3d-695cdcbd4-kchln" event={"ID":"ebb65cb2-c696-42ff-b2b9-7e5407f6cbbd","Type":"ContainerStarted","Data":"d6d8e1494ed1749723c4b5c8ab0fd4370f25cc63a043520e810df95464a921a8"}
Sep 16 19:22:52.873658 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:22:52.873625    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-c3e3d-695cdcbd4-kchln" event={"ID":"ebb65cb2-c696-42ff-b2b9-7e5407f6cbbd","Type":"ContainerStarted","Data":"2e0abdf1fd42c022368ca97b6285c3720fdc6205246061cc7717184d570adef0"}
Sep 16 19:22:52.873860 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:22:52.873711    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/splitter-graph-c3e3d-695cdcbd4-kchln"
Sep 16 19:22:52.893945 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:22:52.893899    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/splitter-graph-c3e3d-695cdcbd4-kchln" podStartSLOduration=1.893885628 podStartE2EDuration="1.893885628s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 19:22:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 19:22:52.891213321 +0000 UTC m=+2303.220715746" watchObservedRunningTime="2026-09-16 19:22:52.893885628 +0000 UTC m=+2303.223388073"
Sep 16 19:22:58.882549 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:22:58.882520    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/splitter-graph-c3e3d-695cdcbd4-kchln"
Sep 16 19:24:36.215290 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:24:36.215185    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-crjw7_ecf273b5-b465-4008-b035-f7ac0e740d58/console-operator/1.log"
Sep 16 19:24:36.221202 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:24:36.221178    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-crjw7_ecf273b5-b465-4008-b035-f7ac0e740d58/console-operator/1.log"
Sep 16 19:26:08.712668 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:26:08.712632    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 19:27:36.257886 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:27:36.257848    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 19:29:36.244290 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:29:36.244173    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-crjw7_ecf273b5-b465-4008-b035-f7ac0e740d58/console-operator/1.log"
Sep 16 19:29:36.250152 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:29:36.250131    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-crjw7_ecf273b5-b465-4008-b035-f7ac0e740d58/console-operator/1.log"
Sep 16 19:31:06.610030 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:31:06.609935    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-c3e3d-695cdcbd4-kchln"]
Sep 16 19:31:06.610557 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:31:06.610290    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/splitter-graph-c3e3d-695cdcbd4-kchln" podUID="ebb65cb2-c696-42ff-b2b9-7e5407f6cbbd" containerName="splitter-graph-c3e3d" containerID="cri-o://d6d8e1494ed1749723c4b5c8ab0fd4370f25cc63a043520e810df95464a921a8" gracePeriod=30
Sep 16 19:31:06.691630 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:31:06.691598    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 19:31:06.765733 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:31:06.765700    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-c3e3d-predictor-866dc5fdb5-sp6mk"]
Sep 16 19:31:06.766009 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:31:06.765980    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-c3e3d-predictor-866dc5fdb5-sp6mk" podUID="ba5c9b0b-507d-4f15-a924-699f630663ab" containerName="kserve-container" containerID="cri-o://d419ad234dff341b449769dbbe03fbac773ccea93bafa209c84899e861147735" gracePeriod=30
Sep 16 19:31:06.766196 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:31:06.765980    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-c3e3d-predictor-866dc5fdb5-sp6mk" podUID="ba5c9b0b-507d-4f15-a924-699f630663ab" containerName="kube-rbac-proxy" containerID="cri-o://4b675cfff62e93427f441be8062bd49d1ff0f5d723d2f8a13fe1390645773a5d" gracePeriod=30
Sep 16 19:31:06.800228 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:31:06.800200    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 19:31:06.867923 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:31:06.867813    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 19:31:07.075191 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:31:07.075161    2584 generic.go:356] "Generic (PLEG): container finished" podID="ba5c9b0b-507d-4f15-a924-699f630663ab" containerID="4b675cfff62e93427f441be8062bd49d1ff0f5d723d2f8a13fe1390645773a5d" exitCode=2
Sep 16 19:31:07.075352 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:31:07.075229    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-c3e3d-predictor-866dc5fdb5-sp6mk" event={"ID":"ba5c9b0b-507d-4f15-a924-699f630663ab","Type":"ContainerDied","Data":"4b675cfff62e93427f441be8062bd49d1ff0f5d723d2f8a13fe1390645773a5d"}
Sep 16 19:31:08.881933 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:31:08.881890    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-c3e3d-695cdcbd4-kchln" podUID="ebb65cb2-c696-42ff-b2b9-7e5407f6cbbd" containerName="splitter-graph-c3e3d" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 16 19:31:09.818572 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:31:09.818546    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-c3e3d-predictor-866dc5fdb5-sp6mk"
Sep 16 19:31:09.946543 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:31:09.946453    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xnr6t\" (UniqueName: \"kubernetes.io/projected/ba5c9b0b-507d-4f15-a924-699f630663ab-kube-api-access-xnr6t\") pod \"ba5c9b0b-507d-4f15-a924-699f630663ab\" (UID: \"ba5c9b0b-507d-4f15-a924-699f630663ab\") "
Sep 16 19:31:09.946543 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:31:09.946500    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ba5c9b0b-507d-4f15-a924-699f630663ab-kube-rbac-proxy-sar-config\") pod \"ba5c9b0b-507d-4f15-a924-699f630663ab\" (UID: \"ba5c9b0b-507d-4f15-a924-699f630663ab\") "
Sep 16 19:31:09.946543 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:31:09.946522    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ba5c9b0b-507d-4f15-a924-699f630663ab-proxy-tls\") pod \"ba5c9b0b-507d-4f15-a924-699f630663ab\" (UID: \"ba5c9b0b-507d-4f15-a924-699f630663ab\") "
Sep 16 19:31:09.947089 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:31:09.946898    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ba5c9b0b-507d-4f15-a924-699f630663ab-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "ba5c9b0b-507d-4f15-a924-699f630663ab" (UID: "ba5c9b0b-507d-4f15-a924-699f630663ab"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 19:31:09.948592 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:31:09.948569    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ba5c9b0b-507d-4f15-a924-699f630663ab-kube-api-access-xnr6t" (OuterVolumeSpecName: "kube-api-access-xnr6t") pod "ba5c9b0b-507d-4f15-a924-699f630663ab" (UID: "ba5c9b0b-507d-4f15-a924-699f630663ab"). InnerVolumeSpecName "kube-api-access-xnr6t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 19:31:09.948592 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:31:09.948576    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ba5c9b0b-507d-4f15-a924-699f630663ab-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "ba5c9b0b-507d-4f15-a924-699f630663ab" (UID: "ba5c9b0b-507d-4f15-a924-699f630663ab"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 19:31:10.047231 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:31:10.047192    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xnr6t\" (UniqueName: \"kubernetes.io/projected/ba5c9b0b-507d-4f15-a924-699f630663ab-kube-api-access-xnr6t\") on node \"ip-10-0-139-182.ec2.internal\" DevicePath \"\""
Sep 16 19:31:10.047231 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:31:10.047226    2584 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ba5c9b0b-507d-4f15-a924-699f630663ab-kube-rbac-proxy-sar-config\") on node \"ip-10-0-139-182.ec2.internal\" DevicePath \"\""
Sep 16 19:31:10.047231 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:31:10.047236    2584 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ba5c9b0b-507d-4f15-a924-699f630663ab-proxy-tls\") on node \"ip-10-0-139-182.ec2.internal\" DevicePath \"\""
Sep 16 19:31:10.091037 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:31:10.091001    2584 generic.go:356] "Generic (PLEG): container finished" podID="ba5c9b0b-507d-4f15-a924-699f630663ab" containerID="d419ad234dff341b449769dbbe03fbac773ccea93bafa209c84899e861147735" exitCode=0
Sep 16 19:31:10.091186 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:31:10.091057    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-c3e3d-predictor-866dc5fdb5-sp6mk"
Sep 16 19:31:10.091186 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:31:10.091080    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-c3e3d-predictor-866dc5fdb5-sp6mk" event={"ID":"ba5c9b0b-507d-4f15-a924-699f630663ab","Type":"ContainerDied","Data":"d419ad234dff341b449769dbbe03fbac773ccea93bafa209c84899e861147735"}
Sep 16 19:31:10.091186 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:31:10.091140    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-c3e3d-predictor-866dc5fdb5-sp6mk" event={"ID":"ba5c9b0b-507d-4f15-a924-699f630663ab","Type":"ContainerDied","Data":"8a32ce810e50df0287c5388d65c039614e2242f230f0ae2e7b08e193c97c1f92"}
Sep 16 19:31:10.091186 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:31:10.091167    2584 scope.go:117] "RemoveContainer" containerID="4b675cfff62e93427f441be8062bd49d1ff0f5d723d2f8a13fe1390645773a5d"
Sep 16 19:31:10.099882 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:31:10.099866    2584 scope.go:117] "RemoveContainer" containerID="d419ad234dff341b449769dbbe03fbac773ccea93bafa209c84899e861147735"
Sep 16 19:31:10.107372 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:31:10.107352    2584 scope.go:117] "RemoveContainer" containerID="4b675cfff62e93427f441be8062bd49d1ff0f5d723d2f8a13fe1390645773a5d"
Sep 16 19:31:10.107629 ip-10-0-139-182 kubenswrapper[2584]: E0916 19:31:10.107608    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4b675cfff62e93427f441be8062bd49d1ff0f5d723d2f8a13fe1390645773a5d\": container with ID starting with 4b675cfff62e93427f441be8062bd49d1ff0f5d723d2f8a13fe1390645773a5d not found: ID does not exist" containerID="4b675cfff62e93427f441be8062bd49d1ff0f5d723d2f8a13fe1390645773a5d"
Sep 16 19:31:10.107674 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:31:10.107638    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4b675cfff62e93427f441be8062bd49d1ff0f5d723d2f8a13fe1390645773a5d"} err="failed to get container status \"4b675cfff62e93427f441be8062bd49d1ff0f5d723d2f8a13fe1390645773a5d\": rpc error: code = NotFound desc = could not find container \"4b675cfff62e93427f441be8062bd49d1ff0f5d723d2f8a13fe1390645773a5d\": container with ID starting with 4b675cfff62e93427f441be8062bd49d1ff0f5d723d2f8a13fe1390645773a5d not found: ID does not exist"
Sep 16 19:31:10.107674 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:31:10.107654    2584 scope.go:117] "RemoveContainer" containerID="d419ad234dff341b449769dbbe03fbac773ccea93bafa209c84899e861147735"
Sep 16 19:31:10.107944 ip-10-0-139-182 kubenswrapper[2584]: E0916 19:31:10.107925    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d419ad234dff341b449769dbbe03fbac773ccea93bafa209c84899e861147735\": container with ID starting with d419ad234dff341b449769dbbe03fbac773ccea93bafa209c84899e861147735 not found: ID does not exist" containerID="d419ad234dff341b449769dbbe03fbac773ccea93bafa209c84899e861147735"
Sep 16 19:31:10.108014 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:31:10.107948    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d419ad234dff341b449769dbbe03fbac773ccea93bafa209c84899e861147735"} err="failed to get container status \"d419ad234dff341b449769dbbe03fbac773ccea93bafa209c84899e861147735\": rpc error: code = NotFound desc = could not find container \"d419ad234dff341b449769dbbe03fbac773ccea93bafa209c84899e861147735\": container with ID starting with d419ad234dff341b449769dbbe03fbac773ccea93bafa209c84899e861147735 not found: ID does not exist"
Sep 16 19:31:10.117960 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:31:10.117934    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-c3e3d-predictor-866dc5fdb5-sp6mk"]
Sep 16 19:31:10.118285 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:31:10.118266    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-c3e3d-predictor-866dc5fdb5-sp6mk"]
Sep 16 19:31:12.116123 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:31:12.116095    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ba5c9b0b-507d-4f15-a924-699f630663ab" path="/var/lib/kubelet/pods/ba5c9b0b-507d-4f15-a924-699f630663ab/volumes"
Sep 16 19:31:13.882282 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:31:13.882237    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-c3e3d-695cdcbd4-kchln" podUID="ebb65cb2-c696-42ff-b2b9-7e5407f6cbbd" containerName="splitter-graph-c3e3d" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 16 19:31:18.881304 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:31:18.881264    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-c3e3d-695cdcbd4-kchln" podUID="ebb65cb2-c696-42ff-b2b9-7e5407f6cbbd" containerName="splitter-graph-c3e3d" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 16 19:31:18.881714 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:31:18.881393    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/splitter-graph-c3e3d-695cdcbd4-kchln"
Sep 16 19:31:23.882370 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:31:23.882325    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-c3e3d-695cdcbd4-kchln" podUID="ebb65cb2-c696-42ff-b2b9-7e5407f6cbbd" containerName="splitter-graph-c3e3d" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 16 19:31:28.881592 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:31:28.881547    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-c3e3d-695cdcbd4-kchln" podUID="ebb65cb2-c696-42ff-b2b9-7e5407f6cbbd" containerName="splitter-graph-c3e3d" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 16 19:31:33.881912 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:31:33.881869    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-c3e3d-695cdcbd4-kchln" podUID="ebb65cb2-c696-42ff-b2b9-7e5407f6cbbd" containerName="splitter-graph-c3e3d" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 16 19:31:37.208226 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:31:37.208191    2584 generic.go:356] "Generic (PLEG): container finished" podID="ebb65cb2-c696-42ff-b2b9-7e5407f6cbbd" containerID="d6d8e1494ed1749723c4b5c8ab0fd4370f25cc63a043520e810df95464a921a8" exitCode=0
Sep 16 19:31:37.208672 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:31:37.208234    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-c3e3d-695cdcbd4-kchln" event={"ID":"ebb65cb2-c696-42ff-b2b9-7e5407f6cbbd","Type":"ContainerDied","Data":"d6d8e1494ed1749723c4b5c8ab0fd4370f25cc63a043520e810df95464a921a8"}
Sep 16 19:31:37.255852 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:31:37.255816    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-c3e3d-695cdcbd4-kchln"
Sep 16 19:31:37.378779 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:31:37.378681    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ebb65cb2-c696-42ff-b2b9-7e5407f6cbbd-proxy-tls\") pod \"ebb65cb2-c696-42ff-b2b9-7e5407f6cbbd\" (UID: \"ebb65cb2-c696-42ff-b2b9-7e5407f6cbbd\") "
Sep 16 19:31:37.378779 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:31:37.378734    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ebb65cb2-c696-42ff-b2b9-7e5407f6cbbd-openshift-service-ca-bundle\") pod \"ebb65cb2-c696-42ff-b2b9-7e5407f6cbbd\" (UID: \"ebb65cb2-c696-42ff-b2b9-7e5407f6cbbd\") "
Sep 16 19:31:37.379137 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:31:37.379112    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ebb65cb2-c696-42ff-b2b9-7e5407f6cbbd-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "ebb65cb2-c696-42ff-b2b9-7e5407f6cbbd" (UID: "ebb65cb2-c696-42ff-b2b9-7e5407f6cbbd"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 19:31:37.380739 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:31:37.380720    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ebb65cb2-c696-42ff-b2b9-7e5407f6cbbd-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "ebb65cb2-c696-42ff-b2b9-7e5407f6cbbd" (UID: "ebb65cb2-c696-42ff-b2b9-7e5407f6cbbd"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 19:31:37.480304 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:31:37.480248    2584 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ebb65cb2-c696-42ff-b2b9-7e5407f6cbbd-proxy-tls\") on node \"ip-10-0-139-182.ec2.internal\" DevicePath \"\""
Sep 16 19:31:37.480304 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:31:37.480302    2584 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ebb65cb2-c696-42ff-b2b9-7e5407f6cbbd-openshift-service-ca-bundle\") on node \"ip-10-0-139-182.ec2.internal\" DevicePath \"\""
Sep 16 19:31:38.213205 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:31:38.213167    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-c3e3d-695cdcbd4-kchln" event={"ID":"ebb65cb2-c696-42ff-b2b9-7e5407f6cbbd","Type":"ContainerDied","Data":"2e0abdf1fd42c022368ca97b6285c3720fdc6205246061cc7717184d570adef0"}
Sep 16 19:31:38.213639 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:31:38.213234    2584 scope.go:117] "RemoveContainer" containerID="d6d8e1494ed1749723c4b5c8ab0fd4370f25cc63a043520e810df95464a921a8"
Sep 16 19:31:38.213639 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:31:38.213175    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-c3e3d-695cdcbd4-kchln"
Sep 16 19:31:38.242249 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:31:38.242220    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-c3e3d-695cdcbd4-kchln"]
Sep 16 19:31:38.247252 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:31:38.247229    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-c3e3d-695cdcbd4-kchln"]
Sep 16 19:31:40.118714 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:31:40.118458    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ebb65cb2-c696-42ff-b2b9-7e5407f6cbbd" path="/var/lib/kubelet/pods/ebb65cb2-c696-42ff-b2b9-7e5407f6cbbd/volumes"
Sep 16 19:34:13.208506 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:34:13.208433    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 19:34:36.271180 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:34:36.271149    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-crjw7_ecf273b5-b465-4008-b035-f7ac0e740d58/console-operator/1.log"
Sep 16 19:34:36.278115 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:34:36.278087    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-crjw7_ecf273b5-b465-4008-b035-f7ac0e740d58/console-operator/1.log"
Sep 16 19:35:42.804206 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:35:42.804167    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 19:38:40.470300 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:38:40.470252    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/switch-graph-0263c-6d797d88cf-8bs4n"]
Sep 16 19:38:40.470850 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:38:40.470499    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/switch-graph-0263c-6d797d88cf-8bs4n" podUID="50df7308-89a9-4106-854e-f259be58ad45" containerName="switch-graph-0263c" containerID="cri-o://0e33648777ae79675e0b2ce490e66cc62d9569906c0a90e82bf6154caa0f45e3" gracePeriod=30
Sep 16 19:38:40.612931 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:38:40.612887    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-0263c-predictor-7bfb46579c-n6fgr"]
Sep 16 19:38:40.613414 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:38:40.613363    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-0263c-predictor-7bfb46579c-n6fgr" podUID="cb9ea1f8-55c4-49d4-8699-a23e889dd5fb" containerName="kserve-container" containerID="cri-o://790d21a48ba7fd6a9b17013a6163a78f7d8179ca1ab6e39136c0187f4bfda729" gracePeriod=30
Sep 16 19:38:40.613622 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:38:40.613457    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-0263c-predictor-7bfb46579c-n6fgr" podUID="cb9ea1f8-55c4-49d4-8699-a23e889dd5fb" containerName="kube-rbac-proxy" containerID="cri-o://a269bc4b2c6503b0fe3992b908f6320f268bfb87a77f8ed504534cb88f6164d8" gracePeriod=30
Sep 16 19:38:40.974329 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:38:40.974290    2584 generic.go:356] "Generic (PLEG): container finished" podID="cb9ea1f8-55c4-49d4-8699-a23e889dd5fb" containerID="a269bc4b2c6503b0fe3992b908f6320f268bfb87a77f8ed504534cb88f6164d8" exitCode=2
Sep 16 19:38:40.974497 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:38:40.974358    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-0263c-predictor-7bfb46579c-n6fgr" event={"ID":"cb9ea1f8-55c4-49d4-8699-a23e889dd5fb","Type":"ContainerDied","Data":"a269bc4b2c6503b0fe3992b908f6320f268bfb87a77f8ed504534cb88f6164d8"}
Sep 16 19:38:41.239541 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:38:41.239510    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 19:38:41.304639 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:38:41.304605    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 19:38:41.413077 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:38:41.413045    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 19:38:41.476475 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:38:41.476438    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 19:38:43.758479 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:38:43.758441    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-0263c-6d797d88cf-8bs4n" podUID="50df7308-89a9-4106-854e-f259be58ad45" containerName="switch-graph-0263c" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 16 19:38:43.990020 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:38:43.989988    2584 generic.go:356] "Generic (PLEG): container finished" podID="cb9ea1f8-55c4-49d4-8699-a23e889dd5fb" containerID="790d21a48ba7fd6a9b17013a6163a78f7d8179ca1ab6e39136c0187f4bfda729" exitCode=0
Sep 16 19:38:43.990208 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:38:43.990048    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-0263c-predictor-7bfb46579c-n6fgr" event={"ID":"cb9ea1f8-55c4-49d4-8699-a23e889dd5fb","Type":"ContainerDied","Data":"790d21a48ba7fd6a9b17013a6163a78f7d8179ca1ab6e39136c0187f4bfda729"}
Sep 16 19:38:44.067740 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:38:44.067712    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-0263c-predictor-7bfb46579c-n6fgr"
Sep 16 19:38:44.140174 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:38:44.140139    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/cb9ea1f8-55c4-49d4-8699-a23e889dd5fb-proxy-tls\") pod \"cb9ea1f8-55c4-49d4-8699-a23e889dd5fb\" (UID: \"cb9ea1f8-55c4-49d4-8699-a23e889dd5fb\") "
Sep 16 19:38:44.140346 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:38:44.140179    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/cb9ea1f8-55c4-49d4-8699-a23e889dd5fb-kube-rbac-proxy-sar-config\") pod \"cb9ea1f8-55c4-49d4-8699-a23e889dd5fb\" (UID: \"cb9ea1f8-55c4-49d4-8699-a23e889dd5fb\") "
Sep 16 19:38:44.140346 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:38:44.140207    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7hzbl\" (UniqueName: \"kubernetes.io/projected/cb9ea1f8-55c4-49d4-8699-a23e889dd5fb-kube-api-access-7hzbl\") pod \"cb9ea1f8-55c4-49d4-8699-a23e889dd5fb\" (UID: \"cb9ea1f8-55c4-49d4-8699-a23e889dd5fb\") "
Sep 16 19:38:44.140585 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:38:44.140560    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cb9ea1f8-55c4-49d4-8699-a23e889dd5fb-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "cb9ea1f8-55c4-49d4-8699-a23e889dd5fb" (UID: "cb9ea1f8-55c4-49d4-8699-a23e889dd5fb"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 19:38:44.142315 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:38:44.142288    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cb9ea1f8-55c4-49d4-8699-a23e889dd5fb-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "cb9ea1f8-55c4-49d4-8699-a23e889dd5fb" (UID: "cb9ea1f8-55c4-49d4-8699-a23e889dd5fb"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 19:38:44.142408 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:38:44.142348    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cb9ea1f8-55c4-49d4-8699-a23e889dd5fb-kube-api-access-7hzbl" (OuterVolumeSpecName: "kube-api-access-7hzbl") pod "cb9ea1f8-55c4-49d4-8699-a23e889dd5fb" (UID: "cb9ea1f8-55c4-49d4-8699-a23e889dd5fb"). InnerVolumeSpecName "kube-api-access-7hzbl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 19:38:44.240800 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:38:44.240763    2584 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/cb9ea1f8-55c4-49d4-8699-a23e889dd5fb-proxy-tls\") on node \"ip-10-0-139-182.ec2.internal\" DevicePath \"\""
Sep 16 19:38:44.240800 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:38:44.240795    2584 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/cb9ea1f8-55c4-49d4-8699-a23e889dd5fb-kube-rbac-proxy-sar-config\") on node \"ip-10-0-139-182.ec2.internal\" DevicePath \"\""
Sep 16 19:38:44.240800 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:38:44.240805    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7hzbl\" (UniqueName: \"kubernetes.io/projected/cb9ea1f8-55c4-49d4-8699-a23e889dd5fb-kube-api-access-7hzbl\") on node \"ip-10-0-139-182.ec2.internal\" DevicePath \"\""
Sep 16 19:38:44.995622 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:38:44.995587    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-0263c-predictor-7bfb46579c-n6fgr" event={"ID":"cb9ea1f8-55c4-49d4-8699-a23e889dd5fb","Type":"ContainerDied","Data":"2ea1ae2218ae51208cd9336d66fee719da4c1b5bc80e8806a4243080e7f47094"}
Sep 16 19:38:44.995622 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:38:44.995612    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-0263c-predictor-7bfb46579c-n6fgr"
Sep 16 19:38:44.996132 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:38:44.995655    2584 scope.go:117] "RemoveContainer" containerID="a269bc4b2c6503b0fe3992b908f6320f268bfb87a77f8ed504534cb88f6164d8"
Sep 16 19:38:45.004949 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:38:45.004924    2584 scope.go:117] "RemoveContainer" containerID="790d21a48ba7fd6a9b17013a6163a78f7d8179ca1ab6e39136c0187f4bfda729"
Sep 16 19:38:45.018385 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:38:45.018355    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-0263c-predictor-7bfb46579c-n6fgr"]
Sep 16 19:38:45.022559 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:38:45.022531    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-0263c-predictor-7bfb46579c-n6fgr"]
Sep 16 19:38:46.116023 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:38:46.115987    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cb9ea1f8-55c4-49d4-8699-a23e889dd5fb" path="/var/lib/kubelet/pods/cb9ea1f8-55c4-49d4-8699-a23e889dd5fb/volumes"
Sep 16 19:38:48.758366 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:38:48.758318    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-0263c-6d797d88cf-8bs4n" podUID="50df7308-89a9-4106-854e-f259be58ad45" containerName="switch-graph-0263c" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 16 19:38:53.758054 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:38:53.758012    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-0263c-6d797d88cf-8bs4n" podUID="50df7308-89a9-4106-854e-f259be58ad45" containerName="switch-graph-0263c" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 16 19:38:53.758551 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:38:53.758143    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/switch-graph-0263c-6d797d88cf-8bs4n"
Sep 16 19:38:57.615157 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:38:57.615119    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-0263c-6d797d88cf-8bs4n_50df7308-89a9-4106-854e-f259be58ad45/switch-graph-0263c/0.log"
Sep 16 19:38:58.457039 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:38:58.457004    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-0263c-6d797d88cf-8bs4n_50df7308-89a9-4106-854e-f259be58ad45/switch-graph-0263c/0.log"
Sep 16 19:38:58.758898 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:38:58.758784    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-0263c-6d797d88cf-8bs4n" podUID="50df7308-89a9-4106-854e-f259be58ad45" containerName="switch-graph-0263c" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 16 19:38:59.311253 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:38:59.311214    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-0263c-6d797d88cf-8bs4n_50df7308-89a9-4106-854e-f259be58ad45/switch-graph-0263c/0.log"
Sep 16 19:39:00.152223 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:00.152180    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-0263c-6d797d88cf-8bs4n_50df7308-89a9-4106-854e-f259be58ad45/switch-graph-0263c/0.log"
Sep 16 19:39:00.988612 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:00.988578    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-0263c-6d797d88cf-8bs4n_50df7308-89a9-4106-854e-f259be58ad45/switch-graph-0263c/0.log"
Sep 16 19:39:01.830487 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:01.830447    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-0263c-6d797d88cf-8bs4n_50df7308-89a9-4106-854e-f259be58ad45/switch-graph-0263c/0.log"
Sep 16 19:39:02.660344 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:02.660308    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-0263c-6d797d88cf-8bs4n_50df7308-89a9-4106-854e-f259be58ad45/switch-graph-0263c/0.log"
Sep 16 19:39:03.505693 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:03.505658    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-0263c-6d797d88cf-8bs4n_50df7308-89a9-4106-854e-f259be58ad45/switch-graph-0263c/0.log"
Sep 16 19:39:03.758761 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:03.758668    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-0263c-6d797d88cf-8bs4n" podUID="50df7308-89a9-4106-854e-f259be58ad45" containerName="switch-graph-0263c" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 16 19:39:04.332049 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:04.332015    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-0263c-6d797d88cf-8bs4n_50df7308-89a9-4106-854e-f259be58ad45/switch-graph-0263c/0.log"
Sep 16 19:39:05.146940 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:05.146907    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-0263c-6d797d88cf-8bs4n_50df7308-89a9-4106-854e-f259be58ad45/switch-graph-0263c/0.log"
Sep 16 19:39:05.979129 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:05.979098    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-0263c-6d797d88cf-8bs4n_50df7308-89a9-4106-854e-f259be58ad45/switch-graph-0263c/0.log"
Sep 16 19:39:06.791878 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:06.791845    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-0263c-6d797d88cf-8bs4n_50df7308-89a9-4106-854e-f259be58ad45/switch-graph-0263c/0.log"
Sep 16 19:39:07.635701 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:07.635669    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-0263c-6d797d88cf-8bs4n_50df7308-89a9-4106-854e-f259be58ad45/switch-graph-0263c/0.log"
Sep 16 19:39:08.516911 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:08.516877    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-0263c-6d797d88cf-8bs4n_50df7308-89a9-4106-854e-f259be58ad45/switch-graph-0263c/0.log"
Sep 16 19:39:08.758124 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:08.758074    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-0263c-6d797d88cf-8bs4n" podUID="50df7308-89a9-4106-854e-f259be58ad45" containerName="switch-graph-0263c" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 16 19:39:09.392898 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:09.392863    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-0263c-6d797d88cf-8bs4n_50df7308-89a9-4106-854e-f259be58ad45/switch-graph-0263c/0.log"
Sep 16 19:39:10.617945 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:10.617916    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-0263c-6d797d88cf-8bs4n"
Sep 16 19:39:10.658317 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:10.658282    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/50df7308-89a9-4106-854e-f259be58ad45-proxy-tls\") pod \"50df7308-89a9-4106-854e-f259be58ad45\" (UID: \"50df7308-89a9-4106-854e-f259be58ad45\") "
Sep 16 19:39:10.658491 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:10.658378    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/50df7308-89a9-4106-854e-f259be58ad45-openshift-service-ca-bundle\") pod \"50df7308-89a9-4106-854e-f259be58ad45\" (UID: \"50df7308-89a9-4106-854e-f259be58ad45\") "
Sep 16 19:39:10.658669 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:10.658646    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/50df7308-89a9-4106-854e-f259be58ad45-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "50df7308-89a9-4106-854e-f259be58ad45" (UID: "50df7308-89a9-4106-854e-f259be58ad45"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 19:39:10.660378 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:10.660355    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/50df7308-89a9-4106-854e-f259be58ad45-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "50df7308-89a9-4106-854e-f259be58ad45" (UID: "50df7308-89a9-4106-854e-f259be58ad45"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 19:39:10.759733 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:10.759698    2584 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/50df7308-89a9-4106-854e-f259be58ad45-openshift-service-ca-bundle\") on node \"ip-10-0-139-182.ec2.internal\" DevicePath \"\""
Sep 16 19:39:10.759733 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:10.759733    2584 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/50df7308-89a9-4106-854e-f259be58ad45-proxy-tls\") on node \"ip-10-0-139-182.ec2.internal\" DevicePath \"\""
Sep 16 19:39:11.109320 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:11.109240    2584 generic.go:356] "Generic (PLEG): container finished" podID="50df7308-89a9-4106-854e-f259be58ad45" containerID="0e33648777ae79675e0b2ce490e66cc62d9569906c0a90e82bf6154caa0f45e3" exitCode=0
Sep 16 19:39:11.109320 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:11.109288    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-0263c-6d797d88cf-8bs4n"
Sep 16 19:39:11.109320 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:11.109297    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-0263c-6d797d88cf-8bs4n" event={"ID":"50df7308-89a9-4106-854e-f259be58ad45","Type":"ContainerDied","Data":"0e33648777ae79675e0b2ce490e66cc62d9569906c0a90e82bf6154caa0f45e3"}
Sep 16 19:39:11.109542 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:11.109337    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-0263c-6d797d88cf-8bs4n" event={"ID":"50df7308-89a9-4106-854e-f259be58ad45","Type":"ContainerDied","Data":"d926b46e3a287c94e13d29c5df8fb4b829d8ea61ca721931002b3dac6210d5ff"}
Sep 16 19:39:11.109542 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:11.109358    2584 scope.go:117] "RemoveContainer" containerID="0e33648777ae79675e0b2ce490e66cc62d9569906c0a90e82bf6154caa0f45e3"
Sep 16 19:39:11.118410 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:11.118392    2584 scope.go:117] "RemoveContainer" containerID="0e33648777ae79675e0b2ce490e66cc62d9569906c0a90e82bf6154caa0f45e3"
Sep 16 19:39:11.118651 ip-10-0-139-182 kubenswrapper[2584]: E0916 19:39:11.118632    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0e33648777ae79675e0b2ce490e66cc62d9569906c0a90e82bf6154caa0f45e3\": container with ID starting with 0e33648777ae79675e0b2ce490e66cc62d9569906c0a90e82bf6154caa0f45e3 not found: ID does not exist" containerID="0e33648777ae79675e0b2ce490e66cc62d9569906c0a90e82bf6154caa0f45e3"
Sep 16 19:39:11.118700 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:11.118660    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0e33648777ae79675e0b2ce490e66cc62d9569906c0a90e82bf6154caa0f45e3"} err="failed to get container status \"0e33648777ae79675e0b2ce490e66cc62d9569906c0a90e82bf6154caa0f45e3\": rpc error: code = NotFound desc = could not find container \"0e33648777ae79675e0b2ce490e66cc62d9569906c0a90e82bf6154caa0f45e3\": container with ID starting with 0e33648777ae79675e0b2ce490e66cc62d9569906c0a90e82bf6154caa0f45e3 not found: ID does not exist"
Sep 16 19:39:11.134246 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:11.134214    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/switch-graph-0263c-6d797d88cf-8bs4n"]
Sep 16 19:39:11.139858 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:11.139816    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/switch-graph-0263c-6d797d88cf-8bs4n"]
Sep 16 19:39:12.009358 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:12.009321    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 19:39:12.116048 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:12.116007    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="50df7308-89a9-4106-854e-f259be58ad45" path="/var/lib/kubelet/pods/50df7308-89a9-4106-854e-f259be58ad45/volumes"
Sep 16 19:39:12.143299 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:12.143261    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-9fjpm/must-gather-k9jp7"]
Sep 16 19:39:12.143643 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:12.143625    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="50df7308-89a9-4106-854e-f259be58ad45" containerName="switch-graph-0263c"
Sep 16 19:39:12.143725 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:12.143645    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="50df7308-89a9-4106-854e-f259be58ad45" containerName="switch-graph-0263c"
Sep 16 19:39:12.143725 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:12.143661    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cb9ea1f8-55c4-49d4-8699-a23e889dd5fb" containerName="kube-rbac-proxy"
Sep 16 19:39:12.143725 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:12.143670    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="cb9ea1f8-55c4-49d4-8699-a23e889dd5fb" containerName="kube-rbac-proxy"
Sep 16 19:39:12.143725 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:12.143685    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ba5c9b0b-507d-4f15-a924-699f630663ab" containerName="kube-rbac-proxy"
Sep 16 19:39:12.143725 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:12.143694    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="ba5c9b0b-507d-4f15-a924-699f630663ab" containerName="kube-rbac-proxy"
Sep 16 19:39:12.143725 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:12.143707    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ebb65cb2-c696-42ff-b2b9-7e5407f6cbbd" containerName="splitter-graph-c3e3d"
Sep 16 19:39:12.143725 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:12.143718    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="ebb65cb2-c696-42ff-b2b9-7e5407f6cbbd" containerName="splitter-graph-c3e3d"
Sep 16 19:39:12.144107 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:12.143728    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ba5c9b0b-507d-4f15-a924-699f630663ab" containerName="kserve-container"
Sep 16 19:39:12.144107 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:12.143739    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="ba5c9b0b-507d-4f15-a924-699f630663ab" containerName="kserve-container"
Sep 16 19:39:12.144107 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:12.143754    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cb9ea1f8-55c4-49d4-8699-a23e889dd5fb" containerName="kserve-container"
Sep 16 19:39:12.144107 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:12.143763    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="cb9ea1f8-55c4-49d4-8699-a23e889dd5fb" containerName="kserve-container"
Sep 16 19:39:12.144107 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:12.143870    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="cb9ea1f8-55c4-49d4-8699-a23e889dd5fb" containerName="kserve-container"
Sep 16 19:39:12.144107 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:12.143885    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="cb9ea1f8-55c4-49d4-8699-a23e889dd5fb" containerName="kube-rbac-proxy"
Sep 16 19:39:12.144107 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:12.143896    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="ba5c9b0b-507d-4f15-a924-699f630663ab" containerName="kserve-container"
Sep 16 19:39:12.144107 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:12.143908    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="50df7308-89a9-4106-854e-f259be58ad45" containerName="switch-graph-0263c"
Sep 16 19:39:12.144107 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:12.143919    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="ba5c9b0b-507d-4f15-a924-699f630663ab" containerName="kube-rbac-proxy"
Sep 16 19:39:12.144107 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:12.143930    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="ebb65cb2-c696-42ff-b2b9-7e5407f6cbbd" containerName="splitter-graph-c3e3d"
Sep 16 19:39:12.148587 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:12.148561    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-9fjpm/must-gather-k9jp7"
Sep 16 19:39:12.151292 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:12.151268    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-9fjpm\"/\"kube-root-ca.crt\""
Sep 16 19:39:12.152465 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:12.152426    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-9fjpm\"/\"default-dockercfg-64szc\""
Sep 16 19:39:12.152465 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:12.152445    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-9fjpm\"/\"openshift-service-ca.crt\""
Sep 16 19:39:12.154695 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:12.154674    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-9fjpm/must-gather-k9jp7"]
Sep 16 19:39:12.158488 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:12.158454    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 19:39:12.255331 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:12.255290    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 19:39:12.271523 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:12.271459    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8d862\" (UniqueName: \"kubernetes.io/projected/9241ddbd-327d-46fd-b4df-135e8dcaa984-kube-api-access-8d862\") pod \"must-gather-k9jp7\" (UID: \"9241ddbd-327d-46fd-b4df-135e8dcaa984\") " pod="openshift-must-gather-9fjpm/must-gather-k9jp7"
Sep 16 19:39:12.271523 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:12.271495    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/9241ddbd-327d-46fd-b4df-135e8dcaa984-must-gather-output\") pod \"must-gather-k9jp7\" (UID: \"9241ddbd-327d-46fd-b4df-135e8dcaa984\") " pod="openshift-must-gather-9fjpm/must-gather-k9jp7"
Sep 16 19:39:12.372498 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:12.372463    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8d862\" (UniqueName: \"kubernetes.io/projected/9241ddbd-327d-46fd-b4df-135e8dcaa984-kube-api-access-8d862\") pod \"must-gather-k9jp7\" (UID: \"9241ddbd-327d-46fd-b4df-135e8dcaa984\") " pod="openshift-must-gather-9fjpm/must-gather-k9jp7"
Sep 16 19:39:12.372498 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:12.372499    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/9241ddbd-327d-46fd-b4df-135e8dcaa984-must-gather-output\") pod \"must-gather-k9jp7\" (UID: \"9241ddbd-327d-46fd-b4df-135e8dcaa984\") " pod="openshift-must-gather-9fjpm/must-gather-k9jp7"
Sep 16 19:39:12.372844 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:12.372815    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/9241ddbd-327d-46fd-b4df-135e8dcaa984-must-gather-output\") pod \"must-gather-k9jp7\" (UID: \"9241ddbd-327d-46fd-b4df-135e8dcaa984\") " pod="openshift-must-gather-9fjpm/must-gather-k9jp7"
Sep 16 19:39:12.380970 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:12.380939    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8d862\" (UniqueName: \"kubernetes.io/projected/9241ddbd-327d-46fd-b4df-135e8dcaa984-kube-api-access-8d862\") pod \"must-gather-k9jp7\" (UID: \"9241ddbd-327d-46fd-b4df-135e8dcaa984\") " pod="openshift-must-gather-9fjpm/must-gather-k9jp7"
Sep 16 19:39:12.458855 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:12.458797    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-9fjpm/must-gather-k9jp7"
Sep 16 19:39:12.730647 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:12.730619    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-9fjpm/must-gather-k9jp7"]
Sep 16 19:39:12.733234 ip-10-0-139-182 kubenswrapper[2584]: W0916 19:39:12.733208    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9241ddbd_327d_46fd_b4df_135e8dcaa984.slice/crio-ef43936e31f17bfb3c8bc9593e722d7a761978dfb602c1c145edaa05a9d8bebf WatchSource:0}: Error finding container ef43936e31f17bfb3c8bc9593e722d7a761978dfb602c1c145edaa05a9d8bebf: Status 404 returned error can't find the container with id ef43936e31f17bfb3c8bc9593e722d7a761978dfb602c1c145edaa05a9d8bebf
Sep 16 19:39:12.735245 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:12.735226    2584 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 16 19:39:13.120374 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:13.120341    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-9fjpm/must-gather-k9jp7" event={"ID":"9241ddbd-327d-46fd-b4df-135e8dcaa984","Type":"ContainerStarted","Data":"ef43936e31f17bfb3c8bc9593e722d7a761978dfb602c1c145edaa05a9d8bebf"}
Sep 16 19:39:14.125795 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:14.125754    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-9fjpm/must-gather-k9jp7" event={"ID":"9241ddbd-327d-46fd-b4df-135e8dcaa984","Type":"ContainerStarted","Data":"d17d145823dfe272f011a388e84c154365c9bdd379503f39cdc596aabbc114f5"}
Sep 16 19:39:14.125795 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:14.125801    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-9fjpm/must-gather-k9jp7" event={"ID":"9241ddbd-327d-46fd-b4df-135e8dcaa984","Type":"ContainerStarted","Data":"4ff0557e153ba723e8b42ae86c1cfe08f03b200354e9bae7f2e40cd8990957bc"}
Sep 16 19:39:14.141377 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:14.141312    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-9fjpm/must-gather-k9jp7" podStartSLOduration=1.459694472 podStartE2EDuration="2.141290387s" totalImagesPullingTime="681.595915ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 19:39:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 19:39:14.140740124 +0000 UTC m=+3284.470242531" watchObservedRunningTime="2026-09-16 19:39:14.141290387 +0000 UTC m=+3284.470792801"
Sep 16 19:39:14.944080 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:14.944038    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-t69xl_b9f0f3b9-87ee-44a5-a66c-6542791f3506/global-pull-secret-syncer/0.log"
Sep 16 19:39:14.991100 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:14.991047    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log"
Sep 16 19:39:15.147269 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:15.147236    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-139-182.ec2.internal_228e51ceefc0d99659afe6cee7fbb736/haproxy/0.log"
Sep 16 19:39:17.493238 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:17.493189    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 19:39:19.159081 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:19.159002    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-59fbdd77d-9h5nf_07294397-1424-41ee-b652-aec35f466d18/kube-state-metrics/0.log"
Sep 16 19:39:19.192721 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:19.192684    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-59fbdd77d-9h5nf_07294397-1424-41ee-b652-aec35f466d18/kube-rbac-proxy-main/0.log"
Sep 16 19:39:19.227262 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:19.227178    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-59fbdd77d-9h5nf_07294397-1424-41ee-b652-aec35f466d18/kube-rbac-proxy-self/0.log"
Sep 16 19:39:19.345757 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:19.345668    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6klb4_828d6bf1-3d76-4c9b-af83-07de04f8f20f/konnectivity-agent/0.log" limit=1024
Sep 16 19:39:19.561576 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:19.561547    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-qbpp2_3097f2a3-e38f-4abe-83dc-bcdc64d46c45/node-exporter/0.log"
Sep 16 19:39:19.586121 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:19.586076    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-qbpp2_3097f2a3-e38f-4abe-83dc-bcdc64d46c45/kube-rbac-proxy/0.log"
Sep 16 19:39:19.612819 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:19.612735    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-qbpp2_3097f2a3-e38f-4abe-83dc-bcdc64d46c45/init-textfile/0.log"
Sep 16 19:39:20.194407 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:20.194372    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6c9b6bc889-kslpj_64f114cb-62c3-4547-8385-8259c4d9537d/thanos-query/0.log"
Sep 16 19:39:20.220184 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:20.220153    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6c9b6bc889-kslpj_64f114cb-62c3-4547-8385-8259c4d9537d/kube-rbac-proxy-web/0.log"
Sep 16 19:39:20.246103 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:20.246067    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6c9b6bc889-kslpj_64f114cb-62c3-4547-8385-8259c4d9537d/kube-rbac-proxy/0.log"
Sep 16 19:39:20.273256 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:20.273214    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6c9b6bc889-kslpj_64f114cb-62c3-4547-8385-8259c4d9537d/prom-label-proxy/0.log"
Sep 16 19:39:20.300602 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:20.300562    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6c9b6bc889-kslpj_64f114cb-62c3-4547-8385-8259c4d9537d/kube-rbac-proxy-rules/0.log"
Sep 16 19:39:20.329668 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:20.329639    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6c9b6bc889-kslpj_64f114cb-62c3-4547-8385-8259c4d9537d/kube-rbac-proxy-metrics/0.log"
Sep 16 19:39:22.265441 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:22.265408    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-crjw7_ecf273b5-b465-4008-b035-f7ac0e740d58/console-operator/1.log"
Sep 16 19:39:22.273164 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:22.273128    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-crjw7_ecf273b5-b465-4008-b035-f7ac0e740d58/console-operator/2.log"
Sep 16 19:39:22.718933 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:22.718839    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-54df7c6d-nl4g9_3e77e0b9-7198-4a61-9c43-f84eccf663dd/console/0.log"
Sep 16 19:39:23.496674 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:23.496625    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-9fjpm/perf-node-gather-daemonset-nb8bc"]
Sep 16 19:39:23.501541 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:23.501507    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-9fjpm/perf-node-gather-daemonset-nb8bc"
Sep 16 19:39:23.508387 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:23.508351    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-9fjpm/perf-node-gather-daemonset-nb8bc"]
Sep 16 19:39:23.589057 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:23.589010    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/666616ea-cb5f-4a2d-9caa-0ecec4a9daca-proc\") pod \"perf-node-gather-daemonset-nb8bc\" (UID: \"666616ea-cb5f-4a2d-9caa-0ecec4a9daca\") " pod="openshift-must-gather-9fjpm/perf-node-gather-daemonset-nb8bc"
Sep 16 19:39:23.589266 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:23.589114    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sq5gw\" (UniqueName: \"kubernetes.io/projected/666616ea-cb5f-4a2d-9caa-0ecec4a9daca-kube-api-access-sq5gw\") pod \"perf-node-gather-daemonset-nb8bc\" (UID: \"666616ea-cb5f-4a2d-9caa-0ecec4a9daca\") " pod="openshift-must-gather-9fjpm/perf-node-gather-daemonset-nb8bc"
Sep 16 19:39:23.589266 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:23.589146    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/666616ea-cb5f-4a2d-9caa-0ecec4a9daca-podres\") pod \"perf-node-gather-daemonset-nb8bc\" (UID: \"666616ea-cb5f-4a2d-9caa-0ecec4a9daca\") " pod="openshift-must-gather-9fjpm/perf-node-gather-daemonset-nb8bc"
Sep 16 19:39:23.589266 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:23.589181    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/666616ea-cb5f-4a2d-9caa-0ecec4a9daca-sys\") pod \"perf-node-gather-daemonset-nb8bc\" (UID: \"666616ea-cb5f-4a2d-9caa-0ecec4a9daca\") " pod="openshift-must-gather-9fjpm/perf-node-gather-daemonset-nb8bc"
Sep 16 19:39:23.589266 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:23.589215    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/666616ea-cb5f-4a2d-9caa-0ecec4a9daca-lib-modules\") pod \"perf-node-gather-daemonset-nb8bc\" (UID: \"666616ea-cb5f-4a2d-9caa-0ecec4a9daca\") " pod="openshift-must-gather-9fjpm/perf-node-gather-daemonset-nb8bc"
Sep 16 19:39:23.690545 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:23.690497    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sq5gw\" (UniqueName: \"kubernetes.io/projected/666616ea-cb5f-4a2d-9caa-0ecec4a9daca-kube-api-access-sq5gw\") pod \"perf-node-gather-daemonset-nb8bc\" (UID: \"666616ea-cb5f-4a2d-9caa-0ecec4a9daca\") " pod="openshift-must-gather-9fjpm/perf-node-gather-daemonset-nb8bc"
Sep 16 19:39:23.690850 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:23.690813    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/666616ea-cb5f-4a2d-9caa-0ecec4a9daca-podres\") pod \"perf-node-gather-daemonset-nb8bc\" (UID: \"666616ea-cb5f-4a2d-9caa-0ecec4a9daca\") " pod="openshift-must-gather-9fjpm/perf-node-gather-daemonset-nb8bc"
Sep 16 19:39:23.691045 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:23.691025    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/666616ea-cb5f-4a2d-9caa-0ecec4a9daca-sys\") pod \"perf-node-gather-daemonset-nb8bc\" (UID: \"666616ea-cb5f-4a2d-9caa-0ecec4a9daca\") " pod="openshift-must-gather-9fjpm/perf-node-gather-daemonset-nb8bc"
Sep 16 19:39:23.691190 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:23.691149    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/666616ea-cb5f-4a2d-9caa-0ecec4a9daca-sys\") pod \"perf-node-gather-daemonset-nb8bc\" (UID: \"666616ea-cb5f-4a2d-9caa-0ecec4a9daca\") " pod="openshift-must-gather-9fjpm/perf-node-gather-daemonset-nb8bc"
Sep 16 19:39:23.691380 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:23.691173    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/666616ea-cb5f-4a2d-9caa-0ecec4a9daca-lib-modules\") pod \"perf-node-gather-daemonset-nb8bc\" (UID: \"666616ea-cb5f-4a2d-9caa-0ecec4a9daca\") " pod="openshift-must-gather-9fjpm/perf-node-gather-daemonset-nb8bc"
Sep 16 19:39:23.691499 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:23.691293    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/666616ea-cb5f-4a2d-9caa-0ecec4a9daca-lib-modules\") pod \"perf-node-gather-daemonset-nb8bc\" (UID: \"666616ea-cb5f-4a2d-9caa-0ecec4a9daca\") " pod="openshift-must-gather-9fjpm/perf-node-gather-daemonset-nb8bc"
Sep 16 19:39:23.691557 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:23.691072    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/666616ea-cb5f-4a2d-9caa-0ecec4a9daca-podres\") pod \"perf-node-gather-daemonset-nb8bc\" (UID: \"666616ea-cb5f-4a2d-9caa-0ecec4a9daca\") " pod="openshift-must-gather-9fjpm/perf-node-gather-daemonset-nb8bc"
Sep 16 19:39:23.691557 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:23.691522    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/666616ea-cb5f-4a2d-9caa-0ecec4a9daca-proc\") pod \"perf-node-gather-daemonset-nb8bc\" (UID: \"666616ea-cb5f-4a2d-9caa-0ecec4a9daca\") " pod="openshift-must-gather-9fjpm/perf-node-gather-daemonset-nb8bc"
Sep 16 19:39:23.691557 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:23.691480    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/666616ea-cb5f-4a2d-9caa-0ecec4a9daca-proc\") pod \"perf-node-gather-daemonset-nb8bc\" (UID: \"666616ea-cb5f-4a2d-9caa-0ecec4a9daca\") " pod="openshift-must-gather-9fjpm/perf-node-gather-daemonset-nb8bc"
Sep 16 19:39:23.698910 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:23.698871    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sq5gw\" (UniqueName: \"kubernetes.io/projected/666616ea-cb5f-4a2d-9caa-0ecec4a9daca-kube-api-access-sq5gw\") pod \"perf-node-gather-daemonset-nb8bc\" (UID: \"666616ea-cb5f-4a2d-9caa-0ecec4a9daca\") " pod="openshift-must-gather-9fjpm/perf-node-gather-daemonset-nb8bc"
Sep 16 19:39:23.814146 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:23.814109    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-9fjpm/perf-node-gather-daemonset-nb8bc"
Sep 16 19:39:24.101367 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:24.101336    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-9fjpm/perf-node-gather-daemonset-nb8bc"]
Sep 16 19:39:24.104305 ip-10-0-139-182 kubenswrapper[2584]: W0916 19:39:24.104270    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod666616ea_cb5f_4a2d_9caa_0ecec4a9daca.slice/crio-a1f37cdc3508b64109ebf4b19fe75409291d2ae9ee6f93b95d7c3ff0dd2875c7 WatchSource:0}: Error finding container a1f37cdc3508b64109ebf4b19fe75409291d2ae9ee6f93b95d7c3ff0dd2875c7: Status 404 returned error can't find the container with id a1f37cdc3508b64109ebf4b19fe75409291d2ae9ee6f93b95d7c3ff0dd2875c7
Sep 16 19:39:24.188563 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:24.188519    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-9fjpm/perf-node-gather-daemonset-nb8bc" event={"ID":"666616ea-cb5f-4a2d-9caa-0ecec4a9daca","Type":"ContainerStarted","Data":"af92882790105990c14a182853131a09737a0ad646f277b587323fe934cc1680"}
Sep 16 19:39:24.188731 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:24.188590    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-9fjpm/perf-node-gather-daemonset-nb8bc" event={"ID":"666616ea-cb5f-4a2d-9caa-0ecec4a9daca","Type":"ContainerStarted","Data":"a1f37cdc3508b64109ebf4b19fe75409291d2ae9ee6f93b95d7c3ff0dd2875c7"}
Sep 16 19:39:24.188800 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:24.188737    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-9fjpm/perf-node-gather-daemonset-nb8bc"
Sep 16 19:39:24.207249 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:24.207186    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-9fjpm/perf-node-gather-daemonset-nb8bc" podStartSLOduration=1.207162566 podStartE2EDuration="1.207162566s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 19:39:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 19:39:24.204689133 +0000 UTC m=+3294.534191551" watchObservedRunningTime="2026-09-16 19:39:24.207162566 +0000 UTC m=+3294.536664980"
Sep 16 19:39:24.383787 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:24.383699    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-tv99q_7a4aa47d-be1a-4b2c-bd88-3edc40ef5d6a/dns/0.log"
Sep 16 19:39:24.407885 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:24.407855    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-tv99q_7a4aa47d-be1a-4b2c-bd88-3edc40ef5d6a/kube-rbac-proxy/0.log"
Sep 16 19:39:24.526236 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:24.526198    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-bd22w_4a80b1e8-39b4-44a2-80cb-aa52d96e80fb/dns-node-resolver/0.log"
Sep 16 19:39:25.100541 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:25.100511    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-f8jpf_1498d81a-223d-470c-9812-9b2910e3ea35/node-ca/0.log"
Sep 16 19:39:25.880491 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:25.880450    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7ff679ddf8-ctqk5_ec6fcb0d-227c-492a-923a-db5e74797aa6/router/0.log"
Sep 16 19:39:26.260276 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:26.260242    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-s4gvd_80b7ba2b-64e7-4bf3-9f78-d8a2b2b0e66a/serve-healthcheck-canary/0.log"
Sep 16 19:39:26.689473 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:26.689384    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-9gqhw_efe7abc6-7f9b-4141-aaa5-fbe85db5cead/kube-rbac-proxy/0.log"
Sep 16 19:39:26.714025 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:26.713991    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-9gqhw_efe7abc6-7f9b-4141-aaa5-fbe85db5cead/exporter/0.log"
Sep 16 19:39:26.741343 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:26.741306    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-9gqhw_efe7abc6-7f9b-4141-aaa5-fbe85db5cead/extractor/0.log"
Sep 16 19:39:29.174151 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:29.174116    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_kserve-controller-manager-75c4c6c467-jfjnj_b5be7327-d711-4e51-a1cc-01421b6f209b/manager/0.log"
Sep 16 19:39:29.232301 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:29.232268    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_model-serving-api-7b954ffc95-mxzbw_ad86938c-bf26-4b41-8615-27995751a6d4/server/0.log"
Sep 16 19:39:29.397811 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:29.397778    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_odh-model-controller-7b966d7984-5nnrz_0915599e-1581-4f0d-b575-6bc3be699f89/manager/0.log"
Sep 16 19:39:29.488176 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:29.488102    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_seaweedfs-64568bcd49-67qr4_a19cc48d-affd-49a6-a918-1d860c4f9896/seaweedfs/0.log"
Sep 16 19:39:30.205698 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:30.205668    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-9fjpm/perf-node-gather-daemonset-nb8bc"
Sep 16 19:39:34.190649 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:34.190611    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6c8f84f897-qkzj5_65e92d96-65ad-4891-b693-1293389b7855/kube-storage-version-migrator-operator/1.log"
Sep 16 19:39:34.192331 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:34.192301    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6c8f84f897-qkzj5_65e92d96-65ad-4891-b693-1293389b7855/kube-storage-version-migrator-operator/0.log"
Sep 16 19:39:35.440432 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:35.440395    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fbw2l_71637102-fba0-4c38-8055-6dceacddcde9/kube-multus-additional-cni-plugins/0.log"
Sep 16 19:39:35.469618 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:35.469581    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fbw2l_71637102-fba0-4c38-8055-6dceacddcde9/egress-router-binary-copy/0.log"
Sep 16 19:39:35.501102 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:35.501062    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fbw2l_71637102-fba0-4c38-8055-6dceacddcde9/cni-plugins/0.log"
Sep 16 19:39:35.526456 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:35.526423    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fbw2l_71637102-fba0-4c38-8055-6dceacddcde9/bond-cni-plugin/0.log"
Sep 16 19:39:35.551335 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:35.551294    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fbw2l_71637102-fba0-4c38-8055-6dceacddcde9/routeoverride-cni/0.log"
Sep 16 19:39:35.576667 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:35.576634    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fbw2l_71637102-fba0-4c38-8055-6dceacddcde9/whereabouts-cni-bincopy/0.log"
Sep 16 19:39:35.602022 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:35.601992    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fbw2l_71637102-fba0-4c38-8055-6dceacddcde9/whereabouts-cni/0.log"
Sep 16 19:39:35.890396 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:35.890339    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-d9bgl_c42d9b1b-2d14-469c-b986-ae99600b602c/kube-multus/0.log"
Sep 16 19:39:35.948251 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:35.948220    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-bp5mv_9b96048a-0005-410b-a710-909d21fbfde8/network-metrics-daemon/0.log"
Sep 16 19:39:35.972145 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:35.972107    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-bp5mv_9b96048a-0005-410b-a710-909d21fbfde8/kube-rbac-proxy/0.log"
Sep 16 19:39:36.301280 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:36.301166    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-crjw7_ecf273b5-b465-4008-b035-f7ac0e740d58/console-operator/1.log"
Sep 16 19:39:36.329139 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:36.312397    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-crjw7_ecf273b5-b465-4008-b035-f7ac0e740d58/console-operator/1.log"
Sep 16 19:39:37.361355 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:37.361320    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-blv9h_08febeb1-4769-4fd6-911f-a0a02c7c4223/ovn-controller/0.log"
Sep 16 19:39:37.381132 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:37.381095    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-blv9h_08febeb1-4769-4fd6-911f-a0a02c7c4223/ovn-acl-logging/0.log"
Sep 16 19:39:37.414778 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:37.414741    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-blv9h_08febeb1-4769-4fd6-911f-a0a02c7c4223/kube-rbac-proxy-node/0.log"
Sep 16 19:39:37.442705 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:37.442669    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-blv9h_08febeb1-4769-4fd6-911f-a0a02c7c4223/kube-rbac-proxy-ovn-metrics/0.log"
Sep 16 19:39:37.466263 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:37.466232    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-blv9h_08febeb1-4769-4fd6-911f-a0a02c7c4223/northd/0.log"
Sep 16 19:39:37.492539 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:37.492502    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-blv9h_08febeb1-4769-4fd6-911f-a0a02c7c4223/nbdb/0.log"
Sep 16 19:39:37.517243 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:37.517208    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-blv9h_08febeb1-4769-4fd6-911f-a0a02c7c4223/sbdb/0.log"
Sep 16 19:39:37.588021 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:37.587985    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-blv9h_08febeb1-4769-4fd6-911f-a0a02c7c4223/ovnkube-controller/0.log"
Sep 16 19:39:38.725247 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:38.725216    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-f4md6_df3c2b31-63a6-43e4-a4ae-0199c34a6a86/network-check-target-container/0.log"
Sep 16 19:39:39.841421 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:39.841384    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-qqjgk_a29102aa-22ea-4623-84fc-0be1e5506274/iptables-alerter/0.log"
Sep 16 19:39:40.652981 ip-10-0-139-182 kubenswrapper[2584]: I0916 19:39:40.652928    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-node-tuning-operator_tuned-9z8sh_7892f55c-beff-42d9-989e-52eeb778bdc5/tuned/0.log"