Sep 19 08:20:14.475446 ip-10-0-139-157 systemd[1]: Starting Kubernetes Kubelet...
Sep 19 08:20:14.854738 ip-10-0-139-157 kubenswrapper[2578]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 19 08:20:14.854738 ip-10-0-139-157 kubenswrapper[2578]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 19 08:20:14.854738 ip-10-0-139-157 kubenswrapper[2578]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 19 08:20:14.854738 ip-10-0-139-157 kubenswrapper[2578]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 19 08:20:14.854738 ip-10-0-139-157 kubenswrapper[2578]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 19 08:20:14.855414 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.854783    2578 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 19 08:20:14.857642 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.857626    2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 19 08:20:14.857642 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.857641    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 19 08:20:14.857703 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.857645    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 19 08:20:14.857703 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.857648    2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 19 08:20:14.857703 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.857650    2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 19 08:20:14.857703 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.857653    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 19 08:20:14.857703 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.857657    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 19 08:20:14.857703 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.857659    2578 feature_gate.go:328] unrecognized feature gate: Example2
Sep 19 08:20:14.857703 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.857662    2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 19 08:20:14.857703 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.857665    2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 19 08:20:14.857703 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.857667    2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 19 08:20:14.857703 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.857670    2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 19 08:20:14.857703 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.857673    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 19 08:20:14.857703 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.857676    2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 19 08:20:14.857703 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.857679    2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 19 08:20:14.857703 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.857681    2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 19 08:20:14.857703 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.857684    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 19 08:20:14.857703 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.857688    2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 19 08:20:14.857703 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.857692    2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 19 08:20:14.857703 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.857695    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 19 08:20:14.857703 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.857698    2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 19 08:20:14.858195 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.857706    2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 19 08:20:14.858195 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.857710    2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 19 08:20:14.858195 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.857713    2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 19 08:20:14.858195 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.857715    2578 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 19 08:20:14.858195 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.857718    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 19 08:20:14.858195 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.857721    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 19 08:20:14.858195 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.857725    2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 19 08:20:14.858195 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.857728    2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 19 08:20:14.858195 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.857731    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 19 08:20:14.858195 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.857734    2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 19 08:20:14.858195 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.857737    2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 19 08:20:14.858195 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.857740    2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 19 08:20:14.858195 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.857742    2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 19 08:20:14.858195 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.857745    2578 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 19 08:20:14.858195 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.857749    2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 19 08:20:14.858195 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.857751    2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 19 08:20:14.858195 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.857754    2578 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 19 08:20:14.858195 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.857757    2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 19 08:20:14.858195 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.857759    2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 19 08:20:14.858732 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.857762    2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 19 08:20:14.858732 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.857764    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 19 08:20:14.858732 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.857766    2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 19 08:20:14.858732 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.857769    2578 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 19 08:20:14.858732 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.857771    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 19 08:20:14.858732 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.857774    2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 19 08:20:14.858732 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.857779    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 19 08:20:14.858732 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.857782    2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 19 08:20:14.858732 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.857784    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 19 08:20:14.858732 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.857787    2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 19 08:20:14.858732 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.857789    2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 19 08:20:14.858732 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.857792    2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 19 08:20:14.858732 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.857794    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 19 08:20:14.858732 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.857797    2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 19 08:20:14.858732 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.857808    2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 19 08:20:14.858732 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.857811    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 19 08:20:14.858732 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.857814    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 19 08:20:14.858732 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.857816    2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 19 08:20:14.858732 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.857819    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 19 08:20:14.858732 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.857821    2578 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 19 08:20:14.859236 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.857824    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 19 08:20:14.859236 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.857826    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 19 08:20:14.859236 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.857829    2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 19 08:20:14.859236 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.857832    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 19 08:20:14.859236 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.857834    2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 19 08:20:14.859236 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.857837    2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 19 08:20:14.859236 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.857839    2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 19 08:20:14.859236 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.857842    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 19 08:20:14.859236 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.857844    2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 19 08:20:14.859236 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.857847    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 19 08:20:14.859236 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.857850    2578 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 19 08:20:14.859236 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.857852    2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 19 08:20:14.859236 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.857855    2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 19 08:20:14.859236 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.857857    2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 19 08:20:14.859236 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.857860    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 19 08:20:14.859236 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.857863    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 19 08:20:14.859236 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.857866    2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 19 08:20:14.859236 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.857868    2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 19 08:20:14.859236 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.857871    2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 19 08:20:14.859236 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.857873    2578 feature_gate.go:328] unrecognized feature gate: Example
Sep 19 08:20:14.859732 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.857876    2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 19 08:20:14.859732 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.857878    2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 19 08:20:14.859732 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.857881    2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 19 08:20:14.859732 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.857883    2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 19 08:20:14.859732 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.857886    2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 19 08:20:14.859732 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.857888    2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 19 08:20:14.859732 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.857892    2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 19 08:20:14.859732 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.857900    2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 19 08:20:14.859732 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.857903    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 19 08:20:14.859732 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.857906    2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 19 08:20:14.859732 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.857908    2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 19 08:20:14.859732 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.857911    2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 19 08:20:14.859732 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.857913    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 19 08:20:14.859732 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.857916    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 19 08:20:14.859732 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.857920    2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 19 08:20:14.859732 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.857923    2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 19 08:20:14.859732 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.857926    2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 19 08:20:14.859732 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.857929    2578 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 19 08:20:14.859732 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.857932    2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 19 08:20:14.860202 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.858647    2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 19 08:20:14.860202 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.858654    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 19 08:20:14.860202 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.858659    2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 19 08:20:14.860202 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.858662    2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 19 08:20:14.860202 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.858666    2578 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 19 08:20:14.860202 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.858668    2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 19 08:20:14.860202 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.858672    2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 19 08:20:14.860202 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.858675    2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 19 08:20:14.860202 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.858677    2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 19 08:20:14.860202 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.858680    2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 19 08:20:14.860202 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.858684    2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 19 08:20:14.860202 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.858686    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 19 08:20:14.860202 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.858690    2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 19 08:20:14.860202 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.858693    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 19 08:20:14.860202 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.858695    2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 19 08:20:14.860202 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.858698    2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 19 08:20:14.860202 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.858701    2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 19 08:20:14.860202 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.858704    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 19 08:20:14.860202 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.858707    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 19 08:20:14.860671 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.858711    2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 19 08:20:14.860671 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.858714    2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 19 08:20:14.860671 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.858717    2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 19 08:20:14.860671 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.858720    2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 19 08:20:14.860671 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.858722    2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 19 08:20:14.860671 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.858725    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 19 08:20:14.860671 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.858728    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 19 08:20:14.860671 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.858731    2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 19 08:20:14.860671 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.858734    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 19 08:20:14.860671 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.858736    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 19 08:20:14.860671 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.858739    2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 19 08:20:14.860671 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.858742    2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 19 08:20:14.860671 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.858745    2578 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 19 08:20:14.860671 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.858747    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 19 08:20:14.860671 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.858750    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 19 08:20:14.860671 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.858753    2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 19 08:20:14.860671 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.858755    2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 19 08:20:14.860671 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.858759    2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 19 08:20:14.860671 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.858761    2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 19 08:20:14.860671 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.858764    2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 19 08:20:14.861154 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.858766    2578 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 19 08:20:14.861154 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.858769    2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 19 08:20:14.861154 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.858771    2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 19 08:20:14.861154 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.858774    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 19 08:20:14.861154 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.858777    2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 19 08:20:14.861154 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.858779    2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 19 08:20:14.861154 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.858782    2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 19 08:20:14.861154 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.858785    2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 19 08:20:14.861154 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.858788    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 19 08:20:14.861154 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.858790    2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 19 08:20:14.861154 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.858793    2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 19 08:20:14.861154 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.858795    2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 19 08:20:14.861154 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.858798    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 19 08:20:14.861154 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.858801    2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 19 08:20:14.861154 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.858803    2578 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 19 08:20:14.861154 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.858807    2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 19 08:20:14.861154 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.858809    2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 19 08:20:14.861154 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.858812    2578 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 19 08:20:14.861154 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.858814    2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 19 08:20:14.861154 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.858817    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 19 08:20:14.861878 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.858820    2578 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 19 08:20:14.861878 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.858822    2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 19 08:20:14.861878 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.858825    2578 feature_gate.go:328] unrecognized feature gate: Example
Sep 19 08:20:14.861878 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.858827    2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 19 08:20:14.861878 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.858830    2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 19 08:20:14.861878 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.858832    2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 19 08:20:14.861878 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.858835    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 19 08:20:14.861878 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.858837    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 19 08:20:14.861878 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.858840    2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 19 08:20:14.861878 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.858842    2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 19 08:20:14.861878 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.858845    2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 19 08:20:14.861878 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.858847    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 19 08:20:14.861878 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.858851    2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 19 08:20:14.861878 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.858855    2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 19 08:20:14.861878 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.858857    2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 19 08:20:14.861878 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.858860    2578 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 19 08:20:14.861878 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.858863    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 19 08:20:14.861878 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.858865    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 19 08:20:14.861878 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.858868    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 19 08:20:14.861878 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.858871    2578 feature_gate.go:328] unrecognized feature gate: Example2
Sep 19 08:20:14.862546 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.858873    2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 19 08:20:14.862546 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.858877    2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 19 08:20:14.862546 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.858879    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 19 08:20:14.862546 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.858882    2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 19 08:20:14.862546 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.858885    2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 19 08:20:14.862546 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.858888    2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 19 08:20:14.862546 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.858890    2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 19 08:20:14.862546 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.858893    2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 19 08:20:14.862546 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.858896    2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 19 08:20:14.862546 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.858898    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 19 08:20:14.862546 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.858901    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 19 08:20:14.862546 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.858904    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 19 08:20:14.862546 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.858907    2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 19 08:20:14.862546 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.858910    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 19 08:20:14.862546 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.858913    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 19 08:20:14.862546 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.858917    2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 19 08:20:14.862546 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.858920    2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 19 08:20:14.862546 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.858923    2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 19 08:20:14.862546 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.858926    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 19 08:20:14.863186 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.858929    2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 19 08:20:14.863186 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.860496    2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 19 08:20:14.863186 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.860503    2578 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 19 08:20:14.863186 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.860522    2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 19 08:20:14.863186 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.860526    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 19 08:20:14.863186 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.860530    2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 19 08:20:14.863186 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.860533    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 19 08:20:14.863186 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.860536    2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 19 08:20:14.863186 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.860539    2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 19 08:20:14.863186 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.860542    2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 19 08:20:14.863186 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.860545    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 19 08:20:14.863186 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.860549    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 19 08:20:14.863186 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.860552    2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 19 08:20:14.863186 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.860555    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 19 08:20:14.863186 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.860557    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 19 08:20:14.863186 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.860561    2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 19 08:20:14.863186 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.860565    2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 19 08:20:14.863186 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.860568    2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 19 08:20:14.863186 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.860571    2578 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 19 08:20:14.863749 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.860573    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 19 08:20:14.863749 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.860576    2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 19 08:20:14.863749 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.860579    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 19 08:20:14.863749 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.860582    2578 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 19 08:20:14.863749 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.860585    2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 19 08:20:14.863749 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.860588    2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 19 08:20:14.863749 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.860590    2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 19 08:20:14.863749 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.860593    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 19 08:20:14.863749 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.860596    2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 19 08:20:14.863749 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.860599    2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 19 08:20:14.863749 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.860601    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 19 08:20:14.863749 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.860604    2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 19 08:20:14.863749 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.860606    2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 19 08:20:14.863749 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.860609    2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 19 08:20:14.863749 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.860611    2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 19 08:20:14.863749 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.860614    2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 19 08:20:14.863749 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.860616    2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 19 08:20:14.863749 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.860619    2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 19 08:20:14.863749 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.860621    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 19 08:20:14.863749 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.860624    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 19 08:20:14.864428 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.860626    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 19 08:20:14.864428 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.860630    2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 19 08:20:14.864428 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.860633    2578 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 19 08:20:14.864428 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.860636    2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 19 08:20:14.864428 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.860639    2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 19 08:20:14.864428 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.860642    2578 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 19 08:20:14.864428 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.860645    2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 19 08:20:14.864428 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.860647    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 19 08:20:14.864428 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.860650    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 19 08:20:14.864428 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.860652    2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 19 08:20:14.864428 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.860655    2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 19 08:20:14.864428 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.860658    2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 19 08:20:14.864428 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.860660    2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 19 08:20:14.864428 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.860663    2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 19 08:20:14.864428 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.860666    2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 19 08:20:14.864428 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.860669    2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 19 08:20:14.864428 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.860671    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 19 08:20:14.864428 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.860674    2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 19 08:20:14.864428 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.860677    2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 19 08:20:14.864918 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.860680    2578 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 19 08:20:14.864918 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.860682    2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 19 08:20:14.864918 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.860685    2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 19 08:20:14.864918 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.860688    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 19 08:20:14.864918 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.860690    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 19 08:20:14.864918 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.860692    2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 19 08:20:14.864918 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.860695    2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 19 08:20:14.864918 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.860697    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 19 08:20:14.864918 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.860700    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 19 08:20:14.864918 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.860702    2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 19 08:20:14.864918 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.860705    2578 feature_gate.go:328] unrecognized feature gate: Example2
Sep 19 08:20:14.864918 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.860708    2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 19 08:20:14.864918 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.860710    2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 19 08:20:14.864918 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.860713    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 19 08:20:14.864918 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.860715    2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 19 08:20:14.864918 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.860718    2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 19 08:20:14.864918 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.860720    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 19 08:20:14.864918 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.860723    2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 19 08:20:14.864918 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.860725    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 19 08:20:14.864918 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.860729    2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 19 08:20:14.865467 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.860731    2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 19 08:20:14.865467 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.860734    2578 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 19 08:20:14.865467 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.860736    2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 19 08:20:14.865467 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.860739    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 19 08:20:14.865467 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.860743    2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 19 08:20:14.865467 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.860747    2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 19 08:20:14.865467 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.860749    2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 19 08:20:14.865467 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.860752    2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 19 08:20:14.865467 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.860755    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 19 08:20:14.865467 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.860758    2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 19 08:20:14.865467 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.860760    2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 19 08:20:14.865467 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.860763    2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 19 08:20:14.865467 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.860765    2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 19 08:20:14.865467 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.860768    2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 19 08:20:14.865467 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.860770    2578 feature_gate.go:328] unrecognized feature gate: Example
Sep 19 08:20:14.865467 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.860773    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 19 08:20:14.865467 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.860775    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 19 08:20:14.865467 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.860778    2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 19 08:20:14.865467 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.860780    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 19 08:20:14.865937 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.860783    2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 19 08:20:14.865937 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.860785    2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 19 08:20:14.865937 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.860788    2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 19 08:20:14.865937 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.860849    2578 flags.go:64] FLAG: --address="0.0.0.0"
Sep 19 08:20:14.865937 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.860881    2578 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 19 08:20:14.865937 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.860891    2578 flags.go:64] FLAG: --anonymous-auth="true"
Sep 19 08:20:14.865937 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.860896    2578 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 19 08:20:14.865937 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.860900    2578 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 19 08:20:14.865937 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.860903    2578 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 19 08:20:14.865937 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.860907    2578 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 19 08:20:14.865937 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.860912    2578 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 19 08:20:14.865937 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.860915    2578 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 19 08:20:14.865937 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.860918    2578 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 19 08:20:14.865937 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.860921    2578 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 19 08:20:14.865937 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.860925    2578 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 19 08:20:14.865937 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.860928    2578 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 19 08:20:14.865937 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.860931    2578 flags.go:64] FLAG: --cgroup-root=""
Sep 19 08:20:14.865937 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.860934    2578 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 19 08:20:14.865937 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.860937    2578 flags.go:64] FLAG: --client-ca-file=""
Sep 19 08:20:14.865937 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.860940    2578 flags.go:64] FLAG: --cloud-provider="external"
Sep 19 08:20:14.865937 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.860943    2578 flags.go:64] FLAG: --cluster-dns="[]"
Sep 19 08:20:14.865937 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.860947    2578 flags.go:64] FLAG: --cluster-domain=""
Sep 19 08:20:14.865937 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.860949    2578 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 19 08:20:14.865937 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.860953    2578 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 19 08:20:14.866536 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.860956    2578 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 19 08:20:14.866536 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.860959    2578 flags.go:64] FLAG: --container-log-max-files="5"
Sep 19 08:20:14.866536 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.860963    2578 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 19 08:20:14.866536 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.860966    2578 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 19 08:20:14.866536 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.860969    2578 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 19 08:20:14.866536 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.860972    2578 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 19 08:20:14.866536 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.860975    2578 flags.go:64] FLAG: --contention-profiling="false"
Sep 19 08:20:14.866536 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.860977    2578 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 19 08:20:14.866536 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.860980    2578 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 19 08:20:14.866536 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.860983    2578 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 19 08:20:14.866536 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.860986    2578 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 19 08:20:14.866536 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.860990    2578 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 19 08:20:14.866536 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.860993    2578 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 19 08:20:14.866536 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.860996    2578 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 19 08:20:14.866536 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.860998    2578 flags.go:64] FLAG: --enable-load-reader="false"
Sep 19 08:20:14.866536 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.861001    2578 flags.go:64] FLAG: --enable-server="true"
Sep 19 08:20:14.866536 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.861004    2578 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 19 08:20:14.866536 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.861009    2578 flags.go:64] FLAG: --event-burst="100"
Sep 19 08:20:14.866536 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.861012    2578 flags.go:64] FLAG: --event-qps="50"
Sep 19 08:20:14.866536 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.861015    2578 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 19 08:20:14.866536 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.861018    2578 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 19 08:20:14.866536 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.861021    2578 flags.go:64] FLAG: --eviction-hard=""
Sep 19 08:20:14.866536 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.861024    2578 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 19 08:20:14.866536 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.861027    2578 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 19 08:20:14.867119 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.861032    2578 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 19 08:20:14.867119 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.861035    2578 flags.go:64] FLAG: --eviction-soft=""
Sep 19 08:20:14.867119 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.861038    2578 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 19 08:20:14.867119 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.861041    2578 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 19 08:20:14.867119 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.861044    2578 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 19 08:20:14.867119 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.861047    2578 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 19 08:20:14.867119 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.861050    2578 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 19 08:20:14.867119 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.861052    2578 flags.go:64] FLAG: --fail-swap-on="true"
Sep 19 08:20:14.867119 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.861056    2578 flags.go:64] FLAG: --feature-gates=""
Sep 19 08:20:14.867119 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.861060    2578 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 19 08:20:14.867119 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.861063    2578 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 19 08:20:14.867119 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.861066    2578 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 19 08:20:14.867119 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.861069    2578 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 19 08:20:14.867119 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.861072    2578 flags.go:64] FLAG: --healthz-port="10248"
Sep 19 08:20:14.867119 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.861075    2578 flags.go:64] FLAG: --help="false"
Sep 19 08:20:14.867119 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.861078    2578 flags.go:64] FLAG: --hostname-override="ip-10-0-139-157.ec2.internal"
Sep 19 08:20:14.867119 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.861081    2578 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 19 08:20:14.867119 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.861084    2578 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 19 08:20:14.867119 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.861087    2578 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 19 08:20:14.867119 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.861090    2578 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 19 08:20:14.867119 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.861093    2578 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 19 08:20:14.867119 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.861096    2578 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 19 08:20:14.867119 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.861099    2578 flags.go:64] FLAG: --image-service-endpoint=""
Sep 19 08:20:14.867755 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.861102    2578 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 19 08:20:14.867755 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.861105    2578 flags.go:64] FLAG: --kube-api-burst="100"
Sep 19 08:20:14.867755 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.861108    2578 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 19 08:20:14.867755 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.861113    2578 flags.go:64] FLAG: --kube-api-qps="50"
Sep 19 08:20:14.867755 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.861116    2578 flags.go:64] FLAG: --kube-reserved=""
Sep 19 08:20:14.867755 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.861119    2578 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 19 08:20:14.867755 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.861121    2578 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 19 08:20:14.867755 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.861125    2578 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 19 08:20:14.867755 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.861128    2578 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 19 08:20:14.867755 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.861132    2578 flags.go:64] FLAG: --lock-file=""
Sep 19 08:20:14.867755 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.861134    2578 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 19 08:20:14.867755 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.861137    2578 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 19 08:20:14.867755 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.861140    2578 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 19 08:20:14.867755 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.861145    2578 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 19 08:20:14.867755 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.861148    2578 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 19 08:20:14.867755 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.861151    2578 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 19 08:20:14.867755 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.861154    2578 flags.go:64] FLAG: --logging-format="text"
Sep 19 08:20:14.867755 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.861157    2578 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 19 08:20:14.867755 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.861212    2578 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 19 08:20:14.867755 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.861254    2578 flags.go:64] FLAG: --manifest-url=""
Sep 19 08:20:14.867755 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.861260    2578 flags.go:64] FLAG: --manifest-url-header=""
Sep 19 08:20:14.867755 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.861276    2578 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 19 08:20:14.867755 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.861280    2578 flags.go:64] FLAG: --max-open-files="1000000"
Sep 19 08:20:14.867755 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.861285    2578 flags.go:64] FLAG: --max-pods="110"
Sep 19 08:20:14.867755 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.861288    2578 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 19 08:20:14.868390 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.861292    2578 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 19 08:20:14.868390 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.861295    2578 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 19 08:20:14.868390 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.861303    2578 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 19 08:20:14.868390 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.861308    2578 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 19 08:20:14.868390 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.861313    2578 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 19 08:20:14.868390 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.861318    2578 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 19 08:20:14.868390 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.861329    2578 flags.go:64] FLAG: --node-status-max-images="50"
Sep 19 08:20:14.868390 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.861334    2578 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 19 08:20:14.868390 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.862723    2578 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 19 08:20:14.868390 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.862739    2578 flags.go:64] FLAG: --pod-cidr=""
Sep 19 08:20:14.868390 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.862744    2578 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a6abe77af61c5f2e3985a83e7d43538aefc38b1b0444127da14ee0e8cf8f76fc"
Sep 19 08:20:14.868390 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.862752    2578 flags.go:64] FLAG: --pod-manifest-path=""
Sep 19 08:20:14.868390 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.862756    2578 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 19 08:20:14.868390 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.862760    2578 flags.go:64] FLAG: --pods-per-core="0"
Sep 19 08:20:14.868390 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.862764    2578 flags.go:64] FLAG: --port="10250"
Sep 19 08:20:14.868390 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.862768    2578 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 19 08:20:14.868390 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.862772    2578 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-029aefbea5905b4d6"
Sep 19 08:20:14.868390 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.862782    2578 flags.go:64] FLAG: --qos-reserved=""
Sep 19 08:20:14.868390 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.862904    2578 flags.go:64] FLAG: --read-only-port="10255"
Sep 19 08:20:14.868390 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.862914    2578 flags.go:64] FLAG: --register-node="true"
Sep 19 08:20:14.868390 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.862918    2578 flags.go:64] FLAG: --register-with-taints=""
Sep 19 08:20:14.868390 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.862931    2578 flags.go:64] FLAG: --registry-burst="10"
Sep 19 08:20:14.868390 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.862934    2578 flags.go:64] FLAG: --registry-qps="5"
Sep 19 08:20:14.868390 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.862938    2578 flags.go:64] FLAG: --reserved-cpus=""
Sep 19 08:20:14.869001 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.862941    2578 flags.go:64] FLAG: --reserved-memory=""
Sep 19 08:20:14.869001 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.862945    2578 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 19 08:20:14.869001 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.862948    2578 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 19 08:20:14.869001 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.862951    2578 flags.go:64] FLAG: --rotate-certificates="false"
Sep 19 08:20:14.869001 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.862954    2578 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 19 08:20:14.869001 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.862957    2578 flags.go:64] FLAG: --runonce="false"
Sep 19 08:20:14.869001 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.862960    2578 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 19 08:20:14.869001 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.862963    2578 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 19 08:20:14.869001 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.862967    2578 flags.go:64] FLAG: --seccomp-default="false"
Sep 19 08:20:14.869001 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.862970    2578 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 19 08:20:14.869001 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.862973    2578 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 19 08:20:14.869001 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.862977    2578 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 19 08:20:14.869001 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.862980    2578 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 19 08:20:14.869001 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.862983    2578 flags.go:64] FLAG: --storage-driver-password="root"
Sep 19 08:20:14.869001 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.862986    2578 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 19 08:20:14.869001 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.862989    2578 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 19 08:20:14.869001 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.862992    2578 flags.go:64] FLAG: --storage-driver-user="root"
Sep 19 08:20:14.869001 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.862995    2578 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 19 08:20:14.869001 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.862999    2578 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 19 08:20:14.869001 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.863002    2578 flags.go:64] FLAG: --system-cgroups=""
Sep 19 08:20:14.869001 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.863005    2578 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 19 08:20:14.869001 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.863014    2578 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 19 08:20:14.869001 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.863017    2578 flags.go:64] FLAG: --tls-cert-file=""
Sep 19 08:20:14.869001 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.863019    2578 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 19 08:20:14.869001 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.863025    2578 flags.go:64] FLAG: --tls-min-version=""
Sep 19 08:20:14.869647 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.863028    2578 flags.go:64] FLAG: --tls-private-key-file=""
Sep 19 08:20:14.869647 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.863031    2578 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 19 08:20:14.869647 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.863034    2578 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 19 08:20:14.869647 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.863037    2578 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 19 08:20:14.869647 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.863040    2578 flags.go:64] FLAG: --v="2"
Sep 19 08:20:14.869647 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.863045    2578 flags.go:64] FLAG: --version="false"
Sep 19 08:20:14.869647 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.863049    2578 flags.go:64] FLAG: --vmodule=""
Sep 19 08:20:14.869647 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.863053    2578 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 19 08:20:14.869647 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.863056    2578 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 19 08:20:14.869647 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.863186    2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 19 08:20:14.869647 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.863190    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 19 08:20:14.869647 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.863193    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 19 08:20:14.869647 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.863197    2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 19 08:20:14.869647 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.863200    2578 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 19 08:20:14.869647 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.863203    2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 19 08:20:14.869647 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.863206    2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 19 08:20:14.869647 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.863209    2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 19 08:20:14.869647 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.863212    2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 19 08:20:14.869647 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.863214    2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 19 08:20:14.869647 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.863218    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 19 08:20:14.869647 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.863220    2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 19 08:20:14.869647 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.863223    2578 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 19 08:20:14.870210 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.863226    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 19 08:20:14.870210 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.863229    2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 19 08:20:14.870210 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.863232    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 19 08:20:14.870210 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.863237    2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 19 08:20:14.870210 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.863239    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 19 08:20:14.870210 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.863242    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 19 08:20:14.870210 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.863245    2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 19 08:20:14.870210 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.863248    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 19 08:20:14.870210 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.863252    2578 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 19 08:20:14.870210 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.863255    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 19 08:20:14.870210 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.863257    2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 19 08:20:14.870210 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.863260    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 19 08:20:14.870210 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.863262    2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 19 08:20:14.870210 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.863265    2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 19 08:20:14.870210 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.863268    2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 19 08:20:14.870210 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.863271    2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 19 08:20:14.870210 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.863274    2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 19 08:20:14.870210 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.863277    2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 19 08:20:14.870210 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.863279    2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 19 08:20:14.870210 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.863282    2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 19 08:20:14.870720 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.863285    2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 19 08:20:14.870720 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.863287    2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 19 08:20:14.870720 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.863290    2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 19 08:20:14.870720 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.863293    2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 19 08:20:14.870720 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.863295    2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 19 08:20:14.870720 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.863298    2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 19 08:20:14.870720 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.863301    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 19 08:20:14.870720 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.863303    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 19 08:20:14.870720 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.863306    2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 19 08:20:14.870720 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.863309    2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 19 08:20:14.870720 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.863311    2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 19 08:20:14.870720 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.863314    2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 19 08:20:14.870720 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.863316    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 19 08:20:14.870720 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.863319    2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 19 08:20:14.870720 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.863321    2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 19 08:20:14.870720 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.863325    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 19 08:20:14.870720 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.863328    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 19 08:20:14.870720 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.863331    2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 19 08:20:14.870720 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.863336    2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 19 08:20:14.871194 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.863340    2578 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 19 08:20:14.871194 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.863343    2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 19 08:20:14.871194 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.863346    2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 19 08:20:14.871194 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.863349    2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 19 08:20:14.871194 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.863353    2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 19 08:20:14.871194 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.863357    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 19 08:20:14.871194 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.863360    2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 19 08:20:14.871194 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.863363    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 19 08:20:14.871194 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.863366    2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 19 08:20:14.871194 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.863369    2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 19 08:20:14.871194 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.863372    2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 19 08:20:14.871194 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.863375    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 19 08:20:14.871194 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.863378    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 19 08:20:14.871194 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.863381    2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 19 08:20:14.871194 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.863383    2578 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 19 08:20:14.871194 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.863394    2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 19 08:20:14.871194 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.863397    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 19 08:20:14.871194 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.863400    2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 19 08:20:14.871194 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.863403    2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 19 08:20:14.871677 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.863405    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 19 08:20:14.871677 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.863408    2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 19 08:20:14.871677 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.863411    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 19 08:20:14.871677 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.863414    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 19 08:20:14.871677 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.863416    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 19 08:20:14.871677 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.863419    2578 feature_gate.go:328] unrecognized feature gate: Example
Sep 19 08:20:14.871677 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.863422    2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 19 08:20:14.871677 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.863425    2578 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 19 08:20:14.871677 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.863428    2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 19 08:20:14.871677 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.863433    2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 19 08:20:14.871677 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.863436    2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 19 08:20:14.871677 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.863438    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 19 08:20:14.871677 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.863441    2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 19 08:20:14.871677 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.863444    2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 19 08:20:14.871677 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.863447    2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 19 08:20:14.871677 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.863450    2578 feature_gate.go:328] unrecognized feature gate: Example2
Sep 19 08:20:14.871677 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.863454    2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 19 08:20:14.871677 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.863458    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 19 08:20:14.871677 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.863460    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 19 08:20:14.871677 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.863463    2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 19 08:20:14.872151 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.863466    2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 19 08:20:14.872151 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.863469    2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 19 08:20:14.872151 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.863471    2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 19 08:20:14.872151 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.863474    2578 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 19 08:20:14.872151 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.863477    2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 19 08:20:14.872151 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.863479    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 19 08:20:14.872151 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.863482    2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 19 08:20:14.872151 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.863485    2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 19 08:20:14.872151 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.864129    2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 19 08:20:14.872151 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.871734    2578 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 19 08:20:14.872151 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.871749    2578 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 19 08:20:14.872625 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.872614    2578 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 19 08:20:14.872657 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.872625    2578 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 19 08:20:14.872687 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.872681    2578 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 19 08:20:14.872687 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.872687    2578 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 19 08:20:14.872740 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.872690    2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 19 08:20:14.872740 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.872694    2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 19 08:20:14.872740 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.872697    2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 19 08:20:14.872740 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.872700    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 19 08:20:14.872740 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.872703    2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 19 08:20:14.872740 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.872707    2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 19 08:20:14.872740 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.872710    2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 19 08:20:14.872740 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.872712    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 19 08:20:14.872740 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.872715    2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 19 08:20:14.872740 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.872718    2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 19 08:20:14.872740 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.872721    2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 19 08:20:14.872740 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.872724    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 19 08:20:14.872740 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.872726    2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 19 08:20:14.872740 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.872729    2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 19 08:20:14.872740 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.872732    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 19 08:20:14.872740 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.872736    2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 19 08:20:14.872740 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.872741    2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 19 08:20:14.872740 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.872745    2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 19 08:20:14.872740 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.872749    2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 19 08:20:14.873265 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.872752    2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 19 08:20:14.873265 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.872755    2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 19 08:20:14.873265 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.872758    2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 19 08:20:14.873265 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.872760    2578 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 19 08:20:14.873265 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.872763    2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 19 08:20:14.873265 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.872765    2578 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 19 08:20:14.873265 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.872768    2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 19 08:20:14.873265 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.872771    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 19 08:20:14.873265 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.872774    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 19 08:20:14.873265 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.872777    2578 feature_gate.go:328] unrecognized feature gate: Example2
Sep 19 08:20:14.873265 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.872780    2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 19 08:20:14.873265 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.872782    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 19 08:20:14.873265 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.872785    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 19 08:20:14.873265 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.872787    2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 19 08:20:14.873265 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.872790    2578 feature_gate.go:328] unrecognized feature gate: Example
Sep 19 08:20:14.873265 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.872793    2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 19 08:20:14.873265 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.872795    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 19 08:20:14.873265 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.872798    2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 19 08:20:14.873265 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.872800    2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 19 08:20:14.873265 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.872803    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 19 08:20:14.873754 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.872805    2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 19 08:20:14.873754 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.872808    2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 19 08:20:14.873754 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.872811    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 19 08:20:14.873754 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.872813    2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 19 08:20:14.873754 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.872816    2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 19 08:20:14.873754 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.872818    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 19 08:20:14.873754 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.872821    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 19 08:20:14.873754 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.872823    2578 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 19 08:20:14.873754 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.872826    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 19 08:20:14.873754 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.872829    2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 19 08:20:14.873754 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.872831    2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 19 08:20:14.873754 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.872833    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 19 08:20:14.873754 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.872836    2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 19 08:20:14.873754 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.872839    2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 19 08:20:14.873754 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.872842    2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 19 08:20:14.873754 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.872845    2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 19 08:20:14.873754 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.872847    2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 19 08:20:14.873754 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.872850    2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 19 08:20:14.873754 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.872853    2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 19 08:20:14.874238 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.872855    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 19 08:20:14.874238 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.872858    2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 19 08:20:14.874238 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.872861    2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 19 08:20:14.874238 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.872864    2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 19 08:20:14.874238 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.872867    2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 19 08:20:14.874238 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.872870    2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 19 08:20:14.874238 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.872872    2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 19 08:20:14.874238 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.872875    2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 19 08:20:14.874238 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.872877    2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 19 08:20:14.874238 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.872880    2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 19 08:20:14.874238 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.872882    2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 19 08:20:14.874238 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.872885    2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 19 08:20:14.874238 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.872888    2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 19 08:20:14.874238 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.872892    2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 19 08:20:14.874238 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.872896    2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 19 08:20:14.874238 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.872899    2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 19 08:20:14.874238 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.872902    2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 19 08:20:14.874238 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.872905    2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 19 08:20:14.874238 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.872908    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 19 08:20:14.874694 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.872911    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 19 08:20:14.874694 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.872913    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 19 08:20:14.874694 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.872916    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 19 08:20:14.874694 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.872919    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 19 08:20:14.874694 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.872921    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 19 08:20:14.874694 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.872924    2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 19 08:20:14.874694 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.872926    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 19 08:20:14.874694 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.872930    2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 19 08:20:14.874694 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.872933    2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 19 08:20:14.874694 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.872936    2578 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 19 08:20:14.874694 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.872939    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 19 08:20:14.874694 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.872942    2578 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 19 08:20:14.874694 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.872945    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 19 08:20:14.874694 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.872948    2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 19 08:20:14.874694 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.872950    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 19 08:20:14.874694 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.872953    2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 19 08:20:14.874694 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.872956    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 19 08:20:14.874694 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.872959    2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 19 08:20:14.874694 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.872962    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 19 08:20:14.874694 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.872964    2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 19 08:20:14.875197 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.872970    2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 19 08:20:14.875197 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.873077    2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 19 08:20:14.875197 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.873081    2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 19 08:20:14.875197 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.873085    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 19 08:20:14.875197 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.873088    2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 19 08:20:14.875197 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.873091    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 19 08:20:14.875197 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.873094    2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 19 08:20:14.875197 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.873097    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 19 08:20:14.875197 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.873100    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 19 08:20:14.875197 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.873103    2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 19 08:20:14.875197 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.873107    2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 19 08:20:14.875197 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.873110    2578 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 19 08:20:14.875197 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.873113    2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 19 08:20:14.875197 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.873116    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 19 08:20:14.875197 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.873120    2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 19 08:20:14.875577 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.873122    2578 feature_gate.go:328] unrecognized feature gate: Example
Sep 19 08:20:14.875577 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.873125    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 19 08:20:14.875577 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.873128    2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 19 08:20:14.875577 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.873131    2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 19 08:20:14.875577 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.873134    2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 19 08:20:14.875577 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.873138    2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 19 08:20:14.875577 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.873141    2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 19 08:20:14.875577 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.873143    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 19 08:20:14.875577 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.873146    2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 19 08:20:14.875577 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.873149    2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 19 08:20:14.875577 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.873151    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 19 08:20:14.875577 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.873154    2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 19 08:20:14.875577 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.873157    2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 19 08:20:14.875577 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.873160    2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 19 08:20:14.875577 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.873163    2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 19 08:20:14.875577 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.873165    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 19 08:20:14.875577 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.873168    2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 19 08:20:14.875577 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.873186    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 19 08:20:14.875577 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.873189    2578 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 19 08:20:14.875577 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.873192    2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 19 08:20:14.876071 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.873195    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 19 08:20:14.876071 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.873197    2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 19 08:20:14.876071 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.873200    2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 19 08:20:14.876071 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.873203    2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 19 08:20:14.876071 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.873206    2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 19 08:20:14.876071 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.873209    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 19 08:20:14.876071 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.873211    2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 19 08:20:14.876071 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.873214    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 19 08:20:14.876071 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.873217    2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 19 08:20:14.876071 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.873219    2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 19 08:20:14.876071 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.873222    2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 19 08:20:14.876071 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.873225    2578 feature_gate.go:328] unrecognized feature gate: Example2
Sep 19 08:20:14.876071 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.873227    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 19 08:20:14.876071 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.873230    2578 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 19 08:20:14.876071 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.873232    2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 19 08:20:14.876071 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.873235    2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 19 08:20:14.876071 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.873238    2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 19 08:20:14.876071 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.873240    2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 19 08:20:14.876071 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.873243    2578 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 19 08:20:14.876071 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.873246    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 19 08:20:14.876571 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.873248    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 19 08:20:14.876571 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.873251    2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 19 08:20:14.876571 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.873253    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 19 08:20:14.876571 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.873256    2578 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 19 08:20:14.876571 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.873258    2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 19 08:20:14.876571 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.873261    2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 19 08:20:14.876571 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.873263    2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 19 08:20:14.876571 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.873267    2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 19 08:20:14.876571 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.873270    2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 19 08:20:14.876571 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.873273    2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 19 08:20:14.876571 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.873275    2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 19 08:20:14.876571 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.873278    2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 19 08:20:14.876571 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.873281    2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 19 08:20:14.876571 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.873283    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 19 08:20:14.876571 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.873286    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 19 08:20:14.876571 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.873288    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 19 08:20:14.876571 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.873291    2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 19 08:20:14.876571 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.873294    2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 19 08:20:14.876571 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.873297    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 19 08:20:14.876571 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.873300    2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 19 08:20:14.877059 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.873302    2578 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 19 08:20:14.877059 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.873305    2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 19 08:20:14.877059 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.873307    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 19 08:20:14.877059 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.873310    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 19 08:20:14.877059 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.873312    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 19 08:20:14.877059 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.873315    2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 19 08:20:14.877059 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.873319    2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 19 08:20:14.877059 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.873322    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 19 08:20:14.877059 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.873325    2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 19 08:20:14.877059 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.873327    2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 19 08:20:14.877059 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.873330    2578 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 19 08:20:14.877059 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.873332    2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 19 08:20:14.877059 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.873335    2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 19 08:20:14.877059 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.873337    2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 19 08:20:14.877059 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.873340    2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 19 08:20:14.877059 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.873342    2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 19 08:20:14.877059 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.873345    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 19 08:20:14.877059 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.873349    2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 19 08:20:14.877059 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.873352    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 19 08:20:14.877562 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.873355    2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 19 08:20:14.877562 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.873359    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 19 08:20:14.877562 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.873361    2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 19 08:20:14.877562 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.873364    2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 19 08:20:14.877562 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.873367    2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 19 08:20:14.877562 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:14.873369    2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 19 08:20:14.877562 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.873374    2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 19 08:20:14.877562 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.873497    2578 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 19 08:20:14.877562 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.875470    2578 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 19 08:20:14.877562 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.876220    2578 server.go:1019] "Starting client certificate rotation"
Sep 19 08:20:14.877562 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.876314    2578 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 19 08:20:14.877562 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.876898    2578 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 19 08:20:14.878716 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.878700    2578 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 19 08:20:14.880791 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.880776    2578 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 19 08:20:14.890686 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.890126    2578 log.go:25] "Validated CRI v1 runtime API"
Sep 19 08:20:14.892318 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.892303    2578 log.go:25] "Validated CRI v1 image API"
Sep 19 08:20:14.893100 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.893083    2578 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 19 08:20:14.896130 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.896111    2578 fs.go:135] Filesystem UUIDs: map[4fbdadd9-c68a-4c59-80b9-d103df8a5860:/dev/nvme0n1p3 6065e10d-7037-40d1-8de3-42aa597a398d:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2]
Sep 19 08:20:14.896202 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.896130    2578 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 19 08:20:14.901835 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.901729    2578 manager.go:216] Machine: {Timestamp:2026-09-19 08:20:14.900641755 +0000 UTC m=+0.310477436 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3094476 MemoryCapacity:33164476416 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2986b6c21c1c7d94e4fab440cdfadd SystemUUID:ec2986b6-c21c-1c7d-94e4-fab440cdfadd BootID:74044061-0f19-45d7-85ed-afef23357bf6 Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582238208 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:f2:8e:43:c5 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:f2:8e:43:c5 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:9e:a6:c2:88:84:39 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164476416 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 19 08:20:14.901835 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.901829    2578 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 19 08:20:14.901954 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.901908    2578 manager.go:232] Version: {KernelVersion:5.14.0-570.141.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260914-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 19 08:20:14.904698 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.904668    2578 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 19 08:20:14.904846 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.904702    2578 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-139-157.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 19 08:20:14.905411 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.905401    2578 topology_manager.go:138] "Creating topology manager with none policy"
Sep 19 08:20:14.905450 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.905412    2578 container_manager_linux.go:309] "Creating device plugin manager"
Sep 19 08:20:14.905450 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.905419    2578 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 19 08:20:14.905970 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.905959    2578 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 19 08:20:14.907275 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.907265    2578 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 19 08:20:14.908553 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.908538    2578 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 19 08:20:14.911163 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.911152    2578 state_mem.go:36] "Initialized new in-memory state store"
Sep 19 08:20:14.911292 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.911283    2578 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 19 08:20:14.911329 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.911314    2578 kubelet.go:486] "Attempting to sync node with API server"
Sep 19 08:20:14.911357 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.911333    2578 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 19 08:20:14.911357 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.911348    2578 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 19 08:20:14.911420 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.911359    2578 kubelet.go:398] "Adding apiserver pod source"
Sep 19 08:20:14.911420 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.911371    2578 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 19 08:20:14.911734 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.911723    2578 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 19 08:20:14.911767 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.911741    2578 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 19 08:20:14.914336 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.914318    2578 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.13-6.rhaos4.21.git6253162.el9" apiVersion="v1"
Sep 19 08:20:14.916341 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.916328    2578 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 19 08:20:14.916380 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.916358    2578 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 19 08:20:14.916532 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.916523    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 19 08:20:14.916565 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.916538    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 19 08:20:14.916565 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.916544    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 19 08:20:14.916565 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.916550    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 19 08:20:14.916565 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.916558    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 19 08:20:14.916669 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.916567    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 19 08:20:14.917079 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.917071    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 19 08:20:14.917110 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.917080    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 19 08:20:14.917110 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.917087    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 19 08:20:14.917110 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.917093    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 19 08:20:14.917110 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.917106    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 19 08:20:14.917220 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.917115    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 19 08:20:14.917791 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.917782    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 19 08:20:14.917791 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.917791    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 19 08:20:14.921415 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.921402    2578 server.go:1268] "Started kubelet"
Sep 19 08:20:14.921514 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.921485    2578 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 19 08:20:14.921636 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.921560    2578 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 19 08:20:14.921678 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.921662    2578 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 19 08:20:14.921993 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.921959    2578 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 19 08:20:14.922322 ip-10-0-139-157 systemd[1]: Started Kubernetes Kubelet.
Sep 19 08:20:14.923706 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.923690    2578 server.go:310] "Adding debug handlers to kubelet server"
Sep 19 08:20:14.927752 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:14.927724    2578 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-139-157.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 19 08:20:14.927862 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:14.927838    2578 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 19 08:20:14.927920 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.927908    2578 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-139-157.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 19 08:20:14.928641 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:14.928626    2578 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 19 08:20:14.928892 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:14.927720    2578 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-157.ec2.internal.18d6ab16783ef47d  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-157.ec2.internal,UID:ip-10-0-139-157.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-139-157.ec2.internal,},FirstTimestamp:2026-09-19 08:20:14.921372797 +0000 UTC m=+0.331208479,LastTimestamp:2026-09-19 08:20:14.921372797 +0000 UTC m=+0.331208479,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-157.ec2.internal,}"
Sep 19 08:20:14.931323 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.931300    2578 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 19 08:20:14.931323 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.931313    2578 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 19 08:20:14.931480 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:14.931409    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-157.ec2.internal\" not found"
Sep 19 08:20:14.932064 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.932016    2578 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 19 08:20:14.932064 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.932019    2578 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 19 08:20:14.932064 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.932056    2578 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 19 08:20:14.932269 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.932104    2578 reconstruct.go:97] "Volume reconstruction finished"
Sep 19 08:20:14.932269 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.932112    2578 reconciler.go:29] "Reconciler: start to sync state"
Sep 19 08:20:14.933009 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.932991    2578 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 19 08:20:14.933009 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.933010    2578 factory.go:55] Registering systemd factory
Sep 19 08:20:14.933136 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.933021    2578 factory.go:223] Registration of the systemd container factory successfully
Sep 19 08:20:14.933248 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.933235    2578 factory.go:153] Registering CRI-O factory
Sep 19 08:20:14.933317 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.933249    2578 factory.go:223] Registration of the crio container factory successfully
Sep 19 08:20:14.933317 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.933264    2578 factory.go:103] Registering Raw factory
Sep 19 08:20:14.933317 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.933274    2578 manager.go:1196] Started watching for new ooms in manager
Sep 19 08:20:14.933664 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.933651    2578 manager.go:349] Starting recovery of all containers
Sep 19 08:20:14.935755 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:14.935727    2578 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-139-157.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 19 08:20:14.935885 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:14.935863    2578 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 19 08:20:14.944530 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.944403    2578 manager.go:354] Recovery completed
Sep 19 08:20:14.948356 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.948342    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 19 08:20:14.950718 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.950697    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-157.ec2.internal" event="NodeHasSufficientMemory"
Sep 19 08:20:14.950808 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.950736    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-157.ec2.internal" event="NodeHasNoDiskPressure"
Sep 19 08:20:14.950808 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.950753    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-157.ec2.internal" event="NodeHasSufficientPID"
Sep 19 08:20:14.951372 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.951354    2578 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 19 08:20:14.951372 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.951366    2578 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 19 08:20:14.951471 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.951380    2578 state_mem.go:36] "Initialized new in-memory state store"
Sep 19 08:20:14.951471 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.951383    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 19 08:20:14.952843 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:14.952774    2578 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-157.ec2.internal.18d6ab1679feb882  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-157.ec2.internal,UID:ip-10-0-139-157.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-139-157.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-139-157.ec2.internal,},FirstTimestamp:2026-09-19 08:20:14.95071757 +0000 UTC m=+0.360553252,LastTimestamp:2026-09-19 08:20:14.95071757 +0000 UTC m=+0.360553252,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-157.ec2.internal,}"
Sep 19 08:20:14.953675 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.953648    2578 policy_none.go:49] "None policy: Start"
Sep 19 08:20:14.953675 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.953668    2578 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 19 08:20:14.953850 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.953682    2578 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 19 08:20:14.954888 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.954863    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-157.ec2.internal" event="NodeHasSufficientMemory"
Sep 19 08:20:14.954964 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.954915    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-157.ec2.internal" event="NodeHasNoDiskPressure"
Sep 19 08:20:14.954964 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.954932    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-157.ec2.internal" event="NodeHasSufficientPID"
Sep 19 08:20:14.955738 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.955725    2578 policy_none.go:47] "Start"
Sep 19 08:20:14.960054 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.960031    2578 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-7rkfc"
Sep 19 08:20:14.962441 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:14.962375    2578 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-157.ec2.internal.18d6ab1679ff1e8c  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-157.ec2.internal,UID:ip-10-0-139-157.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-139-157.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-139-157.ec2.internal,},FirstTimestamp:2026-09-19 08:20:14.950743692 +0000 UTC m=+0.360579375,LastTimestamp:2026-09-19 08:20:14.950743692 +0000 UTC m=+0.360579375,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-157.ec2.internal,}"
Sep 19 08:20:14.964758 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:14.964697    2578 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-157.ec2.internal.18d6ab1679ff582f  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-157.ec2.internal,UID:ip-10-0-139-157.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-139-157.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-139-157.ec2.internal,},FirstTimestamp:2026-09-19 08:20:14.950758447 +0000 UTC m=+0.360594131,LastTimestamp:2026-09-19 08:20:14.950758447 +0000 UTC m=+0.360594131,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-157.ec2.internal,}"
Sep 19 08:20:14.967351 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.967336    2578 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-7rkfc"
Sep 19 08:20:14.971368 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:14.971290    2578 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-139-157.ec2.internal.18d6ab1679feb882\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-157.ec2.internal.18d6ab1679feb882  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-157.ec2.internal,UID:ip-10-0-139-157.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-139-157.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-139-157.ec2.internal,},FirstTimestamp:2026-09-19 08:20:14.95071757 +0000 UTC m=+0.360553252,LastTimestamp:2026-09-19 08:20:14.954897849 +0000 UTC m=+0.364733543,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-157.ec2.internal,}"
Sep 19 08:20:15.002530 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.991813    2578 manager.go:337] "Starting Device Plugin manager"
Sep 19 08:20:15.002530 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:14.991857    2578 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 19 08:20:15.002530 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.991869    2578 server.go:87] "Starting device plugin registration server"
Sep 19 08:20:15.002530 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.992264    2578 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 19 08:20:15.002530 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.992278    2578 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 19 08:20:15.002530 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.992358    2578 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 19 08:20:15.002530 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.992438    2578 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 19 08:20:15.002530 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:14.992447    2578 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 19 08:20:15.002530 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:14.993139    2578 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 19 08:20:15.002530 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:14.993197    2578 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-139-157.ec2.internal\" not found"
Sep 19 08:20:15.072430 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:15.072409    2578 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 19 08:20:15.073543 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:15.073526    2578 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 19 08:20:15.073604 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:15.073585    2578 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 19 08:20:15.073645 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:15.073615    2578 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 19 08:20:15.073722 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:15.073707    2578 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 19 08:20:15.076276 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:15.076260    2578 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 19 08:20:15.093311 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:15.093296    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 19 08:20:15.094167 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:15.094148    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-157.ec2.internal" event="NodeHasSufficientMemory"
Sep 19 08:20:15.094258 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:15.094193    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-157.ec2.internal" event="NodeHasNoDiskPressure"
Sep 19 08:20:15.094258 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:15.094205    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-157.ec2.internal" event="NodeHasSufficientPID"
Sep 19 08:20:15.094258 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:15.094230    2578 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-139-157.ec2.internal"
Sep 19 08:20:15.102719 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:15.102705    2578 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-139-157.ec2.internal"
Sep 19 08:20:15.102790 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:15.102724    2578 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-139-157.ec2.internal\": node \"ip-10-0-139-157.ec2.internal\" not found"
Sep 19 08:20:15.124493 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:15.124432    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-157.ec2.internal\" not found"
Sep 19 08:20:15.174618 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:15.174040    2578 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-157.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-139-157.ec2.internal"]
Sep 19 08:20:15.174699 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:15.174658    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 19 08:20:15.175536 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:15.175517    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-157.ec2.internal" event="NodeHasSufficientMemory"
Sep 19 08:20:15.175626 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:15.175544    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-157.ec2.internal" event="NodeHasNoDiskPressure"
Sep 19 08:20:15.175626 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:15.175554    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-157.ec2.internal" event="NodeHasSufficientPID"
Sep 19 08:20:15.177840 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:15.177828    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 19 08:20:15.177974 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:15.177960    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-157.ec2.internal"
Sep 19 08:20:15.178033 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:15.177994    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 19 08:20:15.178576 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:15.178563    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-157.ec2.internal" event="NodeHasSufficientMemory"
Sep 19 08:20:15.178648 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:15.178582    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-157.ec2.internal" event="NodeHasNoDiskPressure"
Sep 19 08:20:15.178648 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:15.178591    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-157.ec2.internal" event="NodeHasSufficientPID"
Sep 19 08:20:15.178648 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:15.178565    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-157.ec2.internal" event="NodeHasSufficientMemory"
Sep 19 08:20:15.178762 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:15.178651    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-157.ec2.internal" event="NodeHasNoDiskPressure"
Sep 19 08:20:15.178762 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:15.178660    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-157.ec2.internal" event="NodeHasSufficientPID"
Sep 19 08:20:15.180832 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:15.180818    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-157.ec2.internal"
Sep 19 08:20:15.180903 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:15.180842    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 19 08:20:15.181530 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:15.181516    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-157.ec2.internal" event="NodeHasSufficientMemory"
Sep 19 08:20:15.181615 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:15.181542    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-157.ec2.internal" event="NodeHasNoDiskPressure"
Sep 19 08:20:15.181615 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:15.181555    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-157.ec2.internal" event="NodeHasSufficientPID"
Sep 19 08:20:15.207881 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:15.207863    2578 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-157.ec2.internal\" not found" node="ip-10-0-139-157.ec2.internal"
Sep 19 08:20:15.212302 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:15.212288    2578 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-157.ec2.internal\" not found" node="ip-10-0-139-157.ec2.internal"
Sep 19 08:20:15.225419 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:15.225401    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-157.ec2.internal\" not found"
Sep 19 08:20:15.233748 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:15.233723    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/6b8b6e7e22ce47a3909ac23b630738ee-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-157.ec2.internal\" (UID: \"6b8b6e7e22ce47a3909ac23b630738ee\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-157.ec2.internal"
Sep 19 08:20:15.233828 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:15.233756    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6b8b6e7e22ce47a3909ac23b630738ee-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-157.ec2.internal\" (UID: \"6b8b6e7e22ce47a3909ac23b630738ee\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-157.ec2.internal"
Sep 19 08:20:15.233828 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:15.233773    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ecf1d89d174b946c499fe0ba49cfe70a-config\") pod \"kube-apiserver-proxy-ip-10-0-139-157.ec2.internal\" (UID: \"ecf1d89d174b946c499fe0ba49cfe70a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-157.ec2.internal"
Sep 19 08:20:15.325903 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:15.325880    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-157.ec2.internal\" not found"
Sep 19 08:20:15.334258 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:15.334239    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/6b8b6e7e22ce47a3909ac23b630738ee-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-157.ec2.internal\" (UID: \"6b8b6e7e22ce47a3909ac23b630738ee\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-157.ec2.internal"
Sep 19 08:20:15.334320 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:15.334262    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6b8b6e7e22ce47a3909ac23b630738ee-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-157.ec2.internal\" (UID: \"6b8b6e7e22ce47a3909ac23b630738ee\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-157.ec2.internal"
Sep 19 08:20:15.334320 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:15.334280    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ecf1d89d174b946c499fe0ba49cfe70a-config\") pod \"kube-apiserver-proxy-ip-10-0-139-157.ec2.internal\" (UID: \"ecf1d89d174b946c499fe0ba49cfe70a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-157.ec2.internal"
Sep 19 08:20:15.334394 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:15.334337    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/6b8b6e7e22ce47a3909ac23b630738ee-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-157.ec2.internal\" (UID: \"6b8b6e7e22ce47a3909ac23b630738ee\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-157.ec2.internal"
Sep 19 08:20:15.334394 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:15.334352    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ecf1d89d174b946c499fe0ba49cfe70a-config\") pod \"kube-apiserver-proxy-ip-10-0-139-157.ec2.internal\" (UID: \"ecf1d89d174b946c499fe0ba49cfe70a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-157.ec2.internal"
Sep 19 08:20:15.334394 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:15.334355    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6b8b6e7e22ce47a3909ac23b630738ee-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-157.ec2.internal\" (UID: \"6b8b6e7e22ce47a3909ac23b630738ee\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-157.ec2.internal"
Sep 19 08:20:15.426696 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:15.426633    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-157.ec2.internal\" not found"
Sep 19 08:20:15.511184 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:15.511146    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-157.ec2.internal"
Sep 19 08:20:15.516350 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:15.516281    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-157.ec2.internal"
Sep 19 08:20:15.527425 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:15.527407    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-157.ec2.internal\" not found"
Sep 19 08:20:15.627967 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:15.627932    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-157.ec2.internal\" not found"
Sep 19 08:20:15.728480 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:15.728402    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-157.ec2.internal\" not found"
Sep 19 08:20:15.828912 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:15.828881    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-157.ec2.internal\" not found"
Sep 19 08:20:15.839038 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:15.839019    2578 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 19 08:20:15.876330 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:15.876308    2578 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 19 08:20:15.876735 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:15.876447    2578 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 19 08:20:15.876735 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:15.876463    2578 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 19 08:20:15.929012 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:15.928974    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-157.ec2.internal\" not found"
Sep 19 08:20:15.972331 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:15.972301    2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-18 08:15:15 +0000 UTC" deadline="2028-02-23 16:48:37.055281934 +0000 UTC"
Sep 19 08:20:15.972331 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:15.972328    2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12536h28m21.082958244s"
Sep 19 08:20:16.029628 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:16.029572    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-157.ec2.internal\" not found"
Sep 19 08:20:16.043429 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:16.043408    2578 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 19 08:20:16.074382 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:16.074361    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 19 08:20:16.075921 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:16.075902    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-157.ec2.internal" event="NodeHasSufficientMemory"
Sep 19 08:20:16.075921 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:16.075934    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-157.ec2.internal" event="NodeHasNoDiskPressure"
Sep 19 08:20:16.076048 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:16.075944    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-157.ec2.internal" event="NodeHasSufficientPID"
Sep 19 08:20:16.130322 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:16.130298    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-157.ec2.internal\" not found"
Sep 19 08:20:16.230768 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:16.230738    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-157.ec2.internal\" not found"
Sep 19 08:20:16.268940 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:16.268911    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podecf1d89d174b946c499fe0ba49cfe70a.slice/crio-389d7ecd981a9ef6bd4428958fa0fa566e8a80ccf4428ac03954d50a657d0c3c WatchSource:0}: Error finding container 389d7ecd981a9ef6bd4428958fa0fa566e8a80ccf4428ac03954d50a657d0c3c: Status 404 returned error can't find the container with id 389d7ecd981a9ef6bd4428958fa0fa566e8a80ccf4428ac03954d50a657d0c3c
Sep 19 08:20:16.269233 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:16.269213    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b8b6e7e22ce47a3909ac23b630738ee.slice/crio-da9b6af5043945b6590adcd9e90efa0475dad98ae28af8265bb0c477b0a06003 WatchSource:0}: Error finding container da9b6af5043945b6590adcd9e90efa0475dad98ae28af8265bb0c477b0a06003: Status 404 returned error can't find the container with id da9b6af5043945b6590adcd9e90efa0475dad98ae28af8265bb0c477b0a06003
Sep 19 08:20:16.276660 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:16.276641    2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 19 08:20:16.331724 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:16.331673    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-157.ec2.internal\" not found"
Sep 19 08:20:16.432362 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:16.432328    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-157.ec2.internal\" not found"
Sep 19 08:20:16.523671 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:16.523643    2578 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 19 08:20:16.532659 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:16.532641    2578 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-157.ec2.internal"
Sep 19 08:20:16.544651 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:16.544630    2578 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 19 08:20:16.545528 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:16.545516    2578 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-157.ec2.internal"
Sep 19 08:20:16.554227 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:16.554209    2578 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 19 08:20:16.912354 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:16.912323    2578 apiserver.go:52] "Watching apiserver"
Sep 19 08:20:16.921122 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:16.921097    2578 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 19 08:20:16.921610 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:16.921583    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-wln5n","openshift-multus/network-metrics-daemon-md9ln","openshift-network-operator/iptables-alerter-jfkwf","kube-system/global-pull-secret-syncer-xq8d8","kube-system/kube-apiserver-proxy-ip-10-0-139-157.ec2.internal","openshift-cluster-node-tuning-operator/tuned-5x6rt","openshift-dns/node-resolver-lqdbb","openshift-multus/multus-additional-cni-plugins-9frbh","openshift-network-diagnostics/network-check-target-l6m8x","openshift-ovn-kubernetes/ovnkube-node-zd2zp","kube-system/konnectivity-agent-7tt6m","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6vp2f","openshift-image-registry/node-ca-hb9xj","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-157.ec2.internal"]
Sep 19 08:20:16.925235 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:16.924418    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-lqdbb"
Sep 19 08:20:16.926588 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:16.926564    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-7tt6m"
Sep 19 08:20:16.929530 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:16.929493    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 19 08:20:16.929651 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:16.929542    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-mtzfs\""
Sep 19 08:20:16.929960 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:16.929831    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 19 08:20:16.931213 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:16.931191    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-svqc2\""
Sep 19 08:20:16.931476 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:16.931459    2578 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 19 08:20:16.932508 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:16.932489    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 19 08:20:16.932664 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:16.932644    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 19 08:20:16.932723 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:16.932665    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-jfkwf"
Sep 19 08:20:16.935676 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:16.935646    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 19 08:20:16.935887 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:16.935870    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 19 08:20:16.936055 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:16.936041    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-djbpz\""
Sep 19 08:20:16.936527 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:16.936212    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 19 08:20:16.936527 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:16.936421    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xq8d8"
Sep 19 08:20:16.936527 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:16.936478    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xq8d8" podUID="19642aea-2c2e-4208-8329-09f03b1e5d5b"
Sep 19 08:20:16.937862 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:16.937844    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-9frbh"
Sep 19 08:20:16.940064 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:16.940044    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 19 08:20:16.940361 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:16.940343    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 19 08:20:16.940566 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:16.940552    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 19 08:20:16.940788 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:16.940774    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 19 08:20:16.940973 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:16.940955    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 19 08:20:16.941130 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:16.941114    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-qlwpn\""
Sep 19 08:20:16.941736 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:16.941717    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-5x6rt"
Sep 19 08:20:16.941929 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:16.941858    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l6m8x"
Sep 19 08:20:16.942006 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:16.941941    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l6m8x" podUID="8324d8db-b082-475a-8bb7-8c578404b02e"
Sep 19 08:20:16.943518 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:16.943495    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2btqj\" (UniqueName: \"kubernetes.io/projected/637bbfb9-45ba-481d-b9fe-87f60051712c-kube-api-access-2btqj\") pod \"node-resolver-lqdbb\" (UID: \"637bbfb9-45ba-481d-b9fe-87f60051712c\") " pod="openshift-dns/node-resolver-lqdbb"
Sep 19 08:20:16.943606 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:16.943532    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/0904bd86-22aa-4c03-af38-ee3e765096be-konnectivity-ca\") pod \"konnectivity-agent-7tt6m\" (UID: \"0904bd86-22aa-4c03-af38-ee3e765096be\") " pod="kube-system/konnectivity-agent-7tt6m"
Sep 19 08:20:16.943606 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:16.943562    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/637bbfb9-45ba-481d-b9fe-87f60051712c-tmp-dir\") pod \"node-resolver-lqdbb\" (UID: \"637bbfb9-45ba-481d-b9fe-87f60051712c\") " pod="openshift-dns/node-resolver-lqdbb"
Sep 19 08:20:16.943606 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:16.943586    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/0904bd86-22aa-4c03-af38-ee3e765096be-agent-certs\") pod \"konnectivity-agent-7tt6m\" (UID: \"0904bd86-22aa-4c03-af38-ee3e765096be\") " pod="kube-system/konnectivity-agent-7tt6m"
Sep 19 08:20:16.943756 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:16.943611    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/dcc90f06-1e07-4b6b-add7-46f0cdd95ecf-iptables-alerter-script\") pod \"iptables-alerter-jfkwf\" (UID: \"dcc90f06-1e07-4b6b-add7-46f0cdd95ecf\") " pod="openshift-network-operator/iptables-alerter-jfkwf"
Sep 19 08:20:16.943756 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:16.943634    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/dcc90f06-1e07-4b6b-add7-46f0cdd95ecf-host-slash\") pod \"iptables-alerter-jfkwf\" (UID: \"dcc90f06-1e07-4b6b-add7-46f0cdd95ecf\") " pod="openshift-network-operator/iptables-alerter-jfkwf"
Sep 19 08:20:16.943756 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:16.943694    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dfjs8\" (UniqueName: \"kubernetes.io/projected/dcc90f06-1e07-4b6b-add7-46f0cdd95ecf-kube-api-access-dfjs8\") pod \"iptables-alerter-jfkwf\" (UID: \"dcc90f06-1e07-4b6b-add7-46f0cdd95ecf\") " pod="openshift-network-operator/iptables-alerter-jfkwf"
Sep 19 08:20:16.943756 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:16.943733    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/637bbfb9-45ba-481d-b9fe-87f60051712c-hosts-file\") pod \"node-resolver-lqdbb\" (UID: \"637bbfb9-45ba-481d-b9fe-87f60051712c\") " pod="openshift-dns/node-resolver-lqdbb"
Sep 19 08:20:16.943951 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:16.943845    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-8dkdg\""
Sep 19 08:20:16.943951 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:16.943909    2578 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 19 08:20:16.944085 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:16.944065    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 19 08:20:16.944144 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:16.944097    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 19 08:20:16.944804 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:16.944787    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-hb9xj"
Sep 19 08:20:16.946839 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:16.946822    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-md9ln"
Sep 19 08:20:16.946944 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:16.946919    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-md9ln" podUID="79f6ab54-a988-4015-81c2-da35716eab52"
Sep 19 08:20:16.947682 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:16.947380    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 19 08:20:16.947682 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:16.947569    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-kgl4c\""
Sep 19 08:20:16.947800 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:16.947769    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 19 08:20:16.948091 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:16.947950    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 19 08:20:16.949719 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:16.949604    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-zd2zp"
Sep 19 08:20:16.952102 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:16.952078    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 19 08:20:16.952341 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:16.952325    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 19 08:20:16.952632 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:16.952614    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 19 08:20:16.952817 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:16.952802    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 19 08:20:16.953660 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:16.953641    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-wln5n"
Sep 19 08:20:16.953787 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:16.953729    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 19 08:20:16.953854 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:16.953821    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 19 08:20:16.954590 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:16.954488    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-jg9qf\""
Sep 19 08:20:16.955874 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:16.955855    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 19 08:20:16.957461 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:16.956670    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-vfnl4\""
Sep 19 08:20:16.957461 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:16.956765    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6vp2f"
Sep 19 08:20:16.958952 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:16.958937    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 19 08:20:16.959246 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:16.959228    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 19 08:20:16.959357 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:16.959339    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-gwm5c\""
Sep 19 08:20:16.959604 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:16.959570    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 19 08:20:16.970622 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:16.970597    2578 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-rlql7"
Sep 19 08:20:16.977051 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:16.976727    2578 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-rlql7"
Sep 19 08:20:17.033449 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.033426    2578 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 19 08:20:17.044187 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.043942    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/07ddac45-12a1-42e8-9566-55f08854c02e-run-ovn\") pod \"ovnkube-node-zd2zp\" (UID: \"07ddac45-12a1-42e8-9566-55f08854c02e\") " pod="openshift-ovn-kubernetes/ovnkube-node-zd2zp"
Sep 19 08:20:17.044187 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.043994    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/07ddac45-12a1-42e8-9566-55f08854c02e-host-run-ovn-kubernetes\") pod \"ovnkube-node-zd2zp\" (UID: \"07ddac45-12a1-42e8-9566-55f08854c02e\") " pod="openshift-ovn-kubernetes/ovnkube-node-zd2zp"
Sep 19 08:20:17.044187 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.044023    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c3a67fda-81e8-4d20-83d4-a37e277382b4-multus-socket-dir-parent\") pod \"multus-wln5n\" (UID: \"c3a67fda-81e8-4d20-83d4-a37e277382b4\") " pod="openshift-multus/multus-wln5n"
Sep 19 08:20:17.044187 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.044045    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c3a67fda-81e8-4d20-83d4-a37e277382b4-hostroot\") pod \"multus-wln5n\" (UID: \"c3a67fda-81e8-4d20-83d4-a37e277382b4\") " pod="openshift-multus/multus-wln5n"
Sep 19 08:20:17.044187 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.044068    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c3a67fda-81e8-4d20-83d4-a37e277382b4-multus-daemon-config\") pod \"multus-wln5n\" (UID: \"c3a67fda-81e8-4d20-83d4-a37e277382b4\") " pod="openshift-multus/multus-wln5n"
Sep 19 08:20:17.044187 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.044097    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/dcc90f06-1e07-4b6b-add7-46f0cdd95ecf-iptables-alerter-script\") pod \"iptables-alerter-jfkwf\" (UID: \"dcc90f06-1e07-4b6b-add7-46f0cdd95ecf\") " pod="openshift-network-operator/iptables-alerter-jfkwf"
Sep 19 08:20:17.044187 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.044120    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/07ddac45-12a1-42e8-9566-55f08854c02e-var-lib-openvswitch\") pod \"ovnkube-node-zd2zp\" (UID: \"07ddac45-12a1-42e8-9566-55f08854c02e\") " pod="openshift-ovn-kubernetes/ovnkube-node-zd2zp"
Sep 19 08:20:17.044187 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.044146    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/07ddac45-12a1-42e8-9566-55f08854c02e-env-overrides\") pod \"ovnkube-node-zd2zp\" (UID: \"07ddac45-12a1-42e8-9566-55f08854c02e\") " pod="openshift-ovn-kubernetes/ovnkube-node-zd2zp"
Sep 19 08:20:17.044625 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.044216    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/65dc9622-6c75-457d-ad22-6841b2673aae-serviceca\") pod \"node-ca-hb9xj\" (UID: \"65dc9622-6c75-457d-ad22-6841b2673aae\") " pod="openshift-image-registry/node-ca-hb9xj"
Sep 19 08:20:17.044625 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.044247    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/647eb10e-629e-4cbc-bad4-234bbb862d6e-cni-binary-copy\") pod \"multus-additional-cni-plugins-9frbh\" (UID: \"647eb10e-629e-4cbc-bad4-234bbb862d6e\") " pod="openshift-multus/multus-additional-cni-plugins-9frbh"
Sep 19 08:20:17.044625 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.044265    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/90d27f61-9e28-4cc5-bc29-12f69fda90a0-host\") pod \"tuned-5x6rt\" (UID: \"90d27f61-9e28-4cc5-bc29-12f69fda90a0\") " pod="openshift-cluster-node-tuning-operator/tuned-5x6rt"
Sep 19 08:20:17.044625 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.044295    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c3a67fda-81e8-4d20-83d4-a37e277382b4-host-run-netns\") pod \"multus-wln5n\" (UID: \"c3a67fda-81e8-4d20-83d4-a37e277382b4\") " pod="openshift-multus/multus-wln5n"
Sep 19 08:20:17.044625 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.044317    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/78faeeea-464e-4ad2-9987-47fec1907637-kubelet-dir\") pod \"aws-ebs-csi-driver-node-6vp2f\" (UID: \"78faeeea-464e-4ad2-9987-47fec1907637\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6vp2f"
Sep 19 08:20:17.044625 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.044332    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/78faeeea-464e-4ad2-9987-47fec1907637-socket-dir\") pod \"aws-ebs-csi-driver-node-6vp2f\" (UID: \"78faeeea-464e-4ad2-9987-47fec1907637\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6vp2f"
Sep 19 08:20:17.044625 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.044345    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/78faeeea-464e-4ad2-9987-47fec1907637-registration-dir\") pod \"aws-ebs-csi-driver-node-6vp2f\" (UID: \"78faeeea-464e-4ad2-9987-47fec1907637\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6vp2f"
Sep 19 08:20:17.044625 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.044362    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/90d27f61-9e28-4cc5-bc29-12f69fda90a0-etc-systemd\") pod \"tuned-5x6rt\" (UID: \"90d27f61-9e28-4cc5-bc29-12f69fda90a0\") " pod="openshift-cluster-node-tuning-operator/tuned-5x6rt"
Sep 19 08:20:17.044994 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.044971    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/dcc90f06-1e07-4b6b-add7-46f0cdd95ecf-iptables-alerter-script\") pod \"iptables-alerter-jfkwf\" (UID: \"dcc90f06-1e07-4b6b-add7-46f0cdd95ecf\") " pod="openshift-network-operator/iptables-alerter-jfkwf"
Sep 19 08:20:17.045545 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.045088    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dfjs8\" (UniqueName: \"kubernetes.io/projected/dcc90f06-1e07-4b6b-add7-46f0cdd95ecf-kube-api-access-dfjs8\") pod \"iptables-alerter-jfkwf\" (UID: \"dcc90f06-1e07-4b6b-add7-46f0cdd95ecf\") " pod="openshift-network-operator/iptables-alerter-jfkwf"
Sep 19 08:20:17.045545 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.045146    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/90d27f61-9e28-4cc5-bc29-12f69fda90a0-etc-modprobe-d\") pod \"tuned-5x6rt\" (UID: \"90d27f61-9e28-4cc5-bc29-12f69fda90a0\") " pod="openshift-cluster-node-tuning-operator/tuned-5x6rt"
Sep 19 08:20:17.045545 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.045193    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/07ddac45-12a1-42e8-9566-55f08854c02e-run-systemd\") pod \"ovnkube-node-zd2zp\" (UID: \"07ddac45-12a1-42e8-9566-55f08854c02e\") " pod="openshift-ovn-kubernetes/ovnkube-node-zd2zp"
Sep 19 08:20:17.045545 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.045240    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c3a67fda-81e8-4d20-83d4-a37e277382b4-cni-binary-copy\") pod \"multus-wln5n\" (UID: \"c3a67fda-81e8-4d20-83d4-a37e277382b4\") " pod="openshift-multus/multus-wln5n"
Sep 19 08:20:17.045545 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.045277    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/78faeeea-464e-4ad2-9987-47fec1907637-device-dir\") pod \"aws-ebs-csi-driver-node-6vp2f\" (UID: \"78faeeea-464e-4ad2-9987-47fec1907637\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6vp2f"
Sep 19 08:20:17.045545 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.045311    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/647eb10e-629e-4cbc-bad4-234bbb862d6e-cnibin\") pod \"multus-additional-cni-plugins-9frbh\" (UID: \"647eb10e-629e-4cbc-bad4-234bbb862d6e\") " pod="openshift-multus/multus-additional-cni-plugins-9frbh"
Sep 19 08:20:17.045545 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.045333    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/90d27f61-9e28-4cc5-bc29-12f69fda90a0-run\") pod \"tuned-5x6rt\" (UID: \"90d27f61-9e28-4cc5-bc29-12f69fda90a0\") " pod="openshift-cluster-node-tuning-operator/tuned-5x6rt"
Sep 19 08:20:17.045545 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.045355    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c3a67fda-81e8-4d20-83d4-a37e277382b4-multus-conf-dir\") pod \"multus-wln5n\" (UID: \"c3a67fda-81e8-4d20-83d4-a37e277382b4\") " pod="openshift-multus/multus-wln5n"
Sep 19 08:20:17.045545 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.045380    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/78faeeea-464e-4ad2-9987-47fec1907637-sys-fs\") pod \"aws-ebs-csi-driver-node-6vp2f\" (UID: \"78faeeea-464e-4ad2-9987-47fec1907637\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6vp2f"
Sep 19 08:20:17.045545 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.045422    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/647eb10e-629e-4cbc-bad4-234bbb862d6e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-9frbh\" (UID: \"647eb10e-629e-4cbc-bad4-234bbb862d6e\") " pod="openshift-multus/multus-additional-cni-plugins-9frbh"
Sep 19 08:20:17.045545 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.045447    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/647eb10e-629e-4cbc-bad4-234bbb862d6e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-9frbh\" (UID: \"647eb10e-629e-4cbc-bad4-234bbb862d6e\") " pod="openshift-multus/multus-additional-cni-plugins-9frbh"
Sep 19 08:20:17.045545 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.045472    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/65dc9622-6c75-457d-ad22-6841b2673aae-host\") pod \"node-ca-hb9xj\" (UID: \"65dc9622-6c75-457d-ad22-6841b2673aae\") " pod="openshift-image-registry/node-ca-hb9xj"
Sep 19 08:20:17.045545 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.045503    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c3a67fda-81e8-4d20-83d4-a37e277382b4-host-var-lib-cni-bin\") pod \"multus-wln5n\" (UID: \"c3a67fda-81e8-4d20-83d4-a37e277382b4\") " pod="openshift-multus/multus-wln5n"
Sep 19 08:20:17.045545 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.045540    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/647eb10e-629e-4cbc-bad4-234bbb862d6e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-9frbh\" (UID: \"647eb10e-629e-4cbc-bad4-234bbb862d6e\") " pod="openshift-multus/multus-additional-cni-plugins-9frbh"
Sep 19 08:20:17.046274 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.045569    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/90d27f61-9e28-4cc5-bc29-12f69fda90a0-lib-modules\") pod \"tuned-5x6rt\" (UID: \"90d27f61-9e28-4cc5-bc29-12f69fda90a0\") " pod="openshift-cluster-node-tuning-operator/tuned-5x6rt"
Sep 19 08:20:17.046274 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.045602    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/07ddac45-12a1-42e8-9566-55f08854c02e-run-openvswitch\") pod \"ovnkube-node-zd2zp\" (UID: \"07ddac45-12a1-42e8-9566-55f08854c02e\") " pod="openshift-ovn-kubernetes/ovnkube-node-zd2zp"
Sep 19 08:20:17.046274 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.045637    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/07ddac45-12a1-42e8-9566-55f08854c02e-node-log\") pod \"ovnkube-node-zd2zp\" (UID: \"07ddac45-12a1-42e8-9566-55f08854c02e\") " pod="openshift-ovn-kubernetes/ovnkube-node-zd2zp"
Sep 19 08:20:17.046274 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.045657    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c3a67fda-81e8-4d20-83d4-a37e277382b4-multus-cni-dir\") pod \"multus-wln5n\" (UID: \"c3a67fda-81e8-4d20-83d4-a37e277382b4\") " pod="openshift-multus/multus-wln5n"
Sep 19 08:20:17.046274 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.045684    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wz4bl\" (UniqueName: \"kubernetes.io/projected/c3a67fda-81e8-4d20-83d4-a37e277382b4-kube-api-access-wz4bl\") pod \"multus-wln5n\" (UID: \"c3a67fda-81e8-4d20-83d4-a37e277382b4\") " pod="openshift-multus/multus-wln5n"
Sep 19 08:20:17.046274 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.045708    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-shczr\" (UniqueName: \"kubernetes.io/projected/07ddac45-12a1-42e8-9566-55f08854c02e-kube-api-access-shczr\") pod \"ovnkube-node-zd2zp\" (UID: \"07ddac45-12a1-42e8-9566-55f08854c02e\") " pod="openshift-ovn-kubernetes/ovnkube-node-zd2zp"
Sep 19 08:20:17.046274 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.045737    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/90d27f61-9e28-4cc5-bc29-12f69fda90a0-etc-tuned\") pod \"tuned-5x6rt\" (UID: \"90d27f61-9e28-4cc5-bc29-12f69fda90a0\") " pod="openshift-cluster-node-tuning-operator/tuned-5x6rt"
Sep 19 08:20:17.046274 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.045764    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/07ddac45-12a1-42e8-9566-55f08854c02e-systemd-units\") pod \"ovnkube-node-zd2zp\" (UID: \"07ddac45-12a1-42e8-9566-55f08854c02e\") " pod="openshift-ovn-kubernetes/ovnkube-node-zd2zp"
Sep 19 08:20:17.046274 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.045787    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/07ddac45-12a1-42e8-9566-55f08854c02e-host-run-netns\") pod \"ovnkube-node-zd2zp\" (UID: \"07ddac45-12a1-42e8-9566-55f08854c02e\") " pod="openshift-ovn-kubernetes/ovnkube-node-zd2zp"
Sep 19 08:20:17.046274 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.045810    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c3a67fda-81e8-4d20-83d4-a37e277382b4-host-run-k8s-cni-cncf-io\") pod \"multus-wln5n\" (UID: \"c3a67fda-81e8-4d20-83d4-a37e277382b4\") " pod="openshift-multus/multus-wln5n"
Sep 19 08:20:17.046274 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.045832    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/90d27f61-9e28-4cc5-bc29-12f69fda90a0-var-lib-kubelet\") pod \"tuned-5x6rt\" (UID: \"90d27f61-9e28-4cc5-bc29-12f69fda90a0\") " pod="openshift-cluster-node-tuning-operator/tuned-5x6rt"
Sep 19 08:20:17.046274 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.045856    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r7rcd\" (UniqueName: \"kubernetes.io/projected/8324d8db-b082-475a-8bb7-8c578404b02e-kube-api-access-r7rcd\") pod \"network-check-target-l6m8x\" (UID: \"8324d8db-b082-475a-8bb7-8c578404b02e\") " pod="openshift-network-diagnostics/network-check-target-l6m8x"
Sep 19 08:20:17.046274 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.045890    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/19642aea-2c2e-4208-8329-09f03b1e5d5b-kubelet-config\") pod \"global-pull-secret-syncer-xq8d8\" (UID: \"19642aea-2c2e-4208-8329-09f03b1e5d5b\") " pod="kube-system/global-pull-secret-syncer-xq8d8"
Sep 19 08:20:17.046274 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.045917    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/647eb10e-629e-4cbc-bad4-234bbb862d6e-system-cni-dir\") pod \"multus-additional-cni-plugins-9frbh\" (UID: \"647eb10e-629e-4cbc-bad4-234bbb862d6e\") " pod="openshift-multus/multus-additional-cni-plugins-9frbh"
Sep 19 08:20:17.046274 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.045942    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/dcc90f06-1e07-4b6b-add7-46f0cdd95ecf-host-slash\") pod \"iptables-alerter-jfkwf\" (UID: \"dcc90f06-1e07-4b6b-add7-46f0cdd95ecf\") " pod="openshift-network-operator/iptables-alerter-jfkwf"
Sep 19 08:20:17.046274 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.045967    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/07ddac45-12a1-42e8-9566-55f08854c02e-etc-openvswitch\") pod \"ovnkube-node-zd2zp\" (UID: \"07ddac45-12a1-42e8-9566-55f08854c02e\") " pod="openshift-ovn-kubernetes/ovnkube-node-zd2zp"
Sep 19 08:20:17.047025 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.045990    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/07ddac45-12a1-42e8-9566-55f08854c02e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-zd2zp\" (UID: \"07ddac45-12a1-42e8-9566-55f08854c02e\") " pod="openshift-ovn-kubernetes/ovnkube-node-zd2zp"
Sep 19 08:20:17.047025 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.045997    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/dcc90f06-1e07-4b6b-add7-46f0cdd95ecf-host-slash\") pod \"iptables-alerter-jfkwf\" (UID: \"dcc90f06-1e07-4b6b-add7-46f0cdd95ecf\") " pod="openshift-network-operator/iptables-alerter-jfkwf"
Sep 19 08:20:17.047025 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.046013    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/79f6ab54-a988-4015-81c2-da35716eab52-metrics-certs\") pod \"network-metrics-daemon-md9ln\" (UID: \"79f6ab54-a988-4015-81c2-da35716eab52\") " pod="openshift-multus/network-metrics-daemon-md9ln"
Sep 19 08:20:17.047025 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.046037    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c3a67fda-81e8-4d20-83d4-a37e277382b4-cnibin\") pod \"multus-wln5n\" (UID: \"c3a67fda-81e8-4d20-83d4-a37e277382b4\") " pod="openshift-multus/multus-wln5n"
Sep 19 08:20:17.047025 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.046066    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/78faeeea-464e-4ad2-9987-47fec1907637-etc-selinux\") pod \"aws-ebs-csi-driver-node-6vp2f\" (UID: \"78faeeea-464e-4ad2-9987-47fec1907637\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6vp2f"
Sep 19 08:20:17.047025 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.046090    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vldf6\" (UniqueName: \"kubernetes.io/projected/78faeeea-464e-4ad2-9987-47fec1907637-kube-api-access-vldf6\") pod \"aws-ebs-csi-driver-node-6vp2f\" (UID: \"78faeeea-464e-4ad2-9987-47fec1907637\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6vp2f"
Sep 19 08:20:17.047025 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.046137    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/637bbfb9-45ba-481d-b9fe-87f60051712c-hosts-file\") pod \"node-resolver-lqdbb\" (UID: \"637bbfb9-45ba-481d-b9fe-87f60051712c\") " pod="openshift-dns/node-resolver-lqdbb"
Sep 19 08:20:17.047025 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.046164    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c3a67fda-81e8-4d20-83d4-a37e277382b4-host-var-lib-kubelet\") pod \"multus-wln5n\" (UID: \"c3a67fda-81e8-4d20-83d4-a37e277382b4\") " pod="openshift-multus/multus-wln5n"
Sep 19 08:20:17.047025 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.046199    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/90d27f61-9e28-4cc5-bc29-12f69fda90a0-etc-sysctl-conf\") pod \"tuned-5x6rt\" (UID: \"90d27f61-9e28-4cc5-bc29-12f69fda90a0\") " pod="openshift-cluster-node-tuning-operator/tuned-5x6rt"
Sep 19 08:20:17.047025 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.046216    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/0904bd86-22aa-4c03-af38-ee3e765096be-agent-certs\") pod \"konnectivity-agent-7tt6m\" (UID: \"0904bd86-22aa-4c03-af38-ee3e765096be\") " pod="kube-system/konnectivity-agent-7tt6m"
Sep 19 08:20:17.047025 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.046227    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/637bbfb9-45ba-481d-b9fe-87f60051712c-hosts-file\") pod \"node-resolver-lqdbb\" (UID: \"637bbfb9-45ba-481d-b9fe-87f60051712c\") " pod="openshift-dns/node-resolver-lqdbb"
Sep 19 08:20:17.047025 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.046230    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/07ddac45-12a1-42e8-9566-55f08854c02e-log-socket\") pod \"ovnkube-node-zd2zp\" (UID: \"07ddac45-12a1-42e8-9566-55f08854c02e\") " pod="openshift-ovn-kubernetes/ovnkube-node-zd2zp"
Sep 19 08:20:17.047025 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.046339    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/19642aea-2c2e-4208-8329-09f03b1e5d5b-dbus\") pod \"global-pull-secret-syncer-xq8d8\" (UID: \"19642aea-2c2e-4208-8329-09f03b1e5d5b\") " pod="kube-system/global-pull-secret-syncer-xq8d8"
Sep 19 08:20:17.047025 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.046357    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c3a67fda-81e8-4d20-83d4-a37e277382b4-host-run-multus-certs\") pod \"multus-wln5n\" (UID: \"c3a67fda-81e8-4d20-83d4-a37e277382b4\") " pod="openshift-multus/multus-wln5n"
Sep 19 08:20:17.047025 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.046372    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/0904bd86-22aa-4c03-af38-ee3e765096be-konnectivity-ca\") pod \"konnectivity-agent-7tt6m\" (UID: \"0904bd86-22aa-4c03-af38-ee3e765096be\") " pod="kube-system/konnectivity-agent-7tt6m"
Sep 19 08:20:17.047025 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.046387    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l5wdh\" (UniqueName: \"kubernetes.io/projected/90d27f61-9e28-4cc5-bc29-12f69fda90a0-kube-api-access-l5wdh\") pod \"tuned-5x6rt\" (UID: \"90d27f61-9e28-4cc5-bc29-12f69fda90a0\") " pod="openshift-cluster-node-tuning-operator/tuned-5x6rt"
Sep 19 08:20:17.047025 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.046410    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/07ddac45-12a1-42e8-9566-55f08854c02e-ovn-node-metrics-cert\") pod \"ovnkube-node-zd2zp\" (UID: \"07ddac45-12a1-42e8-9566-55f08854c02e\") " pod="openshift-ovn-kubernetes/ovnkube-node-zd2zp"
Sep 19 08:20:17.047855 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.046441    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/07ddac45-12a1-42e8-9566-55f08854c02e-ovnkube-script-lib\") pod \"ovnkube-node-zd2zp\" (UID: \"07ddac45-12a1-42e8-9566-55f08854c02e\") " pod="openshift-ovn-kubernetes/ovnkube-node-zd2zp"
Sep 19 08:20:17.047855 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.046456    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c3a67fda-81e8-4d20-83d4-a37e277382b4-system-cni-dir\") pod \"multus-wln5n\" (UID: \"c3a67fda-81e8-4d20-83d4-a37e277382b4\") " pod="openshift-multus/multus-wln5n"
Sep 19 08:20:17.047855 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.046469    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/90d27f61-9e28-4cc5-bc29-12f69fda90a0-etc-sysctl-d\") pod \"tuned-5x6rt\" (UID: \"90d27f61-9e28-4cc5-bc29-12f69fda90a0\") " pod="openshift-cluster-node-tuning-operator/tuned-5x6rt"
Sep 19 08:20:17.047855 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.046486    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/07ddac45-12a1-42e8-9566-55f08854c02e-host-slash\") pod \"ovnkube-node-zd2zp\" (UID: \"07ddac45-12a1-42e8-9566-55f08854c02e\") " pod="openshift-ovn-kubernetes/ovnkube-node-zd2zp"
Sep 19 08:20:17.047855 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.046521    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jptt6\" (UniqueName: \"kubernetes.io/projected/65dc9622-6c75-457d-ad22-6841b2673aae-kube-api-access-jptt6\") pod \"node-ca-hb9xj\" (UID: \"65dc9622-6c75-457d-ad22-6841b2673aae\") " pod="openshift-image-registry/node-ca-hb9xj"
Sep 19 08:20:17.047855 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.046552    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/19642aea-2c2e-4208-8329-09f03b1e5d5b-original-pull-secret\") pod \"global-pull-secret-syncer-xq8d8\" (UID: \"19642aea-2c2e-4208-8329-09f03b1e5d5b\") " pod="kube-system/global-pull-secret-syncer-xq8d8"
Sep 19 08:20:17.047855 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.046567    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/90d27f61-9e28-4cc5-bc29-12f69fda90a0-etc-kubernetes\") pod \"tuned-5x6rt\" (UID: \"90d27f61-9e28-4cc5-bc29-12f69fda90a0\") " pod="openshift-cluster-node-tuning-operator/tuned-5x6rt"
Sep 19 08:20:17.047855 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.046588    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/90d27f61-9e28-4cc5-bc29-12f69fda90a0-tmp\") pod \"tuned-5x6rt\" (UID: \"90d27f61-9e28-4cc5-bc29-12f69fda90a0\") " pod="openshift-cluster-node-tuning-operator/tuned-5x6rt"
Sep 19 08:20:17.047855 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.046611    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/07ddac45-12a1-42e8-9566-55f08854c02e-host-kubelet\") pod \"ovnkube-node-zd2zp\" (UID: \"07ddac45-12a1-42e8-9566-55f08854c02e\") " pod="openshift-ovn-kubernetes/ovnkube-node-zd2zp"
Sep 19 08:20:17.047855 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.046639    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6kh4x\" (UniqueName: \"kubernetes.io/projected/79f6ab54-a988-4015-81c2-da35716eab52-kube-api-access-6kh4x\") pod \"network-metrics-daemon-md9ln\" (UID: \"79f6ab54-a988-4015-81c2-da35716eab52\") " pod="openshift-multus/network-metrics-daemon-md9ln"
Sep 19 08:20:17.047855 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.046672    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c3a67fda-81e8-4d20-83d4-a37e277382b4-etc-kubernetes\") pod \"multus-wln5n\" (UID: \"c3a67fda-81e8-4d20-83d4-a37e277382b4\") " pod="openshift-multus/multus-wln5n"
Sep 19 08:20:17.047855 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.046670    2578 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 19 08:20:17.047855 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.046702    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/07ddac45-12a1-42e8-9566-55f08854c02e-ovnkube-config\") pod \"ovnkube-node-zd2zp\" (UID: \"07ddac45-12a1-42e8-9566-55f08854c02e\") " pod="openshift-ovn-kubernetes/ovnkube-node-zd2zp"
Sep 19 08:20:17.047855 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.046725    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/90d27f61-9e28-4cc5-bc29-12f69fda90a0-etc-sysconfig\") pod \"tuned-5x6rt\" (UID: \"90d27f61-9e28-4cc5-bc29-12f69fda90a0\") " pod="openshift-cluster-node-tuning-operator/tuned-5x6rt"
Sep 19 08:20:17.047855 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.046750    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/637bbfb9-45ba-481d-b9fe-87f60051712c-tmp-dir\") pod \"node-resolver-lqdbb\" (UID: \"637bbfb9-45ba-481d-b9fe-87f60051712c\") " pod="openshift-dns/node-resolver-lqdbb"
Sep 19 08:20:17.047855 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.046770    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qhpbr\" (UniqueName: \"kubernetes.io/projected/647eb10e-629e-4cbc-bad4-234bbb862d6e-kube-api-access-qhpbr\") pod \"multus-additional-cni-plugins-9frbh\" (UID: \"647eb10e-629e-4cbc-bad4-234bbb862d6e\") " pod="openshift-multus/multus-additional-cni-plugins-9frbh"
Sep 19 08:20:17.047855 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.046789    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/07ddac45-12a1-42e8-9566-55f08854c02e-host-cni-bin\") pod \"ovnkube-node-zd2zp\" (UID: \"07ddac45-12a1-42e8-9566-55f08854c02e\") " pod="openshift-ovn-kubernetes/ovnkube-node-zd2zp"
Sep 19 08:20:17.048689 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.046807    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/07ddac45-12a1-42e8-9566-55f08854c02e-host-cni-netd\") pod \"ovnkube-node-zd2zp\" (UID: \"07ddac45-12a1-42e8-9566-55f08854c02e\") " pod="openshift-ovn-kubernetes/ovnkube-node-zd2zp"
Sep 19 08:20:17.048689 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.046827    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c3a67fda-81e8-4d20-83d4-a37e277382b4-host-var-lib-cni-multus\") pod \"multus-wln5n\" (UID: \"c3a67fda-81e8-4d20-83d4-a37e277382b4\") " pod="openshift-multus/multus-wln5n"
Sep 19 08:20:17.048689 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.046849    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2btqj\" (UniqueName: \"kubernetes.io/projected/637bbfb9-45ba-481d-b9fe-87f60051712c-kube-api-access-2btqj\") pod \"node-resolver-lqdbb\" (UID: \"637bbfb9-45ba-481d-b9fe-87f60051712c\") " pod="openshift-dns/node-resolver-lqdbb"
Sep 19 08:20:17.048689 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.046866    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/90d27f61-9e28-4cc5-bc29-12f69fda90a0-sys\") pod \"tuned-5x6rt\" (UID: \"90d27f61-9e28-4cc5-bc29-12f69fda90a0\") " pod="openshift-cluster-node-tuning-operator/tuned-5x6rt"
Sep 19 08:20:17.048689 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.046935    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/0904bd86-22aa-4c03-af38-ee3e765096be-konnectivity-ca\") pod \"konnectivity-agent-7tt6m\" (UID: \"0904bd86-22aa-4c03-af38-ee3e765096be\") " pod="kube-system/konnectivity-agent-7tt6m"
Sep 19 08:20:17.048689 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.047036    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/637bbfb9-45ba-481d-b9fe-87f60051712c-tmp-dir\") pod \"node-resolver-lqdbb\" (UID: \"637bbfb9-45ba-481d-b9fe-87f60051712c\") " pod="openshift-dns/node-resolver-lqdbb"
Sep 19 08:20:17.052845 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.052812    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/0904bd86-22aa-4c03-af38-ee3e765096be-agent-certs\") pod \"konnectivity-agent-7tt6m\" (UID: \"0904bd86-22aa-4c03-af38-ee3e765096be\") " pod="kube-system/konnectivity-agent-7tt6m"
Sep 19 08:20:17.056605 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.056584    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dfjs8\" (UniqueName: \"kubernetes.io/projected/dcc90f06-1e07-4b6b-add7-46f0cdd95ecf-kube-api-access-dfjs8\") pod \"iptables-alerter-jfkwf\" (UID: \"dcc90f06-1e07-4b6b-add7-46f0cdd95ecf\") " pod="openshift-network-operator/iptables-alerter-jfkwf"
Sep 19 08:20:17.056923 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.056895    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2btqj\" (UniqueName: \"kubernetes.io/projected/637bbfb9-45ba-481d-b9fe-87f60051712c-kube-api-access-2btqj\") pod \"node-resolver-lqdbb\" (UID: \"637bbfb9-45ba-481d-b9fe-87f60051712c\") " pod="openshift-dns/node-resolver-lqdbb"
Sep 19 08:20:17.081809 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.081686    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-157.ec2.internal" event={"ID":"ecf1d89d174b946c499fe0ba49cfe70a","Type":"ContainerStarted","Data":"389d7ecd981a9ef6bd4428958fa0fa566e8a80ccf4428ac03954d50a657d0c3c"}
Sep 19 08:20:17.083872 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.083847    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-157.ec2.internal" event={"ID":"6b8b6e7e22ce47a3909ac23b630738ee","Type":"ContainerStarted","Data":"da9b6af5043945b6590adcd9e90efa0475dad98ae28af8265bb0c477b0a06003"}
Sep 19 08:20:17.147603 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.147573    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/90d27f61-9e28-4cc5-bc29-12f69fda90a0-etc-sysctl-conf\") pod \"tuned-5x6rt\" (UID: \"90d27f61-9e28-4cc5-bc29-12f69fda90a0\") " pod="openshift-cluster-node-tuning-operator/tuned-5x6rt"
Sep 19 08:20:17.147758 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.147618    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/07ddac45-12a1-42e8-9566-55f08854c02e-log-socket\") pod \"ovnkube-node-zd2zp\" (UID: \"07ddac45-12a1-42e8-9566-55f08854c02e\") " pod="openshift-ovn-kubernetes/ovnkube-node-zd2zp"
Sep 19 08:20:17.147758 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.147641    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/19642aea-2c2e-4208-8329-09f03b1e5d5b-dbus\") pod \"global-pull-secret-syncer-xq8d8\" (UID: \"19642aea-2c2e-4208-8329-09f03b1e5d5b\") " pod="kube-system/global-pull-secret-syncer-xq8d8"
Sep 19 08:20:17.147758 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.147661    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c3a67fda-81e8-4d20-83d4-a37e277382b4-host-run-multus-certs\") pod \"multus-wln5n\" (UID: \"c3a67fda-81e8-4d20-83d4-a37e277382b4\") " pod="openshift-multus/multus-wln5n"
Sep 19 08:20:17.147758 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.147683    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l5wdh\" (UniqueName: \"kubernetes.io/projected/90d27f61-9e28-4cc5-bc29-12f69fda90a0-kube-api-access-l5wdh\") pod \"tuned-5x6rt\" (UID: \"90d27f61-9e28-4cc5-bc29-12f69fda90a0\") " pod="openshift-cluster-node-tuning-operator/tuned-5x6rt"
Sep 19 08:20:17.147758 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.147702    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/07ddac45-12a1-42e8-9566-55f08854c02e-ovn-node-metrics-cert\") pod \"ovnkube-node-zd2zp\" (UID: \"07ddac45-12a1-42e8-9566-55f08854c02e\") " pod="openshift-ovn-kubernetes/ovnkube-node-zd2zp"
Sep 19 08:20:17.147758 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.147723    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/07ddac45-12a1-42e8-9566-55f08854c02e-ovnkube-script-lib\") pod \"ovnkube-node-zd2zp\" (UID: \"07ddac45-12a1-42e8-9566-55f08854c02e\") " pod="openshift-ovn-kubernetes/ovnkube-node-zd2zp"
Sep 19 08:20:17.147758 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.147732    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/07ddac45-12a1-42e8-9566-55f08854c02e-log-socket\") pod \"ovnkube-node-zd2zp\" (UID: \"07ddac45-12a1-42e8-9566-55f08854c02e\") " pod="openshift-ovn-kubernetes/ovnkube-node-zd2zp"
Sep 19 08:20:17.147758 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.147745    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c3a67fda-81e8-4d20-83d4-a37e277382b4-system-cni-dir\") pod \"multus-wln5n\" (UID: \"c3a67fda-81e8-4d20-83d4-a37e277382b4\") " pod="openshift-multus/multus-wln5n"
Sep 19 08:20:17.147758 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.147740    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/90d27f61-9e28-4cc5-bc29-12f69fda90a0-etc-sysctl-conf\") pod \"tuned-5x6rt\" (UID: \"90d27f61-9e28-4cc5-bc29-12f69fda90a0\") " pod="openshift-cluster-node-tuning-operator/tuned-5x6rt"
Sep 19 08:20:17.148036 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.147766    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/90d27f61-9e28-4cc5-bc29-12f69fda90a0-etc-sysctl-d\") pod \"tuned-5x6rt\" (UID: \"90d27f61-9e28-4cc5-bc29-12f69fda90a0\") " pod="openshift-cluster-node-tuning-operator/tuned-5x6rt"
Sep 19 08:20:17.148036 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.147787    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/07ddac45-12a1-42e8-9566-55f08854c02e-host-slash\") pod \"ovnkube-node-zd2zp\" (UID: \"07ddac45-12a1-42e8-9566-55f08854c02e\") " pod="openshift-ovn-kubernetes/ovnkube-node-zd2zp"
Sep 19 08:20:17.148036 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.147809    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jptt6\" (UniqueName: \"kubernetes.io/projected/65dc9622-6c75-457d-ad22-6841b2673aae-kube-api-access-jptt6\") pod \"node-ca-hb9xj\" (UID: \"65dc9622-6c75-457d-ad22-6841b2673aae\") " pod="openshift-image-registry/node-ca-hb9xj"
Sep 19 08:20:17.148036 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.147830    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/19642aea-2c2e-4208-8329-09f03b1e5d5b-original-pull-secret\") pod \"global-pull-secret-syncer-xq8d8\" (UID: \"19642aea-2c2e-4208-8329-09f03b1e5d5b\") " pod="kube-system/global-pull-secret-syncer-xq8d8"
Sep 19 08:20:17.148036 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.147851    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/90d27f61-9e28-4cc5-bc29-12f69fda90a0-etc-kubernetes\") pod \"tuned-5x6rt\" (UID: \"90d27f61-9e28-4cc5-bc29-12f69fda90a0\") " pod="openshift-cluster-node-tuning-operator/tuned-5x6rt"
Sep 19 08:20:17.148036 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.147873    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/90d27f61-9e28-4cc5-bc29-12f69fda90a0-tmp\") pod \"tuned-5x6rt\" (UID: \"90d27f61-9e28-4cc5-bc29-12f69fda90a0\") " pod="openshift-cluster-node-tuning-operator/tuned-5x6rt"
Sep 19 08:20:17.148036 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.147894    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/07ddac45-12a1-42e8-9566-55f08854c02e-host-kubelet\") pod \"ovnkube-node-zd2zp\" (UID: \"07ddac45-12a1-42e8-9566-55f08854c02e\") " pod="openshift-ovn-kubernetes/ovnkube-node-zd2zp"
Sep 19 08:20:17.148036 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.147917    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6kh4x\" (UniqueName: \"kubernetes.io/projected/79f6ab54-a988-4015-81c2-da35716eab52-kube-api-access-6kh4x\") pod \"network-metrics-daemon-md9ln\" (UID: \"79f6ab54-a988-4015-81c2-da35716eab52\") " pod="openshift-multus/network-metrics-daemon-md9ln"
Sep 19 08:20:17.148036 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.147939    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c3a67fda-81e8-4d20-83d4-a37e277382b4-etc-kubernetes\") pod \"multus-wln5n\" (UID: \"c3a67fda-81e8-4d20-83d4-a37e277382b4\") " pod="openshift-multus/multus-wln5n"
Sep 19 08:20:17.148036 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.147998    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/19642aea-2c2e-4208-8329-09f03b1e5d5b-dbus\") pod \"global-pull-secret-syncer-xq8d8\" (UID: \"19642aea-2c2e-4208-8329-09f03b1e5d5b\") " pod="kube-system/global-pull-secret-syncer-xq8d8"
Sep 19 08:20:17.148036 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.147998    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/07ddac45-12a1-42e8-9566-55f08854c02e-ovnkube-config\") pod \"ovnkube-node-zd2zp\" (UID: \"07ddac45-12a1-42e8-9566-55f08854c02e\") " pod="openshift-ovn-kubernetes/ovnkube-node-zd2zp"
Sep 19 08:20:17.148402 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.148040    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/90d27f61-9e28-4cc5-bc29-12f69fda90a0-etc-sysconfig\") pod \"tuned-5x6rt\" (UID: \"90d27f61-9e28-4cc5-bc29-12f69fda90a0\") " pod="openshift-cluster-node-tuning-operator/tuned-5x6rt"
Sep 19 08:20:17.148402 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.148069    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qhpbr\" (UniqueName: \"kubernetes.io/projected/647eb10e-629e-4cbc-bad4-234bbb862d6e-kube-api-access-qhpbr\") pod \"multus-additional-cni-plugins-9frbh\" (UID: \"647eb10e-629e-4cbc-bad4-234bbb862d6e\") " pod="openshift-multus/multus-additional-cni-plugins-9frbh"
Sep 19 08:20:17.148402 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.148095    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/07ddac45-12a1-42e8-9566-55f08854c02e-host-cni-bin\") pod \"ovnkube-node-zd2zp\" (UID: \"07ddac45-12a1-42e8-9566-55f08854c02e\") " pod="openshift-ovn-kubernetes/ovnkube-node-zd2zp"
Sep 19 08:20:17.148402 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.148118    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/07ddac45-12a1-42e8-9566-55f08854c02e-host-cni-netd\") pod \"ovnkube-node-zd2zp\" (UID: \"07ddac45-12a1-42e8-9566-55f08854c02e\") " pod="openshift-ovn-kubernetes/ovnkube-node-zd2zp"
Sep 19 08:20:17.148402 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.148143    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c3a67fda-81e8-4d20-83d4-a37e277382b4-host-var-lib-cni-multus\") pod \"multus-wln5n\" (UID: \"c3a67fda-81e8-4d20-83d4-a37e277382b4\") " pod="openshift-multus/multus-wln5n"
Sep 19 08:20:17.148402 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.148187    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/90d27f61-9e28-4cc5-bc29-12f69fda90a0-sys\") pod \"tuned-5x6rt\" (UID: \"90d27f61-9e28-4cc5-bc29-12f69fda90a0\") " pod="openshift-cluster-node-tuning-operator/tuned-5x6rt"
Sep 19 08:20:17.148402 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.148211    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/07ddac45-12a1-42e8-9566-55f08854c02e-run-ovn\") pod \"ovnkube-node-zd2zp\" (UID: \"07ddac45-12a1-42e8-9566-55f08854c02e\") " pod="openshift-ovn-kubernetes/ovnkube-node-zd2zp"
Sep 19 08:20:17.148402 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.148236    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/07ddac45-12a1-42e8-9566-55f08854c02e-host-run-ovn-kubernetes\") pod \"ovnkube-node-zd2zp\" (UID: \"07ddac45-12a1-42e8-9566-55f08854c02e\") " pod="openshift-ovn-kubernetes/ovnkube-node-zd2zp"
Sep 19 08:20:17.148402 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.148263    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c3a67fda-81e8-4d20-83d4-a37e277382b4-multus-socket-dir-parent\") pod \"multus-wln5n\" (UID: \"c3a67fda-81e8-4d20-83d4-a37e277382b4\") " pod="openshift-multus/multus-wln5n"
Sep 19 08:20:17.148402 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.148287    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c3a67fda-81e8-4d20-83d4-a37e277382b4-hostroot\") pod \"multus-wln5n\" (UID: \"c3a67fda-81e8-4d20-83d4-a37e277382b4\") " pod="openshift-multus/multus-wln5n"
Sep 19 08:20:17.148402 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.148310    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c3a67fda-81e8-4d20-83d4-a37e277382b4-multus-daemon-config\") pod \"multus-wln5n\" (UID: \"c3a67fda-81e8-4d20-83d4-a37e277382b4\") " pod="openshift-multus/multus-wln5n"
Sep 19 08:20:17.148402 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.148333    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/07ddac45-12a1-42e8-9566-55f08854c02e-var-lib-openvswitch\") pod \"ovnkube-node-zd2zp\" (UID: \"07ddac45-12a1-42e8-9566-55f08854c02e\") " pod="openshift-ovn-kubernetes/ovnkube-node-zd2zp"
Sep 19 08:20:17.148402 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.148358    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/07ddac45-12a1-42e8-9566-55f08854c02e-env-overrides\") pod \"ovnkube-node-zd2zp\" (UID: \"07ddac45-12a1-42e8-9566-55f08854c02e\") " pod="openshift-ovn-kubernetes/ovnkube-node-zd2zp"
Sep 19 08:20:17.148402 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.148384    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/65dc9622-6c75-457d-ad22-6841b2673aae-serviceca\") pod \"node-ca-hb9xj\" (UID: \"65dc9622-6c75-457d-ad22-6841b2673aae\") " pod="openshift-image-registry/node-ca-hb9xj"
Sep 19 08:20:17.148402 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.148387    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/07ddac45-12a1-42e8-9566-55f08854c02e-ovnkube-script-lib\") pod \"ovnkube-node-zd2zp\" (UID: \"07ddac45-12a1-42e8-9566-55f08854c02e\") " pod="openshift-ovn-kubernetes/ovnkube-node-zd2zp"
Sep 19 08:20:17.148402 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.148409    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/647eb10e-629e-4cbc-bad4-234bbb862d6e-cni-binary-copy\") pod \"multus-additional-cni-plugins-9frbh\" (UID: \"647eb10e-629e-4cbc-bad4-234bbb862d6e\") " pod="openshift-multus/multus-additional-cni-plugins-9frbh"
Sep 19 08:20:17.148917 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.148432    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/90d27f61-9e28-4cc5-bc29-12f69fda90a0-host\") pod \"tuned-5x6rt\" (UID: \"90d27f61-9e28-4cc5-bc29-12f69fda90a0\") " pod="openshift-cluster-node-tuning-operator/tuned-5x6rt"
Sep 19 08:20:17.148917 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.148456    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c3a67fda-81e8-4d20-83d4-a37e277382b4-host-run-netns\") pod \"multus-wln5n\" (UID: \"c3a67fda-81e8-4d20-83d4-a37e277382b4\") " pod="openshift-multus/multus-wln5n"
Sep 19 08:20:17.148917 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.148481    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/78faeeea-464e-4ad2-9987-47fec1907637-kubelet-dir\") pod \"aws-ebs-csi-driver-node-6vp2f\" (UID: \"78faeeea-464e-4ad2-9987-47fec1907637\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6vp2f"
Sep 19 08:20:17.148917 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.148506    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/78faeeea-464e-4ad2-9987-47fec1907637-socket-dir\") pod \"aws-ebs-csi-driver-node-6vp2f\" (UID: \"78faeeea-464e-4ad2-9987-47fec1907637\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6vp2f"
Sep 19 08:20:17.148917 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.148505    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/07ddac45-12a1-42e8-9566-55f08854c02e-ovnkube-config\") pod \"ovnkube-node-zd2zp\" (UID: \"07ddac45-12a1-42e8-9566-55f08854c02e\") " pod="openshift-ovn-kubernetes/ovnkube-node-zd2zp"
Sep 19 08:20:17.148917 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.148560    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c3a67fda-81e8-4d20-83d4-a37e277382b4-host-run-multus-certs\") pod \"multus-wln5n\" (UID: \"c3a67fda-81e8-4d20-83d4-a37e277382b4\") " pod="openshift-multus/multus-wln5n"
Sep 19 08:20:17.148917 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.148603    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c3a67fda-81e8-4d20-83d4-a37e277382b4-system-cni-dir\") pod \"multus-wln5n\" (UID: \"c3a67fda-81e8-4d20-83d4-a37e277382b4\") " pod="openshift-multus/multus-wln5n"
Sep 19 08:20:17.148917 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.148639    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/07ddac45-12a1-42e8-9566-55f08854c02e-host-cni-bin\") pod \"ovnkube-node-zd2zp\" (UID: \"07ddac45-12a1-42e8-9566-55f08854c02e\") " pod="openshift-ovn-kubernetes/ovnkube-node-zd2zp"
Sep 19 08:20:17.148917 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.148656    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/90d27f61-9e28-4cc5-bc29-12f69fda90a0-sys\") pod \"tuned-5x6rt\" (UID: \"90d27f61-9e28-4cc5-bc29-12f69fda90a0\") " pod="openshift-cluster-node-tuning-operator/tuned-5x6rt"
Sep 19 08:20:17.148917 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.148678    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c3a67fda-81e8-4d20-83d4-a37e277382b4-host-var-lib-cni-multus\") pod \"multus-wln5n\" (UID: \"c3a67fda-81e8-4d20-83d4-a37e277382b4\") " pod="openshift-multus/multus-wln5n"
Sep 19 08:20:17.148917 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.148715    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/07ddac45-12a1-42e8-9566-55f08854c02e-run-ovn\") pod \"ovnkube-node-zd2zp\" (UID: \"07ddac45-12a1-42e8-9566-55f08854c02e\") " pod="openshift-ovn-kubernetes/ovnkube-node-zd2zp"
Sep 19 08:20:17.148917 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.148750    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/07ddac45-12a1-42e8-9566-55f08854c02e-host-run-ovn-kubernetes\") pod \"ovnkube-node-zd2zp\" (UID: \"07ddac45-12a1-42e8-9566-55f08854c02e\") " pod="openshift-ovn-kubernetes/ovnkube-node-zd2zp"
Sep 19 08:20:17.148917 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.148773    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/90d27f61-9e28-4cc5-bc29-12f69fda90a0-etc-sysctl-d\") pod \"tuned-5x6rt\" (UID: \"90d27f61-9e28-4cc5-bc29-12f69fda90a0\") " pod="openshift-cluster-node-tuning-operator/tuned-5x6rt"
Sep 19 08:20:17.148917 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.148801    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c3a67fda-81e8-4d20-83d4-a37e277382b4-multus-socket-dir-parent\") pod \"multus-wln5n\" (UID: \"c3a67fda-81e8-4d20-83d4-a37e277382b4\") " pod="openshift-multus/multus-wln5n"
Sep 19 08:20:17.148917 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.148804    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/07ddac45-12a1-42e8-9566-55f08854c02e-host-slash\") pod \"ovnkube-node-zd2zp\" (UID: \"07ddac45-12a1-42e8-9566-55f08854c02e\") " pod="openshift-ovn-kubernetes/ovnkube-node-zd2zp"
Sep 19 08:20:17.148917 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.148836    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/07ddac45-12a1-42e8-9566-55f08854c02e-host-kubelet\") pod \"ovnkube-node-zd2zp\" (UID: \"07ddac45-12a1-42e8-9566-55f08854c02e\") " pod="openshift-ovn-kubernetes/ovnkube-node-zd2zp"
Sep 19 08:20:17.149476 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.149016    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c3a67fda-81e8-4d20-83d4-a37e277382b4-etc-kubernetes\") pod \"multus-wln5n\" (UID: \"c3a67fda-81e8-4d20-83d4-a37e277382b4\") " pod="openshift-multus/multus-wln5n"
Sep 19 08:20:17.149476 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.149050    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/78faeeea-464e-4ad2-9987-47fec1907637-registration-dir\") pod \"aws-ebs-csi-driver-node-6vp2f\" (UID: \"78faeeea-464e-4ad2-9987-47fec1907637\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6vp2f"
Sep 19 08:20:17.149476 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.149077    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/90d27f61-9e28-4cc5-bc29-12f69fda90a0-etc-systemd\") pod \"tuned-5x6rt\" (UID: \"90d27f61-9e28-4cc5-bc29-12f69fda90a0\") " pod="openshift-cluster-node-tuning-operator/tuned-5x6rt"
Sep 19 08:20:17.149476 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:17.149103    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 19 08:20:17.149476 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:17.149166    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/19642aea-2c2e-4208-8329-09f03b1e5d5b-original-pull-secret podName:19642aea-2c2e-4208-8329-09f03b1e5d5b nodeName:}" failed. No retries permitted until 2026-09-19 08:20:17.649145558 +0000 UTC m=+3.058981241 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/19642aea-2c2e-4208-8329-09f03b1e5d5b-original-pull-secret") pod "global-pull-secret-syncer-xq8d8" (UID: "19642aea-2c2e-4208-8329-09f03b1e5d5b") : object "kube-system"/"original-pull-secret" not registered
Sep 19 08:20:17.149476 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.149202    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/90d27f61-9e28-4cc5-bc29-12f69fda90a0-etc-modprobe-d\") pod \"tuned-5x6rt\" (UID: \"90d27f61-9e28-4cc5-bc29-12f69fda90a0\") " pod="openshift-cluster-node-tuning-operator/tuned-5x6rt"
Sep 19 08:20:17.149476 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.148605    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/90d27f61-9e28-4cc5-bc29-12f69fda90a0-etc-sysconfig\") pod \"tuned-5x6rt\" (UID: \"90d27f61-9e28-4cc5-bc29-12f69fda90a0\") " pod="openshift-cluster-node-tuning-operator/tuned-5x6rt"
Sep 19 08:20:17.149476 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.149243    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/90d27f61-9e28-4cc5-bc29-12f69fda90a0-etc-kubernetes\") pod \"tuned-5x6rt\" (UID: \"90d27f61-9e28-4cc5-bc29-12f69fda90a0\") " pod="openshift-cluster-node-tuning-operator/tuned-5x6rt"
Sep 19 08:20:17.149476 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.149294    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/65dc9622-6c75-457d-ad22-6841b2673aae-serviceca\") pod \"node-ca-hb9xj\" (UID: \"65dc9622-6c75-457d-ad22-6841b2673aae\") " pod="openshift-image-registry/node-ca-hb9xj"
Sep 19 08:20:17.149476 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.149365    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c3a67fda-81e8-4d20-83d4-a37e277382b4-hostroot\") pod \"multus-wln5n\" (UID: \"c3a67fda-81e8-4d20-83d4-a37e277382b4\") " pod="openshift-multus/multus-wln5n"
Sep 19 08:20:17.149476 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.149370    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/647eb10e-629e-4cbc-bad4-234bbb862d6e-cni-binary-copy\") pod \"multus-additional-cni-plugins-9frbh\" (UID: \"647eb10e-629e-4cbc-bad4-234bbb862d6e\") " pod="openshift-multus/multus-additional-cni-plugins-9frbh"
Sep 19 08:20:17.149476 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.149425    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/07ddac45-12a1-42e8-9566-55f08854c02e-var-lib-openvswitch\") pod \"ovnkube-node-zd2zp\" (UID: \"07ddac45-12a1-42e8-9566-55f08854c02e\") " pod="openshift-ovn-kubernetes/ovnkube-node-zd2zp"
Sep 19 08:20:17.149476 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.149440    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/90d27f61-9e28-4cc5-bc29-12f69fda90a0-host\") pod \"tuned-5x6rt\" (UID: \"90d27f61-9e28-4cc5-bc29-12f69fda90a0\") " pod="openshift-cluster-node-tuning-operator/tuned-5x6rt"
Sep 19 08:20:17.149476 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.149443    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/90d27f61-9e28-4cc5-bc29-12f69fda90a0-etc-systemd\") pod \"tuned-5x6rt\" (UID: \"90d27f61-9e28-4cc5-bc29-12f69fda90a0\") " pod="openshift-cluster-node-tuning-operator/tuned-5x6rt"
Sep 19 08:20:17.149476 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.149104    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/90d27f61-9e28-4cc5-bc29-12f69fda90a0-etc-modprobe-d\") pod \"tuned-5x6rt\" (UID: \"90d27f61-9e28-4cc5-bc29-12f69fda90a0\") " pod="openshift-cluster-node-tuning-operator/tuned-5x6rt"
Sep 19 08:20:17.150155 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.149486    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/07ddac45-12a1-42e8-9566-55f08854c02e-run-systemd\") pod \"ovnkube-node-zd2zp\" (UID: \"07ddac45-12a1-42e8-9566-55f08854c02e\") " pod="openshift-ovn-kubernetes/ovnkube-node-zd2zp"
Sep 19 08:20:17.150155 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.149513    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c3a67fda-81e8-4d20-83d4-a37e277382b4-cni-binary-copy\") pod \"multus-wln5n\" (UID: \"c3a67fda-81e8-4d20-83d4-a37e277382b4\") " pod="openshift-multus/multus-wln5n"
Sep 19 08:20:17.150155 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.149535    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/78faeeea-464e-4ad2-9987-47fec1907637-device-dir\") pod \"aws-ebs-csi-driver-node-6vp2f\" (UID: \"78faeeea-464e-4ad2-9987-47fec1907637\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6vp2f"
Sep 19 08:20:17.150155 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.149559    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/647eb10e-629e-4cbc-bad4-234bbb862d6e-cnibin\") pod \"multus-additional-cni-plugins-9frbh\" (UID: \"647eb10e-629e-4cbc-bad4-234bbb862d6e\") " pod="openshift-multus/multus-additional-cni-plugins-9frbh"
Sep 19 08:20:17.150155 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.149584    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/90d27f61-9e28-4cc5-bc29-12f69fda90a0-run\") pod \"tuned-5x6rt\" (UID: \"90d27f61-9e28-4cc5-bc29-12f69fda90a0\") " pod="openshift-cluster-node-tuning-operator/tuned-5x6rt"
Sep 19 08:20:17.150155 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.149607    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c3a67fda-81e8-4d20-83d4-a37e277382b4-multus-conf-dir\") pod \"multus-wln5n\" (UID: \"c3a67fda-81e8-4d20-83d4-a37e277382b4\") " pod="openshift-multus/multus-wln5n"
Sep 19 08:20:17.150155 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.149651    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/78faeeea-464e-4ad2-9987-47fec1907637-sys-fs\") pod \"aws-ebs-csi-driver-node-6vp2f\" (UID: \"78faeeea-464e-4ad2-9987-47fec1907637\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6vp2f"
Sep 19 08:20:17.150155 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.149663    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/78faeeea-464e-4ad2-9987-47fec1907637-socket-dir\") pod \"aws-ebs-csi-driver-node-6vp2f\" (UID: \"78faeeea-464e-4ad2-9987-47fec1907637\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6vp2f"
Sep 19 08:20:17.150155 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.149677    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/647eb10e-629e-4cbc-bad4-234bbb862d6e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-9frbh\" (UID: \"647eb10e-629e-4cbc-bad4-234bbb862d6e\") " pod="openshift-multus/multus-additional-cni-plugins-9frbh"
Sep 19 08:20:17.150155 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.149704    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/647eb10e-629e-4cbc-bad4-234bbb862d6e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-9frbh\" (UID: \"647eb10e-629e-4cbc-bad4-234bbb862d6e\") " pod="openshift-multus/multus-additional-cni-plugins-9frbh"
Sep 19 08:20:17.150155 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.149720    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c3a67fda-81e8-4d20-83d4-a37e277382b4-host-run-netns\") pod \"multus-wln5n\" (UID: \"c3a67fda-81e8-4d20-83d4-a37e277382b4\") " pod="openshift-multus/multus-wln5n"
Sep 19 08:20:17.150155 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.149732    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/65dc9622-6c75-457d-ad22-6841b2673aae-host\") pod \"node-ca-hb9xj\" (UID: \"65dc9622-6c75-457d-ad22-6841b2673aae\") " pod="openshift-image-registry/node-ca-hb9xj"
Sep 19 08:20:17.150155 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.149757    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c3a67fda-81e8-4d20-83d4-a37e277382b4-host-var-lib-cni-bin\") pod \"multus-wln5n\" (UID: \"c3a67fda-81e8-4d20-83d4-a37e277382b4\") " pod="openshift-multus/multus-wln5n"
Sep 19 08:20:17.150155 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.149773    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/78faeeea-464e-4ad2-9987-47fec1907637-kubelet-dir\") pod \"aws-ebs-csi-driver-node-6vp2f\" (UID: \"78faeeea-464e-4ad2-9987-47fec1907637\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6vp2f"
Sep 19 08:20:17.150155 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.149782    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/647eb10e-629e-4cbc-bad4-234bbb862d6e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-9frbh\" (UID: \"647eb10e-629e-4cbc-bad4-234bbb862d6e\") " pod="openshift-multus/multus-additional-cni-plugins-9frbh"
Sep 19 08:20:17.150155 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.149809    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/90d27f61-9e28-4cc5-bc29-12f69fda90a0-lib-modules\") pod \"tuned-5x6rt\" (UID: \"90d27f61-9e28-4cc5-bc29-12f69fda90a0\") " pod="openshift-cluster-node-tuning-operator/tuned-5x6rt"
Sep 19 08:20:17.150155 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.149836    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/07ddac45-12a1-42e8-9566-55f08854c02e-run-openvswitch\") pod \"ovnkube-node-zd2zp\" (UID: \"07ddac45-12a1-42e8-9566-55f08854c02e\") " pod="openshift-ovn-kubernetes/ovnkube-node-zd2zp"
Sep 19 08:20:17.150957 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.149843    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/07ddac45-12a1-42e8-9566-55f08854c02e-env-overrides\") pod \"ovnkube-node-zd2zp\" (UID: \"07ddac45-12a1-42e8-9566-55f08854c02e\") " pod="openshift-ovn-kubernetes/ovnkube-node-zd2zp"
Sep 19 08:20:17.150957 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.149862    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/07ddac45-12a1-42e8-9566-55f08854c02e-node-log\") pod \"ovnkube-node-zd2zp\" (UID: \"07ddac45-12a1-42e8-9566-55f08854c02e\") " pod="openshift-ovn-kubernetes/ovnkube-node-zd2zp"
Sep 19 08:20:17.150957 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.149888    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c3a67fda-81e8-4d20-83d4-a37e277382b4-multus-cni-dir\") pod \"multus-wln5n\" (UID: \"c3a67fda-81e8-4d20-83d4-a37e277382b4\") " pod="openshift-multus/multus-wln5n"
Sep 19 08:20:17.150957 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.149896    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/07ddac45-12a1-42e8-9566-55f08854c02e-host-cni-netd\") pod \"ovnkube-node-zd2zp\" (UID: \"07ddac45-12a1-42e8-9566-55f08854c02e\") " pod="openshift-ovn-kubernetes/ovnkube-node-zd2zp"
Sep 19 08:20:17.150957 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.149916    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wz4bl\" (UniqueName: \"kubernetes.io/projected/c3a67fda-81e8-4d20-83d4-a37e277382b4-kube-api-access-wz4bl\") pod \"multus-wln5n\" (UID: \"c3a67fda-81e8-4d20-83d4-a37e277382b4\") " pod="openshift-multus/multus-wln5n"
Sep 19 08:20:17.150957 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.149929    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c3a67fda-81e8-4d20-83d4-a37e277382b4-multus-daemon-config\") pod \"multus-wln5n\" (UID: \"c3a67fda-81e8-4d20-83d4-a37e277382b4\") " pod="openshift-multus/multus-wln5n"
Sep 19 08:20:17.150957 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.149941    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-shczr\" (UniqueName: \"kubernetes.io/projected/07ddac45-12a1-42e8-9566-55f08854c02e-kube-api-access-shczr\") pod \"ovnkube-node-zd2zp\" (UID: \"07ddac45-12a1-42e8-9566-55f08854c02e\") " pod="openshift-ovn-kubernetes/ovnkube-node-zd2zp"
Sep 19 08:20:17.150957 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.149945    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/78faeeea-464e-4ad2-9987-47fec1907637-registration-dir\") pod \"aws-ebs-csi-driver-node-6vp2f\" (UID: \"78faeeea-464e-4ad2-9987-47fec1907637\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6vp2f"
Sep 19 08:20:17.150957 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.149965    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/90d27f61-9e28-4cc5-bc29-12f69fda90a0-etc-tuned\") pod \"tuned-5x6rt\" (UID: \"90d27f61-9e28-4cc5-bc29-12f69fda90a0\") " pod="openshift-cluster-node-tuning-operator/tuned-5x6rt"
Sep 19 08:20:17.150957 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.149986    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/07ddac45-12a1-42e8-9566-55f08854c02e-systemd-units\") pod \"ovnkube-node-zd2zp\" (UID: \"07ddac45-12a1-42e8-9566-55f08854c02e\") " pod="openshift-ovn-kubernetes/ovnkube-node-zd2zp"
Sep 19 08:20:17.150957 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.149987    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/07ddac45-12a1-42e8-9566-55f08854c02e-run-systemd\") pod \"ovnkube-node-zd2zp\" (UID: \"07ddac45-12a1-42e8-9566-55f08854c02e\") " pod="openshift-ovn-kubernetes/ovnkube-node-zd2zp"
Sep 19 08:20:17.150957 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.150038    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/90d27f61-9e28-4cc5-bc29-12f69fda90a0-lib-modules\") pod \"tuned-5x6rt\" (UID: \"90d27f61-9e28-4cc5-bc29-12f69fda90a0\") " pod="openshift-cluster-node-tuning-operator/tuned-5x6rt"
Sep 19 08:20:17.150957 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.150039    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/07ddac45-12a1-42e8-9566-55f08854c02e-run-openvswitch\") pod \"ovnkube-node-zd2zp\" (UID: \"07ddac45-12a1-42e8-9566-55f08854c02e\") " pod="openshift-ovn-kubernetes/ovnkube-node-zd2zp"
Sep 19 08:20:17.150957 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.150073    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/647eb10e-629e-4cbc-bad4-234bbb862d6e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-9frbh\" (UID: \"647eb10e-629e-4cbc-bad4-234bbb862d6e\") " pod="openshift-multus/multus-additional-cni-plugins-9frbh"
Sep 19 08:20:17.150957 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.150093    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/78faeeea-464e-4ad2-9987-47fec1907637-device-dir\") pod \"aws-ebs-csi-driver-node-6vp2f\" (UID: \"78faeeea-464e-4ad2-9987-47fec1907637\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6vp2f"
Sep 19 08:20:17.150957 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.150107    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/07ddac45-12a1-42e8-9566-55f08854c02e-node-log\") pod \"ovnkube-node-zd2zp\" (UID: \"07ddac45-12a1-42e8-9566-55f08854c02e\") " pod="openshift-ovn-kubernetes/ovnkube-node-zd2zp"
Sep 19 08:20:17.150957 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.150143    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/647eb10e-629e-4cbc-bad4-234bbb862d6e-cnibin\") pod \"multus-additional-cni-plugins-9frbh\" (UID: \"647eb10e-629e-4cbc-bad4-234bbb862d6e\") " pod="openshift-multus/multus-additional-cni-plugins-9frbh"
Sep 19 08:20:17.150957 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.150214    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c3a67fda-81e8-4d20-83d4-a37e277382b4-multus-cni-dir\") pod \"multus-wln5n\" (UID: \"c3a67fda-81e8-4d20-83d4-a37e277382b4\") " pod="openshift-multus/multus-wln5n"
Sep 19 08:20:17.151773 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.150375    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/90d27f61-9e28-4cc5-bc29-12f69fda90a0-run\") pod \"tuned-5x6rt\" (UID: \"90d27f61-9e28-4cc5-bc29-12f69fda90a0\") " pod="openshift-cluster-node-tuning-operator/tuned-5x6rt"
Sep 19 08:20:17.151773 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.150418    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c3a67fda-81e8-4d20-83d4-a37e277382b4-cni-binary-copy\") pod \"multus-wln5n\" (UID: \"c3a67fda-81e8-4d20-83d4-a37e277382b4\") " pod="openshift-multus/multus-wln5n"
Sep 19 08:20:17.151773 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.150432    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/78faeeea-464e-4ad2-9987-47fec1907637-sys-fs\") pod \"aws-ebs-csi-driver-node-6vp2f\" (UID: \"78faeeea-464e-4ad2-9987-47fec1907637\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6vp2f"
Sep 19 08:20:17.151773 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.150463    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/07ddac45-12a1-42e8-9566-55f08854c02e-host-run-netns\") pod \"ovnkube-node-zd2zp\" (UID: \"07ddac45-12a1-42e8-9566-55f08854c02e\") " pod="openshift-ovn-kubernetes/ovnkube-node-zd2zp"
Sep 19 08:20:17.151773 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.150480    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/65dc9622-6c75-457d-ad22-6841b2673aae-host\") pod \"node-ca-hb9xj\" (UID: \"65dc9622-6c75-457d-ad22-6841b2673aae\") " pod="openshift-image-registry/node-ca-hb9xj"
Sep 19 08:20:17.151773 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.150492    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c3a67fda-81e8-4d20-83d4-a37e277382b4-host-run-k8s-cni-cncf-io\") pod \"multus-wln5n\" (UID: \"c3a67fda-81e8-4d20-83d4-a37e277382b4\") " pod="openshift-multus/multus-wln5n"
Sep 19 08:20:17.151773 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.150513    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c3a67fda-81e8-4d20-83d4-a37e277382b4-multus-conf-dir\") pod \"multus-wln5n\" (UID: \"c3a67fda-81e8-4d20-83d4-a37e277382b4\") " pod="openshift-multus/multus-wln5n"
Sep 19 08:20:17.151773 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.150519    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/90d27f61-9e28-4cc5-bc29-12f69fda90a0-var-lib-kubelet\") pod \"tuned-5x6rt\" (UID: \"90d27f61-9e28-4cc5-bc29-12f69fda90a0\") " pod="openshift-cluster-node-tuning-operator/tuned-5x6rt"
Sep 19 08:20:17.151773 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.150519    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c3a67fda-81e8-4d20-83d4-a37e277382b4-host-var-lib-cni-bin\") pod \"multus-wln5n\" (UID: \"c3a67fda-81e8-4d20-83d4-a37e277382b4\") " pod="openshift-multus/multus-wln5n"
Sep 19 08:20:17.151773 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.150481    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/07ddac45-12a1-42e8-9566-55f08854c02e-systemd-units\") pod \"ovnkube-node-zd2zp\" (UID: \"07ddac45-12a1-42e8-9566-55f08854c02e\") " pod="openshift-ovn-kubernetes/ovnkube-node-zd2zp"
Sep 19 08:20:17.151773 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.150535    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/647eb10e-629e-4cbc-bad4-234bbb862d6e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-9frbh\" (UID: \"647eb10e-629e-4cbc-bad4-234bbb862d6e\") " pod="openshift-multus/multus-additional-cni-plugins-9frbh"
Sep 19 08:20:17.151773 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.150564    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r7rcd\" (UniqueName: \"kubernetes.io/projected/8324d8db-b082-475a-8bb7-8c578404b02e-kube-api-access-r7rcd\") pod \"network-check-target-l6m8x\" (UID: \"8324d8db-b082-475a-8bb7-8c578404b02e\") " pod="openshift-network-diagnostics/network-check-target-l6m8x"
Sep 19 08:20:17.151773 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.150590    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/90d27f61-9e28-4cc5-bc29-12f69fda90a0-var-lib-kubelet\") pod \"tuned-5x6rt\" (UID: \"90d27f61-9e28-4cc5-bc29-12f69fda90a0\") " pod="openshift-cluster-node-tuning-operator/tuned-5x6rt"
Sep 19 08:20:17.151773 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.150590    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/19642aea-2c2e-4208-8329-09f03b1e5d5b-kubelet-config\") pod \"global-pull-secret-syncer-xq8d8\" (UID: \"19642aea-2c2e-4208-8329-09f03b1e5d5b\") " pod="kube-system/global-pull-secret-syncer-xq8d8"
Sep 19 08:20:17.151773 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.150624    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/647eb10e-629e-4cbc-bad4-234bbb862d6e-system-cni-dir\") pod \"multus-additional-cni-plugins-9frbh\" (UID: \"647eb10e-629e-4cbc-bad4-234bbb862d6e\") " pod="openshift-multus/multus-additional-cni-plugins-9frbh"
Sep 19 08:20:17.151773 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.150629    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/19642aea-2c2e-4208-8329-09f03b1e5d5b-kubelet-config\") pod \"global-pull-secret-syncer-xq8d8\" (UID: \"19642aea-2c2e-4208-8329-09f03b1e5d5b\") " pod="kube-system/global-pull-secret-syncer-xq8d8"
Sep 19 08:20:17.151773 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.150649    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/07ddac45-12a1-42e8-9566-55f08854c02e-etc-openvswitch\") pod \"ovnkube-node-zd2zp\" (UID: \"07ddac45-12a1-42e8-9566-55f08854c02e\") " pod="openshift-ovn-kubernetes/ovnkube-node-zd2zp"
Sep 19 08:20:17.152580 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.150673    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/07ddac45-12a1-42e8-9566-55f08854c02e-host-run-netns\") pod \"ovnkube-node-zd2zp\" (UID: \"07ddac45-12a1-42e8-9566-55f08854c02e\") " pod="openshift-ovn-kubernetes/ovnkube-node-zd2zp"
Sep 19 08:20:17.152580 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.150674    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/07ddac45-12a1-42e8-9566-55f08854c02e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-zd2zp\" (UID: \"07ddac45-12a1-42e8-9566-55f08854c02e\") " pod="openshift-ovn-kubernetes/ovnkube-node-zd2zp"
Sep 19 08:20:17.152580 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.150703    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/79f6ab54-a988-4015-81c2-da35716eab52-metrics-certs\") pod \"network-metrics-daemon-md9ln\" (UID: \"79f6ab54-a988-4015-81c2-da35716eab52\") " pod="openshift-multus/network-metrics-daemon-md9ln"
Sep 19 08:20:17.152580 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.150710    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/07ddac45-12a1-42e8-9566-55f08854c02e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-zd2zp\" (UID: \"07ddac45-12a1-42e8-9566-55f08854c02e\") " pod="openshift-ovn-kubernetes/ovnkube-node-zd2zp"
Sep 19 08:20:17.152580 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.150752    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c3a67fda-81e8-4d20-83d4-a37e277382b4-cnibin\") pod \"multus-wln5n\" (UID: \"c3a67fda-81e8-4d20-83d4-a37e277382b4\") " pod="openshift-multus/multus-wln5n"
Sep 19 08:20:17.152580 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.150777    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/78faeeea-464e-4ad2-9987-47fec1907637-etc-selinux\") pod \"aws-ebs-csi-driver-node-6vp2f\" (UID: \"78faeeea-464e-4ad2-9987-47fec1907637\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6vp2f"
Sep 19 08:20:17.152580 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.150803    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vldf6\" (UniqueName: \"kubernetes.io/projected/78faeeea-464e-4ad2-9987-47fec1907637-kube-api-access-vldf6\") pod \"aws-ebs-csi-driver-node-6vp2f\" (UID: \"78faeeea-464e-4ad2-9987-47fec1907637\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6vp2f"
Sep 19 08:20:17.152580 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.150828    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c3a67fda-81e8-4d20-83d4-a37e277382b4-host-var-lib-kubelet\") pod \"multus-wln5n\" (UID: \"c3a67fda-81e8-4d20-83d4-a37e277382b4\") " pod="openshift-multus/multus-wln5n"
Sep 19 08:20:17.152580 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.150893    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c3a67fda-81e8-4d20-83d4-a37e277382b4-host-var-lib-kubelet\") pod \"multus-wln5n\" (UID: \"c3a67fda-81e8-4d20-83d4-a37e277382b4\") " pod="openshift-multus/multus-wln5n"
Sep 19 08:20:17.152580 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.150976    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/647eb10e-629e-4cbc-bad4-234bbb862d6e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-9frbh\" (UID: \"647eb10e-629e-4cbc-bad4-234bbb862d6e\") " pod="openshift-multus/multus-additional-cni-plugins-9frbh"
Sep 19 08:20:17.152580 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.151044    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c3a67fda-81e8-4d20-83d4-a37e277382b4-cnibin\") pod \"multus-wln5n\" (UID: \"c3a67fda-81e8-4d20-83d4-a37e277382b4\") " pod="openshift-multus/multus-wln5n"
Sep 19 08:20:17.152580 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.151083    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c3a67fda-81e8-4d20-83d4-a37e277382b4-host-run-k8s-cni-cncf-io\") pod \"multus-wln5n\" (UID: \"c3a67fda-81e8-4d20-83d4-a37e277382b4\") " pod="openshift-multus/multus-wln5n"
Sep 19 08:20:17.152580 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:17.151150    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 19 08:20:17.152580 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.151158    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/78faeeea-464e-4ad2-9987-47fec1907637-etc-selinux\") pod \"aws-ebs-csi-driver-node-6vp2f\" (UID: \"78faeeea-464e-4ad2-9987-47fec1907637\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6vp2f"
Sep 19 08:20:17.152580 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.151189    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/647eb10e-629e-4cbc-bad4-234bbb862d6e-system-cni-dir\") pod \"multus-additional-cni-plugins-9frbh\" (UID: \"647eb10e-629e-4cbc-bad4-234bbb862d6e\") " pod="openshift-multus/multus-additional-cni-plugins-9frbh"
Sep 19 08:20:17.152580 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:17.151214    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79f6ab54-a988-4015-81c2-da35716eab52-metrics-certs podName:79f6ab54-a988-4015-81c2-da35716eab52 nodeName:}" failed. No retries permitted until 2026-09-19 08:20:17.651199773 +0000 UTC m=+3.061035448 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/79f6ab54-a988-4015-81c2-da35716eab52-metrics-certs") pod "network-metrics-daemon-md9ln" (UID: "79f6ab54-a988-4015-81c2-da35716eab52") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 19 08:20:17.152580 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.151235    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/07ddac45-12a1-42e8-9566-55f08854c02e-etc-openvswitch\") pod \"ovnkube-node-zd2zp\" (UID: \"07ddac45-12a1-42e8-9566-55f08854c02e\") " pod="openshift-ovn-kubernetes/ovnkube-node-zd2zp"
Sep 19 08:20:17.153289 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.151786    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/90d27f61-9e28-4cc5-bc29-12f69fda90a0-tmp\") pod \"tuned-5x6rt\" (UID: \"90d27f61-9e28-4cc5-bc29-12f69fda90a0\") " pod="openshift-cluster-node-tuning-operator/tuned-5x6rt"
Sep 19 08:20:17.153917 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.153650    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/90d27f61-9e28-4cc5-bc29-12f69fda90a0-etc-tuned\") pod \"tuned-5x6rt\" (UID: \"90d27f61-9e28-4cc5-bc29-12f69fda90a0\") " pod="openshift-cluster-node-tuning-operator/tuned-5x6rt"
Sep 19 08:20:17.153917 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.153702    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/07ddac45-12a1-42e8-9566-55f08854c02e-ovn-node-metrics-cert\") pod \"ovnkube-node-zd2zp\" (UID: \"07ddac45-12a1-42e8-9566-55f08854c02e\") " pod="openshift-ovn-kubernetes/ovnkube-node-zd2zp"
Sep 19 08:20:17.157248 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.157226    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l5wdh\" (UniqueName: \"kubernetes.io/projected/90d27f61-9e28-4cc5-bc29-12f69fda90a0-kube-api-access-l5wdh\") pod \"tuned-5x6rt\" (UID: \"90d27f61-9e28-4cc5-bc29-12f69fda90a0\") " pod="openshift-cluster-node-tuning-operator/tuned-5x6rt"
Sep 19 08:20:17.157955 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:17.157935    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 19 08:20:17.158043 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:17.157959    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 19 08:20:17.158043 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:17.157971    2578 projected.go:196] Error preparing data for projected volume kube-api-access-r7rcd for pod openshift-network-diagnostics/network-check-target-l6m8x: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 19 08:20:17.158043 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:17.158027    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8324d8db-b082-475a-8bb7-8c578404b02e-kube-api-access-r7rcd podName:8324d8db-b082-475a-8bb7-8c578404b02e nodeName:}" failed. No retries permitted until 2026-09-19 08:20:17.658010437 +0000 UTC m=+3.067846129 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-r7rcd" (UniqueName: "kubernetes.io/projected/8324d8db-b082-475a-8bb7-8c578404b02e-kube-api-access-r7rcd") pod "network-check-target-l6m8x" (UID: "8324d8db-b082-475a-8bb7-8c578404b02e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 19 08:20:17.160470 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.160433    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6kh4x\" (UniqueName: \"kubernetes.io/projected/79f6ab54-a988-4015-81c2-da35716eab52-kube-api-access-6kh4x\") pod \"network-metrics-daemon-md9ln\" (UID: \"79f6ab54-a988-4015-81c2-da35716eab52\") " pod="openshift-multus/network-metrics-daemon-md9ln"
Sep 19 08:20:17.161057 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.161014    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wz4bl\" (UniqueName: \"kubernetes.io/projected/c3a67fda-81e8-4d20-83d4-a37e277382b4-kube-api-access-wz4bl\") pod \"multus-wln5n\" (UID: \"c3a67fda-81e8-4d20-83d4-a37e277382b4\") " pod="openshift-multus/multus-wln5n"
Sep 19 08:20:17.161251 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.161231    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qhpbr\" (UniqueName: \"kubernetes.io/projected/647eb10e-629e-4cbc-bad4-234bbb862d6e-kube-api-access-qhpbr\") pod \"multus-additional-cni-plugins-9frbh\" (UID: \"647eb10e-629e-4cbc-bad4-234bbb862d6e\") " pod="openshift-multus/multus-additional-cni-plugins-9frbh"
Sep 19 08:20:17.163403 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.163205    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vldf6\" (UniqueName: \"kubernetes.io/projected/78faeeea-464e-4ad2-9987-47fec1907637-kube-api-access-vldf6\") pod \"aws-ebs-csi-driver-node-6vp2f\" (UID: \"78faeeea-464e-4ad2-9987-47fec1907637\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6vp2f"
Sep 19 08:20:17.163659 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.163638    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-shczr\" (UniqueName: \"kubernetes.io/projected/07ddac45-12a1-42e8-9566-55f08854c02e-kube-api-access-shczr\") pod \"ovnkube-node-zd2zp\" (UID: \"07ddac45-12a1-42e8-9566-55f08854c02e\") " pod="openshift-ovn-kubernetes/ovnkube-node-zd2zp"
Sep 19 08:20:17.165358 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.165320    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jptt6\" (UniqueName: \"kubernetes.io/projected/65dc9622-6c75-457d-ad22-6841b2673aae-kube-api-access-jptt6\") pod \"node-ca-hb9xj\" (UID: \"65dc9622-6c75-457d-ad22-6841b2673aae\") " pod="openshift-image-registry/node-ca-hb9xj"
Sep 19 08:20:17.242219 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.242186    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-lqdbb"
Sep 19 08:20:17.253129 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:17.253098    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod637bbfb9_45ba_481d_b9fe_87f60051712c.slice/crio-7c513a9ba60d3c7df1574dbe586f675daea34ecfefb1d62e0b784af41e265d0e WatchSource:0}: Error finding container 7c513a9ba60d3c7df1574dbe586f675daea34ecfefb1d62e0b784af41e265d0e: Status 404 returned error can't find the container with id 7c513a9ba60d3c7df1574dbe586f675daea34ecfefb1d62e0b784af41e265d0e
Sep 19 08:20:17.254507 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.254485    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-7tt6m"
Sep 19 08:20:17.264153 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:17.264110    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0904bd86_22aa_4c03_af38_ee3e765096be.slice/crio-1c59f9ffbafd952c4db87506e431699a06d2e2b1f5f82e3eef2bfc6daffa3ff0 WatchSource:0}: Error finding container 1c59f9ffbafd952c4db87506e431699a06d2e2b1f5f82e3eef2bfc6daffa3ff0: Status 404 returned error can't find the container with id 1c59f9ffbafd952c4db87506e431699a06d2e2b1f5f82e3eef2bfc6daffa3ff0
Sep 19 08:20:17.265532 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.265510    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-jfkwf"
Sep 19 08:20:17.274157 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.274096    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-9frbh"
Sep 19 08:20:17.276320 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:17.276295    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddcc90f06_1e07_4b6b_add7_46f0cdd95ecf.slice/crio-5f315eb4c067f98b02219fa0743ff12187ab114c0473d2cf72071d03d85f8e47 WatchSource:0}: Error finding container 5f315eb4c067f98b02219fa0743ff12187ab114c0473d2cf72071d03d85f8e47: Status 404 returned error can't find the container with id 5f315eb4c067f98b02219fa0743ff12187ab114c0473d2cf72071d03d85f8e47
Sep 19 08:20:17.280605 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.280583    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-5x6rt"
Sep 19 08:20:17.285430 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:17.285405    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod647eb10e_629e_4cbc_bad4_234bbb862d6e.slice/crio-b7d54567a502d0ae53a3ea357de4fbf443dfd9a6f69b51f1399824b481512d1a WatchSource:0}: Error finding container b7d54567a502d0ae53a3ea357de4fbf443dfd9a6f69b51f1399824b481512d1a: Status 404 returned error can't find the container with id b7d54567a502d0ae53a3ea357de4fbf443dfd9a6f69b51f1399824b481512d1a
Sep 19 08:20:17.288115 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.288096    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-hb9xj"
Sep 19 08:20:17.292359 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:17.292337    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90d27f61_9e28_4cc5_bc29_12f69fda90a0.slice/crio-52bc51bd038a49e53348449e245aeca5cc2c38826a8405c4dccc1b49cd07f5f7 WatchSource:0}: Error finding container 52bc51bd038a49e53348449e245aeca5cc2c38826a8405c4dccc1b49cd07f5f7: Status 404 returned error can't find the container with id 52bc51bd038a49e53348449e245aeca5cc2c38826a8405c4dccc1b49cd07f5f7
Sep 19 08:20:17.297212 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:17.297188    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65dc9622_6c75_457d_ad22_6841b2673aae.slice/crio-ff83a060b1d863c70cb35bd16ac6f909741ea989e9e4ee203302abc8e8f00d70 WatchSource:0}: Error finding container ff83a060b1d863c70cb35bd16ac6f909741ea989e9e4ee203302abc8e8f00d70: Status 404 returned error can't find the container with id ff83a060b1d863c70cb35bd16ac6f909741ea989e9e4ee203302abc8e8f00d70
Sep 19 08:20:17.298049 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.298032    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-zd2zp"
Sep 19 08:20:17.306771 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.306751    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-wln5n"
Sep 19 08:20:17.315903 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:17.315875    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6vp2f"
Sep 19 08:20:18.494709 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:18.494416    2578 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 19 08:20:18.687413 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:18.687385    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/19642aea-2c2e-4208-8329-09f03b1e5d5b-original-pull-secret\") pod \"global-pull-secret-syncer-xq8d8\" (UID: \"19642aea-2c2e-4208-8329-09f03b1e5d5b\") " pod="kube-system/global-pull-secret-syncer-xq8d8"
Sep 19 08:20:18.687557 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:18.687444    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/79f6ab54-a988-4015-81c2-da35716eab52-metrics-certs\") pod \"network-metrics-daemon-md9ln\" (UID: \"79f6ab54-a988-4015-81c2-da35716eab52\") " pod="openshift-multus/network-metrics-daemon-md9ln"
Sep 19 08:20:18.687557 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:18.687525    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 19 08:20:18.687557 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:18.687551    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 19 08:20:18.687703 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:18.687582    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/19642aea-2c2e-4208-8329-09f03b1e5d5b-original-pull-secret podName:19642aea-2c2e-4208-8329-09f03b1e5d5b nodeName:}" failed. No retries permitted until 2026-09-19 08:20:19.687568561 +0000 UTC m=+4.064278400 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/19642aea-2c2e-4208-8329-09f03b1e5d5b-original-pull-secret") pod "global-pull-secret-syncer-xq8d8" (UID: "19642aea-2c2e-4208-8329-09f03b1e5d5b") : object "kube-system"/"original-pull-secret" not registered
Sep 19 08:20:18.687703 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:18.687609    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79f6ab54-a988-4015-81c2-da35716eab52-metrics-certs podName:79f6ab54-a988-4015-81c2-da35716eab52 nodeName:}" failed. No retries permitted until 2026-09-19 08:20:19.687590174 +0000 UTC m=+4.064300023 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/79f6ab54-a988-4015-81c2-da35716eab52-metrics-certs") pod "network-metrics-daemon-md9ln" (UID: "79f6ab54-a988-4015-81c2-da35716eab52") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 19 08:20:18.787805 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:18.787734    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r7rcd\" (UniqueName: \"kubernetes.io/projected/8324d8db-b082-475a-8bb7-8c578404b02e-kube-api-access-r7rcd\") pod \"network-check-target-l6m8x\" (UID: \"8324d8db-b082-475a-8bb7-8c578404b02e\") " pod="openshift-network-diagnostics/network-check-target-l6m8x"
Sep 19 08:20:18.787934 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:18.787912    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 19 08:20:18.787934 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:18.787931    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 19 08:20:18.788027 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:18.787942    2578 projected.go:196] Error preparing data for projected volume kube-api-access-r7rcd for pod openshift-network-diagnostics/network-check-target-l6m8x: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 19 08:20:18.788027 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:18.787997    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8324d8db-b082-475a-8bb7-8c578404b02e-kube-api-access-r7rcd podName:8324d8db-b082-475a-8bb7-8c578404b02e nodeName:}" failed. No retries permitted until 2026-09-19 08:20:19.787983278 +0000 UTC m=+4.164693123 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-r7rcd" (UniqueName: "kubernetes.io/projected/8324d8db-b082-475a-8bb7-8c578404b02e-kube-api-access-r7rcd") pod "network-check-target-l6m8x" (UID: "8324d8db-b082-475a-8bb7-8c578404b02e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 19 08:20:19.012715 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:19.012659    2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-18 08:15:18 +0000 UTC" deadline="2028-07-01 07:22:18.590604852 +0000 UTC"
Sep 19 08:20:19.012715 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:19.012707    2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15623h1m59.577900889s"
Sep 19 08:20:19.083623 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:19.083533    2578 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 19 08:20:19.107581 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:19.107553    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-md9ln"
Sep 19 08:20:19.107746 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:19.107682    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-md9ln" podUID="79f6ab54-a988-4015-81c2-da35716eab52"
Sep 19 08:20:19.119980 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:19.119920    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-hb9xj" event={"ID":"65dc9622-6c75-457d-ad22-6841b2673aae","Type":"ContainerStarted","Data":"ff83a060b1d863c70cb35bd16ac6f909741ea989e9e4ee203302abc8e8f00d70"}
Sep 19 08:20:19.121407 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:19.121381    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-5x6rt" event={"ID":"90d27f61-9e28-4cc5-bc29-12f69fda90a0","Type":"ContainerStarted","Data":"52bc51bd038a49e53348449e245aeca5cc2c38826a8405c4dccc1b49cd07f5f7"}
Sep 19 08:20:19.122725 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:19.122701    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9frbh" event={"ID":"647eb10e-629e-4cbc-bad4-234bbb862d6e","Type":"ContainerStarted","Data":"b7d54567a502d0ae53a3ea357de4fbf443dfd9a6f69b51f1399824b481512d1a"}
Sep 19 08:20:19.123946 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:19.123922    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-jfkwf" event={"ID":"dcc90f06-1e07-4b6b-add7-46f0cdd95ecf","Type":"ContainerStarted","Data":"5f315eb4c067f98b02219fa0743ff12187ab114c0473d2cf72071d03d85f8e47"}
Sep 19 08:20:19.125104 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:19.125083    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-7tt6m" event={"ID":"0904bd86-22aa-4c03-af38-ee3e765096be","Type":"ContainerStarted","Data":"1c59f9ffbafd952c4db87506e431699a06d2e2b1f5f82e3eef2bfc6daffa3ff0"}
Sep 19 08:20:19.126316 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:19.126271    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-lqdbb" event={"ID":"637bbfb9-45ba-481d-b9fe-87f60051712c","Type":"ContainerStarted","Data":"7c513a9ba60d3c7df1574dbe586f675daea34ecfefb1d62e0b784af41e265d0e"}
Sep 19 08:20:19.450518 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:19.450481    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3a67fda_81e8_4d20_83d4_a37e277382b4.slice/crio-c83cdf9ffb5179360127afa46f069fd87b678632104b7edfd0fd150f842d1d12 WatchSource:0}: Error finding container c83cdf9ffb5179360127afa46f069fd87b678632104b7edfd0fd150f842d1d12: Status 404 returned error can't find the container with id c83cdf9ffb5179360127afa46f069fd87b678632104b7edfd0fd150f842d1d12
Sep 19 08:20:19.455308 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:19.455267    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07ddac45_12a1_42e8_9566_55f08854c02e.slice/crio-23aaa2ebe1c4ca936dbcd2e8f21dad1d5c245ab52c8e8b9c31fbeb327ab0d01a WatchSource:0}: Error finding container 23aaa2ebe1c4ca936dbcd2e8f21dad1d5c245ab52c8e8b9c31fbeb327ab0d01a: Status 404 returned error can't find the container with id 23aaa2ebe1c4ca936dbcd2e8f21dad1d5c245ab52c8e8b9c31fbeb327ab0d01a
Sep 19 08:20:19.458048 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:19.458026    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78faeeea_464e_4ad2_9987_47fec1907637.slice/crio-a67c6932b35e5e29618a35e3c352062d54317a61a5fbed23e842d7bd5e33eefd WatchSource:0}: Error finding container a67c6932b35e5e29618a35e3c352062d54317a61a5fbed23e842d7bd5e33eefd: Status 404 returned error can't find the container with id a67c6932b35e5e29618a35e3c352062d54317a61a5fbed23e842d7bd5e33eefd
Sep 19 08:20:19.697716 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:19.697638    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/79f6ab54-a988-4015-81c2-da35716eab52-metrics-certs\") pod \"network-metrics-daemon-md9ln\" (UID: \"79f6ab54-a988-4015-81c2-da35716eab52\") " pod="openshift-multus/network-metrics-daemon-md9ln"
Sep 19 08:20:19.697716 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:19.697700    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/19642aea-2c2e-4208-8329-09f03b1e5d5b-original-pull-secret\") pod \"global-pull-secret-syncer-xq8d8\" (UID: \"19642aea-2c2e-4208-8329-09f03b1e5d5b\") " pod="kube-system/global-pull-secret-syncer-xq8d8"
Sep 19 08:20:19.697920 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:19.697902    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 19 08:20:19.697986 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:19.697971    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/19642aea-2c2e-4208-8329-09f03b1e5d5b-original-pull-secret podName:19642aea-2c2e-4208-8329-09f03b1e5d5b nodeName:}" failed. No retries permitted until 2026-09-19 08:20:21.697952063 +0000 UTC m=+6.074661918 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/19642aea-2c2e-4208-8329-09f03b1e5d5b-original-pull-secret") pod "global-pull-secret-syncer-xq8d8" (UID: "19642aea-2c2e-4208-8329-09f03b1e5d5b") : object "kube-system"/"original-pull-secret" not registered
Sep 19 08:20:19.698199 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:19.698180    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 19 08:20:19.698281 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:19.698242    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79f6ab54-a988-4015-81c2-da35716eab52-metrics-certs podName:79f6ab54-a988-4015-81c2-da35716eab52 nodeName:}" failed. No retries permitted until 2026-09-19 08:20:21.698225086 +0000 UTC m=+6.074934939 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/79f6ab54-a988-4015-81c2-da35716eab52-metrics-certs") pod "network-metrics-daemon-md9ln" (UID: "79f6ab54-a988-4015-81c2-da35716eab52") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 19 08:20:19.798291 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:19.798254    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r7rcd\" (UniqueName: \"kubernetes.io/projected/8324d8db-b082-475a-8bb7-8c578404b02e-kube-api-access-r7rcd\") pod \"network-check-target-l6m8x\" (UID: \"8324d8db-b082-475a-8bb7-8c578404b02e\") " pod="openshift-network-diagnostics/network-check-target-l6m8x"
Sep 19 08:20:19.798493 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:19.798475    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 19 08:20:19.798570 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:19.798502    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 19 08:20:19.798570 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:19.798514    2578 projected.go:196] Error preparing data for projected volume kube-api-access-r7rcd for pod openshift-network-diagnostics/network-check-target-l6m8x: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 19 08:20:19.798673 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:19.798571    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8324d8db-b082-475a-8bb7-8c578404b02e-kube-api-access-r7rcd podName:8324d8db-b082-475a-8bb7-8c578404b02e nodeName:}" failed. No retries permitted until 2026-09-19 08:20:21.798553666 +0000 UTC m=+6.175263513 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-r7rcd" (UniqueName: "kubernetes.io/projected/8324d8db-b082-475a-8bb7-8c578404b02e-kube-api-access-r7rcd") pod "network-check-target-l6m8x" (UID: "8324d8db-b082-475a-8bb7-8c578404b02e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 19 08:20:20.013087 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:20.013014    2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-18 08:15:18 +0000 UTC" deadline="2028-04-28 12:09:28.422898354 +0000 UTC"
Sep 19 08:20:20.013087 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:20.013055    2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14091h49m8.409848287s"
Sep 19 08:20:20.112636 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:20.112608    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xq8d8"
Sep 19 08:20:20.112778 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:20.112734    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xq8d8" podUID="19642aea-2c2e-4208-8329-09f03b1e5d5b"
Sep 19 08:20:20.112854 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:20.112833    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l6m8x"
Sep 19 08:20:20.112922 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:20.112903    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l6m8x" podUID="8324d8db-b082-475a-8bb7-8c578404b02e"
Sep 19 08:20:20.136236 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:20.136197    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-157.ec2.internal" event={"ID":"6b8b6e7e22ce47a3909ac23b630738ee","Type":"ContainerStarted","Data":"70d3fdda6162b4a5f337fe478fc97ad28baa79102be28e242bb0d80af425dea9"}
Sep 19 08:20:20.144324 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:20.144003    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-157.ec2.internal" event={"ID":"ecf1d89d174b946c499fe0ba49cfe70a","Type":"ContainerStarted","Data":"b36746d0dee70ed65d27c0331079b34506addc4cbb34955382a530dc7f61d85f"}
Sep 19 08:20:20.146843 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:20.146786    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6vp2f" event={"ID":"78faeeea-464e-4ad2-9987-47fec1907637","Type":"ContainerStarted","Data":"a67c6932b35e5e29618a35e3c352062d54317a61a5fbed23e842d7bd5e33eefd"}
Sep 19 08:20:20.153326 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:20.153223    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zd2zp" event={"ID":"07ddac45-12a1-42e8-9566-55f08854c02e","Type":"ContainerStarted","Data":"23aaa2ebe1c4ca936dbcd2e8f21dad1d5c245ab52c8e8b9c31fbeb327ab0d01a"}
Sep 19 08:20:20.160436 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:20.160409    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-wln5n" event={"ID":"c3a67fda-81e8-4d20-83d4-a37e277382b4","Type":"ContainerStarted","Data":"c83cdf9ffb5179360127afa46f069fd87b678632104b7edfd0fd150f842d1d12"}
Sep 19 08:20:21.107802 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:21.107765    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-md9ln"
Sep 19 08:20:21.108370 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:21.107903    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-md9ln" podUID="79f6ab54-a988-4015-81c2-da35716eab52"
Sep 19 08:20:21.717214 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:21.716938    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/79f6ab54-a988-4015-81c2-da35716eab52-metrics-certs\") pod \"network-metrics-daemon-md9ln\" (UID: \"79f6ab54-a988-4015-81c2-da35716eab52\") " pod="openshift-multus/network-metrics-daemon-md9ln"
Sep 19 08:20:21.717214 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:21.716998    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/19642aea-2c2e-4208-8329-09f03b1e5d5b-original-pull-secret\") pod \"global-pull-secret-syncer-xq8d8\" (UID: \"19642aea-2c2e-4208-8329-09f03b1e5d5b\") " pod="kube-system/global-pull-secret-syncer-xq8d8"
Sep 19 08:20:21.717214 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:21.717113    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 19 08:20:21.717214 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:21.717115    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 19 08:20:21.717214 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:21.717172    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/19642aea-2c2e-4208-8329-09f03b1e5d5b-original-pull-secret podName:19642aea-2c2e-4208-8329-09f03b1e5d5b nodeName:}" failed. No retries permitted until 2026-09-19 08:20:25.717156192 +0000 UTC m=+10.093866037 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/19642aea-2c2e-4208-8329-09f03b1e5d5b-original-pull-secret") pod "global-pull-secret-syncer-xq8d8" (UID: "19642aea-2c2e-4208-8329-09f03b1e5d5b") : object "kube-system"/"original-pull-secret" not registered
Sep 19 08:20:21.717214 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:21.717190    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79f6ab54-a988-4015-81c2-da35716eab52-metrics-certs podName:79f6ab54-a988-4015-81c2-da35716eab52 nodeName:}" failed. No retries permitted until 2026-09-19 08:20:25.717180915 +0000 UTC m=+10.093890760 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/79f6ab54-a988-4015-81c2-da35716eab52-metrics-certs") pod "network-metrics-daemon-md9ln" (UID: "79f6ab54-a988-4015-81c2-da35716eab52") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 19 08:20:21.817911 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:21.817637    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r7rcd\" (UniqueName: \"kubernetes.io/projected/8324d8db-b082-475a-8bb7-8c578404b02e-kube-api-access-r7rcd\") pod \"network-check-target-l6m8x\" (UID: \"8324d8db-b082-475a-8bb7-8c578404b02e\") " pod="openshift-network-diagnostics/network-check-target-l6m8x"
Sep 19 08:20:21.817911 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:21.817813    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 19 08:20:21.817911 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:21.817835    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 19 08:20:21.817911 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:21.817847    2578 projected.go:196] Error preparing data for projected volume kube-api-access-r7rcd for pod openshift-network-diagnostics/network-check-target-l6m8x: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 19 08:20:21.817911 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:21.817908    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8324d8db-b082-475a-8bb7-8c578404b02e-kube-api-access-r7rcd podName:8324d8db-b082-475a-8bb7-8c578404b02e nodeName:}" failed. No retries permitted until 2026-09-19 08:20:25.817890581 +0000 UTC m=+10.194600437 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-r7rcd" (UniqueName: "kubernetes.io/projected/8324d8db-b082-475a-8bb7-8c578404b02e-kube-api-access-r7rcd") pod "network-check-target-l6m8x" (UID: "8324d8db-b082-475a-8bb7-8c578404b02e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 19 08:20:22.112107 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:22.111697    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xq8d8"
Sep 19 08:20:22.112107 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:22.111821    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xq8d8" podUID="19642aea-2c2e-4208-8329-09f03b1e5d5b"
Sep 19 08:20:22.112107 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:22.111851    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l6m8x"
Sep 19 08:20:22.112107 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:22.111948    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l6m8x" podUID="8324d8db-b082-475a-8bb7-8c578404b02e"
Sep 19 08:20:23.107179 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:23.107148    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-md9ln"
Sep 19 08:20:23.107360 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:23.107289    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-md9ln" podUID="79f6ab54-a988-4015-81c2-da35716eab52"
Sep 19 08:20:24.107567 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:24.107535    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xq8d8"
Sep 19 08:20:24.107567 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:24.107553    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l6m8x"
Sep 19 08:20:24.108042 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:24.107670    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xq8d8" podUID="19642aea-2c2e-4208-8329-09f03b1e5d5b"
Sep 19 08:20:24.108042 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:24.107794    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l6m8x" podUID="8324d8db-b082-475a-8bb7-8c578404b02e"
Sep 19 08:20:25.107446 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:25.107359    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-md9ln"
Sep 19 08:20:25.107590 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:25.107521    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-md9ln" podUID="79f6ab54-a988-4015-81c2-da35716eab52"
Sep 19 08:20:25.174395 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:25.174184    2578 generic.go:356] "Generic (PLEG): container finished" podID="6b8b6e7e22ce47a3909ac23b630738ee" containerID="70d3fdda6162b4a5f337fe478fc97ad28baa79102be28e242bb0d80af425dea9" exitCode=0
Sep 19 08:20:25.174395 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:25.174233    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-157.ec2.internal" event={"ID":"6b8b6e7e22ce47a3909ac23b630738ee","Type":"ContainerDied","Data":"70d3fdda6162b4a5f337fe478fc97ad28baa79102be28e242bb0d80af425dea9"}
Sep 19 08:20:25.189957 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:25.189906    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-157.ec2.internal" podStartSLOduration=8.189885929 podStartE2EDuration="8.189885929s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 08:20:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 08:20:20.165452809 +0000 UTC m=+4.542162674" watchObservedRunningTime="2026-09-19 08:20:25.189885929 +0000 UTC m=+9.566595788"
Sep 19 08:20:25.747622 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:25.747498    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/79f6ab54-a988-4015-81c2-da35716eab52-metrics-certs\") pod \"network-metrics-daemon-md9ln\" (UID: \"79f6ab54-a988-4015-81c2-da35716eab52\") " pod="openshift-multus/network-metrics-daemon-md9ln"
Sep 19 08:20:25.747622 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:25.747553    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/19642aea-2c2e-4208-8329-09f03b1e5d5b-original-pull-secret\") pod \"global-pull-secret-syncer-xq8d8\" (UID: \"19642aea-2c2e-4208-8329-09f03b1e5d5b\") " pod="kube-system/global-pull-secret-syncer-xq8d8"
Sep 19 08:20:25.747834 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:25.747662    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 19 08:20:25.747834 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:25.747734    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79f6ab54-a988-4015-81c2-da35716eab52-metrics-certs podName:79f6ab54-a988-4015-81c2-da35716eab52 nodeName:}" failed. No retries permitted until 2026-09-19 08:20:33.74771441 +0000 UTC m=+18.124424262 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/79f6ab54-a988-4015-81c2-da35716eab52-metrics-certs") pod "network-metrics-daemon-md9ln" (UID: "79f6ab54-a988-4015-81c2-da35716eab52") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 19 08:20:25.747834 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:25.747666    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 19 08:20:25.747834 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:25.747823    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/19642aea-2c2e-4208-8329-09f03b1e5d5b-original-pull-secret podName:19642aea-2c2e-4208-8329-09f03b1e5d5b nodeName:}" failed. No retries permitted until 2026-09-19 08:20:33.747802183 +0000 UTC m=+18.124512041 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/19642aea-2c2e-4208-8329-09f03b1e5d5b-original-pull-secret") pod "global-pull-secret-syncer-xq8d8" (UID: "19642aea-2c2e-4208-8329-09f03b1e5d5b") : object "kube-system"/"original-pull-secret" not registered
Sep 19 08:20:25.848751 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:25.848642    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r7rcd\" (UniqueName: \"kubernetes.io/projected/8324d8db-b082-475a-8bb7-8c578404b02e-kube-api-access-r7rcd\") pod \"network-check-target-l6m8x\" (UID: \"8324d8db-b082-475a-8bb7-8c578404b02e\") " pod="openshift-network-diagnostics/network-check-target-l6m8x"
Sep 19 08:20:25.848938 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:25.848828    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 19 08:20:25.848938 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:25.848850    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 19 08:20:25.848938 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:25.848867    2578 projected.go:196] Error preparing data for projected volume kube-api-access-r7rcd for pod openshift-network-diagnostics/network-check-target-l6m8x: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 19 08:20:25.848938 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:25.848926    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8324d8db-b082-475a-8bb7-8c578404b02e-kube-api-access-r7rcd podName:8324d8db-b082-475a-8bb7-8c578404b02e nodeName:}" failed. No retries permitted until 2026-09-19 08:20:33.848907062 +0000 UTC m=+18.225616914 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-r7rcd" (UniqueName: "kubernetes.io/projected/8324d8db-b082-475a-8bb7-8c578404b02e-kube-api-access-r7rcd") pod "network-check-target-l6m8x" (UID: "8324d8db-b082-475a-8bb7-8c578404b02e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 19 08:20:26.110665 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:26.110593    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xq8d8"
Sep 19 08:20:26.111061 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:26.110709    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xq8d8" podUID="19642aea-2c2e-4208-8329-09f03b1e5d5b"
Sep 19 08:20:26.111061 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:26.111056    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l6m8x"
Sep 19 08:20:26.111150 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:26.111137    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l6m8x" podUID="8324d8db-b082-475a-8bb7-8c578404b02e"
Sep 19 08:20:27.107362 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:27.107320    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-md9ln"
Sep 19 08:20:27.107677 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:27.107465    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-md9ln" podUID="79f6ab54-a988-4015-81c2-da35716eab52"
Sep 19 08:20:28.108274 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:28.108240    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l6m8x"
Sep 19 08:20:28.108723 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:28.108383    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l6m8x" podUID="8324d8db-b082-475a-8bb7-8c578404b02e"
Sep 19 08:20:28.108723 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:28.108467    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xq8d8"
Sep 19 08:20:28.108723 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:28.108608    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xq8d8" podUID="19642aea-2c2e-4208-8329-09f03b1e5d5b"
Sep 19 08:20:29.107478 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:29.107444    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-md9ln"
Sep 19 08:20:29.107649 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:29.107581    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-md9ln" podUID="79f6ab54-a988-4015-81c2-da35716eab52"
Sep 19 08:20:30.107949 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:30.107912    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xq8d8"
Sep 19 08:20:30.108391 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:30.107912    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l6m8x"
Sep 19 08:20:30.108391 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:30.108045    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xq8d8" podUID="19642aea-2c2e-4208-8329-09f03b1e5d5b"
Sep 19 08:20:30.108391 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:30.108137    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l6m8x" podUID="8324d8db-b082-475a-8bb7-8c578404b02e"
Sep 19 08:20:31.107612 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:31.107579    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-md9ln"
Sep 19 08:20:31.107766 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:31.107708    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-md9ln" podUID="79f6ab54-a988-4015-81c2-da35716eab52"
Sep 19 08:20:32.107421 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:32.107387    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xq8d8"
Sep 19 08:20:32.107421 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:32.107435    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l6m8x"
Sep 19 08:20:32.107879 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:32.107545    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xq8d8" podUID="19642aea-2c2e-4208-8329-09f03b1e5d5b"
Sep 19 08:20:32.107879 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:32.107622    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l6m8x" podUID="8324d8db-b082-475a-8bb7-8c578404b02e"
Sep 19 08:20:33.107238 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:33.107208    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-md9ln"
Sep 19 08:20:33.107466 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:33.107352    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-md9ln" podUID="79f6ab54-a988-4015-81c2-da35716eab52"
Sep 19 08:20:33.802334 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:33.802286    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/79f6ab54-a988-4015-81c2-da35716eab52-metrics-certs\") pod \"network-metrics-daemon-md9ln\" (UID: \"79f6ab54-a988-4015-81c2-da35716eab52\") " pod="openshift-multus/network-metrics-daemon-md9ln"
Sep 19 08:20:33.802624 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:33.802359    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/19642aea-2c2e-4208-8329-09f03b1e5d5b-original-pull-secret\") pod \"global-pull-secret-syncer-xq8d8\" (UID: \"19642aea-2c2e-4208-8329-09f03b1e5d5b\") " pod="kube-system/global-pull-secret-syncer-xq8d8"
Sep 19 08:20:33.802624 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:33.802429    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 19 08:20:33.802624 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:33.802492    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79f6ab54-a988-4015-81c2-da35716eab52-metrics-certs podName:79f6ab54-a988-4015-81c2-da35716eab52 nodeName:}" failed. No retries permitted until 2026-09-19 08:20:49.802476744 +0000 UTC m=+34.179186587 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/79f6ab54-a988-4015-81c2-da35716eab52-metrics-certs") pod "network-metrics-daemon-md9ln" (UID: "79f6ab54-a988-4015-81c2-da35716eab52") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 19 08:20:33.802624 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:33.802438    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 19 08:20:33.802624 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:33.802540    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/19642aea-2c2e-4208-8329-09f03b1e5d5b-original-pull-secret podName:19642aea-2c2e-4208-8329-09f03b1e5d5b nodeName:}" failed. No retries permitted until 2026-09-19 08:20:49.802532038 +0000 UTC m=+34.179241878 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/19642aea-2c2e-4208-8329-09f03b1e5d5b-original-pull-secret") pod "global-pull-secret-syncer-xq8d8" (UID: "19642aea-2c2e-4208-8329-09f03b1e5d5b") : object "kube-system"/"original-pull-secret" not registered
Sep 19 08:20:33.902764 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:33.902722    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r7rcd\" (UniqueName: \"kubernetes.io/projected/8324d8db-b082-475a-8bb7-8c578404b02e-kube-api-access-r7rcd\") pod \"network-check-target-l6m8x\" (UID: \"8324d8db-b082-475a-8bb7-8c578404b02e\") " pod="openshift-network-diagnostics/network-check-target-l6m8x"
Sep 19 08:20:33.902952 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:33.902893    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 19 08:20:33.902952 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:33.902921    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 19 08:20:33.902952 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:33.902935    2578 projected.go:196] Error preparing data for projected volume kube-api-access-r7rcd for pod openshift-network-diagnostics/network-check-target-l6m8x: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 19 08:20:33.903071 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:33.902991    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8324d8db-b082-475a-8bb7-8c578404b02e-kube-api-access-r7rcd podName:8324d8db-b082-475a-8bb7-8c578404b02e nodeName:}" failed. No retries permitted until 2026-09-19 08:20:49.902976186 +0000 UTC m=+34.279686030 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-r7rcd" (UniqueName: "kubernetes.io/projected/8324d8db-b082-475a-8bb7-8c578404b02e-kube-api-access-r7rcd") pod "network-check-target-l6m8x" (UID: "8324d8db-b082-475a-8bb7-8c578404b02e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 19 08:20:34.107775 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:34.107698    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xq8d8"
Sep 19 08:20:34.108211 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:34.107808    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xq8d8" podUID="19642aea-2c2e-4208-8329-09f03b1e5d5b"
Sep 19 08:20:34.108211 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:34.107859    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l6m8x"
Sep 19 08:20:34.108211 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:34.107993    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l6m8x" podUID="8324d8db-b082-475a-8bb7-8c578404b02e"
Sep 19 08:20:35.107110 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:35.107074    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-md9ln"
Sep 19 08:20:35.107294 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:35.107217    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-md9ln" podUID="79f6ab54-a988-4015-81c2-da35716eab52"
Sep 19 08:20:36.108166 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:36.107256    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xq8d8"
Sep 19 08:20:36.108166 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:36.107256    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l6m8x"
Sep 19 08:20:36.109254 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:36.109116    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xq8d8" podUID="19642aea-2c2e-4208-8329-09f03b1e5d5b"
Sep 19 08:20:36.109254 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:36.109220    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l6m8x" podUID="8324d8db-b082-475a-8bb7-8c578404b02e"
Sep 19 08:20:36.196092 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:36.196030    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-lqdbb" event={"ID":"637bbfb9-45ba-481d-b9fe-87f60051712c","Type":"ContainerStarted","Data":"f057fa646db630e9d14b1c90e7ecee7ae49692aeb0c5d90e267cc4afea76058d"}
Sep 19 08:20:36.197427 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:36.197384    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-7tt6m" event={"ID":"0904bd86-22aa-4c03-af38-ee3e765096be","Type":"ContainerStarted","Data":"04fe3d93ea5f2c72ab50d26e9f4ef3a9d8d7e341ea7795a572d3b1bcec04da31"}
Sep 19 08:20:36.198651 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:36.198632    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zd2zp" event={"ID":"07ddac45-12a1-42e8-9566-55f08854c02e","Type":"ContainerStarted","Data":"d19e1afbc24d8a09b566e861cadaad3e38c7c052494385f49392f0fbea8761fc"}
Sep 19 08:20:36.202521 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:36.202502    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-157.ec2.internal" event={"ID":"6b8b6e7e22ce47a3909ac23b630738ee","Type":"ContainerStarted","Data":"a8e98f6c67ab76c5ae42ca43e174ff653d82a56b45373ba54492592b9e1bf900"}
Sep 19 08:20:36.203789 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:36.203772    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-hb9xj" event={"ID":"65dc9622-6c75-457d-ad22-6841b2673aae","Type":"ContainerStarted","Data":"4d1b67940f09d9ab051779964cbcfe81fafb37b5e44bb3671b5545802bd42693"}
Sep 19 08:20:36.205322 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:36.205273    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-5x6rt" event={"ID":"90d27f61-9e28-4cc5-bc29-12f69fda90a0","Type":"ContainerStarted","Data":"b25441dcfa6dfc55c3cbe48b7a3d612162e7292b023e14dc6cb9beabe7a2f8fb"}
Sep 19 08:20:36.207099 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:36.207077    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9frbh" event={"ID":"647eb10e-629e-4cbc-bad4-234bbb862d6e","Type":"ContainerStarted","Data":"b6f7d098881c97d26cf55dcde61c7ec1a0692eb50a86a40a7b5546e9cfd92cc5"}
Sep 19 08:20:36.225949 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:36.225904    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-lqdbb" podStartSLOduration=2.961442756 podStartE2EDuration="20.225889573s" totalImagesPullingTime="17.264446817s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 08:20:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 08:20:36.225581721 +0000 UTC m=+20.602291595" watchObservedRunningTime="2026-09-19 08:20:36.225889573 +0000 UTC m=+20.602599434"
Sep 19 08:20:36.241908 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:36.241870    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-5x6rt" podStartSLOduration=3.016353146 podStartE2EDuration="20.241856142s" totalImagesPullingTime="17.225502996s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 08:20:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 08:20:36.241569106 +0000 UTC m=+20.618278972" watchObservedRunningTime="2026-09-19 08:20:36.241856142 +0000 UTC m=+20.618566003"
Sep 19 08:20:36.254702 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:36.254661    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-hb9xj" podStartSLOduration=11.163884085 podStartE2EDuration="20.254648692s" totalImagesPullingTime="9.090764607s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 08:20:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 08:20:36.254334912 +0000 UTC m=+20.631044773" watchObservedRunningTime="2026-09-19 08:20:36.254648692 +0000 UTC m=+20.631358550"
Sep 19 08:20:36.284996 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:36.284953    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-157.ec2.internal" podStartSLOduration=19.284937412 podStartE2EDuration="19.284937412s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 08:20:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 08:20:36.279132119 +0000 UTC m=+20.655841979" watchObservedRunningTime="2026-09-19 08:20:36.284937412 +0000 UTC m=+20.661647273"
Sep 19 08:20:37.107309 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:37.107275    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-md9ln"
Sep 19 08:20:37.107495 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:37.107410    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-md9ln" podUID="79f6ab54-a988-4015-81c2-da35716eab52"
Sep 19 08:20:37.210258 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:37.210232    2578 generic.go:356] "Generic (PLEG): container finished" podID="647eb10e-629e-4cbc-bad4-234bbb862d6e" containerID="b6f7d098881c97d26cf55dcde61c7ec1a0692eb50a86a40a7b5546e9cfd92cc5" exitCode=0
Sep 19 08:20:37.211019 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:37.210324    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9frbh" event={"ID":"647eb10e-629e-4cbc-bad4-234bbb862d6e","Type":"ContainerDied","Data":"b6f7d098881c97d26cf55dcde61c7ec1a0692eb50a86a40a7b5546e9cfd92cc5"}
Sep 19 08:20:37.211821 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:37.211802    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6vp2f" event={"ID":"78faeeea-464e-4ad2-9987-47fec1907637","Type":"ContainerStarted","Data":"bdc799c2af244962bd44cc109152953e89d64eb9c8978b01166f0f6af0900a9e"}
Sep 19 08:20:37.214572 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:37.214555    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zd2zp_07ddac45-12a1-42e8-9566-55f08854c02e/ovn-acl-logging/0.log"
Sep 19 08:20:37.214878 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:37.214857    2578 generic.go:356] "Generic (PLEG): container finished" podID="07ddac45-12a1-42e8-9566-55f08854c02e" containerID="38b16217af4c47453286373c70c7d7c55bf77a5d04166bc3c224d0f286420c8e" exitCode=1
Sep 19 08:20:37.214974 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:37.214924    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zd2zp" event={"ID":"07ddac45-12a1-42e8-9566-55f08854c02e","Type":"ContainerStarted","Data":"0d63cdeaebd6b1af99623056aa1ae7b889034fb7398d85ea9a943af8465fc06b"}
Sep 19 08:20:37.215036 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:37.215006    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zd2zp" event={"ID":"07ddac45-12a1-42e8-9566-55f08854c02e","Type":"ContainerStarted","Data":"268ed859163c73ec0baa27f85e60f2f5dd94017874fbd145600229fb6d497c77"}
Sep 19 08:20:37.215089 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:37.215056    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zd2zp" event={"ID":"07ddac45-12a1-42e8-9566-55f08854c02e","Type":"ContainerStarted","Data":"89b578d7f367e15f728c999700a03291da70c9c66d1ac695d91cb221f2a2d78a"}
Sep 19 08:20:37.215137 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:37.215104    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zd2zp" event={"ID":"07ddac45-12a1-42e8-9566-55f08854c02e","Type":"ContainerStarted","Data":"3bcff8733160037a306e80475435a0a4a31f8de790e646d157ec98a86941645f"}
Sep 19 08:20:37.215192 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:37.215152    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zd2zp" event={"ID":"07ddac45-12a1-42e8-9566-55f08854c02e","Type":"ContainerDied","Data":"38b16217af4c47453286373c70c7d7c55bf77a5d04166bc3c224d0f286420c8e"}
Sep 19 08:20:37.217007 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:37.216983    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-wln5n" event={"ID":"c3a67fda-81e8-4d20-83d4-a37e277382b4","Type":"ContainerStarted","Data":"b0e282fed1830888ff644a9a6e1f9887f09c07583dcd4a01c110c5aff5859f4e"}
Sep 19 08:20:37.232856 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:37.232819    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-7tt6m" podStartSLOduration=3.979405936 podStartE2EDuration="21.232804933s" totalImagesPullingTime="17.253398997s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 08:20:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 08:20:36.324853364 +0000 UTC m=+20.701563229" watchObservedRunningTime="2026-09-19 08:20:37.232804933 +0000 UTC m=+21.609514777"
Sep 19 08:20:37.245427 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:37.245408    2578 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 19 08:20:37.251582 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:37.251523    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-wln5n" podStartSLOduration=4.477005735 podStartE2EDuration="21.251508998s" totalImagesPullingTime="16.774503263s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 08:20:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 08:20:37.251256354 +0000 UTC m=+21.627966214" watchObservedRunningTime="2026-09-19 08:20:37.251508998 +0000 UTC m=+21.628218858"
Sep 19 08:20:38.037185 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:38.036928    2578 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-19T08:20:37.245421732Z","UUID":"151a1583-7ec2-41d7-a0f5-0ebca344733c","Handler":null,"Name":"","Endpoint":""}
Sep 19 08:20:38.038691 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:38.038666    2578 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 19 08:20:38.038821 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:38.038704    2578 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 19 08:20:38.111787 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:38.111223    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xq8d8"
Sep 19 08:20:38.111787 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:38.111357    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xq8d8" podUID="19642aea-2c2e-4208-8329-09f03b1e5d5b"
Sep 19 08:20:38.111787 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:38.111223    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l6m8x"
Sep 19 08:20:38.111787 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:38.111752    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l6m8x" podUID="8324d8db-b082-475a-8bb7-8c578404b02e"
Sep 19 08:20:38.220423 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:38.220381    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-jfkwf" event={"ID":"dcc90f06-1e07-4b6b-add7-46f0cdd95ecf","Type":"ContainerStarted","Data":"292d1e1fbd98bc081c62a41c748761d0033b0dbbf3c69bb24ccafa201d437a22"}
Sep 19 08:20:38.222491 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:38.222468    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6vp2f" event={"ID":"78faeeea-464e-4ad2-9987-47fec1907637","Type":"ContainerStarted","Data":"fa55d10f05fc13122d0d9a1197f9b9131c17dd3aa019ad2c3dbec0e8e4689969"}
Sep 19 08:20:38.236244 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:38.236206    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-jfkwf" podStartSLOduration=4.995078535 podStartE2EDuration="22.236193155s" totalImagesPullingTime="17.24111462s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 08:20:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 08:20:38.235772626 +0000 UTC m=+22.612482488" watchObservedRunningTime="2026-09-19 08:20:38.236193155 +0000 UTC m=+22.612903015"
Sep 19 08:20:38.770615 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:38.770561    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-7tt6m"
Sep 19 08:20:39.107645 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:39.107578    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-md9ln"
Sep 19 08:20:39.107785 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:39.107681    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-md9ln" podUID="79f6ab54-a988-4015-81c2-da35716eab52"
Sep 19 08:20:39.227676 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:39.227638    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6vp2f" event={"ID":"78faeeea-464e-4ad2-9987-47fec1907637","Type":"ContainerStarted","Data":"7a2233c0f5149914aaf12c18181b3a9dc067b735c51d2e5ad0c014f743c875ee"}
Sep 19 08:20:39.233633 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:39.233581    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zd2zp_07ddac45-12a1-42e8-9566-55f08854c02e/ovn-acl-logging/0.log"
Sep 19 08:20:39.234119 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:39.234084    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zd2zp" event={"ID":"07ddac45-12a1-42e8-9566-55f08854c02e","Type":"ContainerStarted","Data":"590bb3c63cce54620db99aee21966e855705120ec5be196f838353e9241e4ff2"}
Sep 19 08:20:39.247257 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:39.247217    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6vp2f" podStartSLOduration=4.879437908 podStartE2EDuration="23.247205236s" totalImagesPullingTime="18.367767328s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 08:20:16 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 08:20:39.246942511 +0000 UTC m=+23.623652372" watchObservedRunningTime="2026-09-19 08:20:39.247205236 +0000 UTC m=+23.623915112"
Sep 19 08:20:40.107896 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:40.107867    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l6m8x"
Sep 19 08:20:40.108167 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:40.107992    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l6m8x" podUID="8324d8db-b082-475a-8bb7-8c578404b02e"
Sep 19 08:20:40.108167 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:40.108029    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xq8d8"
Sep 19 08:20:40.108167 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:40.108094    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xq8d8" podUID="19642aea-2c2e-4208-8329-09f03b1e5d5b"
Sep 19 08:20:40.791523 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:40.791493    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-7tt6m"
Sep 19 08:20:40.792103 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:40.792085    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-7tt6m"
Sep 19 08:20:41.107850 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:41.107785    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-md9ln"
Sep 19 08:20:41.107974 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:41.107910    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-md9ln" podUID="79f6ab54-a988-4015-81c2-da35716eab52"
Sep 19 08:20:41.239627 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:41.239602    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-7tt6m"
Sep 19 08:20:42.107199 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:42.106989    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l6m8x"
Sep 19 08:20:42.107606 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:42.107042    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xq8d8"
Sep 19 08:20:42.107606 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:42.107287    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l6m8x" podUID="8324d8db-b082-475a-8bb7-8c578404b02e"
Sep 19 08:20:42.107606 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:42.107350    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xq8d8" podUID="19642aea-2c2e-4208-8329-09f03b1e5d5b"
Sep 19 08:20:42.242559 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:42.242532    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9frbh" event={"ID":"647eb10e-629e-4cbc-bad4-234bbb862d6e","Type":"ContainerStarted","Data":"b83d6c4251f4da2b646e22abde00f94c04aadeb07668fd7e41d9770502ebb38b"}
Sep 19 08:20:42.246224 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:42.246207    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zd2zp_07ddac45-12a1-42e8-9566-55f08854c02e/ovn-acl-logging/0.log"
Sep 19 08:20:42.246691 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:42.246659    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zd2zp" event={"ID":"07ddac45-12a1-42e8-9566-55f08854c02e","Type":"ContainerStarted","Data":"805ce0ef869199b26a8a3a7c37d9e097b8f0f0ae3f3b7449e926c523381672d6"}
Sep 19 08:20:42.247061 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:42.247044    2578 scope.go:117] "RemoveContainer" containerID="38b16217af4c47453286373c70c7d7c55bf77a5d04166bc3c224d0f286420c8e"
Sep 19 08:20:43.107877 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:43.107791    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-md9ln"
Sep 19 08:20:43.108487 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:43.107934    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-md9ln" podUID="79f6ab54-a988-4015-81c2-da35716eab52"
Sep 19 08:20:43.252910 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:43.252880    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zd2zp_07ddac45-12a1-42e8-9566-55f08854c02e/ovn-acl-logging/0.log"
Sep 19 08:20:43.253269 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:43.253245    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zd2zp" event={"ID":"07ddac45-12a1-42e8-9566-55f08854c02e","Type":"ContainerStarted","Data":"0148de0c1f3f4c203331225da746e2960bc190dd67b4b32e952a5cb2f1a2848e"}
Sep 19 08:20:43.253531 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:43.253513    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-zd2zp"
Sep 19 08:20:43.253626 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:43.253598    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-zd2zp"
Sep 19 08:20:43.253675 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:43.253642    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-zd2zp"
Sep 19 08:20:43.255445 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:43.255420    2578 generic.go:356] "Generic (PLEG): container finished" podID="647eb10e-629e-4cbc-bad4-234bbb862d6e" containerID="b83d6c4251f4da2b646e22abde00f94c04aadeb07668fd7e41d9770502ebb38b" exitCode=0
Sep 19 08:20:43.255535 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:43.255466    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9frbh" event={"ID":"647eb10e-629e-4cbc-bad4-234bbb862d6e","Type":"ContainerDied","Data":"b83d6c4251f4da2b646e22abde00f94c04aadeb07668fd7e41d9770502ebb38b"}
Sep 19 08:20:43.268521 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:43.268489    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-zd2zp"
Sep 19 08:20:43.268678 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:43.268663    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-zd2zp"
Sep 19 08:20:43.280333 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:43.280277    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-zd2zp" podStartSLOduration=10.790884843 podStartE2EDuration="27.280263563s" totalImagesPullingTime="16.48937872s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 08:20:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 08:20:43.279961324 +0000 UTC m=+27.656671186" watchObservedRunningTime="2026-09-19 08:20:43.280263563 +0000 UTC m=+27.656973424"
Sep 19 08:20:43.705488 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:43.705433    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-l6m8x"]
Sep 19 08:20:43.705647 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:43.705629    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l6m8x"
Sep 19 08:20:43.705795 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:43.705769    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l6m8x" podUID="8324d8db-b082-475a-8bb7-8c578404b02e"
Sep 19 08:20:43.708674 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:43.708651    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-xq8d8"]
Sep 19 08:20:43.708774 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:43.708745    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xq8d8"
Sep 19 08:20:43.708834 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:43.708817    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xq8d8" podUID="19642aea-2c2e-4208-8329-09f03b1e5d5b"
Sep 19 08:20:43.709170 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:43.709148    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-md9ln"]
Sep 19 08:20:43.709256 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:43.709244    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-md9ln"
Sep 19 08:20:43.709371 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:43.709353    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-md9ln" podUID="79f6ab54-a988-4015-81c2-da35716eab52"
Sep 19 08:20:44.259929 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:44.259900    2578 generic.go:356] "Generic (PLEG): container finished" podID="647eb10e-629e-4cbc-bad4-234bbb862d6e" containerID="6d72e353f283d111b8c0b566a0529f9c89fcd4e4e98f5fab355776a260042a73" exitCode=0
Sep 19 08:20:44.260458 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:44.259985    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9frbh" event={"ID":"647eb10e-629e-4cbc-bad4-234bbb862d6e","Type":"ContainerDied","Data":"6d72e353f283d111b8c0b566a0529f9c89fcd4e4e98f5fab355776a260042a73"}
Sep 19 08:20:45.107832 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:45.107802    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l6m8x"
Sep 19 08:20:45.107832 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:45.107833    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-md9ln"
Sep 19 08:20:45.108028 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:45.107799    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xq8d8"
Sep 19 08:20:45.108028 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:45.107904    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l6m8x" podUID="8324d8db-b082-475a-8bb7-8c578404b02e"
Sep 19 08:20:45.108028 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:45.107985    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xq8d8" podUID="19642aea-2c2e-4208-8329-09f03b1e5d5b"
Sep 19 08:20:45.108162 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:45.108079    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-md9ln" podUID="79f6ab54-a988-4015-81c2-da35716eab52"
Sep 19 08:20:45.264778 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:45.264697    2578 generic.go:356] "Generic (PLEG): container finished" podID="647eb10e-629e-4cbc-bad4-234bbb862d6e" containerID="d202a3bf2148eedafe8a0126ce5fd3af0a44d088e6d856a2ea8cf9de6d905d1f" exitCode=0
Sep 19 08:20:45.264778 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:45.264759    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9frbh" event={"ID":"647eb10e-629e-4cbc-bad4-234bbb862d6e","Type":"ContainerDied","Data":"d202a3bf2148eedafe8a0126ce5fd3af0a44d088e6d856a2ea8cf9de6d905d1f"}
Sep 19 08:20:47.108038 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:47.107847    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l6m8x"
Sep 19 08:20:47.108463 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:47.107866    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xq8d8"
Sep 19 08:20:47.108463 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:47.108138    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l6m8x" podUID="8324d8db-b082-475a-8bb7-8c578404b02e"
Sep 19 08:20:47.108463 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:47.108207    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xq8d8" podUID="19642aea-2c2e-4208-8329-09f03b1e5d5b"
Sep 19 08:20:47.108463 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:47.107909    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-md9ln"
Sep 19 08:20:47.108463 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:47.108331    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-md9ln" podUID="79f6ab54-a988-4015-81c2-da35716eab52"
Sep 19 08:20:48.919094 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:48.919067    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-157.ec2.internal" event="NodeReady"
Sep 19 08:20:48.919539 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:48.919219    2578 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 19 08:20:48.953633 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:48.953559    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-8566d48d94-7ksk8"]
Sep 19 08:20:48.973774 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:48.973726    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-ldghd"]
Sep 19 08:20:48.973958 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:48.973856    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-8566d48d94-7ksk8"
Sep 19 08:20:48.978746 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:48.977263    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 19 08:20:48.978746 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:48.977354    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 19 08:20:48.978746 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:48.977268    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 19 08:20:48.978746 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:48.977596    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-8bzsz\""
Sep 19 08:20:48.987560 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:48.987541    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 19 08:20:48.988182 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:48.988159    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-8566d48d94-7ksk8"]
Sep 19 08:20:48.988285 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:48.988197    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-6thbg"]
Sep 19 08:20:48.988285 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:48.988274    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-ldghd"
Sep 19 08:20:48.990726 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:48.990660    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 19 08:20:48.990849 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:48.990740    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-cdf7v\""
Sep 19 08:20:48.990928 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:48.990869    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 19 08:20:49.004526 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:49.004505    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-ldghd"]
Sep 19 08:20:49.004647 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:49.004569    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-6thbg"]
Sep 19 08:20:49.004647 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:49.004628    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-6thbg"
Sep 19 08:20:49.007001 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:49.006985    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 19 08:20:49.007142 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:49.007122    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 19 08:20:49.007286 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:49.007251    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-2fzkh\""
Sep 19 08:20:49.007495 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:49.007481    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 19 08:20:49.107950 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:49.107912    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-md9ln"
Sep 19 08:20:49.107950 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:49.107932    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xq8d8"
Sep 19 08:20:49.108210 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:49.107927    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l6m8x"
Sep 19 08:20:49.110624 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:49.110602    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 19 08:20:49.110772 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:49.110734    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-2vrmn\""
Sep 19 08:20:49.110772 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:49.110764    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 19 08:20:49.110901 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:49.110888    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-czqpg\""
Sep 19 08:20:49.110993 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:49.110979    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 19 08:20:49.111038 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:49.111027    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 19 08:20:49.115897 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:49.115871    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:20:49.117956 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:49.117937    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/61a85fdc-d1d2-4a56-a247-968786b293a5-tmp-dir\") pod \"dns-default-ldghd\" (UID: \"61a85fdc-d1d2-4a56-a247-968786b293a5\") " pod="openshift-dns/dns-default-ldghd"
Sep 19 08:20:49.118059 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:49.117967    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3135bdcd-43c1-4e88-9ec1-0a9a5d2ecdb0-cert\") pod \"ingress-canary-6thbg\" (UID: \"3135bdcd-43c1-4e88-9ec1-0a9a5d2ecdb0\") " pod="openshift-ingress-canary/ingress-canary-6thbg"
Sep 19 08:20:49.118059 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:49.118032    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5dca0cc5-6589-4585-978d-ca12d129c1a0-image-registry-private-configuration\") pod \"image-registry-8566d48d94-7ksk8\" (UID: \"5dca0cc5-6589-4585-978d-ca12d129c1a0\") " pod="openshift-image-registry/image-registry-8566d48d94-7ksk8"
Sep 19 08:20:49.118167 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:49.118058    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rf4n7\" (UniqueName: \"kubernetes.io/projected/5dca0cc5-6589-4585-978d-ca12d129c1a0-kube-api-access-rf4n7\") pod \"image-registry-8566d48d94-7ksk8\" (UID: \"5dca0cc5-6589-4585-978d-ca12d129c1a0\") " pod="openshift-image-registry/image-registry-8566d48d94-7ksk8"
Sep 19 08:20:49.118167 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:49.118097    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5dca0cc5-6589-4585-978d-ca12d129c1a0-registry-certificates\") pod \"image-registry-8566d48d94-7ksk8\" (UID: \"5dca0cc5-6589-4585-978d-ca12d129c1a0\") " pod="openshift-image-registry/image-registry-8566d48d94-7ksk8"
Sep 19 08:20:49.118167 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:49.118133    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pwrwn\" (UniqueName: \"kubernetes.io/projected/3135bdcd-43c1-4e88-9ec1-0a9a5d2ecdb0-kube-api-access-pwrwn\") pod \"ingress-canary-6thbg\" (UID: \"3135bdcd-43c1-4e88-9ec1-0a9a5d2ecdb0\") " pod="openshift-ingress-canary/ingress-canary-6thbg"
Sep 19 08:20:49.118343 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:49.118212    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5dca0cc5-6589-4585-978d-ca12d129c1a0-registry-tls\") pod \"image-registry-8566d48d94-7ksk8\" (UID: \"5dca0cc5-6589-4585-978d-ca12d129c1a0\") " pod="openshift-image-registry/image-registry-8566d48d94-7ksk8"
Sep 19 08:20:49.118343 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:49.118241    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/61a85fdc-d1d2-4a56-a247-968786b293a5-config-volume\") pod \"dns-default-ldghd\" (UID: \"61a85fdc-d1d2-4a56-a247-968786b293a5\") " pod="openshift-dns/dns-default-ldghd"
Sep 19 08:20:49.118343 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:49.118316    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5dca0cc5-6589-4585-978d-ca12d129c1a0-ca-trust-extracted\") pod \"image-registry-8566d48d94-7ksk8\" (UID: \"5dca0cc5-6589-4585-978d-ca12d129c1a0\") " pod="openshift-image-registry/image-registry-8566d48d94-7ksk8"
Sep 19 08:20:49.118490 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:49.118354    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5dca0cc5-6589-4585-978d-ca12d129c1a0-trusted-ca\") pod \"image-registry-8566d48d94-7ksk8\" (UID: \"5dca0cc5-6589-4585-978d-ca12d129c1a0\") " pod="openshift-image-registry/image-registry-8566d48d94-7ksk8"
Sep 19 08:20:49.118490 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:49.118377    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5dca0cc5-6589-4585-978d-ca12d129c1a0-installation-pull-secrets\") pod \"image-registry-8566d48d94-7ksk8\" (UID: \"5dca0cc5-6589-4585-978d-ca12d129c1a0\") " pod="openshift-image-registry/image-registry-8566d48d94-7ksk8"
Sep 19 08:20:49.118490 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:49.118400    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5dca0cc5-6589-4585-978d-ca12d129c1a0-bound-sa-token\") pod \"image-registry-8566d48d94-7ksk8\" (UID: \"5dca0cc5-6589-4585-978d-ca12d129c1a0\") " pod="openshift-image-registry/image-registry-8566d48d94-7ksk8"
Sep 19 08:20:49.118490 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:49.118437    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/61a85fdc-d1d2-4a56-a247-968786b293a5-metrics-tls\") pod \"dns-default-ldghd\" (UID: \"61a85fdc-d1d2-4a56-a247-968786b293a5\") " pod="openshift-dns/dns-default-ldghd"
Sep 19 08:20:49.118490 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:49.118470    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xlz4m\" (UniqueName: \"kubernetes.io/projected/61a85fdc-d1d2-4a56-a247-968786b293a5-kube-api-access-xlz4m\") pod \"dns-default-ldghd\" (UID: \"61a85fdc-d1d2-4a56-a247-968786b293a5\") " pod="openshift-dns/dns-default-ldghd"
Sep 19 08:20:49.208281 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:49.208187    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:20:49.219093 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:49.219057    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/61a85fdc-d1d2-4a56-a247-968786b293a5-metrics-tls\") pod \"dns-default-ldghd\" (UID: \"61a85fdc-d1d2-4a56-a247-968786b293a5\") " pod="openshift-dns/dns-default-ldghd"
Sep 19 08:20:49.219226 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:49.219096    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xlz4m\" (UniqueName: \"kubernetes.io/projected/61a85fdc-d1d2-4a56-a247-968786b293a5-kube-api-access-xlz4m\") pod \"dns-default-ldghd\" (UID: \"61a85fdc-d1d2-4a56-a247-968786b293a5\") " pod="openshift-dns/dns-default-ldghd"
Sep 19 08:20:49.219226 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:49.219116    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/61a85fdc-d1d2-4a56-a247-968786b293a5-tmp-dir\") pod \"dns-default-ldghd\" (UID: \"61a85fdc-d1d2-4a56-a247-968786b293a5\") " pod="openshift-dns/dns-default-ldghd"
Sep 19 08:20:49.219226 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:49.219144    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3135bdcd-43c1-4e88-9ec1-0a9a5d2ecdb0-cert\") pod \"ingress-canary-6thbg\" (UID: \"3135bdcd-43c1-4e88-9ec1-0a9a5d2ecdb0\") " pod="openshift-ingress-canary/ingress-canary-6thbg"
Sep 19 08:20:49.219226 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:49.219198    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5dca0cc5-6589-4585-978d-ca12d129c1a0-image-registry-private-configuration\") pod \"image-registry-8566d48d94-7ksk8\" (UID: \"5dca0cc5-6589-4585-978d-ca12d129c1a0\") " pod="openshift-image-registry/image-registry-8566d48d94-7ksk8"
Sep 19 08:20:49.219226 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:49.219216    2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 19 08:20:49.219478 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:49.219227    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rf4n7\" (UniqueName: \"kubernetes.io/projected/5dca0cc5-6589-4585-978d-ca12d129c1a0-kube-api-access-rf4n7\") pod \"image-registry-8566d48d94-7ksk8\" (UID: \"5dca0cc5-6589-4585-978d-ca12d129c1a0\") " pod="openshift-image-registry/image-registry-8566d48d94-7ksk8"
Sep 19 08:20:49.219478 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:49.219256    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5dca0cc5-6589-4585-978d-ca12d129c1a0-registry-certificates\") pod \"image-registry-8566d48d94-7ksk8\" (UID: \"5dca0cc5-6589-4585-978d-ca12d129c1a0\") " pod="openshift-image-registry/image-registry-8566d48d94-7ksk8"
Sep 19 08:20:49.219478 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:49.219281    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/61a85fdc-d1d2-4a56-a247-968786b293a5-metrics-tls podName:61a85fdc-d1d2-4a56-a247-968786b293a5 nodeName:}" failed. No retries permitted until 2026-09-19 08:20:49.719261549 +0000 UTC m=+34.095971403 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/61a85fdc-d1d2-4a56-a247-968786b293a5-metrics-tls") pod "dns-default-ldghd" (UID: "61a85fdc-d1d2-4a56-a247-968786b293a5") : secret "dns-default-metrics-tls" not found
Sep 19 08:20:49.219478 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:49.219328    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pwrwn\" (UniqueName: \"kubernetes.io/projected/3135bdcd-43c1-4e88-9ec1-0a9a5d2ecdb0-kube-api-access-pwrwn\") pod \"ingress-canary-6thbg\" (UID: \"3135bdcd-43c1-4e88-9ec1-0a9a5d2ecdb0\") " pod="openshift-ingress-canary/ingress-canary-6thbg"
Sep 19 08:20:49.219478 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:49.219383    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5dca0cc5-6589-4585-978d-ca12d129c1a0-registry-tls\") pod \"image-registry-8566d48d94-7ksk8\" (UID: \"5dca0cc5-6589-4585-978d-ca12d129c1a0\") " pod="openshift-image-registry/image-registry-8566d48d94-7ksk8"
Sep 19 08:20:49.219478 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:49.219405    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/61a85fdc-d1d2-4a56-a247-968786b293a5-config-volume\") pod \"dns-default-ldghd\" (UID: \"61a85fdc-d1d2-4a56-a247-968786b293a5\") " pod="openshift-dns/dns-default-ldghd"
Sep 19 08:20:49.219478 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:49.219445    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5dca0cc5-6589-4585-978d-ca12d129c1a0-ca-trust-extracted\") pod \"image-registry-8566d48d94-7ksk8\" (UID: \"5dca0cc5-6589-4585-978d-ca12d129c1a0\") " pod="openshift-image-registry/image-registry-8566d48d94-7ksk8"
Sep 19 08:20:49.219757 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:49.219477    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5dca0cc5-6589-4585-978d-ca12d129c1a0-trusted-ca\") pod \"image-registry-8566d48d94-7ksk8\" (UID: \"5dca0cc5-6589-4585-978d-ca12d129c1a0\") " pod="openshift-image-registry/image-registry-8566d48d94-7ksk8"
Sep 19 08:20:49.219757 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:49.219497    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/61a85fdc-d1d2-4a56-a247-968786b293a5-tmp-dir\") pod \"dns-default-ldghd\" (UID: \"61a85fdc-d1d2-4a56-a247-968786b293a5\") " pod="openshift-dns/dns-default-ldghd"
Sep 19 08:20:49.219757 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:49.219506    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5dca0cc5-6589-4585-978d-ca12d129c1a0-installation-pull-secrets\") pod \"image-registry-8566d48d94-7ksk8\" (UID: \"5dca0cc5-6589-4585-978d-ca12d129c1a0\") " pod="openshift-image-registry/image-registry-8566d48d94-7ksk8"
Sep 19 08:20:49.219757 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:49.219528    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5dca0cc5-6589-4585-978d-ca12d129c1a0-bound-sa-token\") pod \"image-registry-8566d48d94-7ksk8\" (UID: \"5dca0cc5-6589-4585-978d-ca12d129c1a0\") " pod="openshift-image-registry/image-registry-8566d48d94-7ksk8"
Sep 19 08:20:49.219757 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:49.219675    2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 19 08:20:49.219757 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:49.219690    2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-8566d48d94-7ksk8: secret "image-registry-tls" not found
Sep 19 08:20:49.219757 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:49.219751    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5dca0cc5-6589-4585-978d-ca12d129c1a0-registry-tls podName:5dca0cc5-6589-4585-978d-ca12d129c1a0 nodeName:}" failed. No retries permitted until 2026-09-19 08:20:49.71973078 +0000 UTC m=+34.096440625 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5dca0cc5-6589-4585-978d-ca12d129c1a0-registry-tls") pod "image-registry-8566d48d94-7ksk8" (UID: "5dca0cc5-6589-4585-978d-ca12d129c1a0") : secret "image-registry-tls" not found
Sep 19 08:20:49.219991 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:49.219840    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5dca0cc5-6589-4585-978d-ca12d129c1a0-registry-certificates\") pod \"image-registry-8566d48d94-7ksk8\" (UID: \"5dca0cc5-6589-4585-978d-ca12d129c1a0\") " pod="openshift-image-registry/image-registry-8566d48d94-7ksk8"
Sep 19 08:20:49.220242 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:49.220201    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/61a85fdc-d1d2-4a56-a247-968786b293a5-config-volume\") pod \"dns-default-ldghd\" (UID: \"61a85fdc-d1d2-4a56-a247-968786b293a5\") " pod="openshift-dns/dns-default-ldghd"
Sep 19 08:20:49.220378 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:49.220291    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5dca0cc5-6589-4585-978d-ca12d129c1a0-ca-trust-extracted\") pod \"image-registry-8566d48d94-7ksk8\" (UID: \"5dca0cc5-6589-4585-978d-ca12d129c1a0\") " pod="openshift-image-registry/image-registry-8566d48d94-7ksk8"
Sep 19 08:20:49.220869 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:49.220851    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5dca0cc5-6589-4585-978d-ca12d129c1a0-trusted-ca\") pod \"image-registry-8566d48d94-7ksk8\" (UID: \"5dca0cc5-6589-4585-978d-ca12d129c1a0\") " pod="openshift-image-registry/image-registry-8566d48d94-7ksk8"
Sep 19 08:20:49.224315 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:49.224259    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5dca0cc5-6589-4585-978d-ca12d129c1a0-installation-pull-secrets\") pod \"image-registry-8566d48d94-7ksk8\" (UID: \"5dca0cc5-6589-4585-978d-ca12d129c1a0\") " pod="openshift-image-registry/image-registry-8566d48d94-7ksk8"
Sep 19 08:20:49.224315 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:49.224288    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5dca0cc5-6589-4585-978d-ca12d129c1a0-image-registry-private-configuration\") pod \"image-registry-8566d48d94-7ksk8\" (UID: \"5dca0cc5-6589-4585-978d-ca12d129c1a0\") " pod="openshift-image-registry/image-registry-8566d48d94-7ksk8"
Sep 19 08:20:49.224483 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:49.224354    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3135bdcd-43c1-4e88-9ec1-0a9a5d2ecdb0-cert\") pod \"ingress-canary-6thbg\" (UID: \"3135bdcd-43c1-4e88-9ec1-0a9a5d2ecdb0\") " pod="openshift-ingress-canary/ingress-canary-6thbg"
Sep 19 08:20:49.228149 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:49.228111    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pwrwn\" (UniqueName: \"kubernetes.io/projected/3135bdcd-43c1-4e88-9ec1-0a9a5d2ecdb0-kube-api-access-pwrwn\") pod \"ingress-canary-6thbg\" (UID: \"3135bdcd-43c1-4e88-9ec1-0a9a5d2ecdb0\") " pod="openshift-ingress-canary/ingress-canary-6thbg"
Sep 19 08:20:49.228149 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:49.228116    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xlz4m\" (UniqueName: \"kubernetes.io/projected/61a85fdc-d1d2-4a56-a247-968786b293a5-kube-api-access-xlz4m\") pod \"dns-default-ldghd\" (UID: \"61a85fdc-d1d2-4a56-a247-968786b293a5\") " pod="openshift-dns/dns-default-ldghd"
Sep 19 08:20:49.228328 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:49.228246    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rf4n7\" (UniqueName: \"kubernetes.io/projected/5dca0cc5-6589-4585-978d-ca12d129c1a0-kube-api-access-rf4n7\") pod \"image-registry-8566d48d94-7ksk8\" (UID: \"5dca0cc5-6589-4585-978d-ca12d129c1a0\") " pod="openshift-image-registry/image-registry-8566d48d94-7ksk8"
Sep 19 08:20:49.228393 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:49.228286    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5dca0cc5-6589-4585-978d-ca12d129c1a0-bound-sa-token\") pod \"image-registry-8566d48d94-7ksk8\" (UID: \"5dca0cc5-6589-4585-978d-ca12d129c1a0\") " pod="openshift-image-registry/image-registry-8566d48d94-7ksk8"
Sep 19 08:20:49.314405 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:49.314376    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-6thbg"
Sep 19 08:20:49.723567 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:49.723525    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/61a85fdc-d1d2-4a56-a247-968786b293a5-metrics-tls\") pod \"dns-default-ldghd\" (UID: \"61a85fdc-d1d2-4a56-a247-968786b293a5\") " pod="openshift-dns/dns-default-ldghd"
Sep 19 08:20:49.723787 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:49.723639    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5dca0cc5-6589-4585-978d-ca12d129c1a0-registry-tls\") pod \"image-registry-8566d48d94-7ksk8\" (UID: \"5dca0cc5-6589-4585-978d-ca12d129c1a0\") " pod="openshift-image-registry/image-registry-8566d48d94-7ksk8"
Sep 19 08:20:49.723787 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:49.723668    2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 19 08:20:49.723787 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:49.723737    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/61a85fdc-d1d2-4a56-a247-968786b293a5-metrics-tls podName:61a85fdc-d1d2-4a56-a247-968786b293a5 nodeName:}" failed. No retries permitted until 2026-09-19 08:20:50.723717022 +0000 UTC m=+35.100426865 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/61a85fdc-d1d2-4a56-a247-968786b293a5-metrics-tls") pod "dns-default-ldghd" (UID: "61a85fdc-d1d2-4a56-a247-968786b293a5") : secret "dns-default-metrics-tls" not found
Sep 19 08:20:49.723787 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:49.723749    2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 19 08:20:49.723787 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:49.723762    2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-8566d48d94-7ksk8: secret "image-registry-tls" not found
Sep 19 08:20:49.724008 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:49.723832    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5dca0cc5-6589-4585-978d-ca12d129c1a0-registry-tls podName:5dca0cc5-6589-4585-978d-ca12d129c1a0 nodeName:}" failed. No retries permitted until 2026-09-19 08:20:50.723817138 +0000 UTC m=+35.100526980 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5dca0cc5-6589-4585-978d-ca12d129c1a0-registry-tls") pod "image-registry-8566d48d94-7ksk8" (UID: "5dca0cc5-6589-4585-978d-ca12d129c1a0") : secret "image-registry-tls" not found
Sep 19 08:20:49.824751 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:49.824725    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/19642aea-2c2e-4208-8329-09f03b1e5d5b-original-pull-secret\") pod \"global-pull-secret-syncer-xq8d8\" (UID: \"19642aea-2c2e-4208-8329-09f03b1e5d5b\") " pod="kube-system/global-pull-secret-syncer-xq8d8"
Sep 19 08:20:49.824874 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:49.824807    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/79f6ab54-a988-4015-81c2-da35716eab52-metrics-certs\") pod \"network-metrics-daemon-md9ln\" (UID: \"79f6ab54-a988-4015-81c2-da35716eab52\") " pod="openshift-multus/network-metrics-daemon-md9ln"
Sep 19 08:20:49.824936 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:49.824888    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 19 08:20:49.824999 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:49.824937    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79f6ab54-a988-4015-81c2-da35716eab52-metrics-certs podName:79f6ab54-a988-4015-81c2-da35716eab52 nodeName:}" failed. No retries permitted until 2026-09-19 08:21:21.824924609 +0000 UTC m=+66.201634454 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/79f6ab54-a988-4015-81c2-da35716eab52-metrics-certs") pod "network-metrics-daemon-md9ln" (UID: "79f6ab54-a988-4015-81c2-da35716eab52") : secret "metrics-daemon-secret" not found
Sep 19 08:20:49.827057 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:49.827020    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/19642aea-2c2e-4208-8329-09f03b1e5d5b-original-pull-secret\") pod \"global-pull-secret-syncer-xq8d8\" (UID: \"19642aea-2c2e-4208-8329-09f03b1e5d5b\") " pod="kube-system/global-pull-secret-syncer-xq8d8"
Sep 19 08:20:49.925602 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:49.925422    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r7rcd\" (UniqueName: \"kubernetes.io/projected/8324d8db-b082-475a-8bb7-8c578404b02e-kube-api-access-r7rcd\") pod \"network-check-target-l6m8x\" (UID: \"8324d8db-b082-475a-8bb7-8c578404b02e\") " pod="openshift-network-diagnostics/network-check-target-l6m8x"
Sep 19 08:20:49.928455 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:49.928436    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r7rcd\" (UniqueName: \"kubernetes.io/projected/8324d8db-b082-475a-8bb7-8c578404b02e-kube-api-access-r7rcd\") pod \"network-check-target-l6m8x\" (UID: \"8324d8db-b082-475a-8bb7-8c578404b02e\") " pod="openshift-network-diagnostics/network-check-target-l6m8x"
Sep 19 08:20:50.027804 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:50.027745    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l6m8x"
Sep 19 08:20:50.034974 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:50.034949    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xq8d8"
Sep 19 08:20:50.101615 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:50.101588    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-6thbg"]
Sep 19 08:20:50.106018 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:50.105974    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3135bdcd_43c1_4e88_9ec1_0a9a5d2ecdb0.slice/crio-905b77ce06305eebd5e27e1f8ee0215d078b6fb03c52e2656edb2285dfc331ab WatchSource:0}: Error finding container 905b77ce06305eebd5e27e1f8ee0215d078b6fb03c52e2656edb2285dfc331ab: Status 404 returned error can't find the container with id 905b77ce06305eebd5e27e1f8ee0215d078b6fb03c52e2656edb2285dfc331ab
Sep 19 08:20:50.276815 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:50.276786    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-6thbg" event={"ID":"3135bdcd-43c1-4e88-9ec1-0a9a5d2ecdb0","Type":"ContainerStarted","Data":"905b77ce06305eebd5e27e1f8ee0215d078b6fb03c52e2656edb2285dfc331ab"}
Sep 19 08:20:50.374114 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:50.374087    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-l6m8x"]
Sep 19 08:20:50.380501 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:50.380471    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-xq8d8"]
Sep 19 08:20:50.383669 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:50.383638    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8324d8db_b082_475a_8bb7_8c578404b02e.slice/crio-b5035a76e325fc7eff0739106880fd0c5fdd4c1252ae8fa3e0e7bdafd7057da4 WatchSource:0}: Error finding container b5035a76e325fc7eff0739106880fd0c5fdd4c1252ae8fa3e0e7bdafd7057da4: Status 404 returned error can't find the container with id b5035a76e325fc7eff0739106880fd0c5fdd4c1252ae8fa3e0e7bdafd7057da4
Sep 19 08:20:50.384698 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:20:50.384671    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19642aea_2c2e_4208_8329_09f03b1e5d5b.slice/crio-158702fd5c3b42e2bca793b048793f49eb18e7fe0ca635f525509ca21bb146c3 WatchSource:0}: Error finding container 158702fd5c3b42e2bca793b048793f49eb18e7fe0ca635f525509ca21bb146c3: Status 404 returned error can't find the container with id 158702fd5c3b42e2bca793b048793f49eb18e7fe0ca635f525509ca21bb146c3
Sep 19 08:20:50.732309 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:50.732223    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5dca0cc5-6589-4585-978d-ca12d129c1a0-registry-tls\") pod \"image-registry-8566d48d94-7ksk8\" (UID: \"5dca0cc5-6589-4585-978d-ca12d129c1a0\") " pod="openshift-image-registry/image-registry-8566d48d94-7ksk8"
Sep 19 08:20:50.732309 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:50.732273    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/61a85fdc-d1d2-4a56-a247-968786b293a5-metrics-tls\") pod \"dns-default-ldghd\" (UID: \"61a85fdc-d1d2-4a56-a247-968786b293a5\") " pod="openshift-dns/dns-default-ldghd"
Sep 19 08:20:50.732568 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:50.732400    2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 19 08:20:50.732568 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:50.732406    2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 19 08:20:50.732568 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:50.732460    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/61a85fdc-d1d2-4a56-a247-968786b293a5-metrics-tls podName:61a85fdc-d1d2-4a56-a247-968786b293a5 nodeName:}" failed. No retries permitted until 2026-09-19 08:20:52.732441851 +0000 UTC m=+37.109151696 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/61a85fdc-d1d2-4a56-a247-968786b293a5-metrics-tls") pod "dns-default-ldghd" (UID: "61a85fdc-d1d2-4a56-a247-968786b293a5") : secret "dns-default-metrics-tls" not found
Sep 19 08:20:50.732568 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:50.732470    2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-8566d48d94-7ksk8: secret "image-registry-tls" not found
Sep 19 08:20:50.732568 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:50.732549    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5dca0cc5-6589-4585-978d-ca12d129c1a0-registry-tls podName:5dca0cc5-6589-4585-978d-ca12d129c1a0 nodeName:}" failed. No retries permitted until 2026-09-19 08:20:52.732514805 +0000 UTC m=+37.109224664 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5dca0cc5-6589-4585-978d-ca12d129c1a0-registry-tls") pod "image-registry-8566d48d94-7ksk8" (UID: "5dca0cc5-6589-4585-978d-ca12d129c1a0") : secret "image-registry-tls" not found
Sep 19 08:20:51.283386 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:51.283198    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-l6m8x" event={"ID":"8324d8db-b082-475a-8bb7-8c578404b02e","Type":"ContainerStarted","Data":"b5035a76e325fc7eff0739106880fd0c5fdd4c1252ae8fa3e0e7bdafd7057da4"}
Sep 19 08:20:51.286253 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:51.286177    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-xq8d8" event={"ID":"19642aea-2c2e-4208-8329-09f03b1e5d5b","Type":"ContainerStarted","Data":"158702fd5c3b42e2bca793b048793f49eb18e7fe0ca635f525509ca21bb146c3"}
Sep 19 08:20:51.292558 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:51.291651    2578 generic.go:356] "Generic (PLEG): container finished" podID="647eb10e-629e-4cbc-bad4-234bbb862d6e" containerID="fecb0b8b9f96dceef9fe0e5c5c2d53f20a41c10d4ea27314933ff5da7e6421fb" exitCode=0
Sep 19 08:20:51.292558 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:51.291704    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9frbh" event={"ID":"647eb10e-629e-4cbc-bad4-234bbb862d6e","Type":"ContainerDied","Data":"fecb0b8b9f96dceef9fe0e5c5c2d53f20a41c10d4ea27314933ff5da7e6421fb"}
Sep 19 08:20:52.299601 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:52.299558    2578 generic.go:356] "Generic (PLEG): container finished" podID="647eb10e-629e-4cbc-bad4-234bbb862d6e" containerID="7723d3e2310d8468b37194eacdb755f289d34328c890bdd370ecaf01e24d6e97" exitCode=0
Sep 19 08:20:52.300371 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:52.299629    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9frbh" event={"ID":"647eb10e-629e-4cbc-bad4-234bbb862d6e","Type":"ContainerDied","Data":"7723d3e2310d8468b37194eacdb755f289d34328c890bdd370ecaf01e24d6e97"}
Sep 19 08:20:52.526036 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:52.525978    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:20:52.595860 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:52.595779    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:20:52.695379 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:52.695342    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:20:52.748631 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:52.748593    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5dca0cc5-6589-4585-978d-ca12d129c1a0-registry-tls\") pod \"image-registry-8566d48d94-7ksk8\" (UID: \"5dca0cc5-6589-4585-978d-ca12d129c1a0\") " pod="openshift-image-registry/image-registry-8566d48d94-7ksk8"
Sep 19 08:20:52.748818 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:52.748664    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/61a85fdc-d1d2-4a56-a247-968786b293a5-metrics-tls\") pod \"dns-default-ldghd\" (UID: \"61a85fdc-d1d2-4a56-a247-968786b293a5\") " pod="openshift-dns/dns-default-ldghd"
Sep 19 08:20:52.748818 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:52.748786    2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 19 08:20:52.748818 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:52.748814    2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-8566d48d94-7ksk8: secret "image-registry-tls" not found
Sep 19 08:20:52.748985 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:52.748877    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5dca0cc5-6589-4585-978d-ca12d129c1a0-registry-tls podName:5dca0cc5-6589-4585-978d-ca12d129c1a0 nodeName:}" failed. No retries permitted until 2026-09-19 08:20:56.748856825 +0000 UTC m=+41.125566687 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5dca0cc5-6589-4585-978d-ca12d129c1a0-registry-tls") pod "image-registry-8566d48d94-7ksk8" (UID: "5dca0cc5-6589-4585-978d-ca12d129c1a0") : secret "image-registry-tls" not found
Sep 19 08:20:52.748985 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:52.748792    2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 19 08:20:52.748985 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:52.748917    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/61a85fdc-d1d2-4a56-a247-968786b293a5-metrics-tls podName:61a85fdc-d1d2-4a56-a247-968786b293a5 nodeName:}" failed. No retries permitted until 2026-09-19 08:20:56.748907337 +0000 UTC m=+41.125617181 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/61a85fdc-d1d2-4a56-a247-968786b293a5-metrics-tls") pod "dns-default-ldghd" (UID: "61a85fdc-d1d2-4a56-a247-968786b293a5") : secret "dns-default-metrics-tls" not found
Sep 19 08:20:52.891161 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:52.891081    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:20:52.998540 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:52.998482    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:20:53.088433 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:53.088399    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:20:53.201540 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:53.201141    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:20:53.302252 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:53.302126    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:20:53.304868 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:53.304821    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-6thbg" event={"ID":"3135bdcd-43c1-4e88-9ec1-0a9a5d2ecdb0","Type":"ContainerStarted","Data":"5f339d7e5fdb7d3f24c49d846c3e261d93fdc7cf9e8b1cb933cb994704726063"}
Sep 19 08:20:53.308840 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:53.308817    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9frbh" event={"ID":"647eb10e-629e-4cbc-bad4-234bbb862d6e","Type":"ContainerStarted","Data":"c9b4c5f6b47db3e0cda976c3649ab411d0818c77de1be6fe8994225969e4f1b5"}
Sep 19 08:20:53.321716 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:53.321665    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-6thbg" podStartSLOduration=2.279744885 podStartE2EDuration="5.321647384s" totalImagesPullingTime="3.041902499s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 08:20:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 08:20:53.321329608 +0000 UTC m=+37.698039470" watchObservedRunningTime="2026-09-19 08:20:53.321647384 +0000 UTC m=+37.698357247"
Sep 19 08:20:53.341862 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:53.341813    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-9frbh" podStartSLOduration=8.563077239 podStartE2EDuration="37.341798712s" totalImagesPullingTime="28.778721473s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 08:20:16 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 08:20:53.340714322 +0000 UTC m=+37.717424184" watchObservedRunningTime="2026-09-19 08:20:53.341798712 +0000 UTC m=+37.718508601"
Sep 19 08:20:53.567975 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:53.567851    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:20:53.665154 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:53.665115    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:20:53.938977 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:53.938882    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:20:53.994561 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:53.994526    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:20:54.534743 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:54.534686    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:20:54.603026 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:54.602990    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:20:54.735796 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:54.735766    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:20:54.837362 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:54.837266    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:20:55.191569 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:55.191487    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:20:55.340584 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:55.340551    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:20:56.318032 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:56.318000    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-l6m8x" event={"ID":"8324d8db-b082-475a-8bb7-8c578404b02e","Type":"ContainerStarted","Data":"6f91cd4110bbce1c77b733cce539fb872e02b831c2deae4094a822fd0cd31338"}
Sep 19 08:20:56.779576 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:56.779537    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/61a85fdc-d1d2-4a56-a247-968786b293a5-metrics-tls\") pod \"dns-default-ldghd\" (UID: \"61a85fdc-d1d2-4a56-a247-968786b293a5\") " pod="openshift-dns/dns-default-ldghd"
Sep 19 08:20:56.779786 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:56.779621    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5dca0cc5-6589-4585-978d-ca12d129c1a0-registry-tls\") pod \"image-registry-8566d48d94-7ksk8\" (UID: \"5dca0cc5-6589-4585-978d-ca12d129c1a0\") " pod="openshift-image-registry/image-registry-8566d48d94-7ksk8"
Sep 19 08:20:56.779786 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:56.779698    2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 19 08:20:56.779786 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:56.779726    2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 19 08:20:56.779786 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:56.779740    2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-8566d48d94-7ksk8: secret "image-registry-tls" not found
Sep 19 08:20:56.779786 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:56.779763    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/61a85fdc-d1d2-4a56-a247-968786b293a5-metrics-tls podName:61a85fdc-d1d2-4a56-a247-968786b293a5 nodeName:}" failed. No retries permitted until 2026-09-19 08:21:04.77974839 +0000 UTC m=+49.156458233 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/61a85fdc-d1d2-4a56-a247-968786b293a5-metrics-tls") pod "dns-default-ldghd" (UID: "61a85fdc-d1d2-4a56-a247-968786b293a5") : secret "dns-default-metrics-tls" not found
Sep 19 08:20:56.779786 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:20:56.779785    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5dca0cc5-6589-4585-978d-ca12d129c1a0-registry-tls podName:5dca0cc5-6589-4585-978d-ca12d129c1a0 nodeName:}" failed. No retries permitted until 2026-09-19 08:21:04.779769007 +0000 UTC m=+49.156478866 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5dca0cc5-6589-4585-978d-ca12d129c1a0-registry-tls") pod "image-registry-8566d48d94-7ksk8" (UID: "5dca0cc5-6589-4585-978d-ca12d129c1a0") : secret "image-registry-tls" not found
Sep 19 08:20:57.323493 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:57.323460    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-xq8d8" event={"ID":"19642aea-2c2e-4208-8329-09f03b1e5d5b","Type":"ContainerStarted","Data":"a87160ccb3d46eb320a140bc53421a7fd59ca015aee204b1bb9431275ff621ad"}
Sep 19 08:20:57.323832 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:57.323578    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-l6m8x"
Sep 19 08:20:57.339165 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:57.339123    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-xq8d8" podStartSLOduration=35.552387237 podStartE2EDuration="41.339110865s" totalImagesPullingTime="5.786723628s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 08:20:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 08:20:57.338818517 +0000 UTC m=+41.715528378" watchObservedRunningTime="2026-09-19 08:20:57.339110865 +0000 UTC m=+41.715820725"
Sep 19 08:20:57.355431 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:20:57.355388    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-l6m8x" podStartSLOduration=35.559586058 podStartE2EDuration="41.355376519s" totalImagesPullingTime="5.795790461s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 08:20:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 08:20:57.355124852 +0000 UTC m=+41.731834727" watchObservedRunningTime="2026-09-19 08:20:57.355376519 +0000 UTC m=+41.732086379"
Sep 19 08:21:01.887404 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:01.887363    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:21:01.987739 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:01.987704    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:21:04.833429 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:04.833381    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/61a85fdc-d1d2-4a56-a247-968786b293a5-metrics-tls\") pod \"dns-default-ldghd\" (UID: \"61a85fdc-d1d2-4a56-a247-968786b293a5\") " pod="openshift-dns/dns-default-ldghd"
Sep 19 08:21:04.833821 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:04.833469    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5dca0cc5-6589-4585-978d-ca12d129c1a0-registry-tls\") pod \"image-registry-8566d48d94-7ksk8\" (UID: \"5dca0cc5-6589-4585-978d-ca12d129c1a0\") " pod="openshift-image-registry/image-registry-8566d48d94-7ksk8"
Sep 19 08:21:04.833821 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:21:04.833547    2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 19 08:21:04.833821 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:21:04.833555    2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 19 08:21:04.833821 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:21:04.833568    2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-8566d48d94-7ksk8: secret "image-registry-tls" not found
Sep 19 08:21:04.833821 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:21:04.833620    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5dca0cc5-6589-4585-978d-ca12d129c1a0-registry-tls podName:5dca0cc5-6589-4585-978d-ca12d129c1a0 nodeName:}" failed. No retries permitted until 2026-09-19 08:21:20.833601336 +0000 UTC m=+65.210311182 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5dca0cc5-6589-4585-978d-ca12d129c1a0-registry-tls") pod "image-registry-8566d48d94-7ksk8" (UID: "5dca0cc5-6589-4585-978d-ca12d129c1a0") : secret "image-registry-tls" not found
Sep 19 08:21:04.833821 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:21:04.833636    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/61a85fdc-d1d2-4a56-a247-968786b293a5-metrics-tls podName:61a85fdc-d1d2-4a56-a247-968786b293a5 nodeName:}" failed. No retries permitted until 2026-09-19 08:21:20.833627283 +0000 UTC m=+65.210337125 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/61a85fdc-d1d2-4a56-a247-968786b293a5-metrics-tls") pod "dns-default-ldghd" (UID: "61a85fdc-d1d2-4a56-a247-968786b293a5") : secret "dns-default-metrics-tls" not found
Sep 19 08:21:05.310324 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:05.310264    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:21:05.368287 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:05.368254    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:21:06.961019 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:06.960983    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-mdm6c"]
Sep 19 08:21:06.971547 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:06.971525    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-mdm6c"
Sep 19 08:21:06.975810 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:06.974526    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-mdm6c"]
Sep 19 08:21:06.975810 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:06.975421    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 19 08:21:06.975810 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:06.975421    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 19 08:21:06.976276 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:06.976256    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Sep 19 08:21:06.976357 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:06.976319    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Sep 19 08:21:06.976691 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:06.976654    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-qfwb5\""
Sep 19 08:21:07.146481 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:07.146441    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/6f5aff65-5f00-4cbe-bb6f-f6139743adac-telemetry-config\") pod \"cluster-monitoring-operator-847b8bd8d7-mdm6c\" (UID: \"6f5aff65-5f00-4cbe-bb6f-f6139743adac\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-mdm6c"
Sep 19 08:21:07.146481 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:07.146484    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kvlvs\" (UniqueName: \"kubernetes.io/projected/6f5aff65-5f00-4cbe-bb6f-f6139743adac-kube-api-access-kvlvs\") pod \"cluster-monitoring-operator-847b8bd8d7-mdm6c\" (UID: \"6f5aff65-5f00-4cbe-bb6f-f6139743adac\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-mdm6c"
Sep 19 08:21:07.146729 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:07.146546    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6f5aff65-5f00-4cbe-bb6f-f6139743adac-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-847b8bd8d7-mdm6c\" (UID: \"6f5aff65-5f00-4cbe-bb6f-f6139743adac\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-mdm6c"
Sep 19 08:21:07.188907 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:07.188872    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:21:07.247064 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:07.246986    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/6f5aff65-5f00-4cbe-bb6f-f6139743adac-telemetry-config\") pod \"cluster-monitoring-operator-847b8bd8d7-mdm6c\" (UID: \"6f5aff65-5f00-4cbe-bb6f-f6139743adac\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-mdm6c"
Sep 19 08:21:07.247064 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:07.247024    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kvlvs\" (UniqueName: \"kubernetes.io/projected/6f5aff65-5f00-4cbe-bb6f-f6139743adac-kube-api-access-kvlvs\") pod \"cluster-monitoring-operator-847b8bd8d7-mdm6c\" (UID: \"6f5aff65-5f00-4cbe-bb6f-f6139743adac\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-mdm6c"
Sep 19 08:21:07.247064 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:07.247062    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6f5aff65-5f00-4cbe-bb6f-f6139743adac-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-847b8bd8d7-mdm6c\" (UID: \"6f5aff65-5f00-4cbe-bb6f-f6139743adac\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-mdm6c"
Sep 19 08:21:07.247267 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:21:07.247172    2578 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 19 08:21:07.247267 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:21:07.247258    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f5aff65-5f00-4cbe-bb6f-f6139743adac-cluster-monitoring-operator-tls podName:6f5aff65-5f00-4cbe-bb6f-f6139743adac nodeName:}" failed. No retries permitted until 2026-09-19 08:21:07.747239389 +0000 UTC m=+52.123949239 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/6f5aff65-5f00-4cbe-bb6f-f6139743adac-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-847b8bd8d7-mdm6c" (UID: "6f5aff65-5f00-4cbe-bb6f-f6139743adac") : secret "cluster-monitoring-operator-tls" not found
Sep 19 08:21:07.247702 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:07.247684    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/6f5aff65-5f00-4cbe-bb6f-f6139743adac-telemetry-config\") pod \"cluster-monitoring-operator-847b8bd8d7-mdm6c\" (UID: \"6f5aff65-5f00-4cbe-bb6f-f6139743adac\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-mdm6c"
Sep 19 08:21:07.259341 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:07.259313    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kvlvs\" (UniqueName: \"kubernetes.io/projected/6f5aff65-5f00-4cbe-bb6f-f6139743adac-kube-api-access-kvlvs\") pod \"cluster-monitoring-operator-847b8bd8d7-mdm6c\" (UID: \"6f5aff65-5f00-4cbe-bb6f-f6139743adac\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-mdm6c"
Sep 19 08:21:07.289267 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:07.289228    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:21:07.749649 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:07.749600    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6f5aff65-5f00-4cbe-bb6f-f6139743adac-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-847b8bd8d7-mdm6c\" (UID: \"6f5aff65-5f00-4cbe-bb6f-f6139743adac\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-mdm6c"
Sep 19 08:21:07.749850 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:21:07.749741    2578 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 19 08:21:07.749850 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:21:07.749817    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f5aff65-5f00-4cbe-bb6f-f6139743adac-cluster-monitoring-operator-tls podName:6f5aff65-5f00-4cbe-bb6f-f6139743adac nodeName:}" failed. No retries permitted until 2026-09-19 08:21:08.749803308 +0000 UTC m=+53.126513147 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/6f5aff65-5f00-4cbe-bb6f-f6139743adac-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-847b8bd8d7-mdm6c" (UID: "6f5aff65-5f00-4cbe-bb6f-f6139743adac") : secret "cluster-monitoring-operator-tls" not found
Sep 19 08:21:08.756227 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:08.756125    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6f5aff65-5f00-4cbe-bb6f-f6139743adac-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-847b8bd8d7-mdm6c\" (UID: \"6f5aff65-5f00-4cbe-bb6f-f6139743adac\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-mdm6c"
Sep 19 08:21:08.756617 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:21:08.756313    2578 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 19 08:21:08.756617 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:21:08.756386    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f5aff65-5f00-4cbe-bb6f-f6139743adac-cluster-monitoring-operator-tls podName:6f5aff65-5f00-4cbe-bb6f-f6139743adac nodeName:}" failed. No retries permitted until 2026-09-19 08:21:10.756369614 +0000 UTC m=+55.133079453 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/6f5aff65-5f00-4cbe-bb6f-f6139743adac-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-847b8bd8d7-mdm6c" (UID: "6f5aff65-5f00-4cbe-bb6f-f6139743adac") : secret "cluster-monitoring-operator-tls" not found
Sep 19 08:21:09.661185 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:09.661145    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:21:09.721421 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:09.721385    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:21:10.038551 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:10.038520    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:21:10.144315 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:10.144267    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:21:10.772143 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:10.772111    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6f5aff65-5f00-4cbe-bb6f-f6139743adac-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-847b8bd8d7-mdm6c\" (UID: \"6f5aff65-5f00-4cbe-bb6f-f6139743adac\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-mdm6c"
Sep 19 08:21:10.772315 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:21:10.772252    2578 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 19 08:21:10.772356 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:21:10.772344    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f5aff65-5f00-4cbe-bb6f-f6139743adac-cluster-monitoring-operator-tls podName:6f5aff65-5f00-4cbe-bb6f-f6139743adac nodeName:}" failed. No retries permitted until 2026-09-19 08:21:14.772328539 +0000 UTC m=+59.149038393 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/6f5aff65-5f00-4cbe-bb6f-f6139743adac-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-847b8bd8d7-mdm6c" (UID: "6f5aff65-5f00-4cbe-bb6f-f6139743adac") : secret "cluster-monitoring-operator-tls" not found
Sep 19 08:21:11.635522 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:11.635493    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-dm5sf"]
Sep 19 08:21:11.637517 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:11.637497    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-dm5sf"
Sep 19 08:21:11.639744 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:11.639720    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 19 08:21:11.639939 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:11.639923    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-jw55w\""
Sep 19 08:21:11.640034 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:11.640017    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 19 08:21:11.640845 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:11.640829    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 19 08:21:11.640958 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:11.640830    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 19 08:21:11.647982 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:11.647962    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:21:11.654512 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:11.654493    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-dm5sf"]
Sep 19 08:21:11.733222 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:11.733190    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:21:11.779526 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:11.779501    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/220642c2-d6c2-4681-a58b-087ba432f53a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dm5sf\" (UID: \"220642c2-d6c2-4681-a58b-087ba432f53a\") " pod="openshift-insights/insights-runtime-extractor-dm5sf"
Sep 19 08:21:11.779625 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:11.779530    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2p25l\" (UniqueName: \"kubernetes.io/projected/220642c2-d6c2-4681-a58b-087ba432f53a-kube-api-access-2p25l\") pod \"insights-runtime-extractor-dm5sf\" (UID: \"220642c2-d6c2-4681-a58b-087ba432f53a\") " pod="openshift-insights/insights-runtime-extractor-dm5sf"
Sep 19 08:21:11.779625 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:11.779568    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/220642c2-d6c2-4681-a58b-087ba432f53a-data-volume\") pod \"insights-runtime-extractor-dm5sf\" (UID: \"220642c2-d6c2-4681-a58b-087ba432f53a\") " pod="openshift-insights/insights-runtime-extractor-dm5sf"
Sep 19 08:21:11.779625 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:11.779602    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/220642c2-d6c2-4681-a58b-087ba432f53a-crio-socket\") pod \"insights-runtime-extractor-dm5sf\" (UID: \"220642c2-d6c2-4681-a58b-087ba432f53a\") " pod="openshift-insights/insights-runtime-extractor-dm5sf"
Sep 19 08:21:11.779722 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:11.779651    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/220642c2-d6c2-4681-a58b-087ba432f53a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-dm5sf\" (UID: \"220642c2-d6c2-4681-a58b-087ba432f53a\") " pod="openshift-insights/insights-runtime-extractor-dm5sf"
Sep 19 08:21:11.880176 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:11.880126    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/220642c2-d6c2-4681-a58b-087ba432f53a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-dm5sf\" (UID: \"220642c2-d6c2-4681-a58b-087ba432f53a\") " pod="openshift-insights/insights-runtime-extractor-dm5sf"
Sep 19 08:21:11.880176 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:11.880186    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/220642c2-d6c2-4681-a58b-087ba432f53a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dm5sf\" (UID: \"220642c2-d6c2-4681-a58b-087ba432f53a\") " pod="openshift-insights/insights-runtime-extractor-dm5sf"
Sep 19 08:21:11.880318 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:11.880202    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2p25l\" (UniqueName: \"kubernetes.io/projected/220642c2-d6c2-4681-a58b-087ba432f53a-kube-api-access-2p25l\") pod \"insights-runtime-extractor-dm5sf\" (UID: \"220642c2-d6c2-4681-a58b-087ba432f53a\") " pod="openshift-insights/insights-runtime-extractor-dm5sf"
Sep 19 08:21:11.880318 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:11.880234    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/220642c2-d6c2-4681-a58b-087ba432f53a-data-volume\") pod \"insights-runtime-extractor-dm5sf\" (UID: \"220642c2-d6c2-4681-a58b-087ba432f53a\") " pod="openshift-insights/insights-runtime-extractor-dm5sf"
Sep 19 08:21:11.880318 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:21:11.880246    2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 19 08:21:11.880318 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:11.880261    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/220642c2-d6c2-4681-a58b-087ba432f53a-crio-socket\") pod \"insights-runtime-extractor-dm5sf\" (UID: \"220642c2-d6c2-4681-a58b-087ba432f53a\") " pod="openshift-insights/insights-runtime-extractor-dm5sf"
Sep 19 08:21:11.880318 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:21:11.880279    2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 19 08:21:11.880464 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:21:11.880331    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/220642c2-d6c2-4681-a58b-087ba432f53a-kube-rbac-proxy-cm podName:220642c2-d6c2-4681-a58b-087ba432f53a nodeName:}" failed. No retries permitted until 2026-09-19 08:21:12.380294273 +0000 UTC m=+56.757004112 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/220642c2-d6c2-4681-a58b-087ba432f53a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-dm5sf" (UID: "220642c2-d6c2-4681-a58b-087ba432f53a") : configmap "kube-rbac-proxy" not found
Sep 19 08:21:11.880464 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:21:11.880355    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/220642c2-d6c2-4681-a58b-087ba432f53a-insights-runtime-extractor-tls podName:220642c2-d6c2-4681-a58b-087ba432f53a nodeName:}" failed. No retries permitted until 2026-09-19 08:21:12.380341619 +0000 UTC m=+56.757051458 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/220642c2-d6c2-4681-a58b-087ba432f53a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-dm5sf" (UID: "220642c2-d6c2-4681-a58b-087ba432f53a") : secret "insights-runtime-extractor-tls" not found
Sep 19 08:21:11.880464 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:11.880365    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/220642c2-d6c2-4681-a58b-087ba432f53a-crio-socket\") pod \"insights-runtime-extractor-dm5sf\" (UID: \"220642c2-d6c2-4681-a58b-087ba432f53a\") " pod="openshift-insights/insights-runtime-extractor-dm5sf"
Sep 19 08:21:11.880673 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:11.880655    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/220642c2-d6c2-4681-a58b-087ba432f53a-data-volume\") pod \"insights-runtime-extractor-dm5sf\" (UID: \"220642c2-d6c2-4681-a58b-087ba432f53a\") " pod="openshift-insights/insights-runtime-extractor-dm5sf"
Sep 19 08:21:11.889028 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:11.888986    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2p25l\" (UniqueName: \"kubernetes.io/projected/220642c2-d6c2-4681-a58b-087ba432f53a-kube-api-access-2p25l\") pod \"insights-runtime-extractor-dm5sf\" (UID: \"220642c2-d6c2-4681-a58b-087ba432f53a\") " pod="openshift-insights/insights-runtime-extractor-dm5sf"
Sep 19 08:21:12.383443 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:12.383412    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/220642c2-d6c2-4681-a58b-087ba432f53a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-dm5sf\" (UID: \"220642c2-d6c2-4681-a58b-087ba432f53a\") " pod="openshift-insights/insights-runtime-extractor-dm5sf"
Sep 19 08:21:12.383602 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:12.383453    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/220642c2-d6c2-4681-a58b-087ba432f53a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dm5sf\" (UID: \"220642c2-d6c2-4681-a58b-087ba432f53a\") " pod="openshift-insights/insights-runtime-extractor-dm5sf"
Sep 19 08:21:12.383602 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:21:12.383519    2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 19 08:21:12.383602 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:21:12.383585    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/220642c2-d6c2-4681-a58b-087ba432f53a-kube-rbac-proxy-cm podName:220642c2-d6c2-4681-a58b-087ba432f53a nodeName:}" failed. No retries permitted until 2026-09-19 08:21:13.383570326 +0000 UTC m=+57.760280165 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/220642c2-d6c2-4681-a58b-087ba432f53a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-dm5sf" (UID: "220642c2-d6c2-4681-a58b-087ba432f53a") : configmap "kube-rbac-proxy" not found
Sep 19 08:21:12.383602 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:21:12.383585    2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 19 08:21:12.383743 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:21:12.383633    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/220642c2-d6c2-4681-a58b-087ba432f53a-insights-runtime-extractor-tls podName:220642c2-d6c2-4681-a58b-087ba432f53a nodeName:}" failed. No retries permitted until 2026-09-19 08:21:13.383622378 +0000 UTC m=+57.760332217 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/220642c2-d6c2-4681-a58b-087ba432f53a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-dm5sf" (UID: "220642c2-d6c2-4681-a58b-087ba432f53a") : secret "insights-runtime-extractor-tls" not found
Sep 19 08:21:13.389781 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:13.389755    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-lqdbb_637bbfb9-45ba-481d-b9fe-87f60051712c/dns-node-resolver/0.log"
Sep 19 08:21:13.390191 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:13.390049    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/220642c2-d6c2-4681-a58b-087ba432f53a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-dm5sf\" (UID: \"220642c2-d6c2-4681-a58b-087ba432f53a\") " pod="openshift-insights/insights-runtime-extractor-dm5sf"
Sep 19 08:21:13.390191 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:21:13.390132    2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 19 08:21:13.390191 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:13.390161    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/220642c2-d6c2-4681-a58b-087ba432f53a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dm5sf\" (UID: \"220642c2-d6c2-4681-a58b-087ba432f53a\") " pod="openshift-insights/insights-runtime-extractor-dm5sf"
Sep 19 08:21:13.390191 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:21:13.390190    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/220642c2-d6c2-4681-a58b-087ba432f53a-kube-rbac-proxy-cm podName:220642c2-d6c2-4681-a58b-087ba432f53a nodeName:}" failed. No retries permitted until 2026-09-19 08:21:15.390175895 +0000 UTC m=+59.766885739 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/220642c2-d6c2-4681-a58b-087ba432f53a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-dm5sf" (UID: "220642c2-d6c2-4681-a58b-087ba432f53a") : configmap "kube-rbac-proxy" not found
Sep 19 08:21:13.390399 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:21:13.390242    2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 19 08:21:13.390399 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:21:13.390285    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/220642c2-d6c2-4681-a58b-087ba432f53a-insights-runtime-extractor-tls podName:220642c2-d6c2-4681-a58b-087ba432f53a nodeName:}" failed. No retries permitted until 2026-09-19 08:21:15.390274507 +0000 UTC m=+59.766984346 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/220642c2-d6c2-4681-a58b-087ba432f53a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-dm5sf" (UID: "220642c2-d6c2-4681-a58b-087ba432f53a") : secret "insights-runtime-extractor-tls" not found
Sep 19 08:21:13.789083 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:13.789058    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-hb9xj_65dc9622-6c75-457d-ad22-6841b2673aae/node-ca/0.log"
Sep 19 08:21:13.968161 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:13.968127    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:21:14.070171 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:14.070097    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:21:14.161823 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:14.161788    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:21:14.589255 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:14.589225    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-6thbg_3135bdcd-43c1-4e88-9ec1-0a9a5d2ecdb0/serve-healthcheck-canary/0.log"
Sep 19 08:21:14.803516 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:14.803477    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6f5aff65-5f00-4cbe-bb6f-f6139743adac-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-847b8bd8d7-mdm6c\" (UID: \"6f5aff65-5f00-4cbe-bb6f-f6139743adac\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-mdm6c"
Sep 19 08:21:14.803678 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:21:14.803615    2578 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 19 08:21:14.803678 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:21:14.803677    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f5aff65-5f00-4cbe-bb6f-f6139743adac-cluster-monitoring-operator-tls podName:6f5aff65-5f00-4cbe-bb6f-f6139743adac nodeName:}" failed. No retries permitted until 2026-09-19 08:21:22.803663922 +0000 UTC m=+67.180373765 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/6f5aff65-5f00-4cbe-bb6f-f6139743adac-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-847b8bd8d7-mdm6c" (UID: "6f5aff65-5f00-4cbe-bb6f-f6139743adac") : secret "cluster-monitoring-operator-tls" not found
Sep 19 08:21:15.009115 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:15.009073    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:21:15.065634 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:15.065598    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:21:15.275330 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:15.275247    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-zd2zp"
Sep 19 08:21:15.408589 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:15.408536    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/220642c2-d6c2-4681-a58b-087ba432f53a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-dm5sf\" (UID: \"220642c2-d6c2-4681-a58b-087ba432f53a\") " pod="openshift-insights/insights-runtime-extractor-dm5sf"
Sep 19 08:21:15.408771 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:15.408606    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/220642c2-d6c2-4681-a58b-087ba432f53a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dm5sf\" (UID: \"220642c2-d6c2-4681-a58b-087ba432f53a\") " pod="openshift-insights/insights-runtime-extractor-dm5sf"
Sep 19 08:21:15.408771 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:21:15.408732    2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 19 08:21:15.408771 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:21:15.408750    2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 19 08:21:15.408941 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:21:15.408799    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/220642c2-d6c2-4681-a58b-087ba432f53a-insights-runtime-extractor-tls podName:220642c2-d6c2-4681-a58b-087ba432f53a nodeName:}" failed. No retries permitted until 2026-09-19 08:21:19.408779952 +0000 UTC m=+63.785489794 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/220642c2-d6c2-4681-a58b-087ba432f53a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-dm5sf" (UID: "220642c2-d6c2-4681-a58b-087ba432f53a") : secret "insights-runtime-extractor-tls" not found
Sep 19 08:21:15.408941 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:21:15.408848    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/220642c2-d6c2-4681-a58b-087ba432f53a-kube-rbac-proxy-cm podName:220642c2-d6c2-4681-a58b-087ba432f53a nodeName:}" failed. No retries permitted until 2026-09-19 08:21:19.408828552 +0000 UTC m=+63.785538411 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/220642c2-d6c2-4681-a58b-087ba432f53a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-dm5sf" (UID: "220642c2-d6c2-4681-a58b-087ba432f53a") : configmap "kube-rbac-proxy" not found
Sep 19 08:21:15.439567 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:15.439529    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:21:15.540032 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:15.539945    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:21:16.980562 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:16.980507    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7cbc45fddc-2bdxn"]
Sep 19 08:21:16.983197 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:16.983179    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7cbc45fddc-2bdxn"
Sep 19 08:21:16.986308 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:16.986277    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-4dfml\""
Sep 19 08:21:16.992007 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:16.991982    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7cbc45fddc-2bdxn"]
Sep 19 08:21:17.021781 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:17.021720    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-66msp\" (UniqueName: \"kubernetes.io/projected/54012075-1fb8-4ac5-9b11-74d546b4216f-kube-api-access-66msp\") pod \"network-check-source-7cbc45fddc-2bdxn\" (UID: \"54012075-1fb8-4ac5-9b11-74d546b4216f\") " pod="openshift-network-diagnostics/network-check-source-7cbc45fddc-2bdxn"
Sep 19 08:21:17.081677 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:17.081641    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-tjr4s"]
Sep 19 08:21:17.083490 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:17.083474    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-tjr4s"
Sep 19 08:21:17.085816 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:17.085790    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Sep 19 08:21:17.085968 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:17.085791    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Sep 19 08:21:17.086077 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:17.086059    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Sep 19 08:21:17.086179 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:17.086060    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-w62tl\""
Sep 19 08:21:17.086291 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:17.086278    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Sep 19 08:21:17.098326 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:17.098284    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-tjr4s"]
Sep 19 08:21:17.122231 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:17.122191    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ebaf628e-811b-4dbc-a814-f257d77db389-serving-cert\") pod \"kube-storage-version-migrator-operator-55889f7b47-tjr4s\" (UID: \"ebaf628e-811b-4dbc-a814-f257d77db389\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-tjr4s"
Sep 19 08:21:17.122231 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:17.122229    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ebaf628e-811b-4dbc-a814-f257d77db389-config\") pod \"kube-storage-version-migrator-operator-55889f7b47-tjr4s\" (UID: \"ebaf628e-811b-4dbc-a814-f257d77db389\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-tjr4s"
Sep 19 08:21:17.122452 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:17.122333    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jfxz9\" (UniqueName: \"kubernetes.io/projected/ebaf628e-811b-4dbc-a814-f257d77db389-kube-api-access-jfxz9\") pod \"kube-storage-version-migrator-operator-55889f7b47-tjr4s\" (UID: \"ebaf628e-811b-4dbc-a814-f257d77db389\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-tjr4s"
Sep 19 08:21:17.122452 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:17.122390    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-66msp\" (UniqueName: \"kubernetes.io/projected/54012075-1fb8-4ac5-9b11-74d546b4216f-kube-api-access-66msp\") pod \"network-check-source-7cbc45fddc-2bdxn\" (UID: \"54012075-1fb8-4ac5-9b11-74d546b4216f\") " pod="openshift-network-diagnostics/network-check-source-7cbc45fddc-2bdxn"
Sep 19 08:21:17.129799 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:17.129757    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-66msp\" (UniqueName: \"kubernetes.io/projected/54012075-1fb8-4ac5-9b11-74d546b4216f-kube-api-access-66msp\") pod \"network-check-source-7cbc45fddc-2bdxn\" (UID: \"54012075-1fb8-4ac5-9b11-74d546b4216f\") " pod="openshift-network-diagnostics/network-check-source-7cbc45fddc-2bdxn"
Sep 19 08:21:17.187532 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:17.187496    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-wcrh2"]
Sep 19 08:21:17.189447 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:17.189429    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-wcrh2"
Sep 19 08:21:17.191694 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:17.191670    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Sep 19 08:21:17.192874 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:17.192851    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-8dhw6\""
Sep 19 08:21:17.192874 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:17.192857    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Sep 19 08:21:17.193158 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:17.193141    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Sep 19 08:21:17.203948 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:17.203920    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-wcrh2"]
Sep 19 08:21:17.223118 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:17.223084    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t28sb\" (UniqueName: \"kubernetes.io/projected/1b6f77bc-f7af-4355-9bb7-3527d89764d8-kube-api-access-t28sb\") pod \"cluster-samples-operator-685d7d68b5-wcrh2\" (UID: \"1b6f77bc-f7af-4355-9bb7-3527d89764d8\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-wcrh2"
Sep 19 08:21:17.223269 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:17.223126    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/1b6f77bc-f7af-4355-9bb7-3527d89764d8-samples-operator-tls\") pod \"cluster-samples-operator-685d7d68b5-wcrh2\" (UID: \"1b6f77bc-f7af-4355-9bb7-3527d89764d8\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-wcrh2"
Sep 19 08:21:17.223269 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:17.223185    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jfxz9\" (UniqueName: \"kubernetes.io/projected/ebaf628e-811b-4dbc-a814-f257d77db389-kube-api-access-jfxz9\") pod \"kube-storage-version-migrator-operator-55889f7b47-tjr4s\" (UID: \"ebaf628e-811b-4dbc-a814-f257d77db389\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-tjr4s"
Sep 19 08:21:17.223419 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:17.223273    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ebaf628e-811b-4dbc-a814-f257d77db389-serving-cert\") pod \"kube-storage-version-migrator-operator-55889f7b47-tjr4s\" (UID: \"ebaf628e-811b-4dbc-a814-f257d77db389\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-tjr4s"
Sep 19 08:21:17.223419 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:17.223294    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ebaf628e-811b-4dbc-a814-f257d77db389-config\") pod \"kube-storage-version-migrator-operator-55889f7b47-tjr4s\" (UID: \"ebaf628e-811b-4dbc-a814-f257d77db389\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-tjr4s"
Sep 19 08:21:17.223830 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:17.223812    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ebaf628e-811b-4dbc-a814-f257d77db389-config\") pod \"kube-storage-version-migrator-operator-55889f7b47-tjr4s\" (UID: \"ebaf628e-811b-4dbc-a814-f257d77db389\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-tjr4s"
Sep 19 08:21:17.225416 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:17.225392    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ebaf628e-811b-4dbc-a814-f257d77db389-serving-cert\") pod \"kube-storage-version-migrator-operator-55889f7b47-tjr4s\" (UID: \"ebaf628e-811b-4dbc-a814-f257d77db389\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-tjr4s"
Sep 19 08:21:17.230974 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:17.230918    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jfxz9\" (UniqueName: \"kubernetes.io/projected/ebaf628e-811b-4dbc-a814-f257d77db389-kube-api-access-jfxz9\") pod \"kube-storage-version-migrator-operator-55889f7b47-tjr4s\" (UID: \"ebaf628e-811b-4dbc-a814-f257d77db389\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-tjr4s"
Sep 19 08:21:17.292237 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:17.292196    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7cbc45fddc-2bdxn"
Sep 19 08:21:17.324555 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:17.324503    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/1b6f77bc-f7af-4355-9bb7-3527d89764d8-samples-operator-tls\") pod \"cluster-samples-operator-685d7d68b5-wcrh2\" (UID: \"1b6f77bc-f7af-4355-9bb7-3527d89764d8\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-wcrh2"
Sep 19 08:21:17.324711 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:17.324642    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t28sb\" (UniqueName: \"kubernetes.io/projected/1b6f77bc-f7af-4355-9bb7-3527d89764d8-kube-api-access-t28sb\") pod \"cluster-samples-operator-685d7d68b5-wcrh2\" (UID: \"1b6f77bc-f7af-4355-9bb7-3527d89764d8\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-wcrh2"
Sep 19 08:21:17.324711 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:21:17.324675    2578 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 19 08:21:17.324777 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:21:17.324761    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1b6f77bc-f7af-4355-9bb7-3527d89764d8-samples-operator-tls podName:1b6f77bc-f7af-4355-9bb7-3527d89764d8 nodeName:}" failed. No retries permitted until 2026-09-19 08:21:17.824744709 +0000 UTC m=+62.201454566 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/1b6f77bc-f7af-4355-9bb7-3527d89764d8-samples-operator-tls") pod "cluster-samples-operator-685d7d68b5-wcrh2" (UID: "1b6f77bc-f7af-4355-9bb7-3527d89764d8") : secret "samples-operator-tls" not found
Sep 19 08:21:17.333436 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:17.333406    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t28sb\" (UniqueName: \"kubernetes.io/projected/1b6f77bc-f7af-4355-9bb7-3527d89764d8-kube-api-access-t28sb\") pod \"cluster-samples-operator-685d7d68b5-wcrh2\" (UID: \"1b6f77bc-f7af-4355-9bb7-3527d89764d8\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-wcrh2"
Sep 19 08:21:17.392270 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:17.392236    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-tjr4s"
Sep 19 08:21:17.559202 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:17.559178    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7cbc45fddc-2bdxn"]
Sep 19 08:21:17.561451 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:21:17.561413    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54012075_1fb8_4ac5_9b11_74d546b4216f.slice/crio-5d4f934bdc7cdee9fff37890c38578588edc747e95e68f26b8fb5eadee74f077 WatchSource:0}: Error finding container 5d4f934bdc7cdee9fff37890c38578588edc747e95e68f26b8fb5eadee74f077: Status 404 returned error can't find the container with id 5d4f934bdc7cdee9fff37890c38578588edc747e95e68f26b8fb5eadee74f077
Sep 19 08:21:17.667481 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:17.667438    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-tjr4s"]
Sep 19 08:21:17.670099 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:21:17.670065    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podebaf628e_811b_4dbc_a814_f257d77db389.slice/crio-034f105175bba83759d1bfa0322daa8c216d927dc85f10dee1befd7a5aa9f4ba WatchSource:0}: Error finding container 034f105175bba83759d1bfa0322daa8c216d927dc85f10dee1befd7a5aa9f4ba: Status 404 returned error can't find the container with id 034f105175bba83759d1bfa0322daa8c216d927dc85f10dee1befd7a5aa9f4ba
Sep 19 08:21:17.827813 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:17.827730    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/1b6f77bc-f7af-4355-9bb7-3527d89764d8-samples-operator-tls\") pod \"cluster-samples-operator-685d7d68b5-wcrh2\" (UID: \"1b6f77bc-f7af-4355-9bb7-3527d89764d8\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-wcrh2"
Sep 19 08:21:17.827945 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:21:17.827870    2578 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 19 08:21:17.827945 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:21:17.827934    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1b6f77bc-f7af-4355-9bb7-3527d89764d8-samples-operator-tls podName:1b6f77bc-f7af-4355-9bb7-3527d89764d8 nodeName:}" failed. No retries permitted until 2026-09-19 08:21:18.827919497 +0000 UTC m=+63.204629341 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/1b6f77bc-f7af-4355-9bb7-3527d89764d8-samples-operator-tls") pod "cluster-samples-operator-685d7d68b5-wcrh2" (UID: "1b6f77bc-f7af-4355-9bb7-3527d89764d8") : secret "samples-operator-tls" not found
Sep 19 08:21:18.376308 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:18.376260    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-tjr4s" event={"ID":"ebaf628e-811b-4dbc-a814-f257d77db389","Type":"ContainerStarted","Data":"034f105175bba83759d1bfa0322daa8c216d927dc85f10dee1befd7a5aa9f4ba"}
Sep 19 08:21:18.378177 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:18.378151    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7cbc45fddc-2bdxn" event={"ID":"54012075-1fb8-4ac5-9b11-74d546b4216f","Type":"ContainerStarted","Data":"e1acc3f0b56eb347aa6b2575441a1d55243f4c628d67275f300948020fa01b13"}
Sep 19 08:21:18.378323 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:18.378198    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7cbc45fddc-2bdxn" event={"ID":"54012075-1fb8-4ac5-9b11-74d546b4216f","Type":"ContainerStarted","Data":"5d4f934bdc7cdee9fff37890c38578588edc747e95e68f26b8fb5eadee74f077"}
Sep 19 08:21:18.392678 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:18.392628    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7cbc45fddc-2bdxn" podStartSLOduration=2.392613877 podStartE2EDuration="2.392613877s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 08:21:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 08:21:18.392395179 +0000 UTC m=+62.769105083" watchObservedRunningTime="2026-09-19 08:21:18.392613877 +0000 UTC m=+62.769323740"
Sep 19 08:21:18.835785 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:18.835741    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/1b6f77bc-f7af-4355-9bb7-3527d89764d8-samples-operator-tls\") pod \"cluster-samples-operator-685d7d68b5-wcrh2\" (UID: \"1b6f77bc-f7af-4355-9bb7-3527d89764d8\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-wcrh2"
Sep 19 08:21:18.835982 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:21:18.835912    2578 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 19 08:21:18.836061 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:21:18.835989    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1b6f77bc-f7af-4355-9bb7-3527d89764d8-samples-operator-tls podName:1b6f77bc-f7af-4355-9bb7-3527d89764d8 nodeName:}" failed. No retries permitted until 2026-09-19 08:21:20.835972368 +0000 UTC m=+65.212682224 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/1b6f77bc-f7af-4355-9bb7-3527d89764d8-samples-operator-tls") pod "cluster-samples-operator-685d7d68b5-wcrh2" (UID: "1b6f77bc-f7af-4355-9bb7-3527d89764d8") : secret "samples-operator-tls" not found
Sep 19 08:21:18.998612 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:18.998572    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:21:19.383650 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:19.383610    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-tjr4s" event={"ID":"ebaf628e-811b-4dbc-a814-f257d77db389","Type":"ContainerStarted","Data":"6af39bfa22ba16e3c74fae3f073c7739f7628f86091d8d10d7a56116d4c6e143"}
Sep 19 08:21:19.399973 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:19.399918    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-tjr4s" podStartSLOduration=0.892325951 podStartE2EDuration="2.399896977s" totalImagesPullingTime="1.507571026s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 08:21:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 08:21:19.397984059 +0000 UTC m=+63.774693920" watchObservedRunningTime="2026-09-19 08:21:19.399896977 +0000 UTC m=+63.776606841"
Sep 19 08:21:19.440286 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:19.440213    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/220642c2-d6c2-4681-a58b-087ba432f53a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-dm5sf\" (UID: \"220642c2-d6c2-4681-a58b-087ba432f53a\") " pod="openshift-insights/insights-runtime-extractor-dm5sf"
Sep 19 08:21:19.440286 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:19.440266    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/220642c2-d6c2-4681-a58b-087ba432f53a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dm5sf\" (UID: \"220642c2-d6c2-4681-a58b-087ba432f53a\") " pod="openshift-insights/insights-runtime-extractor-dm5sf"
Sep 19 08:21:19.440509 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:21:19.440366    2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 19 08:21:19.440509 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:21:19.440442    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/220642c2-d6c2-4681-a58b-087ba432f53a-kube-rbac-proxy-cm podName:220642c2-d6c2-4681-a58b-087ba432f53a nodeName:}" failed. No retries permitted until 2026-09-19 08:21:27.440421012 +0000 UTC m=+71.817130874 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/220642c2-d6c2-4681-a58b-087ba432f53a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-dm5sf" (UID: "220642c2-d6c2-4681-a58b-087ba432f53a") : configmap "kube-rbac-proxy" not found
Sep 19 08:21:19.440795 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:21:19.440777    2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 19 08:21:19.440853 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:21:19.440828    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/220642c2-d6c2-4681-a58b-087ba432f53a-insights-runtime-extractor-tls podName:220642c2-d6c2-4681-a58b-087ba432f53a nodeName:}" failed. No retries permitted until 2026-09-19 08:21:27.440813062 +0000 UTC m=+71.817522909 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/220642c2-d6c2-4681-a58b-087ba432f53a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-dm5sf" (UID: "220642c2-d6c2-4681-a58b-087ba432f53a") : secret "insights-runtime-extractor-tls" not found
Sep 19 08:21:20.849666 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:20.849621    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5dca0cc5-6589-4585-978d-ca12d129c1a0-registry-tls\") pod \"image-registry-8566d48d94-7ksk8\" (UID: \"5dca0cc5-6589-4585-978d-ca12d129c1a0\") " pod="openshift-image-registry/image-registry-8566d48d94-7ksk8"
Sep 19 08:21:20.850088 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:20.849689    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/61a85fdc-d1d2-4a56-a247-968786b293a5-metrics-tls\") pod \"dns-default-ldghd\" (UID: \"61a85fdc-d1d2-4a56-a247-968786b293a5\") " pod="openshift-dns/dns-default-ldghd"
Sep 19 08:21:20.850088 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:20.849729    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/1b6f77bc-f7af-4355-9bb7-3527d89764d8-samples-operator-tls\") pod \"cluster-samples-operator-685d7d68b5-wcrh2\" (UID: \"1b6f77bc-f7af-4355-9bb7-3527d89764d8\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-wcrh2"
Sep 19 08:21:20.850088 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:21:20.849770    2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 19 08:21:20.850088 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:21:20.849789    2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-8566d48d94-7ksk8: secret "image-registry-tls" not found
Sep 19 08:21:20.850088 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:21:20.849848    2578 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 19 08:21:20.850088 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:21:20.849856    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5dca0cc5-6589-4585-978d-ca12d129c1a0-registry-tls podName:5dca0cc5-6589-4585-978d-ca12d129c1a0 nodeName:}" failed. No retries permitted until 2026-09-19 08:21:52.849838298 +0000 UTC m=+97.226548153 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5dca0cc5-6589-4585-978d-ca12d129c1a0-registry-tls") pod "image-registry-8566d48d94-7ksk8" (UID: "5dca0cc5-6589-4585-978d-ca12d129c1a0") : secret "image-registry-tls" not found
Sep 19 08:21:20.850088 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:21:20.849909    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1b6f77bc-f7af-4355-9bb7-3527d89764d8-samples-operator-tls podName:1b6f77bc-f7af-4355-9bb7-3527d89764d8 nodeName:}" failed. No retries permitted until 2026-09-19 08:21:24.849892984 +0000 UTC m=+69.226602845 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/1b6f77bc-f7af-4355-9bb7-3527d89764d8-samples-operator-tls") pod "cluster-samples-operator-685d7d68b5-wcrh2" (UID: "1b6f77bc-f7af-4355-9bb7-3527d89764d8") : secret "samples-operator-tls" not found
Sep 19 08:21:20.850088 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:21:20.849981    2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 19 08:21:20.850088 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:21:20.850048    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/61a85fdc-d1d2-4a56-a247-968786b293a5-metrics-tls podName:61a85fdc-d1d2-4a56-a247-968786b293a5 nodeName:}" failed. No retries permitted until 2026-09-19 08:21:52.850033657 +0000 UTC m=+97.226743500 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/61a85fdc-d1d2-4a56-a247-968786b293a5-metrics-tls") pod "dns-default-ldghd" (UID: "61a85fdc-d1d2-4a56-a247-968786b293a5") : secret "dns-default-metrics-tls" not found
Sep 19 08:21:20.915293 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:20.915251    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:21:21.005049 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:21.005010    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:21:21.106067 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:21.105995    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:21:21.679076 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:21.679041    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:21:21.740191 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:21.740159    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:21:21.856842 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:21.856802    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/79f6ab54-a988-4015-81c2-da35716eab52-metrics-certs\") pod \"network-metrics-daemon-md9ln\" (UID: \"79f6ab54-a988-4015-81c2-da35716eab52\") " pod="openshift-multus/network-metrics-daemon-md9ln"
Sep 19 08:21:21.857191 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:21:21.856946    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 19 08:21:21.857191 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:21:21.857016    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79f6ab54-a988-4015-81c2-da35716eab52-metrics-certs podName:79f6ab54-a988-4015-81c2-da35716eab52 nodeName:}" failed. No retries permitted until 2026-09-19 08:22:25.856999974 +0000 UTC m=+130.233709813 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/79f6ab54-a988-4015-81c2-da35716eab52-metrics-certs") pod "network-metrics-daemon-md9ln" (UID: "79f6ab54-a988-4015-81c2-da35716eab52") : secret "metrics-daemon-secret" not found
Sep 19 08:21:22.291390 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:22.291361    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:21:22.390021 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:22.389991    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:21:22.862609 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:22.862579    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6f5aff65-5f00-4cbe-bb6f-f6139743adac-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-847b8bd8d7-mdm6c\" (UID: \"6f5aff65-5f00-4cbe-bb6f-f6139743adac\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-mdm6c"
Sep 19 08:21:22.862950 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:21:22.862718    2578 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 19 08:21:22.862950 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:21:22.862773    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f5aff65-5f00-4cbe-bb6f-f6139743adac-cluster-monitoring-operator-tls podName:6f5aff65-5f00-4cbe-bb6f-f6139743adac nodeName:}" failed. No retries permitted until 2026-09-19 08:21:38.862760828 +0000 UTC m=+83.239470667 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/6f5aff65-5f00-4cbe-bb6f-f6139743adac-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-847b8bd8d7-mdm6c" (UID: "6f5aff65-5f00-4cbe-bb6f-f6139743adac") : secret "cluster-monitoring-operator-tls" not found
Sep 19 08:21:23.018402 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:23.018359    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:21:23.103214 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:23.103188    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:21:23.730697 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:23.730670    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:21:24.609159 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:24.609126    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-55f6fc7b86-nrcl8"]
Sep 19 08:21:24.611621 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:24.611606    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-55f6fc7b86-nrcl8"
Sep 19 08:21:24.613970 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:24.613945    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Sep 19 08:21:24.613970 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:24.613959    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-l99dt\""
Sep 19 08:21:24.614894 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:24.614879    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Sep 19 08:21:24.614942 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:24.614880    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Sep 19 08:21:24.614984 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:24.614944    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Sep 19 08:21:24.621951 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:24.621927    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-55f6fc7b86-nrcl8"]
Sep 19 08:21:24.648636 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:24.648611    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:21:24.678285 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:24.678263    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/b79ee59a-1e8a-47e8-a143-d7cb4dcf988d-signing-cabundle\") pod \"service-ca-55f6fc7b86-nrcl8\" (UID: \"b79ee59a-1e8a-47e8-a143-d7cb4dcf988d\") " pod="openshift-service-ca/service-ca-55f6fc7b86-nrcl8"
Sep 19 08:21:24.678376 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:24.678334    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/b79ee59a-1e8a-47e8-a143-d7cb4dcf988d-signing-key\") pod \"service-ca-55f6fc7b86-nrcl8\" (UID: \"b79ee59a-1e8a-47e8-a143-d7cb4dcf988d\") " pod="openshift-service-ca/service-ca-55f6fc7b86-nrcl8"
Sep 19 08:21:24.678412 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:24.678394    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qbgc2\" (UniqueName: \"kubernetes.io/projected/b79ee59a-1e8a-47e8-a143-d7cb4dcf988d-kube-api-access-qbgc2\") pod \"service-ca-55f6fc7b86-nrcl8\" (UID: \"b79ee59a-1e8a-47e8-a143-d7cb4dcf988d\") " pod="openshift-service-ca/service-ca-55f6fc7b86-nrcl8"
Sep 19 08:21:24.708820 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:24.708788    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:21:24.778993 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:24.778969    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/b79ee59a-1e8a-47e8-a143-d7cb4dcf988d-signing-key\") pod \"service-ca-55f6fc7b86-nrcl8\" (UID: \"b79ee59a-1e8a-47e8-a143-d7cb4dcf988d\") " pod="openshift-service-ca/service-ca-55f6fc7b86-nrcl8"
Sep 19 08:21:24.779094 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:24.779017    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qbgc2\" (UniqueName: \"kubernetes.io/projected/b79ee59a-1e8a-47e8-a143-d7cb4dcf988d-kube-api-access-qbgc2\") pod \"service-ca-55f6fc7b86-nrcl8\" (UID: \"b79ee59a-1e8a-47e8-a143-d7cb4dcf988d\") " pod="openshift-service-ca/service-ca-55f6fc7b86-nrcl8"
Sep 19 08:21:24.779094 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:24.779068    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/b79ee59a-1e8a-47e8-a143-d7cb4dcf988d-signing-cabundle\") pod \"service-ca-55f6fc7b86-nrcl8\" (UID: \"b79ee59a-1e8a-47e8-a143-d7cb4dcf988d\") " pod="openshift-service-ca/service-ca-55f6fc7b86-nrcl8"
Sep 19 08:21:24.779631 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:24.779615    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/b79ee59a-1e8a-47e8-a143-d7cb4dcf988d-signing-cabundle\") pod \"service-ca-55f6fc7b86-nrcl8\" (UID: \"b79ee59a-1e8a-47e8-a143-d7cb4dcf988d\") " pod="openshift-service-ca/service-ca-55f6fc7b86-nrcl8"
Sep 19 08:21:24.781419 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:24.781398    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/b79ee59a-1e8a-47e8-a143-d7cb4dcf988d-signing-key\") pod \"service-ca-55f6fc7b86-nrcl8\" (UID: \"b79ee59a-1e8a-47e8-a143-d7cb4dcf988d\") " pod="openshift-service-ca/service-ca-55f6fc7b86-nrcl8"
Sep 19 08:21:24.789109 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:24.789089    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qbgc2\" (UniqueName: \"kubernetes.io/projected/b79ee59a-1e8a-47e8-a143-d7cb4dcf988d-kube-api-access-qbgc2\") pod \"service-ca-55f6fc7b86-nrcl8\" (UID: \"b79ee59a-1e8a-47e8-a143-d7cb4dcf988d\") " pod="openshift-service-ca/service-ca-55f6fc7b86-nrcl8"
Sep 19 08:21:24.832316 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:24.832271    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:21:24.879602 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:24.879550    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/1b6f77bc-f7af-4355-9bb7-3527d89764d8-samples-operator-tls\") pod \"cluster-samples-operator-685d7d68b5-wcrh2\" (UID: \"1b6f77bc-f7af-4355-9bb7-3527d89764d8\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-wcrh2"
Sep 19 08:21:24.879729 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:21:24.879714    2578 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 19 08:21:24.879803 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:21:24.879793    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1b6f77bc-f7af-4355-9bb7-3527d89764d8-samples-operator-tls podName:1b6f77bc-f7af-4355-9bb7-3527d89764d8 nodeName:}" failed. No retries permitted until 2026-09-19 08:21:32.879774412 +0000 UTC m=+77.256484265 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/1b6f77bc-f7af-4355-9bb7-3527d89764d8-samples-operator-tls") pod "cluster-samples-operator-685d7d68b5-wcrh2" (UID: "1b6f77bc-f7af-4355-9bb7-3527d89764d8") : secret "samples-operator-tls" not found
Sep 19 08:21:24.920630 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:24.920591    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-55f6fc7b86-nrcl8"
Sep 19 08:21:25.189922 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:25.189847    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-55f6fc7b86-nrcl8"]
Sep 19 08:21:25.193018 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:21:25.192988    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb79ee59a_1e8a_47e8_a143_d7cb4dcf988d.slice/crio-c5909faa39e5de3bdc99e0b5c54b72c65d12fc78530ae199ecbe3b7a701e9ee7 WatchSource:0}: Error finding container c5909faa39e5de3bdc99e0b5c54b72c65d12fc78530ae199ecbe3b7a701e9ee7: Status 404 returned error can't find the container with id c5909faa39e5de3bdc99e0b5c54b72c65d12fc78530ae199ecbe3b7a701e9ee7
Sep 19 08:21:25.408914 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:25.408880    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-55f6fc7b86-nrcl8" event={"ID":"b79ee59a-1e8a-47e8-a143-d7cb4dcf988d","Type":"ContainerStarted","Data":"c5909faa39e5de3bdc99e0b5c54b72c65d12fc78530ae199ecbe3b7a701e9ee7"}
Sep 19 08:21:25.690639 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:25.690600    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:21:27.415985 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:27.415948    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-55f6fc7b86-nrcl8" event={"ID":"b79ee59a-1e8a-47e8-a143-d7cb4dcf988d","Type":"ContainerStarted","Data":"479b62c9ef319e6cc1f5ecc662a766b85513bbe51d25a353eadbeb266c4c7e8a"}
Sep 19 08:21:27.435429 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:27.435384    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-55f6fc7b86-nrcl8" podStartSLOduration=1.8213598420000001 podStartE2EDuration="3.435368372s" totalImagesPullingTime="1.61400853s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 08:21:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 08:21:27.434738125 +0000 UTC m=+71.811448001" watchObservedRunningTime="2026-09-19 08:21:27.435368372 +0000 UTC m=+71.812078608"
Sep 19 08:21:27.503500 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:27.503469    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/220642c2-d6c2-4681-a58b-087ba432f53a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dm5sf\" (UID: \"220642c2-d6c2-4681-a58b-087ba432f53a\") " pod="openshift-insights/insights-runtime-extractor-dm5sf"
Sep 19 08:21:27.503651 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:27.503548    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/220642c2-d6c2-4681-a58b-087ba432f53a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-dm5sf\" (UID: \"220642c2-d6c2-4681-a58b-087ba432f53a\") " pod="openshift-insights/insights-runtime-extractor-dm5sf"
Sep 19 08:21:27.503651 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:21:27.503614    2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 19 08:21:27.503735 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:21:27.503682    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/220642c2-d6c2-4681-a58b-087ba432f53a-insights-runtime-extractor-tls podName:220642c2-d6c2-4681-a58b-087ba432f53a nodeName:}" failed. No retries permitted until 2026-09-19 08:21:43.503662831 +0000 UTC m=+87.880372685 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/220642c2-d6c2-4681-a58b-087ba432f53a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-dm5sf" (UID: "220642c2-d6c2-4681-a58b-087ba432f53a") : secret "insights-runtime-extractor-tls" not found
Sep 19 08:21:27.504172 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:27.504150    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/220642c2-d6c2-4681-a58b-087ba432f53a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-dm5sf\" (UID: \"220642c2-d6c2-4681-a58b-087ba432f53a\") " pod="openshift-insights/insights-runtime-extractor-dm5sf"
Sep 19 08:21:28.328338 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:28.328310    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-l6m8x"
Sep 19 08:21:28.707025 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:28.706939    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:21:28.831055 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:28.831023    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:21:28.978624 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:28.978544    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:21:29.079353 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:29.079322    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:21:29.179287 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:29.179255    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:21:29.283613 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:29.283562    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:21:29.388099 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:29.388060    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:21:29.499761 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:29.499729    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:21:30.480042 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:30.479994    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:21:30.542721 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:30.542682    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:21:30.871927 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:30.871896    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:21:31.072682 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:31.072636    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:21:31.274769 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:31.274734    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:21:31.470349 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:31.470314    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:21:31.879783 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:31.879751    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:21:31.980651 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:31.980616    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:21:32.449678 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:32.449648    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:21:32.504096 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:32.504058    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:21:32.675739 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:32.675699    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:21:32.944680 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:32.944637    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/1b6f77bc-f7af-4355-9bb7-3527d89764d8-samples-operator-tls\") pod \"cluster-samples-operator-685d7d68b5-wcrh2\" (UID: \"1b6f77bc-f7af-4355-9bb7-3527d89764d8\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-wcrh2"
Sep 19 08:21:32.947082 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:32.947055    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/1b6f77bc-f7af-4355-9bb7-3527d89764d8-samples-operator-tls\") pod \"cluster-samples-operator-685d7d68b5-wcrh2\" (UID: \"1b6f77bc-f7af-4355-9bb7-3527d89764d8\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-wcrh2"
Sep 19 08:21:33.081746 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:33.081693    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:21:33.097652 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:33.097633    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-wcrh2"
Sep 19 08:21:33.269519 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:33.269487    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:21:33.323451 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:33.323415    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:21:33.366980 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:33.366951    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-wcrh2"]
Sep 19 08:21:33.436339 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:33.436282    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-wcrh2" event={"ID":"1b6f77bc-f7af-4355-9bb7-3527d89764d8","Type":"ContainerStarted","Data":"dc0230a5c2fb14303d66d5a0ebc68581f48226c0f32d9c10fbc3f9ad63a5921b"}
Sep 19 08:21:33.672822 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:33.672732    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:21:34.476149 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:34.476112    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:21:34.672715 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:34.672676    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:21:35.084698 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:35.084615    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:21:35.182968 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:35.182938    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:21:35.287074 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:35.287042    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:21:35.445392 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:35.445284    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-wcrh2" event={"ID":"1b6f77bc-f7af-4355-9bb7-3527d89764d8","Type":"ContainerStarted","Data":"454c741c5f325d988f70ea2d78eadfade66734f33c9696e40b90fd07ad8e63ed"}
Sep 19 08:21:35.445392 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:35.445368    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-wcrh2" event={"ID":"1b6f77bc-f7af-4355-9bb7-3527d89764d8","Type":"ContainerStarted","Data":"d776017a0469891e2c4f26636167add8916d88274924b7d3f96c70941d857c80"}
Sep 19 08:21:35.464482 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:35.464429    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-wcrh2" podStartSLOduration=17.097182913 podStartE2EDuration="18.464414722s" totalImagesPullingTime="1.367231809s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 08:21:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 08:21:35.463048464 +0000 UTC m=+79.839758347" watchObservedRunningTime="2026-09-19 08:21:35.464414722 +0000 UTC m=+79.841124583"
Sep 19 08:21:35.683726 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:35.683679    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:21:36.076457 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:36.076419    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:21:36.872753 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:36.872705    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:21:37.274486 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:37.274453    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:21:37.336269 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:37.336232    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:21:37.473399 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:37.473364    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:21:37.680504 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:37.680412    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:21:38.071940 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:38.071909    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:21:38.673541 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:38.673504    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:21:38.878542 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:38.878504    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:21:38.892102 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:38.892074    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6f5aff65-5f00-4cbe-bb6f-f6139743adac-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-847b8bd8d7-mdm6c\" (UID: \"6f5aff65-5f00-4cbe-bb6f-f6139743adac\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-mdm6c"
Sep 19 08:21:38.894346 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:38.894325    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6f5aff65-5f00-4cbe-bb6f-f6139743adac-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-847b8bd8d7-mdm6c\" (UID: \"6f5aff65-5f00-4cbe-bb6f-f6139743adac\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-mdm6c"
Sep 19 08:21:39.085597 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:39.085566    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-qfwb5\""
Sep 19 08:21:39.094416 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:39.094397    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-mdm6c"
Sep 19 08:21:39.275622 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:39.275591    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:21:39.358991 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:39.358924    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-mdm6c"]
Sep 19 08:21:39.361997 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:21:39.361973    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f5aff65_5f00_4cbe_bb6f_f6139743adac.slice/crio-f0058a0201e2c38c8dca90e2795995dd6bfb7f852ea78aa4b14981db5d4a04d0 WatchSource:0}: Error finding container f0058a0201e2c38c8dca90e2795995dd6bfb7f852ea78aa4b14981db5d4a04d0: Status 404 returned error can't find the container with id f0058a0201e2c38c8dca90e2795995dd6bfb7f852ea78aa4b14981db5d4a04d0
Sep 19 08:21:39.458358 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:39.458321    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-mdm6c" event={"ID":"6f5aff65-5f00-4cbe-bb6f-f6139743adac","Type":"ContainerStarted","Data":"f0058a0201e2c38c8dca90e2795995dd6bfb7f852ea78aa4b14981db5d4a04d0"}
Sep 19 08:21:39.468778 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:39.468731    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:21:41.213957 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:41.213875    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:21:41.321213 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:41.321178    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:21:41.419493 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:41.419459    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:21:41.467899 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:41.467813    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-mdm6c" event={"ID":"6f5aff65-5f00-4cbe-bb6f-f6139743adac","Type":"ContainerStarted","Data":"87dd0f1927deb38a24d63b1f0a9bcff7b03c28c43d2d57397f49045bf4716217"}
Sep 19 08:21:41.484449 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:41.484406    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-mdm6c" podStartSLOduration=33.938699939 podStartE2EDuration="35.484391997s" totalImagesPullingTime="1.545692058s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 08:21:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 08:21:41.483487661 +0000 UTC m=+85.860197533" watchObservedRunningTime="2026-09-19 08:21:41.484391997 +0000 UTC m=+85.861101857"
Sep 19 08:21:41.553893 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:41.553863    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:21:43.531839 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:43.531788    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/220642c2-d6c2-4681-a58b-087ba432f53a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dm5sf\" (UID: \"220642c2-d6c2-4681-a58b-087ba432f53a\") " pod="openshift-insights/insights-runtime-extractor-dm5sf"
Sep 19 08:21:43.534058 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:43.534039    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/220642c2-d6c2-4681-a58b-087ba432f53a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dm5sf\" (UID: \"220642c2-d6c2-4681-a58b-087ba432f53a\") " pod="openshift-insights/insights-runtime-extractor-dm5sf"
Sep 19 08:21:43.748648 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:43.748616    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-jw55w\""
Sep 19 08:21:43.756021 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:43.756001    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-dm5sf"
Sep 19 08:21:44.029101 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:44.029070    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-dm5sf"]
Sep 19 08:21:44.032227 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:21:44.032199    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod220642c2_d6c2_4681_a58b_087ba432f53a.slice/crio-cf4cee60bd35a822dc0919916c88e583d017b839525968a45a4bd64d881c95f0 WatchSource:0}: Error finding container cf4cee60bd35a822dc0919916c88e583d017b839525968a45a4bd64d881c95f0: Status 404 returned error can't find the container with id cf4cee60bd35a822dc0919916c88e583d017b839525968a45a4bd64d881c95f0
Sep 19 08:21:44.420430 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:44.420352    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-68bf747546-vzlgq"]
Sep 19 08:21:44.422661 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:44.422644    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-68bf747546-vzlgq"
Sep 19 08:21:44.424502 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:44.424477    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7cbf8cb5d9-x5n4c"]
Sep 19 08:21:44.426342 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:44.426318    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7cbf8cb5d9-x5n4c"
Sep 19 08:21:44.426458 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:44.426354    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Sep 19 08:21:44.426458 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:44.426354    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Sep 19 08:21:44.427464 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:44.427444    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Sep 19 08:21:44.427548 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:44.427491    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Sep 19 08:21:44.427548 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:44.427494    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-snck2\""
Sep 19 08:21:44.428757 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:44.428739    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Sep 19 08:21:44.441569 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:44.441546    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-68bf747546-vzlgq"]
Sep 19 08:21:44.446645 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:44.446621    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7cbf8cb5d9-x5n4c"]
Sep 19 08:21:44.478848 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:44.478810    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-dm5sf" event={"ID":"220642c2-d6c2-4681-a58b-087ba432f53a","Type":"ContainerStarted","Data":"47a334b0f7e3eed4bad2ca9e368c8dd66a89cd58965f2d2a4c0de542960f8e5f"}
Sep 19 08:21:44.479007 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:44.478867    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-dm5sf" event={"ID":"220642c2-d6c2-4681-a58b-087ba432f53a","Type":"ContainerStarted","Data":"cf4cee60bd35a822dc0919916c88e583d017b839525968a45a4bd64d881c95f0"}
Sep 19 08:21:44.503773 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:44.503744    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6446c8d75d-tbw4d"]
Sep 19 08:21:44.506269 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:44.506242    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6446c8d75d-tbw4d"
Sep 19 08:21:44.510581 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:44.510554    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Sep 19 08:21:44.510693 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:44.510671    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Sep 19 08:21:44.511760 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:44.511738    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-88mjn\""
Sep 19 08:21:44.517942 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:44.517919    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-gbgd2"]
Sep 19 08:21:44.519903 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:44.519886    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-gbgd2"
Sep 19 08:21:44.522196 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:44.522179    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-rw6fz\""
Sep 19 08:21:44.522529 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:44.522504    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Sep 19 08:21:44.527038 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:44.527014    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6446c8d75d-tbw4d"]
Sep 19 08:21:44.538428 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:44.538404    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/79e21e13-53ba-4395-9b8f-0b1ac1ec7c57-klusterlet-config\") pod \"klusterlet-addon-workmgr-7cbf8cb5d9-x5n4c\" (UID: \"79e21e13-53ba-4395-9b8f-0b1ac1ec7c57\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7cbf8cb5d9-x5n4c"
Sep 19 08:21:44.538777 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:44.538470    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wpq7t\" (UniqueName: \"kubernetes.io/projected/2a006ddb-42d4-4e5c-a4e2-f8e3e99a9a54-kube-api-access-wpq7t\") pod \"managed-serviceaccount-addon-agent-68bf747546-vzlgq\" (UID: \"2a006ddb-42d4-4e5c-a4e2-f8e3e99a9a54\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-68bf747546-vzlgq"
Sep 19 08:21:44.538777 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:44.538534    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/79e21e13-53ba-4395-9b8f-0b1ac1ec7c57-tmp\") pod \"klusterlet-addon-workmgr-7cbf8cb5d9-x5n4c\" (UID: \"79e21e13-53ba-4395-9b8f-0b1ac1ec7c57\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7cbf8cb5d9-x5n4c"
Sep 19 08:21:44.538777 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:44.538556    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v57hp\" (UniqueName: \"kubernetes.io/projected/79e21e13-53ba-4395-9b8f-0b1ac1ec7c57-kube-api-access-v57hp\") pod \"klusterlet-addon-workmgr-7cbf8cb5d9-x5n4c\" (UID: \"79e21e13-53ba-4395-9b8f-0b1ac1ec7c57\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7cbf8cb5d9-x5n4c"
Sep 19 08:21:44.538777 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:44.538595    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2a006ddb-42d4-4e5c-a4e2-f8e3e99a9a54-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-68bf747546-vzlgq\" (UID: \"2a006ddb-42d4-4e5c-a4e2-f8e3e99a9a54\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-68bf747546-vzlgq"
Sep 19 08:21:44.545065 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:44.545034    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:21:44.547772 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:44.547747    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-gbgd2"]
Sep 19 08:21:44.638988 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:44.638958    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/79e21e13-53ba-4395-9b8f-0b1ac1ec7c57-tmp\") pod \"klusterlet-addon-workmgr-7cbf8cb5d9-x5n4c\" (UID: \"79e21e13-53ba-4395-9b8f-0b1ac1ec7c57\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7cbf8cb5d9-x5n4c"
Sep 19 08:21:44.638988 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:44.638992    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v57hp\" (UniqueName: \"kubernetes.io/projected/79e21e13-53ba-4395-9b8f-0b1ac1ec7c57-kube-api-access-v57hp\") pod \"klusterlet-addon-workmgr-7cbf8cb5d9-x5n4c\" (UID: \"79e21e13-53ba-4395-9b8f-0b1ac1ec7c57\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7cbf8cb5d9-x5n4c"
Sep 19 08:21:44.639206 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:44.639029    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/cedc64f2-407a-4344-96be-399fb3c5d499-nginx-conf\") pod \"networking-console-plugin-6446c8d75d-tbw4d\" (UID: \"cedc64f2-407a-4344-96be-399fb3c5d499\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-tbw4d"
Sep 19 08:21:44.639206 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:44.639049    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/cedc64f2-407a-4344-96be-399fb3c5d499-networking-console-plugin-cert\") pod \"networking-console-plugin-6446c8d75d-tbw4d\" (UID: \"cedc64f2-407a-4344-96be-399fb3c5d499\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-tbw4d"
Sep 19 08:21:44.639206 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:44.639066    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2a006ddb-42d4-4e5c-a4e2-f8e3e99a9a54-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-68bf747546-vzlgq\" (UID: \"2a006ddb-42d4-4e5c-a4e2-f8e3e99a9a54\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-68bf747546-vzlgq"
Sep 19 08:21:44.639206 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:44.639109    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/79e21e13-53ba-4395-9b8f-0b1ac1ec7c57-klusterlet-config\") pod \"klusterlet-addon-workmgr-7cbf8cb5d9-x5n4c\" (UID: \"79e21e13-53ba-4395-9b8f-0b1ac1ec7c57\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7cbf8cb5d9-x5n4c"
Sep 19 08:21:44.639428 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:44.639213    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wpq7t\" (UniqueName: \"kubernetes.io/projected/2a006ddb-42d4-4e5c-a4e2-f8e3e99a9a54-kube-api-access-wpq7t\") pod \"managed-serviceaccount-addon-agent-68bf747546-vzlgq\" (UID: \"2a006ddb-42d4-4e5c-a4e2-f8e3e99a9a54\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-68bf747546-vzlgq"
Sep 19 08:21:44.639428 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:44.639252    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/cf1e52ee-e10d-4155-ad44-3e0fddb87ab7-tls-certificates\") pod \"prometheus-operator-admission-webhook-5c6747dc7b-gbgd2\" (UID: \"cf1e52ee-e10d-4155-ad44-3e0fddb87ab7\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-gbgd2"
Sep 19 08:21:44.639428 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:44.639343    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/79e21e13-53ba-4395-9b8f-0b1ac1ec7c57-tmp\") pod \"klusterlet-addon-workmgr-7cbf8cb5d9-x5n4c\" (UID: \"79e21e13-53ba-4395-9b8f-0b1ac1ec7c57\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7cbf8cb5d9-x5n4c"
Sep 19 08:21:44.641631 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:44.641600    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/79e21e13-53ba-4395-9b8f-0b1ac1ec7c57-klusterlet-config\") pod \"klusterlet-addon-workmgr-7cbf8cb5d9-x5n4c\" (UID: \"79e21e13-53ba-4395-9b8f-0b1ac1ec7c57\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7cbf8cb5d9-x5n4c"
Sep 19 08:21:44.641747 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:44.641668    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2a006ddb-42d4-4e5c-a4e2-f8e3e99a9a54-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-68bf747546-vzlgq\" (UID: \"2a006ddb-42d4-4e5c-a4e2-f8e3e99a9a54\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-68bf747546-vzlgq"
Sep 19 08:21:44.652129 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:44.652106    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wpq7t\" (UniqueName: \"kubernetes.io/projected/2a006ddb-42d4-4e5c-a4e2-f8e3e99a9a54-kube-api-access-wpq7t\") pod \"managed-serviceaccount-addon-agent-68bf747546-vzlgq\" (UID: \"2a006ddb-42d4-4e5c-a4e2-f8e3e99a9a54\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-68bf747546-vzlgq"
Sep 19 08:21:44.652230 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:44.652208    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v57hp\" (UniqueName: \"kubernetes.io/projected/79e21e13-53ba-4395-9b8f-0b1ac1ec7c57-kube-api-access-v57hp\") pod \"klusterlet-addon-workmgr-7cbf8cb5d9-x5n4c\" (UID: \"79e21e13-53ba-4395-9b8f-0b1ac1ec7c57\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7cbf8cb5d9-x5n4c"
Sep 19 08:21:44.740092 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:44.740066    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/cedc64f2-407a-4344-96be-399fb3c5d499-nginx-conf\") pod \"networking-console-plugin-6446c8d75d-tbw4d\" (UID: \"cedc64f2-407a-4344-96be-399fb3c5d499\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-tbw4d"
Sep 19 08:21:44.740211 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:44.740100    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/cedc64f2-407a-4344-96be-399fb3c5d499-networking-console-plugin-cert\") pod \"networking-console-plugin-6446c8d75d-tbw4d\" (UID: \"cedc64f2-407a-4344-96be-399fb3c5d499\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-tbw4d"
Sep 19 08:21:44.740211 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:44.740155    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/cf1e52ee-e10d-4155-ad44-3e0fddb87ab7-tls-certificates\") pod \"prometheus-operator-admission-webhook-5c6747dc7b-gbgd2\" (UID: \"cf1e52ee-e10d-4155-ad44-3e0fddb87ab7\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-gbgd2"
Sep 19 08:21:44.740874 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:44.740841    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/cedc64f2-407a-4344-96be-399fb3c5d499-nginx-conf\") pod \"networking-console-plugin-6446c8d75d-tbw4d\" (UID: \"cedc64f2-407a-4344-96be-399fb3c5d499\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-tbw4d"
Sep 19 08:21:44.742843 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:44.742821    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-68bf747546-vzlgq"
Sep 19 08:21:44.743151 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:44.743132    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/cedc64f2-407a-4344-96be-399fb3c5d499-networking-console-plugin-cert\") pod \"networking-console-plugin-6446c8d75d-tbw4d\" (UID: \"cedc64f2-407a-4344-96be-399fb3c5d499\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-tbw4d"
Sep 19 08:21:44.743247 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:44.743226    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/cf1e52ee-e10d-4155-ad44-3e0fddb87ab7-tls-certificates\") pod \"prometheus-operator-admission-webhook-5c6747dc7b-gbgd2\" (UID: \"cf1e52ee-e10d-4155-ad44-3e0fddb87ab7\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-gbgd2"
Sep 19 08:21:44.748713 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:44.748691    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7cbf8cb5d9-x5n4c"
Sep 19 08:21:44.750268 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:44.750242    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:21:44.818354 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:44.817678    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6446c8d75d-tbw4d"
Sep 19 08:21:44.829336 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:44.829311    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-gbgd2"
Sep 19 08:21:44.856629 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:44.856576    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:21:45.014774 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:45.014743    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-68bf747546-vzlgq"]
Sep 19 08:21:45.017642 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:21:45.017615    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2a006ddb_42d4_4e5c_a4e2_f8e3e99a9a54.slice/crio-9144cff55d2fc2dc42ff644b156321978090a2d7b49715de57a0cd0e32e2f477 WatchSource:0}: Error finding container 9144cff55d2fc2dc42ff644b156321978090a2d7b49715de57a0cd0e32e2f477: Status 404 returned error can't find the container with id 9144cff55d2fc2dc42ff644b156321978090a2d7b49715de57a0cd0e32e2f477
Sep 19 08:21:45.019771 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:45.019753    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7cbf8cb5d9-x5n4c"]
Sep 19 08:21:45.022382 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:21:45.022356    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79e21e13_53ba_4395_9b8f_0b1ac1ec7c57.slice/crio-a34337bb392ec2c8f54b6a5db836500b49d734909c2554b275cb3f7adc05d077 WatchSource:0}: Error finding container a34337bb392ec2c8f54b6a5db836500b49d734909c2554b275cb3f7adc05d077: Status 404 returned error can't find the container with id a34337bb392ec2c8f54b6a5db836500b49d734909c2554b275cb3f7adc05d077
Sep 19 08:21:45.087069 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:45.087037    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6446c8d75d-tbw4d"]
Sep 19 08:21:45.089901 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:21:45.089873    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcedc64f2_407a_4344_96be_399fb3c5d499.slice/crio-e4d47b9eda91fce853fbd13db4baeeda65e36bd00fb4be6ee300746a83dd0d22 WatchSource:0}: Error finding container e4d47b9eda91fce853fbd13db4baeeda65e36bd00fb4be6ee300746a83dd0d22: Status 404 returned error can't find the container with id e4d47b9eda91fce853fbd13db4baeeda65e36bd00fb4be6ee300746a83dd0d22
Sep 19 08:21:45.108428 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:45.108397    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-gbgd2"]
Sep 19 08:21:45.111203 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:21:45.111178    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf1e52ee_e10d_4155_ad44_3e0fddb87ab7.slice/crio-553a27c305eaf7c586b2e288a6ffe1e17369aed12fb151995f0ec8b87ed99a34 WatchSource:0}: Error finding container 553a27c305eaf7c586b2e288a6ffe1e17369aed12fb151995f0ec8b87ed99a34: Status 404 returned error can't find the container with id 553a27c305eaf7c586b2e288a6ffe1e17369aed12fb151995f0ec8b87ed99a34
Sep 19 08:21:45.422944 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:45.422814    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:21:45.484194 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:45.484159    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-gbgd2" event={"ID":"cf1e52ee-e10d-4155-ad44-3e0fddb87ab7","Type":"ContainerStarted","Data":"553a27c305eaf7c586b2e288a6ffe1e17369aed12fb151995f0ec8b87ed99a34"}
Sep 19 08:21:45.485927 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:45.485872    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7cbf8cb5d9-x5n4c" event={"ID":"79e21e13-53ba-4395-9b8f-0b1ac1ec7c57","Type":"ContainerStarted","Data":"a34337bb392ec2c8f54b6a5db836500b49d734909c2554b275cb3f7adc05d077"}
Sep 19 08:21:45.487263 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:45.487239    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6446c8d75d-tbw4d" event={"ID":"cedc64f2-407a-4344-96be-399fb3c5d499","Type":"ContainerStarted","Data":"e4d47b9eda91fce853fbd13db4baeeda65e36bd00fb4be6ee300746a83dd0d22"}
Sep 19 08:21:45.489603 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:45.489580    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-dm5sf" event={"ID":"220642c2-d6c2-4681-a58b-087ba432f53a","Type":"ContainerStarted","Data":"f4fe8c2396b3c554191a4849a499788a711fec752f0fc85ef26930aaf51d643c"}
Sep 19 08:21:45.491031 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:45.491005    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-68bf747546-vzlgq" event={"ID":"2a006ddb-42d4-4e5c-a4e2-f8e3e99a9a54","Type":"ContainerStarted","Data":"9144cff55d2fc2dc42ff644b156321978090a2d7b49715de57a0cd0e32e2f477"}
Sep 19 08:21:45.494925 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:45.494809    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:21:46.911847 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:46.911757    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:21:47.020332 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:47.020276    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:21:47.504324 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:47.503904    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-dm5sf" event={"ID":"220642c2-d6c2-4681-a58b-087ba432f53a","Type":"ContainerStarted","Data":"e9b1c14ea1ca1ed0941a59684a9ee4d11471d53b45f683036e1b4b5cd83d789a"}
Sep 19 08:21:47.506274 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:47.506216    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-gbgd2" event={"ID":"cf1e52ee-e10d-4155-ad44-3e0fddb87ab7","Type":"ContainerStarted","Data":"6b008843e70a2d4c2c0e4dffc4b3ed53cddff45e57753511383ff1eb1655bc91"}
Sep 19 08:21:47.506526 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:47.506497    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-gbgd2"
Sep 19 08:21:47.508580 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:47.508536    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6446c8d75d-tbw4d" event={"ID":"cedc64f2-407a-4344-96be-399fb3c5d499","Type":"ContainerStarted","Data":"126b61acc6c1fd338d7b5db10aa6301d4b13d8f5456f26751560654ee89e8c97"}
Sep 19 08:21:47.512382 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:47.512346    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-gbgd2"
Sep 19 08:21:47.527132 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:47.526991    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-dm5sf" podStartSLOduration=34.020794184 podStartE2EDuration="36.526972794s" totalImagesPullingTime="2.50617861s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 08:21:11 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 08:21:47.525315883 +0000 UTC m=+91.902025749" watchObservedRunningTime="2026-09-19 08:21:47.526972794 +0000 UTC m=+91.903682657"
Sep 19 08:21:47.548033 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:47.547967    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6446c8d75d-tbw4d" podStartSLOduration=1.99064504 podStartE2EDuration="3.547952816s" totalImagesPullingTime="1.557307776s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 08:21:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 08:21:47.545749736 +0000 UTC m=+91.922459601" watchObservedRunningTime="2026-09-19 08:21:47.547952816 +0000 UTC m=+91.924662678"
Sep 19 08:21:47.571545 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:47.571484    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-gbgd2" podStartSLOduration=2.03367483 podStartE2EDuration="3.571460914s" totalImagesPullingTime="1.537786084s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 08:21:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 08:21:47.570082003 +0000 UTC m=+91.946791939" watchObservedRunningTime="2026-09-19 08:21:47.571460914 +0000 UTC m=+91.948170775"
Sep 19 08:21:47.669801 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:47.669608    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:21:47.741395 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:47.741353    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:21:48.411709 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:48.411663    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:21:48.486595 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:48.486559    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:21:48.545708 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:48.545664    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:21:50.787970 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:50.787937    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:21:51.525171 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:51.525141    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7cbf8cb5d9-x5n4c" event={"ID":"79e21e13-53ba-4395-9b8f-0b1ac1ec7c57","Type":"ContainerStarted","Data":"0a881d8accf27061b4e7068078175d7f6887005c6b3859ed4169a65763b02222"}
Sep 19 08:21:51.525366 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:51.525341    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7cbf8cb5d9-x5n4c"
Sep 19 08:21:51.526794 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:51.526769    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-68bf747546-vzlgq" event={"ID":"2a006ddb-42d4-4e5c-a4e2-f8e3e99a9a54","Type":"ContainerStarted","Data":"02f05db4bf35b9abd123fc445a33cefdb4dc12b733216fc2352c9992fd360de3"}
Sep 19 08:21:51.527226 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:51.527212    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7cbf8cb5d9-x5n4c"
Sep 19 08:21:51.540988 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:51.540931    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7cbf8cb5d9-x5n4c" podStartSLOduration=2.080523804 podStartE2EDuration="7.540917228s" totalImagesPullingTime="5.460393424s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 08:21:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 08:21:51.540067165 +0000 UTC m=+95.916777026" watchObservedRunningTime="2026-09-19 08:21:51.540917228 +0000 UTC m=+95.917627073"
Sep 19 08:21:51.555100 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:51.555059    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-68bf747546-vzlgq" podStartSLOduration=2.148587421 podStartE2EDuration="7.555047086s" totalImagesPullingTime="5.406459665s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 08:21:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 08:21:51.554037747 +0000 UTC m=+95.930747629" watchObservedRunningTime="2026-09-19 08:21:51.555047086 +0000 UTC m=+95.931756946"
Sep 19 08:21:52.548374 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:52.548345    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:21:52.648834 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:52.648803    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:21:52.770801 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:52.770766    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:21:52.828181 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:52.828097    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-9thpj"]
Sep 19 08:21:52.833651 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:52.833634    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-9thpj"
Sep 19 08:21:52.836071 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:52.836049    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-f7j7t\""
Sep 19 08:21:52.836290 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:52.836259    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 19 08:21:52.836290 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:52.836278    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 19 08:21:52.836555 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:52.836531    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 19 08:21:52.837080 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:52.837064    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 19 08:21:52.876274 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:52.876241    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:21:52.910607 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:52.910568    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b5e8a425-558b-4e10-82b8-764bdd1ad9cc-node-exporter-textfile\") pod \"node-exporter-9thpj\" (UID: \"b5e8a425-558b-4e10-82b8-764bdd1ad9cc\") " pod="openshift-monitoring/node-exporter-9thpj"
Sep 19 08:21:52.910607 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:52.910607    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b5e8a425-558b-4e10-82b8-764bdd1ad9cc-node-exporter-accelerators-collector-config\") pod \"node-exporter-9thpj\" (UID: \"b5e8a425-558b-4e10-82b8-764bdd1ad9cc\") " pod="openshift-monitoring/node-exporter-9thpj"
Sep 19 08:21:52.910810 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:52.910630    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b5e8a425-558b-4e10-82b8-764bdd1ad9cc-node-exporter-tls\") pod \"node-exporter-9thpj\" (UID: \"b5e8a425-558b-4e10-82b8-764bdd1ad9cc\") " pod="openshift-monitoring/node-exporter-9thpj"
Sep 19 08:21:52.910810 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:52.910716    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b5e8a425-558b-4e10-82b8-764bdd1ad9cc-metrics-client-ca\") pod \"node-exporter-9thpj\" (UID: \"b5e8a425-558b-4e10-82b8-764bdd1ad9cc\") " pod="openshift-monitoring/node-exporter-9thpj"
Sep 19 08:21:52.910810 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:52.910744    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b5e8a425-558b-4e10-82b8-764bdd1ad9cc-sys\") pod \"node-exporter-9thpj\" (UID: \"b5e8a425-558b-4e10-82b8-764bdd1ad9cc\") " pod="openshift-monitoring/node-exporter-9thpj"
Sep 19 08:21:52.910810 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:52.910761    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kcz7l\" (UniqueName: \"kubernetes.io/projected/b5e8a425-558b-4e10-82b8-764bdd1ad9cc-kube-api-access-kcz7l\") pod \"node-exporter-9thpj\" (UID: \"b5e8a425-558b-4e10-82b8-764bdd1ad9cc\") " pod="openshift-monitoring/node-exporter-9thpj"
Sep 19 08:21:52.910810 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:52.910783    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b5e8a425-558b-4e10-82b8-764bdd1ad9cc-root\") pod \"node-exporter-9thpj\" (UID: \"b5e8a425-558b-4e10-82b8-764bdd1ad9cc\") " pod="openshift-monitoring/node-exporter-9thpj"
Sep 19 08:21:52.910974 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:52.910816    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5dca0cc5-6589-4585-978d-ca12d129c1a0-registry-tls\") pod \"image-registry-8566d48d94-7ksk8\" (UID: \"5dca0cc5-6589-4585-978d-ca12d129c1a0\") " pod="openshift-image-registry/image-registry-8566d48d94-7ksk8"
Sep 19 08:21:52.910974 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:52.910835    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b5e8a425-558b-4e10-82b8-764bdd1ad9cc-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-9thpj\" (UID: \"b5e8a425-558b-4e10-82b8-764bdd1ad9cc\") " pod="openshift-monitoring/node-exporter-9thpj"
Sep 19 08:21:52.910974 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:52.910865    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/61a85fdc-d1d2-4a56-a247-968786b293a5-metrics-tls\") pod \"dns-default-ldghd\" (UID: \"61a85fdc-d1d2-4a56-a247-968786b293a5\") " pod="openshift-dns/dns-default-ldghd"
Sep 19 08:21:52.910974 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:52.910884    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b5e8a425-558b-4e10-82b8-764bdd1ad9cc-node-exporter-wtmp\") pod \"node-exporter-9thpj\" (UID: \"b5e8a425-558b-4e10-82b8-764bdd1ad9cc\") " pod="openshift-monitoring/node-exporter-9thpj"
Sep 19 08:21:52.913177 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:52.913154    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/61a85fdc-d1d2-4a56-a247-968786b293a5-metrics-tls\") pod \"dns-default-ldghd\" (UID: \"61a85fdc-d1d2-4a56-a247-968786b293a5\") " pod="openshift-dns/dns-default-ldghd"
Sep 19 08:21:52.913292 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:52.913242    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5dca0cc5-6589-4585-978d-ca12d129c1a0-registry-tls\") pod \"image-registry-8566d48d94-7ksk8\" (UID: \"5dca0cc5-6589-4585-978d-ca12d129c1a0\") " pod="openshift-image-registry/image-registry-8566d48d94-7ksk8"
Sep 19 08:21:52.971140 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:52.971099    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:21:53.011579 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:53.011546    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b5e8a425-558b-4e10-82b8-764bdd1ad9cc-node-exporter-textfile\") pod \"node-exporter-9thpj\" (UID: \"b5e8a425-558b-4e10-82b8-764bdd1ad9cc\") " pod="openshift-monitoring/node-exporter-9thpj"
Sep 19 08:21:53.011579 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:53.011582    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b5e8a425-558b-4e10-82b8-764bdd1ad9cc-node-exporter-accelerators-collector-config\") pod \"node-exporter-9thpj\" (UID: \"b5e8a425-558b-4e10-82b8-764bdd1ad9cc\") " pod="openshift-monitoring/node-exporter-9thpj"
Sep 19 08:21:53.011804 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:53.011607    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b5e8a425-558b-4e10-82b8-764bdd1ad9cc-node-exporter-tls\") pod \"node-exporter-9thpj\" (UID: \"b5e8a425-558b-4e10-82b8-764bdd1ad9cc\") " pod="openshift-monitoring/node-exporter-9thpj"
Sep 19 08:21:53.011804 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:53.011649    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b5e8a425-558b-4e10-82b8-764bdd1ad9cc-metrics-client-ca\") pod \"node-exporter-9thpj\" (UID: \"b5e8a425-558b-4e10-82b8-764bdd1ad9cc\") " pod="openshift-monitoring/node-exporter-9thpj"
Sep 19 08:21:53.011804 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:53.011667    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b5e8a425-558b-4e10-82b8-764bdd1ad9cc-sys\") pod \"node-exporter-9thpj\" (UID: \"b5e8a425-558b-4e10-82b8-764bdd1ad9cc\") " pod="openshift-monitoring/node-exporter-9thpj"
Sep 19 08:21:53.011804 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:53.011680    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kcz7l\" (UniqueName: \"kubernetes.io/projected/b5e8a425-558b-4e10-82b8-764bdd1ad9cc-kube-api-access-kcz7l\") pod \"node-exporter-9thpj\" (UID: \"b5e8a425-558b-4e10-82b8-764bdd1ad9cc\") " pod="openshift-monitoring/node-exporter-9thpj"
Sep 19 08:21:53.011804 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:53.011698    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b5e8a425-558b-4e10-82b8-764bdd1ad9cc-root\") pod \"node-exporter-9thpj\" (UID: \"b5e8a425-558b-4e10-82b8-764bdd1ad9cc\") " pod="openshift-monitoring/node-exporter-9thpj"
Sep 19 08:21:53.011804 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:53.011729    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b5e8a425-558b-4e10-82b8-764bdd1ad9cc-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-9thpj\" (UID: \"b5e8a425-558b-4e10-82b8-764bdd1ad9cc\") " pod="openshift-monitoring/node-exporter-9thpj"
Sep 19 08:21:53.011804 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:53.011770    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b5e8a425-558b-4e10-82b8-764bdd1ad9cc-node-exporter-wtmp\") pod \"node-exporter-9thpj\" (UID: \"b5e8a425-558b-4e10-82b8-764bdd1ad9cc\") " pod="openshift-monitoring/node-exporter-9thpj"
Sep 19 08:21:53.012109 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:53.011819    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b5e8a425-558b-4e10-82b8-764bdd1ad9cc-sys\") pod \"node-exporter-9thpj\" (UID: \"b5e8a425-558b-4e10-82b8-764bdd1ad9cc\") " pod="openshift-monitoring/node-exporter-9thpj"
Sep 19 08:21:53.012109 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:53.011880    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b5e8a425-558b-4e10-82b8-764bdd1ad9cc-root\") pod \"node-exporter-9thpj\" (UID: \"b5e8a425-558b-4e10-82b8-764bdd1ad9cc\") " pod="openshift-monitoring/node-exporter-9thpj"
Sep 19 08:21:53.012109 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:53.011902    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b5e8a425-558b-4e10-82b8-764bdd1ad9cc-node-exporter-wtmp\") pod \"node-exporter-9thpj\" (UID: \"b5e8a425-558b-4e10-82b8-764bdd1ad9cc\") " pod="openshift-monitoring/node-exporter-9thpj"
Sep 19 08:21:53.012109 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:53.011928    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b5e8a425-558b-4e10-82b8-764bdd1ad9cc-node-exporter-textfile\") pod \"node-exporter-9thpj\" (UID: \"b5e8a425-558b-4e10-82b8-764bdd1ad9cc\") " pod="openshift-monitoring/node-exporter-9thpj"
Sep 19 08:21:53.012509 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:53.012481    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b5e8a425-558b-4e10-82b8-764bdd1ad9cc-metrics-client-ca\") pod \"node-exporter-9thpj\" (UID: \"b5e8a425-558b-4e10-82b8-764bdd1ad9cc\") " pod="openshift-monitoring/node-exporter-9thpj"
Sep 19 08:21:53.012642 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:53.012551    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b5e8a425-558b-4e10-82b8-764bdd1ad9cc-node-exporter-accelerators-collector-config\") pod \"node-exporter-9thpj\" (UID: \"b5e8a425-558b-4e10-82b8-764bdd1ad9cc\") " pod="openshift-monitoring/node-exporter-9thpj"
Sep 19 08:21:53.014419 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:53.014391    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b5e8a425-558b-4e10-82b8-764bdd1ad9cc-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-9thpj\" (UID: \"b5e8a425-558b-4e10-82b8-764bdd1ad9cc\") " pod="openshift-monitoring/node-exporter-9thpj"
Sep 19 08:21:53.014419 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:53.014407    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b5e8a425-558b-4e10-82b8-764bdd1ad9cc-node-exporter-tls\") pod \"node-exporter-9thpj\" (UID: \"b5e8a425-558b-4e10-82b8-764bdd1ad9cc\") " pod="openshift-monitoring/node-exporter-9thpj"
Sep 19 08:21:53.026085 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:53.026046    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kcz7l\" (UniqueName: \"kubernetes.io/projected/b5e8a425-558b-4e10-82b8-764bdd1ad9cc-kube-api-access-kcz7l\") pod \"node-exporter-9thpj\" (UID: \"b5e8a425-558b-4e10-82b8-764bdd1ad9cc\") " pod="openshift-monitoring/node-exporter-9thpj"
Sep 19 08:21:53.143435 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:53.143347    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-9thpj"
Sep 19 08:21:53.156787 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:21:53.156752    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5e8a425_558b_4e10_82b8_764bdd1ad9cc.slice/crio-e6f539f99cc1392b992fb3acd766cb798a44ae64476814398190cbfbb8c8b587 WatchSource:0}: Error finding container e6f539f99cc1392b992fb3acd766cb798a44ae64476814398190cbfbb8c8b587: Status 404 returned error can't find the container with id e6f539f99cc1392b992fb3acd766cb798a44ae64476814398190cbfbb8c8b587
Sep 19 08:21:53.173891 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:53.173865    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:21:53.189646 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:53.189622    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-8bzsz\""
Sep 19 08:21:53.197903 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:53.197882    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-8566d48d94-7ksk8"
Sep 19 08:21:53.199206 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:53.199189    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-cdf7v\""
Sep 19 08:21:53.207454 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:53.207425    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-ldghd"
Sep 19 08:21:53.275517 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:53.275214    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:21:53.480289 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:53.480188    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-8566d48d94-7ksk8"]
Sep 19 08:21:53.482749 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:21:53.482726    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5dca0cc5_6589_4585_978d_ca12d129c1a0.slice/crio-db143f9b82bf72948f8b79da06dc19668a6e0639f0c022ea591c0fba0330bec8 WatchSource:0}: Error finding container db143f9b82bf72948f8b79da06dc19668a6e0639f0c022ea591c0fba0330bec8: Status 404 returned error can't find the container with id db143f9b82bf72948f8b79da06dc19668a6e0639f0c022ea591c0fba0330bec8
Sep 19 08:21:53.486691 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:53.486668    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-ldghd"]
Sep 19 08:21:53.489177 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:21:53.489154    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61a85fdc_d1d2_4a56_a247_968786b293a5.slice/crio-b48f2ab8f60330c79de53827af77532f946092427e74055e103ddcffeab3e030 WatchSource:0}: Error finding container b48f2ab8f60330c79de53827af77532f946092427e74055e103ddcffeab3e030: Status 404 returned error can't find the container with id b48f2ab8f60330c79de53827af77532f946092427e74055e103ddcffeab3e030
Sep 19 08:21:53.535441 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:53.535414    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-ldghd" event={"ID":"61a85fdc-d1d2-4a56-a247-968786b293a5","Type":"ContainerStarted","Data":"b48f2ab8f60330c79de53827af77532f946092427e74055e103ddcffeab3e030"}
Sep 19 08:21:53.536669 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:53.536646    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-8566d48d94-7ksk8" event={"ID":"5dca0cc5-6589-4585-978d-ca12d129c1a0","Type":"ContainerStarted","Data":"db143f9b82bf72948f8b79da06dc19668a6e0639f0c022ea591c0fba0330bec8"}
Sep 19 08:21:53.537847 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:53.537818    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9thpj" event={"ID":"b5e8a425-558b-4e10-82b8-764bdd1ad9cc","Type":"ContainerStarted","Data":"e6f539f99cc1392b992fb3acd766cb798a44ae64476814398190cbfbb8c8b587"}
Sep 19 08:21:53.546227 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:53.546147    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:21:53.700806 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:53.700764    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:21:53.803759 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:53.803726    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:21:53.903920 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:53.903885    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:21:53.980702 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:53.980491    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:21:54.222007 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:54.221911    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:21:54.544411 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:54.544375    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-8566d48d94-7ksk8" event={"ID":"5dca0cc5-6589-4585-978d-ca12d129c1a0","Type":"ContainerStarted","Data":"db77ac911ea7ec2514c44873b419e79b96c890781b5f4e6fc6031faefa144fc6"}
Sep 19 08:21:54.544629 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:54.544483    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-8566d48d94-7ksk8"
Sep 19 08:21:54.546444 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:54.546418    2578 generic.go:356] "Generic (PLEG): container finished" podID="b5e8a425-558b-4e10-82b8-764bdd1ad9cc" containerID="1937aa6dc9bbeff581611ebb8ba9ddf44db466a3aa3cda5524df2963233ae54d" exitCode=0
Sep 19 08:21:54.546582 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:54.546477    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9thpj" event={"ID":"b5e8a425-558b-4e10-82b8-764bdd1ad9cc","Type":"ContainerDied","Data":"1937aa6dc9bbeff581611ebb8ba9ddf44db466a3aa3cda5524df2963233ae54d"}
Sep 19 08:21:54.567604 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:54.567553    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-8566d48d94-7ksk8" podStartSLOduration=98.567534357 podStartE2EDuration="1m38.567534357s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 08:20:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 08:21:54.564606239 +0000 UTC m=+98.941316100" watchObservedRunningTime="2026-09-19 08:21:54.567534357 +0000 UTC m=+98.944244218"
Sep 19 08:21:55.552567 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:55.552532    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-ldghd" event={"ID":"61a85fdc-d1d2-4a56-a247-968786b293a5","Type":"ContainerStarted","Data":"023aea15cc27eabab4d5c9f8c5d4d1f5e85ca94c742803eec21f83137d6d2bdb"}
Sep 19 08:21:55.553031 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:55.552598    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-ldghd" event={"ID":"61a85fdc-d1d2-4a56-a247-968786b293a5","Type":"ContainerStarted","Data":"ac2c02bd90a403abccc8405f27b0e435e1eda9162f382f2d9ea2499fc97625ae"}
Sep 19 08:21:55.553031 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:55.552646    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-ldghd"
Sep 19 08:21:55.554677 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:55.554654    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9thpj" event={"ID":"b5e8a425-558b-4e10-82b8-764bdd1ad9cc","Type":"ContainerStarted","Data":"9017aaf64990b774bcc25685654d0aaf3acade7d8daa5ff37f932e0af20d8ff8"}
Sep 19 08:21:55.554778 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:55.554693    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9thpj" event={"ID":"b5e8a425-558b-4e10-82b8-764bdd1ad9cc","Type":"ContainerStarted","Data":"af7ad1dc660abf69d604457acaf9f25cdf8955296d8fd2602591aadee470c41b"}
Sep 19 08:21:55.582328 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:55.582265    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-ldghd" podStartSLOduration=65.977844318 podStartE2EDuration="1m7.58224995s" totalImagesPullingTime="1.604405632s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 08:20:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 08:21:55.580679869 +0000 UTC m=+99.957389734" watchObservedRunningTime="2026-09-19 08:21:55.58224995 +0000 UTC m=+99.958959804"
Sep 19 08:21:55.622003 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:55.621952    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-9thpj" podStartSLOduration=2.464426547 podStartE2EDuration="3.621936745s" totalImagesPullingTime="1.084125001s" totalInitContainerRuntime="73.385197ms" isStatefulPod=true podCreationTimestamp="2026-09-19 08:21:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 08:21:55.620724391 +0000 UTC m=+99.997434245" watchObservedRunningTime="2026-09-19 08:21:55.621936745 +0000 UTC m=+99.998646605"
Sep 19 08:21:55.754842 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:55.754806    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:21:55.840698 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:55.840590    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:21:55.987395 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:55.987354    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:21:56.114940 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:56.114844    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:21:57.559236 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:57.559194    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:21:58.011323 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:58.011273    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:21:58.152536 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:58.152503    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:21:58.210464 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:58.210433    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:21:59.126326 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:59.126270    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:21:59.276713 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:59.276677    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:21:59.790710 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:59.790680    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:21:59.849596 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:21:59.849561    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:22:05.349603 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:05.349570    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:22:05.455813 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:05.455778    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:22:05.560403 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:05.560376    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-ldghd"
Sep 19 08:22:07.258846 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:07.258810    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:22:07.363069 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:07.363031    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:22:07.753188 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:07.753113    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-8566d48d94-7ksk8"]
Sep 19 08:22:07.855316 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:07.855256    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:22:07.955523 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:07.955488    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:22:08.788139 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:08.788110    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:22:08.856451 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:08.856422    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:22:13.180385 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:13.180350    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:22:13.234473 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:13.234440    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-6594f4c75-49nzk"]
Sep 19 08:22:13.240180 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:13.240163    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-6594f4c75-49nzk"
Sep 19 08:22:13.242653 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:13.242632    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-gc96l\""
Sep 19 08:22:13.243001 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:13.242980    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 19 08:22:13.243345 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:13.243331    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 19 08:22:13.247667 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:13.247647    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-6594f4c75-49nzk"]
Sep 19 08:22:13.273708 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:13.273681    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w4jz6\" (UniqueName: \"kubernetes.io/projected/5fc3e86d-41cb-41fa-abce-4c8f9688d573-kube-api-access-w4jz6\") pod \"downloads-6594f4c75-49nzk\" (UID: \"5fc3e86d-41cb-41fa-abce-4c8f9688d573\") " pod="openshift-console/downloads-6594f4c75-49nzk"
Sep 19 08:22:13.278312 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:13.278268    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:22:13.342337 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:13.342281    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:22:13.374588 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:13.374559    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w4jz6\" (UniqueName: \"kubernetes.io/projected/5fc3e86d-41cb-41fa-abce-4c8f9688d573-kube-api-access-w4jz6\") pod \"downloads-6594f4c75-49nzk\" (UID: \"5fc3e86d-41cb-41fa-abce-4c8f9688d573\") " pod="openshift-console/downloads-6594f4c75-49nzk"
Sep 19 08:22:13.382802 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:13.382779    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w4jz6\" (UniqueName: \"kubernetes.io/projected/5fc3e86d-41cb-41fa-abce-4c8f9688d573-kube-api-access-w4jz6\") pod \"downloads-6594f4c75-49nzk\" (UID: \"5fc3e86d-41cb-41fa-abce-4c8f9688d573\") " pod="openshift-console/downloads-6594f4c75-49nzk"
Sep 19 08:22:13.506853 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:13.506825    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:22:13.550180 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:13.550148    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-6594f4c75-49nzk"
Sep 19 08:22:13.608376 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:13.608347    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:22:13.725196 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:13.725160    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:22:13.821265 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:13.821185    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-6594f4c75-49nzk"]
Sep 19 08:22:13.917057 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:13.917023    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:22:13.974641 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:13.974604    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:22:14.126341 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:14.126247    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:22:14.624763 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:14.624727    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-6594f4c75-49nzk" event={"ID":"5fc3e86d-41cb-41fa-abce-4c8f9688d573","Type":"ContainerStarted","Data":"9f12d39a2f8ade253071729a406433955a34c853b800b4a86b3fd3ee82754e12"}
Sep 19 08:22:14.724979 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:14.724946    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:22:14.920330 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:14.920230    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:22:15.345649 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:15.345616    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:22:15.446657 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:15.446621    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:22:16.942876 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:16.942839    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:22:16.973042 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:16.973009    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-6thbg_3135bdcd-43c1-4e88-9ec1-0a9a5d2ecdb0/serve-healthcheck-canary/0.log"
Sep 19 08:22:17.044799 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:17.044762    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:22:17.759713 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:17.759682    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-8566d48d94-7ksk8"
Sep 19 08:22:17.845347 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:17.845293    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:22:17.947537 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:17.947502    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:22:18.025565 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:18.025473    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:22:19.979178 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:19.979136    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:22:20.078602 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:20.078568    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:22:20.449677 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:20.449639    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7c65bcd4b6-9dslq"]
Sep 19 08:22:20.500516 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:20.500475    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7c65bcd4b6-9dslq"]
Sep 19 08:22:20.500709 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:20.500613    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c65bcd4b6-9dslq"
Sep 19 08:22:20.503526 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:20.503499    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 19 08:22:20.503658 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:20.503497    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-b5q8k\""
Sep 19 08:22:20.504370 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:20.504199    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 19 08:22:20.504370 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:20.504199    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 19 08:22:20.504370 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:20.504278    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 19 08:22:20.504370 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:20.504317    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 19 08:22:20.534679 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:20.534640    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7cc3aa7c-e18a-4b85-b6c3-2289c4c61e3c-console-config\") pod \"console-7c65bcd4b6-9dslq\" (UID: \"7cc3aa7c-e18a-4b85-b6c3-2289c4c61e3c\") " pod="openshift-console/console-7c65bcd4b6-9dslq"
Sep 19 08:22:20.534679 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:20.534674    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7cc3aa7c-e18a-4b85-b6c3-2289c4c61e3c-service-ca\") pod \"console-7c65bcd4b6-9dslq\" (UID: \"7cc3aa7c-e18a-4b85-b6c3-2289c4c61e3c\") " pod="openshift-console/console-7c65bcd4b6-9dslq"
Sep 19 08:22:20.534899 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:20.534703    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7cc3aa7c-e18a-4b85-b6c3-2289c4c61e3c-oauth-serving-cert\") pod \"console-7c65bcd4b6-9dslq\" (UID: \"7cc3aa7c-e18a-4b85-b6c3-2289c4c61e3c\") " pod="openshift-console/console-7c65bcd4b6-9dslq"
Sep 19 08:22:20.534899 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:20.534765    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-npwrp\" (UniqueName: \"kubernetes.io/projected/7cc3aa7c-e18a-4b85-b6c3-2289c4c61e3c-kube-api-access-npwrp\") pod \"console-7c65bcd4b6-9dslq\" (UID: \"7cc3aa7c-e18a-4b85-b6c3-2289c4c61e3c\") " pod="openshift-console/console-7c65bcd4b6-9dslq"
Sep 19 08:22:20.534899 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:20.534830    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7cc3aa7c-e18a-4b85-b6c3-2289c4c61e3c-console-serving-cert\") pod \"console-7c65bcd4b6-9dslq\" (UID: \"7cc3aa7c-e18a-4b85-b6c3-2289c4c61e3c\") " pod="openshift-console/console-7c65bcd4b6-9dslq"
Sep 19 08:22:20.534899 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:20.534847    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7cc3aa7c-e18a-4b85-b6c3-2289c4c61e3c-console-oauth-config\") pod \"console-7c65bcd4b6-9dslq\" (UID: \"7cc3aa7c-e18a-4b85-b6c3-2289c4c61e3c\") " pod="openshift-console/console-7c65bcd4b6-9dslq"
Sep 19 08:22:20.635209 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:20.635175    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7cc3aa7c-e18a-4b85-b6c3-2289c4c61e3c-console-serving-cert\") pod \"console-7c65bcd4b6-9dslq\" (UID: \"7cc3aa7c-e18a-4b85-b6c3-2289c4c61e3c\") " pod="openshift-console/console-7c65bcd4b6-9dslq"
Sep 19 08:22:20.635209 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:20.635219    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7cc3aa7c-e18a-4b85-b6c3-2289c4c61e3c-console-oauth-config\") pod \"console-7c65bcd4b6-9dslq\" (UID: \"7cc3aa7c-e18a-4b85-b6c3-2289c4c61e3c\") " pod="openshift-console/console-7c65bcd4b6-9dslq"
Sep 19 08:22:20.635479 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:20.635318    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7cc3aa7c-e18a-4b85-b6c3-2289c4c61e3c-console-config\") pod \"console-7c65bcd4b6-9dslq\" (UID: \"7cc3aa7c-e18a-4b85-b6c3-2289c4c61e3c\") " pod="openshift-console/console-7c65bcd4b6-9dslq"
Sep 19 08:22:20.635479 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:20.635343    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7cc3aa7c-e18a-4b85-b6c3-2289c4c61e3c-service-ca\") pod \"console-7c65bcd4b6-9dslq\" (UID: \"7cc3aa7c-e18a-4b85-b6c3-2289c4c61e3c\") " pod="openshift-console/console-7c65bcd4b6-9dslq"
Sep 19 08:22:20.635479 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:20.635379    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7cc3aa7c-e18a-4b85-b6c3-2289c4c61e3c-oauth-serving-cert\") pod \"console-7c65bcd4b6-9dslq\" (UID: \"7cc3aa7c-e18a-4b85-b6c3-2289c4c61e3c\") " pod="openshift-console/console-7c65bcd4b6-9dslq"
Sep 19 08:22:20.635479 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:20.635408    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-npwrp\" (UniqueName: \"kubernetes.io/projected/7cc3aa7c-e18a-4b85-b6c3-2289c4c61e3c-kube-api-access-npwrp\") pod \"console-7c65bcd4b6-9dslq\" (UID: \"7cc3aa7c-e18a-4b85-b6c3-2289c4c61e3c\") " pod="openshift-console/console-7c65bcd4b6-9dslq"
Sep 19 08:22:20.636281 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:20.636251    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7cc3aa7c-e18a-4b85-b6c3-2289c4c61e3c-oauth-serving-cert\") pod \"console-7c65bcd4b6-9dslq\" (UID: \"7cc3aa7c-e18a-4b85-b6c3-2289c4c61e3c\") " pod="openshift-console/console-7c65bcd4b6-9dslq"
Sep 19 08:22:20.636430 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:20.636265    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7cc3aa7c-e18a-4b85-b6c3-2289c4c61e3c-service-ca\") pod \"console-7c65bcd4b6-9dslq\" (UID: \"7cc3aa7c-e18a-4b85-b6c3-2289c4c61e3c\") " pod="openshift-console/console-7c65bcd4b6-9dslq"
Sep 19 08:22:20.636430 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:20.636418    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7cc3aa7c-e18a-4b85-b6c3-2289c4c61e3c-console-config\") pod \"console-7c65bcd4b6-9dslq\" (UID: \"7cc3aa7c-e18a-4b85-b6c3-2289c4c61e3c\") " pod="openshift-console/console-7c65bcd4b6-9dslq"
Sep 19 08:22:20.638060 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:20.638032    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7cc3aa7c-e18a-4b85-b6c3-2289c4c61e3c-console-oauth-config\") pod \"console-7c65bcd4b6-9dslq\" (UID: \"7cc3aa7c-e18a-4b85-b6c3-2289c4c61e3c\") " pod="openshift-console/console-7c65bcd4b6-9dslq"
Sep 19 08:22:20.638172 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:20.638122    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7cc3aa7c-e18a-4b85-b6c3-2289c4c61e3c-console-serving-cert\") pod \"console-7c65bcd4b6-9dslq\" (UID: \"7cc3aa7c-e18a-4b85-b6c3-2289c4c61e3c\") " pod="openshift-console/console-7c65bcd4b6-9dslq"
Sep 19 08:22:20.644362 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:20.644335    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-npwrp\" (UniqueName: \"kubernetes.io/projected/7cc3aa7c-e18a-4b85-b6c3-2289c4c61e3c-kube-api-access-npwrp\") pod \"console-7c65bcd4b6-9dslq\" (UID: \"7cc3aa7c-e18a-4b85-b6c3-2289c4c61e3c\") " pod="openshift-console/console-7c65bcd4b6-9dslq"
Sep 19 08:22:20.812400 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:20.812363    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c65bcd4b6-9dslq"
Sep 19 08:22:21.125496 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:21.125464    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7c65bcd4b6-9dslq"]
Sep 19 08:22:21.129906 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:22:21.129862    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7cc3aa7c_e18a_4b85_b6c3_2289c4c61e3c.slice/crio-3049a5361ca800239e2af0849c82dbf6d78813a4b1a612134ca47f95639c5051 WatchSource:0}: Error finding container 3049a5361ca800239e2af0849c82dbf6d78813a4b1a612134ca47f95639c5051: Status 404 returned error can't find the container with id 3049a5361ca800239e2af0849c82dbf6d78813a4b1a612134ca47f95639c5051
Sep 19 08:22:21.655514 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:21.655438    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c65bcd4b6-9dslq" event={"ID":"7cc3aa7c-e18a-4b85-b6c3-2289c4c61e3c","Type":"ContainerStarted","Data":"3049a5361ca800239e2af0849c82dbf6d78813a4b1a612134ca47f95639c5051"}
Sep 19 08:22:24.744607 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:24.744564    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:22:24.810965 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:24.810899    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:22:25.884962 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:25.884918    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/79f6ab54-a988-4015-81c2-da35716eab52-metrics-certs\") pod \"network-metrics-daemon-md9ln\" (UID: \"79f6ab54-a988-4015-81c2-da35716eab52\") " pod="openshift-multus/network-metrics-daemon-md9ln"
Sep 19 08:22:25.887448 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:25.887426    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/79f6ab54-a988-4015-81c2-da35716eab52-metrics-certs\") pod \"network-metrics-daemon-md9ln\" (UID: \"79f6ab54-a988-4015-81c2-da35716eab52\") " pod="openshift-multus/network-metrics-daemon-md9ln"
Sep 19 08:22:26.022754 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:26.022718    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-2vrmn\""
Sep 19 08:22:26.030753 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:26.030725    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-md9ln"
Sep 19 08:22:27.148497 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:27.148461    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-759694f697-lgcmq"]
Sep 19 08:22:27.150862 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:27.150842    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-759694f697-lgcmq"
Sep 19 08:22:27.158882 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:27.158653    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 19 08:22:27.164228 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:27.164203    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-759694f697-lgcmq"]
Sep 19 08:22:27.164997 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:27.164964    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:22:27.194181 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:27.194144    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cbb765d9-527c-4aef-8254-e2777d4e14ac-service-ca\") pod \"console-759694f697-lgcmq\" (UID: \"cbb765d9-527c-4aef-8254-e2777d4e14ac\") " pod="openshift-console/console-759694f697-lgcmq"
Sep 19 08:22:27.194181 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:27.194186    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cbb765d9-527c-4aef-8254-e2777d4e14ac-oauth-serving-cert\") pod \"console-759694f697-lgcmq\" (UID: \"cbb765d9-527c-4aef-8254-e2777d4e14ac\") " pod="openshift-console/console-759694f697-lgcmq"
Sep 19 08:22:27.194419 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:27.194211    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cbb765d9-527c-4aef-8254-e2777d4e14ac-console-serving-cert\") pod \"console-759694f697-lgcmq\" (UID: \"cbb765d9-527c-4aef-8254-e2777d4e14ac\") " pod="openshift-console/console-759694f697-lgcmq"
Sep 19 08:22:27.194419 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:27.194264    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cbb765d9-527c-4aef-8254-e2777d4e14ac-console-oauth-config\") pod \"console-759694f697-lgcmq\" (UID: \"cbb765d9-527c-4aef-8254-e2777d4e14ac\") " pod="openshift-console/console-759694f697-lgcmq"
Sep 19 08:22:27.194419 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:27.194357    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cbb765d9-527c-4aef-8254-e2777d4e14ac-console-config\") pod \"console-759694f697-lgcmq\" (UID: \"cbb765d9-527c-4aef-8254-e2777d4e14ac\") " pod="openshift-console/console-759694f697-lgcmq"
Sep 19 08:22:27.194419 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:27.194407    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nf8td\" (UniqueName: \"kubernetes.io/projected/cbb765d9-527c-4aef-8254-e2777d4e14ac-kube-api-access-nf8td\") pod \"console-759694f697-lgcmq\" (UID: \"cbb765d9-527c-4aef-8254-e2777d4e14ac\") " pod="openshift-console/console-759694f697-lgcmq"
Sep 19 08:22:27.194580 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:27.194463    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cbb765d9-527c-4aef-8254-e2777d4e14ac-trusted-ca-bundle\") pod \"console-759694f697-lgcmq\" (UID: \"cbb765d9-527c-4aef-8254-e2777d4e14ac\") " pod="openshift-console/console-759694f697-lgcmq"
Sep 19 08:22:27.246287 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:27.246238    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:22:27.295095 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:27.295057    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cbb765d9-527c-4aef-8254-e2777d4e14ac-console-config\") pod \"console-759694f697-lgcmq\" (UID: \"cbb765d9-527c-4aef-8254-e2777d4e14ac\") " pod="openshift-console/console-759694f697-lgcmq"
Sep 19 08:22:27.295321 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:27.295121    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nf8td\" (UniqueName: \"kubernetes.io/projected/cbb765d9-527c-4aef-8254-e2777d4e14ac-kube-api-access-nf8td\") pod \"console-759694f697-lgcmq\" (UID: \"cbb765d9-527c-4aef-8254-e2777d4e14ac\") " pod="openshift-console/console-759694f697-lgcmq"
Sep 19 08:22:27.295321 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:27.295148    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cbb765d9-527c-4aef-8254-e2777d4e14ac-trusted-ca-bundle\") pod \"console-759694f697-lgcmq\" (UID: \"cbb765d9-527c-4aef-8254-e2777d4e14ac\") " pod="openshift-console/console-759694f697-lgcmq"
Sep 19 08:22:27.295321 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:27.295205    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cbb765d9-527c-4aef-8254-e2777d4e14ac-service-ca\") pod \"console-759694f697-lgcmq\" (UID: \"cbb765d9-527c-4aef-8254-e2777d4e14ac\") " pod="openshift-console/console-759694f697-lgcmq"
Sep 19 08:22:27.295490 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:27.295349    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cbb765d9-527c-4aef-8254-e2777d4e14ac-oauth-serving-cert\") pod \"console-759694f697-lgcmq\" (UID: \"cbb765d9-527c-4aef-8254-e2777d4e14ac\") " pod="openshift-console/console-759694f697-lgcmq"
Sep 19 08:22:27.295490 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:27.295410    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cbb765d9-527c-4aef-8254-e2777d4e14ac-console-serving-cert\") pod \"console-759694f697-lgcmq\" (UID: \"cbb765d9-527c-4aef-8254-e2777d4e14ac\") " pod="openshift-console/console-759694f697-lgcmq"
Sep 19 08:22:27.295490 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:27.295439    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cbb765d9-527c-4aef-8254-e2777d4e14ac-console-oauth-config\") pod \"console-759694f697-lgcmq\" (UID: \"cbb765d9-527c-4aef-8254-e2777d4e14ac\") " pod="openshift-console/console-759694f697-lgcmq"
Sep 19 08:22:27.295918 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:27.295859    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cbb765d9-527c-4aef-8254-e2777d4e14ac-console-config\") pod \"console-759694f697-lgcmq\" (UID: \"cbb765d9-527c-4aef-8254-e2777d4e14ac\") " pod="openshift-console/console-759694f697-lgcmq"
Sep 19 08:22:27.296117 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:27.296090    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cbb765d9-527c-4aef-8254-e2777d4e14ac-trusted-ca-bundle\") pod \"console-759694f697-lgcmq\" (UID: \"cbb765d9-527c-4aef-8254-e2777d4e14ac\") " pod="openshift-console/console-759694f697-lgcmq"
Sep 19 08:22:27.296200 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:27.296103    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cbb765d9-527c-4aef-8254-e2777d4e14ac-service-ca\") pod \"console-759694f697-lgcmq\" (UID: \"cbb765d9-527c-4aef-8254-e2777d4e14ac\") " pod="openshift-console/console-759694f697-lgcmq"
Sep 19 08:22:27.296440 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:27.296418    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cbb765d9-527c-4aef-8254-e2777d4e14ac-oauth-serving-cert\") pod \"console-759694f697-lgcmq\" (UID: \"cbb765d9-527c-4aef-8254-e2777d4e14ac\") " pod="openshift-console/console-759694f697-lgcmq"
Sep 19 08:22:27.298590 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:27.298547    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cbb765d9-527c-4aef-8254-e2777d4e14ac-console-serving-cert\") pod \"console-759694f697-lgcmq\" (UID: \"cbb765d9-527c-4aef-8254-e2777d4e14ac\") " pod="openshift-console/console-759694f697-lgcmq"
Sep 19 08:22:27.298590 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:27.298569    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cbb765d9-527c-4aef-8254-e2777d4e14ac-console-oauth-config\") pod \"console-759694f697-lgcmq\" (UID: \"cbb765d9-527c-4aef-8254-e2777d4e14ac\") " pod="openshift-console/console-759694f697-lgcmq"
Sep 19 08:22:27.307464 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:27.307432    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nf8td\" (UniqueName: \"kubernetes.io/projected/cbb765d9-527c-4aef-8254-e2777d4e14ac-kube-api-access-nf8td\") pod \"console-759694f697-lgcmq\" (UID: \"cbb765d9-527c-4aef-8254-e2777d4e14ac\") " pod="openshift-console/console-759694f697-lgcmq"
Sep 19 08:22:27.461582 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:27.461492    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-759694f697-lgcmq"
Sep 19 08:22:27.540750 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:27.540709    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:22:31.889858 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:31.889817    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:22:31.987066 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:31.987029    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:22:32.514079 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:32.514049    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-759694f697-lgcmq"]
Sep 19 08:22:32.516590 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:32.516546    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-md9ln"]
Sep 19 08:22:32.517579 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:22:32.517538    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcbb765d9_527c_4aef_8254_e2777d4e14ac.slice/crio-def703458d3a625c69881a96c30a1b608bbfc1de5d8d09866df725e784b0017e WatchSource:0}: Error finding container def703458d3a625c69881a96c30a1b608bbfc1de5d8d09866df725e784b0017e: Status 404 returned error can't find the container with id def703458d3a625c69881a96c30a1b608bbfc1de5d8d09866df725e784b0017e
Sep 19 08:22:32.520269 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:22:32.520242    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79f6ab54_a988_4015_81c2_da35716eab52.slice/crio-1337dc6edfe012b9ac7c0a2328755d18173cd48737d9aaaeb0ed4bee5adb11c5 WatchSource:0}: Error finding container 1337dc6edfe012b9ac7c0a2328755d18173cd48737d9aaaeb0ed4bee5adb11c5: Status 404 returned error can't find the container with id 1337dc6edfe012b9ac7c0a2328755d18173cd48737d9aaaeb0ed4bee5adb11c5
Sep 19 08:22:32.704481 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:32.704442    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-md9ln" event={"ID":"79f6ab54-a988-4015-81c2-da35716eab52","Type":"ContainerStarted","Data":"1337dc6edfe012b9ac7c0a2328755d18173cd48737d9aaaeb0ed4bee5adb11c5"}
Sep 19 08:22:32.706397 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:32.706367    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-6594f4c75-49nzk" event={"ID":"5fc3e86d-41cb-41fa-abce-4c8f9688d573","Type":"ContainerStarted","Data":"d4042108adb0caa2a436e69d277c1ff1f3cf89e56cd7e9bbed024b0b4f33f22e"}
Sep 19 08:22:32.706737 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:32.706675    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-6594f4c75-49nzk"
Sep 19 08:22:32.708425 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:32.708397    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-759694f697-lgcmq" event={"ID":"cbb765d9-527c-4aef-8254-e2777d4e14ac","Type":"ContainerStarted","Data":"ee543bbf0d59b914b1fa954334f945d64a4cbd3369d04e6796a15dcd46ef921d"}
Sep 19 08:22:32.708559 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:32.708441    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-759694f697-lgcmq" event={"ID":"cbb765d9-527c-4aef-8254-e2777d4e14ac","Type":"ContainerStarted","Data":"def703458d3a625c69881a96c30a1b608bbfc1de5d8d09866df725e784b0017e"}
Sep 19 08:22:32.710742 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:32.710715    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c65bcd4b6-9dslq" event={"ID":"7cc3aa7c-e18a-4b85-b6c3-2289c4c61e3c","Type":"ContainerStarted","Data":"640d477e1f0cc71a01d557ca994e8d6930a4031f3722eae6950aeac952e0a7cc"}
Sep 19 08:22:32.724341 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:32.724292    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-6594f4c75-49nzk"
Sep 19 08:22:32.726756 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:32.726696    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-6594f4c75-49nzk" podStartSLOduration=1.259770801 podStartE2EDuration="19.726680683s" totalImagesPullingTime="18.466909882s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 08:22:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 08:22:32.724423973 +0000 UTC m=+137.101133826" watchObservedRunningTime="2026-09-19 08:22:32.726680683 +0000 UTC m=+137.103390545"
Sep 19 08:22:32.744029 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:32.743981    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7c65bcd4b6-9dslq" podStartSLOduration=1.592452062 podStartE2EDuration="12.743968588s" totalImagesPullingTime="11.151516526s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 08:22:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 08:22:32.74293908 +0000 UTC m=+137.119648942" watchObservedRunningTime="2026-09-19 08:22:32.743968588 +0000 UTC m=+137.120678449"
Sep 19 08:22:32.762964 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:32.762909    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-759694f697-lgcmq" podStartSLOduration=5.762890647 podStartE2EDuration="5.762890647s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 08:22:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 08:22:32.761246904 +0000 UTC m=+137.137956766" watchObservedRunningTime="2026-09-19 08:22:32.762890647 +0000 UTC m=+137.139600508"
Sep 19 08:22:32.771542 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:32.771471    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-8566d48d94-7ksk8" podUID="5dca0cc5-6589-4585-978d-ca12d129c1a0" containerName="registry" containerID="cri-o://db77ac911ea7ec2514c44873b419e79b96c890781b5f4e6fc6031faefa144fc6" gracePeriod=30
Sep 19 08:22:33.033763 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:33.033728    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-8566d48d94-7ksk8"
Sep 19 08:22:33.144135 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:33.143012    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5dca0cc5-6589-4585-978d-ca12d129c1a0-registry-tls\") pod \"5dca0cc5-6589-4585-978d-ca12d129c1a0\" (UID: \"5dca0cc5-6589-4585-978d-ca12d129c1a0\") "
Sep 19 08:22:33.144135 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:33.143066    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5dca0cc5-6589-4585-978d-ca12d129c1a0-ca-trust-extracted\") pod \"5dca0cc5-6589-4585-978d-ca12d129c1a0\" (UID: \"5dca0cc5-6589-4585-978d-ca12d129c1a0\") "
Sep 19 08:22:33.144135 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:33.143112    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5dca0cc5-6589-4585-978d-ca12d129c1a0-registry-certificates\") pod \"5dca0cc5-6589-4585-978d-ca12d129c1a0\" (UID: \"5dca0cc5-6589-4585-978d-ca12d129c1a0\") "
Sep 19 08:22:33.144135 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:33.143154    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rf4n7\" (UniqueName: \"kubernetes.io/projected/5dca0cc5-6589-4585-978d-ca12d129c1a0-kube-api-access-rf4n7\") pod \"5dca0cc5-6589-4585-978d-ca12d129c1a0\" (UID: \"5dca0cc5-6589-4585-978d-ca12d129c1a0\") "
Sep 19 08:22:33.144135 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:33.143199    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5dca0cc5-6589-4585-978d-ca12d129c1a0-image-registry-private-configuration\") pod \"5dca0cc5-6589-4585-978d-ca12d129c1a0\" (UID: \"5dca0cc5-6589-4585-978d-ca12d129c1a0\") "
Sep 19 08:22:33.144135 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:33.143225    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5dca0cc5-6589-4585-978d-ca12d129c1a0-installation-pull-secrets\") pod \"5dca0cc5-6589-4585-978d-ca12d129c1a0\" (UID: \"5dca0cc5-6589-4585-978d-ca12d129c1a0\") "
Sep 19 08:22:33.144135 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:33.143273    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5dca0cc5-6589-4585-978d-ca12d129c1a0-trusted-ca\") pod \"5dca0cc5-6589-4585-978d-ca12d129c1a0\" (UID: \"5dca0cc5-6589-4585-978d-ca12d129c1a0\") "
Sep 19 08:22:33.144135 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:33.143322    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5dca0cc5-6589-4585-978d-ca12d129c1a0-bound-sa-token\") pod \"5dca0cc5-6589-4585-978d-ca12d129c1a0\" (UID: \"5dca0cc5-6589-4585-978d-ca12d129c1a0\") "
Sep 19 08:22:33.145784 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:33.145747    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5dca0cc5-6589-4585-978d-ca12d129c1a0-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "5dca0cc5-6589-4585-978d-ca12d129c1a0" (UID: "5dca0cc5-6589-4585-978d-ca12d129c1a0"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 08:22:33.146018 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:33.145993    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5dca0cc5-6589-4585-978d-ca12d129c1a0-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "5dca0cc5-6589-4585-978d-ca12d129c1a0" (UID: "5dca0cc5-6589-4585-978d-ca12d129c1a0"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 08:22:33.149186 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:33.149149    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5dca0cc5-6589-4585-978d-ca12d129c1a0-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "5dca0cc5-6589-4585-978d-ca12d129c1a0" (UID: "5dca0cc5-6589-4585-978d-ca12d129c1a0"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 08:22:33.149392 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:33.149351    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5dca0cc5-6589-4585-978d-ca12d129c1a0-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "5dca0cc5-6589-4585-978d-ca12d129c1a0" (UID: "5dca0cc5-6589-4585-978d-ca12d129c1a0"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 08:22:33.149703 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:33.149657    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5dca0cc5-6589-4585-978d-ca12d129c1a0-kube-api-access-rf4n7" (OuterVolumeSpecName: "kube-api-access-rf4n7") pod "5dca0cc5-6589-4585-978d-ca12d129c1a0" (UID: "5dca0cc5-6589-4585-978d-ca12d129c1a0"). InnerVolumeSpecName "kube-api-access-rf4n7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 08:22:33.150005 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:33.149966    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5dca0cc5-6589-4585-978d-ca12d129c1a0-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "5dca0cc5-6589-4585-978d-ca12d129c1a0" (UID: "5dca0cc5-6589-4585-978d-ca12d129c1a0"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 08:22:33.156085 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:33.156045    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5dca0cc5-6589-4585-978d-ca12d129c1a0-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "5dca0cc5-6589-4585-978d-ca12d129c1a0" (UID: "5dca0cc5-6589-4585-978d-ca12d129c1a0"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 08:22:33.157289 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:33.157261    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5dca0cc5-6589-4585-978d-ca12d129c1a0-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "5dca0cc5-6589-4585-978d-ca12d129c1a0" (UID: "5dca0cc5-6589-4585-978d-ca12d129c1a0"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 08:22:33.244120 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:33.244083    2578 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5dca0cc5-6589-4585-978d-ca12d129c1a0-trusted-ca\") on node \"ip-10-0-139-157.ec2.internal\" DevicePath \"\""
Sep 19 08:22:33.244120 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:33.244120    2578 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5dca0cc5-6589-4585-978d-ca12d129c1a0-bound-sa-token\") on node \"ip-10-0-139-157.ec2.internal\" DevicePath \"\""
Sep 19 08:22:33.244406 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:33.244134    2578 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5dca0cc5-6589-4585-978d-ca12d129c1a0-registry-tls\") on node \"ip-10-0-139-157.ec2.internal\" DevicePath \"\""
Sep 19 08:22:33.244406 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:33.244148    2578 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5dca0cc5-6589-4585-978d-ca12d129c1a0-ca-trust-extracted\") on node \"ip-10-0-139-157.ec2.internal\" DevicePath \"\""
Sep 19 08:22:33.244406 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:33.244164    2578 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5dca0cc5-6589-4585-978d-ca12d129c1a0-registry-certificates\") on node \"ip-10-0-139-157.ec2.internal\" DevicePath \"\""
Sep 19 08:22:33.244406 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:33.244180    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rf4n7\" (UniqueName: \"kubernetes.io/projected/5dca0cc5-6589-4585-978d-ca12d129c1a0-kube-api-access-rf4n7\") on node \"ip-10-0-139-157.ec2.internal\" DevicePath \"\""
Sep 19 08:22:33.244406 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:33.244197    2578 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5dca0cc5-6589-4585-978d-ca12d129c1a0-image-registry-private-configuration\") on node \"ip-10-0-139-157.ec2.internal\" DevicePath \"\""
Sep 19 08:22:33.244406 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:33.244211    2578 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5dca0cc5-6589-4585-978d-ca12d129c1a0-installation-pull-secrets\") on node \"ip-10-0-139-157.ec2.internal\" DevicePath \"\""
Sep 19 08:22:33.719920 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:33.719880    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-md9ln" event={"ID":"79f6ab54-a988-4015-81c2-da35716eab52","Type":"ContainerStarted","Data":"fec4bee9ee9968aa73f0110e7a317e2fc035659d1de9b67964bdebca74b77cd6"}
Sep 19 08:22:33.721801 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:33.721492    2578 generic.go:356] "Generic (PLEG): container finished" podID="5dca0cc5-6589-4585-978d-ca12d129c1a0" containerID="db77ac911ea7ec2514c44873b419e79b96c890781b5f4e6fc6031faefa144fc6" exitCode=0
Sep 19 08:22:33.722040 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:33.721977    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-8566d48d94-7ksk8" event={"ID":"5dca0cc5-6589-4585-978d-ca12d129c1a0","Type":"ContainerDied","Data":"db77ac911ea7ec2514c44873b419e79b96c890781b5f4e6fc6031faefa144fc6"}
Sep 19 08:22:33.722040 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:33.722003    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-8566d48d94-7ksk8"
Sep 19 08:22:33.722188 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:33.722052    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-8566d48d94-7ksk8" event={"ID":"5dca0cc5-6589-4585-978d-ca12d129c1a0","Type":"ContainerDied","Data":"db143f9b82bf72948f8b79da06dc19668a6e0639f0c022ea591c0fba0330bec8"}
Sep 19 08:22:33.722188 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:33.722082    2578 scope.go:117] "RemoveContainer" containerID="db77ac911ea7ec2514c44873b419e79b96c890781b5f4e6fc6031faefa144fc6"
Sep 19 08:22:33.742989 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:33.742953    2578 scope.go:117] "RemoveContainer" containerID="db77ac911ea7ec2514c44873b419e79b96c890781b5f4e6fc6031faefa144fc6"
Sep 19 08:22:33.743658 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:22:33.743626    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"db77ac911ea7ec2514c44873b419e79b96c890781b5f4e6fc6031faefa144fc6\": container with ID starting with db77ac911ea7ec2514c44873b419e79b96c890781b5f4e6fc6031faefa144fc6 not found: ID does not exist" containerID="db77ac911ea7ec2514c44873b419e79b96c890781b5f4e6fc6031faefa144fc6"
Sep 19 08:22:33.743754 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:33.743688    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"db77ac911ea7ec2514c44873b419e79b96c890781b5f4e6fc6031faefa144fc6"} err="failed to get container status \"db77ac911ea7ec2514c44873b419e79b96c890781b5f4e6fc6031faefa144fc6\": rpc error: code = NotFound desc = could not find container \"db77ac911ea7ec2514c44873b419e79b96c890781b5f4e6fc6031faefa144fc6\": container with ID starting with db77ac911ea7ec2514c44873b419e79b96c890781b5f4e6fc6031faefa144fc6 not found: ID does not exist"
Sep 19 08:22:33.747993 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:33.747971    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-8566d48d94-7ksk8"]
Sep 19 08:22:33.751792 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:33.751771    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-8566d48d94-7ksk8"]
Sep 19 08:22:34.113175 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:34.113137    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5dca0cc5-6589-4585-978d-ca12d129c1a0" path="/var/lib/kubelet/pods/5dca0cc5-6589-4585-978d-ca12d129c1a0/volumes"
Sep 19 08:22:34.728763 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:34.728725    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-md9ln" event={"ID":"79f6ab54-a988-4015-81c2-da35716eab52","Type":"ContainerStarted","Data":"58d2fe1e72c6403711862916b03dbef6cc850a2233c0ed2233a13e4eb1d541d1"}
Sep 19 08:22:34.747282 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:34.747232    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-md9ln" podStartSLOduration=137.714875646 podStartE2EDuration="2m18.747216775s" totalImagesPullingTime="1.032341129s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 08:20:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 08:22:34.746532318 +0000 UTC m=+139.123242178" watchObservedRunningTime="2026-09-19 08:22:34.747216775 +0000 UTC m=+139.123926635"
Sep 19 08:22:34.841774 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:34.841739    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:22:34.912681 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:34.912600    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:22:35.734729 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:35.734698    2578 generic.go:356] "Generic (PLEG): container finished" podID="ebaf628e-811b-4dbc-a814-f257d77db389" containerID="6af39bfa22ba16e3c74fae3f073c7739f7628f86091d8d10d7a56116d4c6e143" exitCode=0
Sep 19 08:22:35.735143 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:35.734781    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-tjr4s" event={"ID":"ebaf628e-811b-4dbc-a814-f257d77db389","Type":"ContainerDied","Data":"6af39bfa22ba16e3c74fae3f073c7739f7628f86091d8d10d7a56116d4c6e143"}
Sep 19 08:22:35.735351 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:35.735328    2578 scope.go:117] "RemoveContainer" containerID="6af39bfa22ba16e3c74fae3f073c7739f7628f86091d8d10d7a56116d4c6e143"
Sep 19 08:22:36.742473 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:36.742432    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-tjr4s" event={"ID":"ebaf628e-811b-4dbc-a814-f257d77db389","Type":"ContainerStarted","Data":"2b20b96f7e123d153ae1aff14b7034046c8c23d350993f6eccc3a199a034f421"}
Sep 19 08:22:37.462100 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:37.462057    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-759694f697-lgcmq"
Sep 19 08:22:37.462292 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:37.462129    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-759694f697-lgcmq"
Sep 19 08:22:37.467515 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:37.467491    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-759694f697-lgcmq"
Sep 19 08:22:37.752909 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:37.752832    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-759694f697-lgcmq"
Sep 19 08:22:37.801409 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:37.801370    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7c65bcd4b6-9dslq"]
Sep 19 08:22:40.812922 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:40.812888    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7c65bcd4b6-9dslq"
Sep 19 08:22:42.969153 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:42.969108    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:22:43.943780 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:43.943731    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:22:44.004905 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:44.004872    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:22:44.737694 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:44.737647    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:22:44.790465 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:44.790431    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:22:46.923066 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:46.923030    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:22:47.026600 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:22:47.026550    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:23:01.880893 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:23:01.880849    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:23:01.981979 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:23:01.981940    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:23:02.831968 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:23:02.831920    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7c65bcd4b6-9dslq" podUID="7cc3aa7c-e18a-4b85-b6c3-2289c4c61e3c" containerName="console" containerID="cri-o://640d477e1f0cc71a01d557ca994e8d6930a4031f3722eae6950aeac952e0a7cc" gracePeriod=15
Sep 19 08:23:03.092818 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:23:03.092760    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c65bcd4b6-9dslq"
Sep 19 08:23:03.181500 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:23:03.181469    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7cc3aa7c-e18a-4b85-b6c3-2289c4c61e3c-service-ca\") pod \"7cc3aa7c-e18a-4b85-b6c3-2289c4c61e3c\" (UID: \"7cc3aa7c-e18a-4b85-b6c3-2289c4c61e3c\") "
Sep 19 08:23:03.181656 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:23:03.181519    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-npwrp\" (UniqueName: \"kubernetes.io/projected/7cc3aa7c-e18a-4b85-b6c3-2289c4c61e3c-kube-api-access-npwrp\") pod \"7cc3aa7c-e18a-4b85-b6c3-2289c4c61e3c\" (UID: \"7cc3aa7c-e18a-4b85-b6c3-2289c4c61e3c\") "
Sep 19 08:23:03.181656 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:23:03.181553    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7cc3aa7c-e18a-4b85-b6c3-2289c4c61e3c-console-oauth-config\") pod \"7cc3aa7c-e18a-4b85-b6c3-2289c4c61e3c\" (UID: \"7cc3aa7c-e18a-4b85-b6c3-2289c4c61e3c\") "
Sep 19 08:23:03.181656 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:23:03.181569    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7cc3aa7c-e18a-4b85-b6c3-2289c4c61e3c-console-serving-cert\") pod \"7cc3aa7c-e18a-4b85-b6c3-2289c4c61e3c\" (UID: \"7cc3aa7c-e18a-4b85-b6c3-2289c4c61e3c\") "
Sep 19 08:23:03.181656 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:23:03.181592    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7cc3aa7c-e18a-4b85-b6c3-2289c4c61e3c-oauth-serving-cert\") pod \"7cc3aa7c-e18a-4b85-b6c3-2289c4c61e3c\" (UID: \"7cc3aa7c-e18a-4b85-b6c3-2289c4c61e3c\") "
Sep 19 08:23:03.181656 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:23:03.181624    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7cc3aa7c-e18a-4b85-b6c3-2289c4c61e3c-console-config\") pod \"7cc3aa7c-e18a-4b85-b6c3-2289c4c61e3c\" (UID: \"7cc3aa7c-e18a-4b85-b6c3-2289c4c61e3c\") "
Sep 19 08:23:03.182034 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:23:03.181911    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7cc3aa7c-e18a-4b85-b6c3-2289c4c61e3c-service-ca" (OuterVolumeSpecName: "service-ca") pod "7cc3aa7c-e18a-4b85-b6c3-2289c4c61e3c" (UID: "7cc3aa7c-e18a-4b85-b6c3-2289c4c61e3c"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 08:23:03.182034 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:23:03.182003    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7cc3aa7c-e18a-4b85-b6c3-2289c4c61e3c-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "7cc3aa7c-e18a-4b85-b6c3-2289c4c61e3c" (UID: "7cc3aa7c-e18a-4b85-b6c3-2289c4c61e3c"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 08:23:03.182185 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:23:03.182082    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7cc3aa7c-e18a-4b85-b6c3-2289c4c61e3c-console-config" (OuterVolumeSpecName: "console-config") pod "7cc3aa7c-e18a-4b85-b6c3-2289c4c61e3c" (UID: "7cc3aa7c-e18a-4b85-b6c3-2289c4c61e3c"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 08:23:03.183880 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:23:03.183860    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7cc3aa7c-e18a-4b85-b6c3-2289c4c61e3c-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "7cc3aa7c-e18a-4b85-b6c3-2289c4c61e3c" (UID: "7cc3aa7c-e18a-4b85-b6c3-2289c4c61e3c"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 08:23:03.184102 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:23:03.184081    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7cc3aa7c-e18a-4b85-b6c3-2289c4c61e3c-kube-api-access-npwrp" (OuterVolumeSpecName: "kube-api-access-npwrp") pod "7cc3aa7c-e18a-4b85-b6c3-2289c4c61e3c" (UID: "7cc3aa7c-e18a-4b85-b6c3-2289c4c61e3c"). InnerVolumeSpecName "kube-api-access-npwrp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 08:23:03.184156 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:23:03.184141    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7cc3aa7c-e18a-4b85-b6c3-2289c4c61e3c-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "7cc3aa7c-e18a-4b85-b6c3-2289c4c61e3c" (UID: "7cc3aa7c-e18a-4b85-b6c3-2289c4c61e3c"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 08:23:03.282652 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:23:03.282624    2578 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7cc3aa7c-e18a-4b85-b6c3-2289c4c61e3c-console-oauth-config\") on node \"ip-10-0-139-157.ec2.internal\" DevicePath \"\""
Sep 19 08:23:03.282652 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:23:03.282649    2578 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7cc3aa7c-e18a-4b85-b6c3-2289c4c61e3c-console-serving-cert\") on node \"ip-10-0-139-157.ec2.internal\" DevicePath \"\""
Sep 19 08:23:03.282863 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:23:03.282661    2578 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7cc3aa7c-e18a-4b85-b6c3-2289c4c61e3c-oauth-serving-cert\") on node \"ip-10-0-139-157.ec2.internal\" DevicePath \"\""
Sep 19 08:23:03.282863 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:23:03.282673    2578 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7cc3aa7c-e18a-4b85-b6c3-2289c4c61e3c-console-config\") on node \"ip-10-0-139-157.ec2.internal\" DevicePath \"\""
Sep 19 08:23:03.282863 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:23:03.282685    2578 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7cc3aa7c-e18a-4b85-b6c3-2289c4c61e3c-service-ca\") on node \"ip-10-0-139-157.ec2.internal\" DevicePath \"\""
Sep 19 08:23:03.282863 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:23:03.282696    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-npwrp\" (UniqueName: \"kubernetes.io/projected/7cc3aa7c-e18a-4b85-b6c3-2289c4c61e3c-kube-api-access-npwrp\") on node \"ip-10-0-139-157.ec2.internal\" DevicePath \"\""
Sep 19 08:23:03.848631 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:23:03.848597    2578 generic.go:356] "Generic (PLEG): container finished" podID="7cc3aa7c-e18a-4b85-b6c3-2289c4c61e3c" containerID="640d477e1f0cc71a01d557ca994e8d6930a4031f3722eae6950aeac952e0a7cc" exitCode=0
Sep 19 08:23:03.848847 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:23:03.848652    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c65bcd4b6-9dslq"
Sep 19 08:23:03.848847 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:23:03.848672    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c65bcd4b6-9dslq" event={"ID":"7cc3aa7c-e18a-4b85-b6c3-2289c4c61e3c","Type":"ContainerDied","Data":"640d477e1f0cc71a01d557ca994e8d6930a4031f3722eae6950aeac952e0a7cc"}
Sep 19 08:23:03.848847 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:23:03.848745    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c65bcd4b6-9dslq" event={"ID":"7cc3aa7c-e18a-4b85-b6c3-2289c4c61e3c","Type":"ContainerDied","Data":"3049a5361ca800239e2af0849c82dbf6d78813a4b1a612134ca47f95639c5051"}
Sep 19 08:23:03.848847 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:23:03.848764    2578 scope.go:117] "RemoveContainer" containerID="640d477e1f0cc71a01d557ca994e8d6930a4031f3722eae6950aeac952e0a7cc"
Sep 19 08:23:03.864814 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:23:03.864797    2578 scope.go:117] "RemoveContainer" containerID="640d477e1f0cc71a01d557ca994e8d6930a4031f3722eae6950aeac952e0a7cc"
Sep 19 08:23:03.865078 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:23:03.865060    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"640d477e1f0cc71a01d557ca994e8d6930a4031f3722eae6950aeac952e0a7cc\": container with ID starting with 640d477e1f0cc71a01d557ca994e8d6930a4031f3722eae6950aeac952e0a7cc not found: ID does not exist" containerID="640d477e1f0cc71a01d557ca994e8d6930a4031f3722eae6950aeac952e0a7cc"
Sep 19 08:23:03.865124 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:23:03.865089    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"640d477e1f0cc71a01d557ca994e8d6930a4031f3722eae6950aeac952e0a7cc"} err="failed to get container status \"640d477e1f0cc71a01d557ca994e8d6930a4031f3722eae6950aeac952e0a7cc\": rpc error: code = NotFound desc = could not find container \"640d477e1f0cc71a01d557ca994e8d6930a4031f3722eae6950aeac952e0a7cc\": container with ID starting with 640d477e1f0cc71a01d557ca994e8d6930a4031f3722eae6950aeac952e0a7cc not found: ID does not exist"
Sep 19 08:23:03.867830 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:23:03.867807    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7c65bcd4b6-9dslq"]
Sep 19 08:23:03.874196 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:23:03.874176    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7c65bcd4b6-9dslq"]
Sep 19 08:23:04.112846 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:23:04.112766    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7cc3aa7c-e18a-4b85-b6c3-2289c4c61e3c" path="/var/lib/kubelet/pods/7cc3aa7c-e18a-4b85-b6c3-2289c4c61e3c/volumes"
Sep 19 08:23:12.023147 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:23:12.023067    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:23:12.258965 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:23:12.258933    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:23:20.599509 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:23:20.599477    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-77b9f76cdc-wkbbl"]
Sep 19 08:23:20.600060 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:23:20.599841    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7cc3aa7c-e18a-4b85-b6c3-2289c4c61e3c" containerName="console"
Sep 19 08:23:20.600060 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:23:20.599859    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="7cc3aa7c-e18a-4b85-b6c3-2289c4c61e3c" containerName="console"
Sep 19 08:23:20.600060 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:23:20.599873    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5dca0cc5-6589-4585-978d-ca12d129c1a0" containerName="registry"
Sep 19 08:23:20.600060 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:23:20.599880    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="5dca0cc5-6589-4585-978d-ca12d129c1a0" containerName="registry"
Sep 19 08:23:20.600060 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:23:20.599959    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="7cc3aa7c-e18a-4b85-b6c3-2289c4c61e3c" containerName="console"
Sep 19 08:23:20.600060 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:23:20.599974    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="5dca0cc5-6589-4585-978d-ca12d129c1a0" containerName="registry"
Sep 19 08:23:20.602632 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:23:20.602610    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77b9f76cdc-wkbbl"
Sep 19 08:23:20.606651 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:23:20.606630    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4031b492-0293-402a-af98-078d9e6e9368-oauth-serving-cert\") pod \"console-77b9f76cdc-wkbbl\" (UID: \"4031b492-0293-402a-af98-078d9e6e9368\") " pod="openshift-console/console-77b9f76cdc-wkbbl"
Sep 19 08:23:20.606757 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:23:20.606662    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h7rpx\" (UniqueName: \"kubernetes.io/projected/4031b492-0293-402a-af98-078d9e6e9368-kube-api-access-h7rpx\") pod \"console-77b9f76cdc-wkbbl\" (UID: \"4031b492-0293-402a-af98-078d9e6e9368\") " pod="openshift-console/console-77b9f76cdc-wkbbl"
Sep 19 08:23:20.606757 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:23:20.606695    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4031b492-0293-402a-af98-078d9e6e9368-console-config\") pod \"console-77b9f76cdc-wkbbl\" (UID: \"4031b492-0293-402a-af98-078d9e6e9368\") " pod="openshift-console/console-77b9f76cdc-wkbbl"
Sep 19 08:23:20.606835 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:23:20.606761    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4031b492-0293-402a-af98-078d9e6e9368-console-serving-cert\") pod \"console-77b9f76cdc-wkbbl\" (UID: \"4031b492-0293-402a-af98-078d9e6e9368\") " pod="openshift-console/console-77b9f76cdc-wkbbl"
Sep 19 08:23:20.606835 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:23:20.606799    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4031b492-0293-402a-af98-078d9e6e9368-console-oauth-config\") pod \"console-77b9f76cdc-wkbbl\" (UID: \"4031b492-0293-402a-af98-078d9e6e9368\") " pod="openshift-console/console-77b9f76cdc-wkbbl"
Sep 19 08:23:20.606910 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:23:20.606839    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4031b492-0293-402a-af98-078d9e6e9368-service-ca\") pod \"console-77b9f76cdc-wkbbl\" (UID: \"4031b492-0293-402a-af98-078d9e6e9368\") " pod="openshift-console/console-77b9f76cdc-wkbbl"
Sep 19 08:23:20.606910 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:23:20.606864    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4031b492-0293-402a-af98-078d9e6e9368-trusted-ca-bundle\") pod \"console-77b9f76cdc-wkbbl\" (UID: \"4031b492-0293-402a-af98-078d9e6e9368\") " pod="openshift-console/console-77b9f76cdc-wkbbl"
Sep 19 08:23:20.614321 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:23:20.614284    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-77b9f76cdc-wkbbl"]
Sep 19 08:23:20.655567 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:23:20.655541    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:23:20.707378 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:23:20.707343    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4031b492-0293-402a-af98-078d9e6e9368-console-config\") pod \"console-77b9f76cdc-wkbbl\" (UID: \"4031b492-0293-402a-af98-078d9e6e9368\") " pod="openshift-console/console-77b9f76cdc-wkbbl"
Sep 19 08:23:20.707378 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:23:20.707381    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4031b492-0293-402a-af98-078d9e6e9368-console-serving-cert\") pod \"console-77b9f76cdc-wkbbl\" (UID: \"4031b492-0293-402a-af98-078d9e6e9368\") " pod="openshift-console/console-77b9f76cdc-wkbbl"
Sep 19 08:23:20.707554 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:23:20.707396    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4031b492-0293-402a-af98-078d9e6e9368-console-oauth-config\") pod \"console-77b9f76cdc-wkbbl\" (UID: \"4031b492-0293-402a-af98-078d9e6e9368\") " pod="openshift-console/console-77b9f76cdc-wkbbl"
Sep 19 08:23:20.707554 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:23:20.707426    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4031b492-0293-402a-af98-078d9e6e9368-service-ca\") pod \"console-77b9f76cdc-wkbbl\" (UID: \"4031b492-0293-402a-af98-078d9e6e9368\") " pod="openshift-console/console-77b9f76cdc-wkbbl"
Sep 19 08:23:20.707640 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:23:20.707542    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4031b492-0293-402a-af98-078d9e6e9368-trusted-ca-bundle\") pod \"console-77b9f76cdc-wkbbl\" (UID: \"4031b492-0293-402a-af98-078d9e6e9368\") " pod="openshift-console/console-77b9f76cdc-wkbbl"
Sep 19 08:23:20.707685 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:23:20.707638    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4031b492-0293-402a-af98-078d9e6e9368-oauth-serving-cert\") pod \"console-77b9f76cdc-wkbbl\" (UID: \"4031b492-0293-402a-af98-078d9e6e9368\") " pod="openshift-console/console-77b9f76cdc-wkbbl"
Sep 19 08:23:20.707685 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:23:20.707669    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h7rpx\" (UniqueName: \"kubernetes.io/projected/4031b492-0293-402a-af98-078d9e6e9368-kube-api-access-h7rpx\") pod \"console-77b9f76cdc-wkbbl\" (UID: \"4031b492-0293-402a-af98-078d9e6e9368\") " pod="openshift-console/console-77b9f76cdc-wkbbl"
Sep 19 08:23:20.708145 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:23:20.708125    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4031b492-0293-402a-af98-078d9e6e9368-console-config\") pod \"console-77b9f76cdc-wkbbl\" (UID: \"4031b492-0293-402a-af98-078d9e6e9368\") " pod="openshift-console/console-77b9f76cdc-wkbbl"
Sep 19 08:23:20.708259 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:23:20.708147    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4031b492-0293-402a-af98-078d9e6e9368-service-ca\") pod \"console-77b9f76cdc-wkbbl\" (UID: \"4031b492-0293-402a-af98-078d9e6e9368\") " pod="openshift-console/console-77b9f76cdc-wkbbl"
Sep 19 08:23:20.708455 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:23:20.708430    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4031b492-0293-402a-af98-078d9e6e9368-oauth-serving-cert\") pod \"console-77b9f76cdc-wkbbl\" (UID: \"4031b492-0293-402a-af98-078d9e6e9368\") " pod="openshift-console/console-77b9f76cdc-wkbbl"
Sep 19 08:23:20.708533 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:23:20.708510    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4031b492-0293-402a-af98-078d9e6e9368-trusted-ca-bundle\") pod \"console-77b9f76cdc-wkbbl\" (UID: \"4031b492-0293-402a-af98-078d9e6e9368\") " pod="openshift-console/console-77b9f76cdc-wkbbl"
Sep 19 08:23:20.710016 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:23:20.709992    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4031b492-0293-402a-af98-078d9e6e9368-console-serving-cert\") pod \"console-77b9f76cdc-wkbbl\" (UID: \"4031b492-0293-402a-af98-078d9e6e9368\") " pod="openshift-console/console-77b9f76cdc-wkbbl"
Sep 19 08:23:20.710016 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:23:20.710015    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4031b492-0293-402a-af98-078d9e6e9368-console-oauth-config\") pod \"console-77b9f76cdc-wkbbl\" (UID: \"4031b492-0293-402a-af98-078d9e6e9368\") " pod="openshift-console/console-77b9f76cdc-wkbbl"
Sep 19 08:23:20.715041 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:23:20.715014    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:23:20.715992 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:23:20.715970    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h7rpx\" (UniqueName: \"kubernetes.io/projected/4031b492-0293-402a-af98-078d9e6e9368-kube-api-access-h7rpx\") pod \"console-77b9f76cdc-wkbbl\" (UID: \"4031b492-0293-402a-af98-078d9e6e9368\") " pod="openshift-console/console-77b9f76cdc-wkbbl"
Sep 19 08:23:20.804875 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:23:20.804842    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:23:20.857218 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:23:20.857147    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:23:20.911398 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:23:20.911371    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77b9f76cdc-wkbbl"
Sep 19 08:23:21.178733 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:23:21.178712    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-77b9f76cdc-wkbbl"]
Sep 19 08:23:21.181076 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:23:21.181046    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4031b492_0293_402a_af98_078d9e6e9368.slice/crio-e6d5f43551f98dad14a1efc79627db1bc1d67b57247e4f42def0f8f4f5e96382 WatchSource:0}: Error finding container e6d5f43551f98dad14a1efc79627db1bc1d67b57247e4f42def0f8f4f5e96382: Status 404 returned error can't find the container with id e6d5f43551f98dad14a1efc79627db1bc1d67b57247e4f42def0f8f4f5e96382
Sep 19 08:23:21.397464 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:23:21.397428    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:23:21.446696 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:23:21.446605    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:23:21.918506 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:23:21.918471    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77b9f76cdc-wkbbl" event={"ID":"4031b492-0293-402a-af98-078d9e6e9368","Type":"ContainerStarted","Data":"0f6eac6a56dc0dd0c01dd8d24923f0ab3d17fd3707001ec650b79fdb0011155e"}
Sep 19 08:23:21.918880 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:23:21.918516    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77b9f76cdc-wkbbl" event={"ID":"4031b492-0293-402a-af98-078d9e6e9368","Type":"ContainerStarted","Data":"e6d5f43551f98dad14a1efc79627db1bc1d67b57247e4f42def0f8f4f5e96382"}
Sep 19 08:23:21.935355 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:23:21.935311    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-77b9f76cdc-wkbbl" podStartSLOduration=1.935278461 podStartE2EDuration="1.935278461s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 08:23:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 08:23:21.934103846 +0000 UTC m=+186.310813708" watchObservedRunningTime="2026-09-19 08:23:21.935278461 +0000 UTC m=+186.311988323"
Sep 19 08:23:30.912051 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:23:30.912014    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-77b9f76cdc-wkbbl"
Sep 19 08:23:30.912539 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:23:30.912071    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-77b9f76cdc-wkbbl"
Sep 19 08:23:30.916791 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:23:30.916764    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-77b9f76cdc-wkbbl"
Sep 19 08:23:30.953180 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:23:30.953150    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-77b9f76cdc-wkbbl"
Sep 19 08:23:30.995035 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:23:30.995000    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-759694f697-lgcmq"]
Sep 19 08:23:31.096633 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:23:31.096586    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:23:31.198129 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:23:31.198037    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:23:31.253390 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:23:31.253355    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:23:47.066833 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:23:47.066791    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:23:47.163030 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:23:47.162990    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:23:56.020213 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:23:56.020156    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-759694f697-lgcmq" podUID="cbb765d9-527c-4aef-8254-e2777d4e14ac" containerName="console" containerID="cri-o://ee543bbf0d59b914b1fa954334f945d64a4cbd3369d04e6796a15dcd46ef921d" gracePeriod=15
Sep 19 08:23:56.249680 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:23:56.249657    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-759694f697-lgcmq"
Sep 19 08:23:56.364233 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:23:56.364147    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nf8td\" (UniqueName: \"kubernetes.io/projected/cbb765d9-527c-4aef-8254-e2777d4e14ac-kube-api-access-nf8td\") pod \"cbb765d9-527c-4aef-8254-e2777d4e14ac\" (UID: \"cbb765d9-527c-4aef-8254-e2777d4e14ac\") "
Sep 19 08:23:56.364233 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:23:56.364180    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cbb765d9-527c-4aef-8254-e2777d4e14ac-console-oauth-config\") pod \"cbb765d9-527c-4aef-8254-e2777d4e14ac\" (UID: \"cbb765d9-527c-4aef-8254-e2777d4e14ac\") "
Sep 19 08:23:56.364233 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:23:56.364206    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cbb765d9-527c-4aef-8254-e2777d4e14ac-oauth-serving-cert\") pod \"cbb765d9-527c-4aef-8254-e2777d4e14ac\" (UID: \"cbb765d9-527c-4aef-8254-e2777d4e14ac\") "
Sep 19 08:23:56.364233 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:23:56.364224    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cbb765d9-527c-4aef-8254-e2777d4e14ac-console-serving-cert\") pod \"cbb765d9-527c-4aef-8254-e2777d4e14ac\" (UID: \"cbb765d9-527c-4aef-8254-e2777d4e14ac\") "
Sep 19 08:23:56.364602 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:23:56.364252    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cbb765d9-527c-4aef-8254-e2777d4e14ac-service-ca\") pod \"cbb765d9-527c-4aef-8254-e2777d4e14ac\" (UID: \"cbb765d9-527c-4aef-8254-e2777d4e14ac\") "
Sep 19 08:23:56.364602 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:23:56.364288    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cbb765d9-527c-4aef-8254-e2777d4e14ac-console-config\") pod \"cbb765d9-527c-4aef-8254-e2777d4e14ac\" (UID: \"cbb765d9-527c-4aef-8254-e2777d4e14ac\") "
Sep 19 08:23:56.364602 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:23:56.364320    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cbb765d9-527c-4aef-8254-e2777d4e14ac-trusted-ca-bundle\") pod \"cbb765d9-527c-4aef-8254-e2777d4e14ac\" (UID: \"cbb765d9-527c-4aef-8254-e2777d4e14ac\") "
Sep 19 08:23:56.364818 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:23:56.364753    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cbb765d9-527c-4aef-8254-e2777d4e14ac-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "cbb765d9-527c-4aef-8254-e2777d4e14ac" (UID: "cbb765d9-527c-4aef-8254-e2777d4e14ac"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 08:23:56.364818 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:23:56.364788    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cbb765d9-527c-4aef-8254-e2777d4e14ac-service-ca" (OuterVolumeSpecName: "service-ca") pod "cbb765d9-527c-4aef-8254-e2777d4e14ac" (UID: "cbb765d9-527c-4aef-8254-e2777d4e14ac"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 08:23:56.364818 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:23:56.364785    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cbb765d9-527c-4aef-8254-e2777d4e14ac-console-config" (OuterVolumeSpecName: "console-config") pod "cbb765d9-527c-4aef-8254-e2777d4e14ac" (UID: "cbb765d9-527c-4aef-8254-e2777d4e14ac"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 08:23:56.365006 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:23:56.364893    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cbb765d9-527c-4aef-8254-e2777d4e14ac-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "cbb765d9-527c-4aef-8254-e2777d4e14ac" (UID: "cbb765d9-527c-4aef-8254-e2777d4e14ac"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 08:23:56.366474 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:23:56.366445    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cbb765d9-527c-4aef-8254-e2777d4e14ac-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "cbb765d9-527c-4aef-8254-e2777d4e14ac" (UID: "cbb765d9-527c-4aef-8254-e2777d4e14ac"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 08:23:56.366563 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:23:56.366472    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cbb765d9-527c-4aef-8254-e2777d4e14ac-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "cbb765d9-527c-4aef-8254-e2777d4e14ac" (UID: "cbb765d9-527c-4aef-8254-e2777d4e14ac"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 08:23:56.366605 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:23:56.366568    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cbb765d9-527c-4aef-8254-e2777d4e14ac-kube-api-access-nf8td" (OuterVolumeSpecName: "kube-api-access-nf8td") pod "cbb765d9-527c-4aef-8254-e2777d4e14ac" (UID: "cbb765d9-527c-4aef-8254-e2777d4e14ac"). InnerVolumeSpecName "kube-api-access-nf8td". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 08:23:56.465041 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:23:56.464999    2578 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cbb765d9-527c-4aef-8254-e2777d4e14ac-console-oauth-config\") on node \"ip-10-0-139-157.ec2.internal\" DevicePath \"\""
Sep 19 08:23:56.465041 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:23:56.465035    2578 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cbb765d9-527c-4aef-8254-e2777d4e14ac-oauth-serving-cert\") on node \"ip-10-0-139-157.ec2.internal\" DevicePath \"\""
Sep 19 08:23:56.465041 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:23:56.465044    2578 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cbb765d9-527c-4aef-8254-e2777d4e14ac-console-serving-cert\") on node \"ip-10-0-139-157.ec2.internal\" DevicePath \"\""
Sep 19 08:23:56.465041 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:23:56.465053    2578 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cbb765d9-527c-4aef-8254-e2777d4e14ac-service-ca\") on node \"ip-10-0-139-157.ec2.internal\" DevicePath \"\""
Sep 19 08:23:56.465276 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:23:56.465064    2578 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cbb765d9-527c-4aef-8254-e2777d4e14ac-console-config\") on node \"ip-10-0-139-157.ec2.internal\" DevicePath \"\""
Sep 19 08:23:56.465276 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:23:56.465072    2578 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cbb765d9-527c-4aef-8254-e2777d4e14ac-trusted-ca-bundle\") on node \"ip-10-0-139-157.ec2.internal\" DevicePath \"\""
Sep 19 08:23:56.465276 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:23:56.465080    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nf8td\" (UniqueName: \"kubernetes.io/projected/cbb765d9-527c-4aef-8254-e2777d4e14ac-kube-api-access-nf8td\") on node \"ip-10-0-139-157.ec2.internal\" DevicePath \"\""
Sep 19 08:23:57.043876 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:23:57.043845    2578 generic.go:356] "Generic (PLEG): container finished" podID="cbb765d9-527c-4aef-8254-e2777d4e14ac" containerID="ee543bbf0d59b914b1fa954334f945d64a4cbd3369d04e6796a15dcd46ef921d" exitCode=0
Sep 19 08:23:57.044266 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:23:57.043886    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-759694f697-lgcmq"
Sep 19 08:23:57.044266 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:23:57.043902    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-759694f697-lgcmq" event={"ID":"cbb765d9-527c-4aef-8254-e2777d4e14ac","Type":"ContainerDied","Data":"ee543bbf0d59b914b1fa954334f945d64a4cbd3369d04e6796a15dcd46ef921d"}
Sep 19 08:23:57.044266 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:23:57.043940    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-759694f697-lgcmq" event={"ID":"cbb765d9-527c-4aef-8254-e2777d4e14ac","Type":"ContainerDied","Data":"def703458d3a625c69881a96c30a1b608bbfc1de5d8d09866df725e784b0017e"}
Sep 19 08:23:57.044266 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:23:57.043959    2578 scope.go:117] "RemoveContainer" containerID="ee543bbf0d59b914b1fa954334f945d64a4cbd3369d04e6796a15dcd46ef921d"
Sep 19 08:23:57.052986 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:23:57.052966    2578 scope.go:117] "RemoveContainer" containerID="ee543bbf0d59b914b1fa954334f945d64a4cbd3369d04e6796a15dcd46ef921d"
Sep 19 08:23:57.053257 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:23:57.053240    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ee543bbf0d59b914b1fa954334f945d64a4cbd3369d04e6796a15dcd46ef921d\": container with ID starting with ee543bbf0d59b914b1fa954334f945d64a4cbd3369d04e6796a15dcd46ef921d not found: ID does not exist" containerID="ee543bbf0d59b914b1fa954334f945d64a4cbd3369d04e6796a15dcd46ef921d"
Sep 19 08:23:57.053356 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:23:57.053263    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ee543bbf0d59b914b1fa954334f945d64a4cbd3369d04e6796a15dcd46ef921d"} err="failed to get container status \"ee543bbf0d59b914b1fa954334f945d64a4cbd3369d04e6796a15dcd46ef921d\": rpc error: code = NotFound desc = could not find container \"ee543bbf0d59b914b1fa954334f945d64a4cbd3369d04e6796a15dcd46ef921d\": container with ID starting with ee543bbf0d59b914b1fa954334f945d64a4cbd3369d04e6796a15dcd46ef921d not found: ID does not exist"
Sep 19 08:23:57.065692 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:23:57.065671    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-759694f697-lgcmq"]
Sep 19 08:23:57.070348 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:23:57.070323    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-759694f697-lgcmq"]
Sep 19 08:23:58.112072 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:23:58.112037    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cbb765d9-527c-4aef-8254-e2777d4e14ac" path="/var/lib/kubelet/pods/cbb765d9-527c-4aef-8254-e2777d4e14ac/volumes"
Sep 19 08:24:37.268677 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:24:37.268638    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:24:37.327088 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:24:37.327051    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:24:45.156924 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:24:45.156846    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-65b4797958-6c5jr"]
Sep 19 08:24:45.157270 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:24:45.157127    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cbb765d9-527c-4aef-8254-e2777d4e14ac" containerName="console"
Sep 19 08:24:45.157270 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:24:45.157137    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="cbb765d9-527c-4aef-8254-e2777d4e14ac" containerName="console"
Sep 19 08:24:45.157270 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:24:45.157197    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="cbb765d9-527c-4aef-8254-e2777d4e14ac" containerName="console"
Sep 19 08:24:45.158906 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:24:45.158890    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-65b4797958-6c5jr"
Sep 19 08:24:45.169970 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:24:45.169944    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-65b4797958-6c5jr"]
Sep 19 08:24:45.212819 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:24:45.212776    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:24:45.270617 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:24:45.270587    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:24:45.322875 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:24:45.322837    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d6783502-5784-4288-93c8-e39e6c27a201-console-config\") pod \"console-65b4797958-6c5jr\" (UID: \"d6783502-5784-4288-93c8-e39e6c27a201\") " pod="openshift-console/console-65b4797958-6c5jr"
Sep 19 08:24:45.322875 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:24:45.322878    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d6783502-5784-4288-93c8-e39e6c27a201-trusted-ca-bundle\") pod \"console-65b4797958-6c5jr\" (UID: \"d6783502-5784-4288-93c8-e39e6c27a201\") " pod="openshift-console/console-65b4797958-6c5jr"
Sep 19 08:24:45.323242 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:24:45.322898    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d6783502-5784-4288-93c8-e39e6c27a201-oauth-serving-cert\") pod \"console-65b4797958-6c5jr\" (UID: \"d6783502-5784-4288-93c8-e39e6c27a201\") " pod="openshift-console/console-65b4797958-6c5jr"
Sep 19 08:24:45.323242 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:24:45.322968    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d6783502-5784-4288-93c8-e39e6c27a201-service-ca\") pod \"console-65b4797958-6c5jr\" (UID: \"d6783502-5784-4288-93c8-e39e6c27a201\") " pod="openshift-console/console-65b4797958-6c5jr"
Sep 19 08:24:45.323242 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:24:45.323057    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d6783502-5784-4288-93c8-e39e6c27a201-console-serving-cert\") pod \"console-65b4797958-6c5jr\" (UID: \"d6783502-5784-4288-93c8-e39e6c27a201\") " pod="openshift-console/console-65b4797958-6c5jr"
Sep 19 08:24:45.323242 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:24:45.323086    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d6783502-5784-4288-93c8-e39e6c27a201-console-oauth-config\") pod \"console-65b4797958-6c5jr\" (UID: \"d6783502-5784-4288-93c8-e39e6c27a201\") " pod="openshift-console/console-65b4797958-6c5jr"
Sep 19 08:24:45.323242 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:24:45.323113    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lg5x9\" (UniqueName: \"kubernetes.io/projected/d6783502-5784-4288-93c8-e39e6c27a201-kube-api-access-lg5x9\") pod \"console-65b4797958-6c5jr\" (UID: \"d6783502-5784-4288-93c8-e39e6c27a201\") " pod="openshift-console/console-65b4797958-6c5jr"
Sep 19 08:24:45.423754 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:24:45.423687    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d6783502-5784-4288-93c8-e39e6c27a201-console-config\") pod \"console-65b4797958-6c5jr\" (UID: \"d6783502-5784-4288-93c8-e39e6c27a201\") " pod="openshift-console/console-65b4797958-6c5jr"
Sep 19 08:24:45.423754 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:24:45.423716    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d6783502-5784-4288-93c8-e39e6c27a201-trusted-ca-bundle\") pod \"console-65b4797958-6c5jr\" (UID: \"d6783502-5784-4288-93c8-e39e6c27a201\") " pod="openshift-console/console-65b4797958-6c5jr"
Sep 19 08:24:45.423754 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:24:45.423733    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d6783502-5784-4288-93c8-e39e6c27a201-oauth-serving-cert\") pod \"console-65b4797958-6c5jr\" (UID: \"d6783502-5784-4288-93c8-e39e6c27a201\") " pod="openshift-console/console-65b4797958-6c5jr"
Sep 19 08:24:45.424003 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:24:45.423760    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d6783502-5784-4288-93c8-e39e6c27a201-service-ca\") pod \"console-65b4797958-6c5jr\" (UID: \"d6783502-5784-4288-93c8-e39e6c27a201\") " pod="openshift-console/console-65b4797958-6c5jr"
Sep 19 08:24:45.424003 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:24:45.423795    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d6783502-5784-4288-93c8-e39e6c27a201-console-serving-cert\") pod \"console-65b4797958-6c5jr\" (UID: \"d6783502-5784-4288-93c8-e39e6c27a201\") " pod="openshift-console/console-65b4797958-6c5jr"
Sep 19 08:24:45.424003 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:24:45.423823    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d6783502-5784-4288-93c8-e39e6c27a201-console-oauth-config\") pod \"console-65b4797958-6c5jr\" (UID: \"d6783502-5784-4288-93c8-e39e6c27a201\") " pod="openshift-console/console-65b4797958-6c5jr"
Sep 19 08:24:45.424003 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:24:45.423901    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lg5x9\" (UniqueName: \"kubernetes.io/projected/d6783502-5784-4288-93c8-e39e6c27a201-kube-api-access-lg5x9\") pod \"console-65b4797958-6c5jr\" (UID: \"d6783502-5784-4288-93c8-e39e6c27a201\") " pod="openshift-console/console-65b4797958-6c5jr"
Sep 19 08:24:45.424577 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:24:45.424554    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d6783502-5784-4288-93c8-e39e6c27a201-console-config\") pod \"console-65b4797958-6c5jr\" (UID: \"d6783502-5784-4288-93c8-e39e6c27a201\") " pod="openshift-console/console-65b4797958-6c5jr"
Sep 19 08:24:45.424713 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:24:45.424609    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d6783502-5784-4288-93c8-e39e6c27a201-oauth-serving-cert\") pod \"console-65b4797958-6c5jr\" (UID: \"d6783502-5784-4288-93c8-e39e6c27a201\") " pod="openshift-console/console-65b4797958-6c5jr"
Sep 19 08:24:45.424862 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:24:45.424840    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d6783502-5784-4288-93c8-e39e6c27a201-service-ca\") pod \"console-65b4797958-6c5jr\" (UID: \"d6783502-5784-4288-93c8-e39e6c27a201\") " pod="openshift-console/console-65b4797958-6c5jr"
Sep 19 08:24:45.424912 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:24:45.424864    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d6783502-5784-4288-93c8-e39e6c27a201-trusted-ca-bundle\") pod \"console-65b4797958-6c5jr\" (UID: \"d6783502-5784-4288-93c8-e39e6c27a201\") " pod="openshift-console/console-65b4797958-6c5jr"
Sep 19 08:24:45.426346 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:24:45.426320    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d6783502-5784-4288-93c8-e39e6c27a201-console-serving-cert\") pod \"console-65b4797958-6c5jr\" (UID: \"d6783502-5784-4288-93c8-e39e6c27a201\") " pod="openshift-console/console-65b4797958-6c5jr"
Sep 19 08:24:45.426453 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:24:45.426417    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d6783502-5784-4288-93c8-e39e6c27a201-console-oauth-config\") pod \"console-65b4797958-6c5jr\" (UID: \"d6783502-5784-4288-93c8-e39e6c27a201\") " pod="openshift-console/console-65b4797958-6c5jr"
Sep 19 08:24:45.431777 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:24:45.431754    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lg5x9\" (UniqueName: \"kubernetes.io/projected/d6783502-5784-4288-93c8-e39e6c27a201-kube-api-access-lg5x9\") pod \"console-65b4797958-6c5jr\" (UID: \"d6783502-5784-4288-93c8-e39e6c27a201\") " pod="openshift-console/console-65b4797958-6c5jr"
Sep 19 08:24:45.467711 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:24:45.467684    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-65b4797958-6c5jr"
Sep 19 08:24:45.733622 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:24:45.733595    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-65b4797958-6c5jr"]
Sep 19 08:24:45.736404 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:24:45.736374    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6783502_5784_4288_93c8_e39e6c27a201.slice/crio-c97840ec0a8bb4663784b20e3fc9a9032b50cdb68bd55457d45e1efba9b27762 WatchSource:0}: Error finding container c97840ec0a8bb4663784b20e3fc9a9032b50cdb68bd55457d45e1efba9b27762: Status 404 returned error can't find the container with id c97840ec0a8bb4663784b20e3fc9a9032b50cdb68bd55457d45e1efba9b27762
Sep 19 08:24:46.220260 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:24:46.220228    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-65b4797958-6c5jr" event={"ID":"d6783502-5784-4288-93c8-e39e6c27a201","Type":"ContainerStarted","Data":"aef69d11046d37433b04d75213470772cd8f7bc5a9d097be86ed47d3cef1d4d9"}
Sep 19 08:24:46.220753 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:24:46.220278    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-65b4797958-6c5jr" event={"ID":"d6783502-5784-4288-93c8-e39e6c27a201","Type":"ContainerStarted","Data":"c97840ec0a8bb4663784b20e3fc9a9032b50cdb68bd55457d45e1efba9b27762"}
Sep 19 08:24:46.237760 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:24:46.237708    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-65b4797958-6c5jr" podStartSLOduration=1.237694039 podStartE2EDuration="1.237694039s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 08:24:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 08:24:46.236357035 +0000 UTC m=+270.613066897" watchObservedRunningTime="2026-09-19 08:24:46.237694039 +0000 UTC m=+270.614403960"
Sep 19 08:24:49.210669 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:24:49.210627    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:24:49.264524 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:24:49.264488    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:24:55.468677 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:24:55.468642    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-65b4797958-6c5jr"
Sep 19 08:24:55.469075 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:24:55.468729    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-65b4797958-6c5jr"
Sep 19 08:24:55.473397 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:24:55.473367    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-65b4797958-6c5jr"
Sep 19 08:24:56.259331 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:24:56.259283    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-65b4797958-6c5jr"
Sep 19 08:24:56.302167 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:24:56.302133    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-77b9f76cdc-wkbbl"]
Sep 19 08:24:56.405485 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:24:56.405450    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:24:56.461660 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:24:56.461627    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:25:01.895795 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:25:01.895758    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:25:01.995390 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:25:01.995356    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:25:02.096031 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:25:02.095984    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:25:15.982579 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:25:15.982552    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zd2zp_07ddac45-12a1-42e8-9566-55f08854c02e/ovn-acl-logging/0.log"
Sep 19 08:25:15.985314 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:25:15.985275    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zd2zp_07ddac45-12a1-42e8-9566-55f08854c02e/ovn-acl-logging/0.log"
Sep 19 08:25:15.989039 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:25:15.989016    2578 kubelet.go:1648] "Image garbage collection succeeded"
Sep 19 08:25:21.321126 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:25:21.321065    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-77b9f76cdc-wkbbl" podUID="4031b492-0293-402a-af98-078d9e6e9368" containerName="console" containerID="cri-o://0f6eac6a56dc0dd0c01dd8d24923f0ab3d17fd3707001ec650b79fdb0011155e" gracePeriod=15
Sep 19 08:25:21.552446 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:25:21.552421    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77b9f76cdc-wkbbl"
Sep 19 08:25:21.581874 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:25:21.581807    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4031b492-0293-402a-af98-078d9e6e9368-service-ca\") pod \"4031b492-0293-402a-af98-078d9e6e9368\" (UID: \"4031b492-0293-402a-af98-078d9e6e9368\") "
Sep 19 08:25:21.581874 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:25:21.581849    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h7rpx\" (UniqueName: \"kubernetes.io/projected/4031b492-0293-402a-af98-078d9e6e9368-kube-api-access-h7rpx\") pod \"4031b492-0293-402a-af98-078d9e6e9368\" (UID: \"4031b492-0293-402a-af98-078d9e6e9368\") "
Sep 19 08:25:21.581874 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:25:21.581866    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4031b492-0293-402a-af98-078d9e6e9368-console-config\") pod \"4031b492-0293-402a-af98-078d9e6e9368\" (UID: \"4031b492-0293-402a-af98-078d9e6e9368\") "
Sep 19 08:25:21.582107 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:25:21.581888    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4031b492-0293-402a-af98-078d9e6e9368-console-serving-cert\") pod \"4031b492-0293-402a-af98-078d9e6e9368\" (UID: \"4031b492-0293-402a-af98-078d9e6e9368\") "
Sep 19 08:25:21.582107 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:25:21.581923    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4031b492-0293-402a-af98-078d9e6e9368-console-oauth-config\") pod \"4031b492-0293-402a-af98-078d9e6e9368\" (UID: \"4031b492-0293-402a-af98-078d9e6e9368\") "
Sep 19 08:25:21.582107 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:25:21.581951    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4031b492-0293-402a-af98-078d9e6e9368-oauth-serving-cert\") pod \"4031b492-0293-402a-af98-078d9e6e9368\" (UID: \"4031b492-0293-402a-af98-078d9e6e9368\") "
Sep 19 08:25:21.582107 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:25:21.582001    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4031b492-0293-402a-af98-078d9e6e9368-trusted-ca-bundle\") pod \"4031b492-0293-402a-af98-078d9e6e9368\" (UID: \"4031b492-0293-402a-af98-078d9e6e9368\") "
Sep 19 08:25:21.582334 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:25:21.582241    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4031b492-0293-402a-af98-078d9e6e9368-service-ca" (OuterVolumeSpecName: "service-ca") pod "4031b492-0293-402a-af98-078d9e6e9368" (UID: "4031b492-0293-402a-af98-078d9e6e9368"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 08:25:21.582334 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:25:21.582271    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4031b492-0293-402a-af98-078d9e6e9368-console-config" (OuterVolumeSpecName: "console-config") pod "4031b492-0293-402a-af98-078d9e6e9368" (UID: "4031b492-0293-402a-af98-078d9e6e9368"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 08:25:21.582552 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:25:21.582528    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4031b492-0293-402a-af98-078d9e6e9368-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "4031b492-0293-402a-af98-078d9e6e9368" (UID: "4031b492-0293-402a-af98-078d9e6e9368"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 08:25:21.582719 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:25:21.582700    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4031b492-0293-402a-af98-078d9e6e9368-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "4031b492-0293-402a-af98-078d9e6e9368" (UID: "4031b492-0293-402a-af98-078d9e6e9368"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 08:25:21.584082 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:25:21.584056    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4031b492-0293-402a-af98-078d9e6e9368-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "4031b492-0293-402a-af98-078d9e6e9368" (UID: "4031b492-0293-402a-af98-078d9e6e9368"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 08:25:21.584518 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:25:21.584496    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4031b492-0293-402a-af98-078d9e6e9368-kube-api-access-h7rpx" (OuterVolumeSpecName: "kube-api-access-h7rpx") pod "4031b492-0293-402a-af98-078d9e6e9368" (UID: "4031b492-0293-402a-af98-078d9e6e9368"). InnerVolumeSpecName "kube-api-access-h7rpx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 08:25:21.584647 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:25:21.584632    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4031b492-0293-402a-af98-078d9e6e9368-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "4031b492-0293-402a-af98-078d9e6e9368" (UID: "4031b492-0293-402a-af98-078d9e6e9368"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 08:25:21.683390 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:25:21.683357    2578 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4031b492-0293-402a-af98-078d9e6e9368-oauth-serving-cert\") on node \"ip-10-0-139-157.ec2.internal\" DevicePath \"\""
Sep 19 08:25:21.683390 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:25:21.683384    2578 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4031b492-0293-402a-af98-078d9e6e9368-trusted-ca-bundle\") on node \"ip-10-0-139-157.ec2.internal\" DevicePath \"\""
Sep 19 08:25:21.683390 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:25:21.683395    2578 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4031b492-0293-402a-af98-078d9e6e9368-service-ca\") on node \"ip-10-0-139-157.ec2.internal\" DevicePath \"\""
Sep 19 08:25:21.683576 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:25:21.683404    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h7rpx\" (UniqueName: \"kubernetes.io/projected/4031b492-0293-402a-af98-078d9e6e9368-kube-api-access-h7rpx\") on node \"ip-10-0-139-157.ec2.internal\" DevicePath \"\""
Sep 19 08:25:21.683576 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:25:21.683414    2578 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4031b492-0293-402a-af98-078d9e6e9368-console-config\") on node \"ip-10-0-139-157.ec2.internal\" DevicePath \"\""
Sep 19 08:25:21.683576 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:25:21.683422    2578 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4031b492-0293-402a-af98-078d9e6e9368-console-serving-cert\") on node \"ip-10-0-139-157.ec2.internal\" DevicePath \"\""
Sep 19 08:25:21.683576 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:25:21.683430    2578 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4031b492-0293-402a-af98-078d9e6e9368-console-oauth-config\") on node \"ip-10-0-139-157.ec2.internal\" DevicePath \"\""
Sep 19 08:25:22.348276 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:25:22.348243    2578 generic.go:356] "Generic (PLEG): container finished" podID="4031b492-0293-402a-af98-078d9e6e9368" containerID="0f6eac6a56dc0dd0c01dd8d24923f0ab3d17fd3707001ec650b79fdb0011155e" exitCode=0
Sep 19 08:25:22.348689 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:25:22.348286    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77b9f76cdc-wkbbl"
Sep 19 08:25:22.348689 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:25:22.348324    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77b9f76cdc-wkbbl" event={"ID":"4031b492-0293-402a-af98-078d9e6e9368","Type":"ContainerDied","Data":"0f6eac6a56dc0dd0c01dd8d24923f0ab3d17fd3707001ec650b79fdb0011155e"}
Sep 19 08:25:22.348689 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:25:22.348375    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77b9f76cdc-wkbbl" event={"ID":"4031b492-0293-402a-af98-078d9e6e9368","Type":"ContainerDied","Data":"e6d5f43551f98dad14a1efc79627db1bc1d67b57247e4f42def0f8f4f5e96382"}
Sep 19 08:25:22.348689 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:25:22.348394    2578 scope.go:117] "RemoveContainer" containerID="0f6eac6a56dc0dd0c01dd8d24923f0ab3d17fd3707001ec650b79fdb0011155e"
Sep 19 08:25:22.356892 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:25:22.356872    2578 scope.go:117] "RemoveContainer" containerID="0f6eac6a56dc0dd0c01dd8d24923f0ab3d17fd3707001ec650b79fdb0011155e"
Sep 19 08:25:22.357214 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:25:22.357181    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0f6eac6a56dc0dd0c01dd8d24923f0ab3d17fd3707001ec650b79fdb0011155e\": container with ID starting with 0f6eac6a56dc0dd0c01dd8d24923f0ab3d17fd3707001ec650b79fdb0011155e not found: ID does not exist" containerID="0f6eac6a56dc0dd0c01dd8d24923f0ab3d17fd3707001ec650b79fdb0011155e"
Sep 19 08:25:22.357353 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:25:22.357221    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0f6eac6a56dc0dd0c01dd8d24923f0ab3d17fd3707001ec650b79fdb0011155e"} err="failed to get container status \"0f6eac6a56dc0dd0c01dd8d24923f0ab3d17fd3707001ec650b79fdb0011155e\": rpc error: code = NotFound desc = could not find container \"0f6eac6a56dc0dd0c01dd8d24923f0ab3d17fd3707001ec650b79fdb0011155e\": container with ID starting with 0f6eac6a56dc0dd0c01dd8d24923f0ab3d17fd3707001ec650b79fdb0011155e not found: ID does not exist"
Sep 19 08:25:22.364795 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:25:22.364768    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-77b9f76cdc-wkbbl"]
Sep 19 08:25:22.367926 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:25:22.367903    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-77b9f76cdc-wkbbl"]
Sep 19 08:25:24.112433 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:25:24.112394    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4031b492-0293-402a-af98-078d9e6e9368" path="/var/lib/kubelet/pods/4031b492-0293-402a-af98-078d9e6e9368/volumes"
Sep 19 08:25:30.206414 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:25:30.206360    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:25:41.230440 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:25:41.230380    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:25:42.312756 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:25:42.312716    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1ajc76j"]
Sep 19 08:25:42.313115 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:25:42.313022    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4031b492-0293-402a-af98-078d9e6e9368" containerName="console"
Sep 19 08:25:42.313115 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:25:42.313038    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="4031b492-0293-402a-af98-078d9e6e9368" containerName="console"
Sep 19 08:25:42.313115 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:25:42.313104    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="4031b492-0293-402a-af98-078d9e6e9368" containerName="console"
Sep 19 08:25:42.314973 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:25:42.314954    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1ajc76j"
Sep 19 08:25:42.317421 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:25:42.317400    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 19 08:25:42.318366 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:25:42.318341    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 19 08:25:42.318492 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:25:42.318389    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-sdmf2\""
Sep 19 08:25:42.324536 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:25:42.324514    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1ajc76j"]
Sep 19 08:25:42.369351 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:25:42.369313    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:25:42.433240 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:25:42.433215    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/68cac027-8fc5-4359-84b9-a03f14e0002b-util\") pod \"1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1ajc76j\" (UID: \"68cac027-8fc5-4359-84b9-a03f14e0002b\") " pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1ajc76j"
Sep 19 08:25:42.433407 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:25:42.433245    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/68cac027-8fc5-4359-84b9-a03f14e0002b-bundle\") pod \"1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1ajc76j\" (UID: \"68cac027-8fc5-4359-84b9-a03f14e0002b\") " pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1ajc76j"
Sep 19 08:25:42.433407 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:25:42.433284    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dhwbt\" (UniqueName: \"kubernetes.io/projected/68cac027-8fc5-4359-84b9-a03f14e0002b-kube-api-access-dhwbt\") pod \"1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1ajc76j\" (UID: \"68cac027-8fc5-4359-84b9-a03f14e0002b\") " pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1ajc76j"
Sep 19 08:25:42.533968 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:25:42.533926    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/68cac027-8fc5-4359-84b9-a03f14e0002b-util\") pod \"1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1ajc76j\" (UID: \"68cac027-8fc5-4359-84b9-a03f14e0002b\") " pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1ajc76j"
Sep 19 08:25:42.533968 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:25:42.533973    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/68cac027-8fc5-4359-84b9-a03f14e0002b-bundle\") pod \"1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1ajc76j\" (UID: \"68cac027-8fc5-4359-84b9-a03f14e0002b\") " pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1ajc76j"
Sep 19 08:25:42.534215 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:25:42.534008    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dhwbt\" (UniqueName: \"kubernetes.io/projected/68cac027-8fc5-4359-84b9-a03f14e0002b-kube-api-access-dhwbt\") pod \"1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1ajc76j\" (UID: \"68cac027-8fc5-4359-84b9-a03f14e0002b\") " pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1ajc76j"
Sep 19 08:25:42.534341 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:25:42.534283    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/68cac027-8fc5-4359-84b9-a03f14e0002b-util\") pod \"1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1ajc76j\" (UID: \"68cac027-8fc5-4359-84b9-a03f14e0002b\") " pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1ajc76j"
Sep 19 08:25:42.534418 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:25:42.534404    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/68cac027-8fc5-4359-84b9-a03f14e0002b-bundle\") pod \"1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1ajc76j\" (UID: \"68cac027-8fc5-4359-84b9-a03f14e0002b\") " pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1ajc76j"
Sep 19 08:25:42.542511 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:25:42.542488    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dhwbt\" (UniqueName: \"kubernetes.io/projected/68cac027-8fc5-4359-84b9-a03f14e0002b-kube-api-access-dhwbt\") pod \"1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1ajc76j\" (UID: \"68cac027-8fc5-4359-84b9-a03f14e0002b\") " pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1ajc76j"
Sep 19 08:25:42.624461 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:25:42.624374    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1ajc76j"
Sep 19 08:25:42.896727 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:25:42.896551    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1ajc76j"]
Sep 19 08:25:42.899400 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:25:42.899372    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68cac027_8fc5_4359_84b9_a03f14e0002b.slice/crio-351c5592a4000879769363a2fcfe3685bb9ace0eb7c1e2cfe012ab9113f87c6b WatchSource:0}: Error finding container 351c5592a4000879769363a2fcfe3685bb9ace0eb7c1e2cfe012ab9113f87c6b: Status 404 returned error can't find the container with id 351c5592a4000879769363a2fcfe3685bb9ace0eb7c1e2cfe012ab9113f87c6b
Sep 19 08:25:42.901445 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:25:42.901424    2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 19 08:25:43.429372 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:25:43.429338    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1ajc76j" event={"ID":"68cac027-8fc5-4359-84b9-a03f14e0002b","Type":"ContainerStarted","Data":"351c5592a4000879769363a2fcfe3685bb9ace0eb7c1e2cfe012ab9113f87c6b"}
Sep 19 08:25:49.455314 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:25:49.455261    2578 generic.go:356] "Generic (PLEG): container finished" podID="68cac027-8fc5-4359-84b9-a03f14e0002b" containerID="337935df6cae6d1b78a169e6fe46da55bb3b67bdf2f507b17b7328c4ff8543e3" exitCode=0
Sep 19 08:25:49.455732 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:25:49.455356    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1ajc76j" event={"ID":"68cac027-8fc5-4359-84b9-a03f14e0002b","Type":"ContainerDied","Data":"337935df6cae6d1b78a169e6fe46da55bb3b67bdf2f507b17b7328c4ff8543e3"}
Sep 19 08:25:53.473616 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:25:53.473584    2578 generic.go:356] "Generic (PLEG): container finished" podID="68cac027-8fc5-4359-84b9-a03f14e0002b" containerID="c5795b8a5986af234575ed3da1b4126dabc3f5679f6161a52c8d981a83f5656d" exitCode=0
Sep 19 08:25:53.473996 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:25:53.473637    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1ajc76j" event={"ID":"68cac027-8fc5-4359-84b9-a03f14e0002b","Type":"ContainerDied","Data":"c5795b8a5986af234575ed3da1b4126dabc3f5679f6161a52c8d981a83f5656d"}
Sep 19 08:26:00.503421 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:26:00.503384    2578 generic.go:356] "Generic (PLEG): container finished" podID="68cac027-8fc5-4359-84b9-a03f14e0002b" containerID="b007aac9b3b10f8845b3db1ddd38be504237ef53989695a7bbf14c0e56ad9129" exitCode=0
Sep 19 08:26:00.503884 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:26:00.503440    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1ajc76j" event={"ID":"68cac027-8fc5-4359-84b9-a03f14e0002b","Type":"ContainerDied","Data":"b007aac9b3b10f8845b3db1ddd38be504237ef53989695a7bbf14c0e56ad9129"}
Sep 19 08:26:01.627540 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:26:01.627513    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1ajc76j"
Sep 19 08:26:01.696164 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:26:01.696132    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/68cac027-8fc5-4359-84b9-a03f14e0002b-util\") pod \"68cac027-8fc5-4359-84b9-a03f14e0002b\" (UID: \"68cac027-8fc5-4359-84b9-a03f14e0002b\") "
Sep 19 08:26:01.696366 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:26:01.696186    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/68cac027-8fc5-4359-84b9-a03f14e0002b-bundle\") pod \"68cac027-8fc5-4359-84b9-a03f14e0002b\" (UID: \"68cac027-8fc5-4359-84b9-a03f14e0002b\") "
Sep 19 08:26:01.696366 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:26:01.696215    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dhwbt\" (UniqueName: \"kubernetes.io/projected/68cac027-8fc5-4359-84b9-a03f14e0002b-kube-api-access-dhwbt\") pod \"68cac027-8fc5-4359-84b9-a03f14e0002b\" (UID: \"68cac027-8fc5-4359-84b9-a03f14e0002b\") "
Sep 19 08:26:01.696850 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:26:01.696825    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/68cac027-8fc5-4359-84b9-a03f14e0002b-bundle" (OuterVolumeSpecName: "bundle") pod "68cac027-8fc5-4359-84b9-a03f14e0002b" (UID: "68cac027-8fc5-4359-84b9-a03f14e0002b"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 08:26:01.698511 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:26:01.698485    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/68cac027-8fc5-4359-84b9-a03f14e0002b-kube-api-access-dhwbt" (OuterVolumeSpecName: "kube-api-access-dhwbt") pod "68cac027-8fc5-4359-84b9-a03f14e0002b" (UID: "68cac027-8fc5-4359-84b9-a03f14e0002b"). InnerVolumeSpecName "kube-api-access-dhwbt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 08:26:01.700580 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:26:01.700559    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/68cac027-8fc5-4359-84b9-a03f14e0002b-util" (OuterVolumeSpecName: "util") pod "68cac027-8fc5-4359-84b9-a03f14e0002b" (UID: "68cac027-8fc5-4359-84b9-a03f14e0002b"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 08:26:01.797127 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:26:01.797025    2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/68cac027-8fc5-4359-84b9-a03f14e0002b-util\") on node \"ip-10-0-139-157.ec2.internal\" DevicePath \"\""
Sep 19 08:26:01.797127 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:26:01.797069    2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/68cac027-8fc5-4359-84b9-a03f14e0002b-bundle\") on node \"ip-10-0-139-157.ec2.internal\" DevicePath \"\""
Sep 19 08:26:01.797127 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:26:01.797080    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dhwbt\" (UniqueName: \"kubernetes.io/projected/68cac027-8fc5-4359-84b9-a03f14e0002b-kube-api-access-dhwbt\") on node \"ip-10-0-139-157.ec2.internal\" DevicePath \"\""
Sep 19 08:26:02.512708 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:26:02.512670    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1ajc76j"
Sep 19 08:26:02.512872 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:26:02.512669    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1ajc76j" event={"ID":"68cac027-8fc5-4359-84b9-a03f14e0002b","Type":"ContainerDied","Data":"351c5592a4000879769363a2fcfe3685bb9ace0eb7c1e2cfe012ab9113f87c6b"}
Sep 19 08:26:02.512872 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:26:02.512799    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="351c5592a4000879769363a2fcfe3685bb9ace0eb7c1e2cfe012ab9113f87c6b"
Sep 19 08:26:03.633751 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:26:03.633715    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:26:03.722082 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:26:03.722042    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:26:05.023024 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:26:05.022987    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-628tz"]
Sep 19 08:26:05.023409 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:26:05.023254    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="68cac027-8fc5-4359-84b9-a03f14e0002b" containerName="extract"
Sep 19 08:26:05.023409 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:26:05.023264    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="68cac027-8fc5-4359-84b9-a03f14e0002b" containerName="extract"
Sep 19 08:26:05.023409 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:26:05.023275    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="68cac027-8fc5-4359-84b9-a03f14e0002b" containerName="util"
Sep 19 08:26:05.023409 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:26:05.023289    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="68cac027-8fc5-4359-84b9-a03f14e0002b" containerName="util"
Sep 19 08:26:05.023409 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:26:05.023314    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="68cac027-8fc5-4359-84b9-a03f14e0002b" containerName="pull"
Sep 19 08:26:05.023409 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:26:05.023319    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="68cac027-8fc5-4359-84b9-a03f14e0002b" containerName="pull"
Sep 19 08:26:05.023409 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:26:05.023382    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="68cac027-8fc5-4359-84b9-a03f14e0002b" containerName="extract"
Sep 19 08:26:05.026409 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:26:05.026392    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-628tz"
Sep 19 08:26:05.030219 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:26:05.030193    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"custom-metrics-autoscaler-operator-dockercfg-ppz7z\""
Sep 19 08:26:05.030556 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:26:05.030538    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"kube-root-ca.crt\""
Sep 19 08:26:05.033645 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:26:05.033628    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"openshift-service-ca.crt\""
Sep 19 08:26:05.033871 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:26:05.033848    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"kedaorg-certs\""
Sep 19 08:26:05.045795 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:26:05.045765    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-628tz"]
Sep 19 08:26:05.120670 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:26:05.120626    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/25c9ac5f-26d4-489e-9e2d-84a40edc88cf-certificates\") pod \"custom-metrics-autoscaler-operator-566c9cd69c-628tz\" (UID: \"25c9ac5f-26d4-489e-9e2d-84a40edc88cf\") " pod="openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-628tz"
Sep 19 08:26:05.120670 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:26:05.120680    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ncvx5\" (UniqueName: \"kubernetes.io/projected/25c9ac5f-26d4-489e-9e2d-84a40edc88cf-kube-api-access-ncvx5\") pod \"custom-metrics-autoscaler-operator-566c9cd69c-628tz\" (UID: \"25c9ac5f-26d4-489e-9e2d-84a40edc88cf\") " pod="openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-628tz"
Sep 19 08:26:05.221234 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:26:05.221200    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ncvx5\" (UniqueName: \"kubernetes.io/projected/25c9ac5f-26d4-489e-9e2d-84a40edc88cf-kube-api-access-ncvx5\") pod \"custom-metrics-autoscaler-operator-566c9cd69c-628tz\" (UID: \"25c9ac5f-26d4-489e-9e2d-84a40edc88cf\") " pod="openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-628tz"
Sep 19 08:26:05.221406 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:26:05.221262    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/25c9ac5f-26d4-489e-9e2d-84a40edc88cf-certificates\") pod \"custom-metrics-autoscaler-operator-566c9cd69c-628tz\" (UID: \"25c9ac5f-26d4-489e-9e2d-84a40edc88cf\") " pod="openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-628tz"
Sep 19 08:26:05.223543 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:26:05.223516    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/25c9ac5f-26d4-489e-9e2d-84a40edc88cf-certificates\") pod \"custom-metrics-autoscaler-operator-566c9cd69c-628tz\" (UID: \"25c9ac5f-26d4-489e-9e2d-84a40edc88cf\") " pod="openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-628tz"
Sep 19 08:26:05.230499 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:26:05.230470    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ncvx5\" (UniqueName: \"kubernetes.io/projected/25c9ac5f-26d4-489e-9e2d-84a40edc88cf-kube-api-access-ncvx5\") pod \"custom-metrics-autoscaler-operator-566c9cd69c-628tz\" (UID: \"25c9ac5f-26d4-489e-9e2d-84a40edc88cf\") " pod="openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-628tz"
Sep 19 08:26:05.336674 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:26:05.336576    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-628tz"
Sep 19 08:26:05.611118 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:26:05.611060    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-628tz"]
Sep 19 08:26:05.613477 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:26:05.613449    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25c9ac5f_26d4_489e_9e2d_84a40edc88cf.slice/crio-e8ef4e7029971c4b42f2aa54c1162e0134070eac3f4f8c54b3ff7efbadef16c1 WatchSource:0}: Error finding container e8ef4e7029971c4b42f2aa54c1162e0134070eac3f4f8c54b3ff7efbadef16c1: Status 404 returned error can't find the container with id e8ef4e7029971c4b42f2aa54c1162e0134070eac3f4f8c54b3ff7efbadef16c1
Sep 19 08:26:06.531372 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:26:06.531318    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-628tz" event={"ID":"25c9ac5f-26d4-489e-9e2d-84a40edc88cf","Type":"ContainerStarted","Data":"e8ef4e7029971c4b42f2aa54c1162e0134070eac3f4f8c54b3ff7efbadef16c1"}
Sep 19 08:26:09.151924 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:26:09.151886    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:26:09.432482 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:26:09.432378    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:26:09.436746 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:26:09.436721    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-operator-579c8958f7-9q84f"]
Sep 19 08:26:09.439849 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:26:09.439833    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-operator-579c8958f7-9q84f"
Sep 19 08:26:09.446530 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:26:09.446509    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"keda-ocp-cabundle\""
Sep 19 08:26:09.446655 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:26:09.446533    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-operator-certs\""
Sep 19 08:26:09.446655 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:26:09.446548    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-operator-dockercfg-4qw4k\""
Sep 19 08:26:09.455501 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:26:09.455474    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-operator-579c8958f7-9q84f"]
Sep 19 08:26:09.548534 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:26:09.548486    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-628tz" event={"ID":"25c9ac5f-26d4-489e-9e2d-84a40edc88cf","Type":"ContainerStarted","Data":"e2c1dce5a9457551595b06378e94b68f12d206a12ce2badcd8ce79931cf7485c"}
Sep 19 08:26:09.548705 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:26:09.548640    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-628tz"
Sep 19 08:26:09.554795 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:26:09.554759    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4hwws\" (UniqueName: \"kubernetes.io/projected/e7548679-13cf-403e-ac2f-1331641074ca-kube-api-access-4hwws\") pod \"keda-operator-579c8958f7-9q84f\" (UID: \"e7548679-13cf-403e-ac2f-1331641074ca\") " pod="openshift-keda/keda-operator-579c8958f7-9q84f"
Sep 19 08:26:09.554928 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:26:09.554803    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/e7548679-13cf-403e-ac2f-1331641074ca-cabundle0\") pod \"keda-operator-579c8958f7-9q84f\" (UID: \"e7548679-13cf-403e-ac2f-1331641074ca\") " pod="openshift-keda/keda-operator-579c8958f7-9q84f"
Sep 19 08:26:09.554928 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:26:09.554840    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/e7548679-13cf-403e-ac2f-1331641074ca-certificates\") pod \"keda-operator-579c8958f7-9q84f\" (UID: \"e7548679-13cf-403e-ac2f-1331641074ca\") " pod="openshift-keda/keda-operator-579c8958f7-9q84f"
Sep 19 08:26:09.566742 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:26:09.566683    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-628tz" podStartSLOduration=2.544345527 podStartE2EDuration="5.566664345s" totalImagesPullingTime="3.022318818s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 08:26:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 08:26:09.565844056 +0000 UTC m=+353.942553918" watchObservedRunningTime="2026-09-19 08:26:09.566664345 +0000 UTC m=+353.943374207"
Sep 19 08:26:09.650638 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:26:09.650573    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:26:09.655578 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:26:09.655541    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4hwws\" (UniqueName: \"kubernetes.io/projected/e7548679-13cf-403e-ac2f-1331641074ca-kube-api-access-4hwws\") pod \"keda-operator-579c8958f7-9q84f\" (UID: \"e7548679-13cf-403e-ac2f-1331641074ca\") " pod="openshift-keda/keda-operator-579c8958f7-9q84f"
Sep 19 08:26:09.655741 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:26:09.655618    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/e7548679-13cf-403e-ac2f-1331641074ca-cabundle0\") pod \"keda-operator-579c8958f7-9q84f\" (UID: \"e7548679-13cf-403e-ac2f-1331641074ca\") " pod="openshift-keda/keda-operator-579c8958f7-9q84f"
Sep 19 08:26:09.655741 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:26:09.655674    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/e7548679-13cf-403e-ac2f-1331641074ca-certificates\") pod \"keda-operator-579c8958f7-9q84f\" (UID: \"e7548679-13cf-403e-ac2f-1331641074ca\") " pod="openshift-keda/keda-operator-579c8958f7-9q84f"
Sep 19 08:26:09.656454 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:26:09.656427    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-metrics-apiserver-77857b7b98-kdjx4"]
Sep 19 08:26:09.656570 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:26:09.656473    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/e7548679-13cf-403e-ac2f-1331641074ca-cabundle0\") pod \"keda-operator-579c8958f7-9q84f\" (UID: \"e7548679-13cf-403e-ac2f-1331641074ca\") " pod="openshift-keda/keda-operator-579c8958f7-9q84f"
Sep 19 08:26:09.658793 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:26:09.658772    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/e7548679-13cf-403e-ac2f-1331641074ca-certificates\") pod \"keda-operator-579c8958f7-9q84f\" (UID: \"e7548679-13cf-403e-ac2f-1331641074ca\") " pod="openshift-keda/keda-operator-579c8958f7-9q84f"
Sep 19 08:26:09.659703 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:26:09.659684    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-metrics-apiserver-77857b7b98-kdjx4"
Sep 19 08:26:09.662485 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:26:09.662466    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-metrics-apiserver-certs\""
Sep 19 08:26:09.674335 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:26:09.674276    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-metrics-apiserver-77857b7b98-kdjx4"]
Sep 19 08:26:09.677733 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:26:09.677707    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4hwws\" (UniqueName: \"kubernetes.io/projected/e7548679-13cf-403e-ac2f-1331641074ca-kube-api-access-4hwws\") pod \"keda-operator-579c8958f7-9q84f\" (UID: \"e7548679-13cf-403e-ac2f-1331641074ca\") " pod="openshift-keda/keda-operator-579c8958f7-9q84f"
Sep 19 08:26:09.749893 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:26:09.749808    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-operator-579c8958f7-9q84f"
Sep 19 08:26:09.753865 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:26:09.753826    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:26:09.756812 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:26:09.756790    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v44w4\" (UniqueName: \"kubernetes.io/projected/3be0da99-460a-4882-9323-0f141ffa3aed-kube-api-access-v44w4\") pod \"keda-metrics-apiserver-77857b7b98-kdjx4\" (UID: \"3be0da99-460a-4882-9323-0f141ffa3aed\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-kdjx4"
Sep 19 08:26:09.756889 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:26:09.756831    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/3be0da99-460a-4882-9323-0f141ffa3aed-temp-vol\") pod \"keda-metrics-apiserver-77857b7b98-kdjx4\" (UID: \"3be0da99-460a-4882-9323-0f141ffa3aed\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-kdjx4"
Sep 19 08:26:09.756889 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:26:09.756856    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/3be0da99-460a-4882-9323-0f141ffa3aed-certificates\") pod \"keda-metrics-apiserver-77857b7b98-kdjx4\" (UID: \"3be0da99-460a-4882-9323-0f141ffa3aed\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-kdjx4"
Sep 19 08:26:09.813907 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:26:09.813391    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-admission-6c4d864b86-p6wl7"]
Sep 19 08:26:09.818534 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:26:09.818511    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-admission-6c4d864b86-p6wl7"
Sep 19 08:26:09.822853 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:26:09.821126    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-admission-webhooks-certs\""
Sep 19 08:26:09.827156 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:26:09.827095    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-admission-6c4d864b86-p6wl7"]
Sep 19 08:26:09.833799 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:26:09.833674    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:26:09.857699 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:26:09.857665    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v44w4\" (UniqueName: \"kubernetes.io/projected/3be0da99-460a-4882-9323-0f141ffa3aed-kube-api-access-v44w4\") pod \"keda-metrics-apiserver-77857b7b98-kdjx4\" (UID: \"3be0da99-460a-4882-9323-0f141ffa3aed\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-kdjx4"
Sep 19 08:26:09.857850 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:26:09.857731    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/3be0da99-460a-4882-9323-0f141ffa3aed-temp-vol\") pod \"keda-metrics-apiserver-77857b7b98-kdjx4\" (UID: \"3be0da99-460a-4882-9323-0f141ffa3aed\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-kdjx4"
Sep 19 08:26:09.857850 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:26:09.857779    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/3be0da99-460a-4882-9323-0f141ffa3aed-certificates\") pod \"keda-metrics-apiserver-77857b7b98-kdjx4\" (UID: \"3be0da99-460a-4882-9323-0f141ffa3aed\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-kdjx4"
Sep 19 08:26:09.857920 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:26:09.857896    2578 projected.go:266] Couldn't get secret openshift-keda/keda-metrics-apiserver-certs: secret "keda-metrics-apiserver-certs" not found
Sep 19 08:26:09.857954 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:26:09.857922    2578 projected.go:196] Error preparing data for projected volume certificates for pod openshift-keda/keda-metrics-apiserver-77857b7b98-kdjx4: secret "keda-metrics-apiserver-certs" not found
Sep 19 08:26:09.857988 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:26:09.857976    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3be0da99-460a-4882-9323-0f141ffa3aed-certificates podName:3be0da99-460a-4882-9323-0f141ffa3aed nodeName:}" failed. No retries permitted until 2026-09-19 08:26:10.357956634 +0000 UTC m=+354.734666489 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/3be0da99-460a-4882-9323-0f141ffa3aed-certificates") pod "keda-metrics-apiserver-77857b7b98-kdjx4" (UID: "3be0da99-460a-4882-9323-0f141ffa3aed") : secret "keda-metrics-apiserver-certs" not found
Sep 19 08:26:09.858288 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:26:09.858260    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/3be0da99-460a-4882-9323-0f141ffa3aed-temp-vol\") pod \"keda-metrics-apiserver-77857b7b98-kdjx4\" (UID: \"3be0da99-460a-4882-9323-0f141ffa3aed\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-kdjx4"
Sep 19 08:26:09.869760 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:26:09.869704    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v44w4\" (UniqueName: \"kubernetes.io/projected/3be0da99-460a-4882-9323-0f141ffa3aed-kube-api-access-v44w4\") pod \"keda-metrics-apiserver-77857b7b98-kdjx4\" (UID: \"3be0da99-460a-4882-9323-0f141ffa3aed\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-kdjx4"
Sep 19 08:26:09.901838 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:26:09.901802    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:26:09.959095 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:26:09.959060    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g2rqm\" (UniqueName: \"kubernetes.io/projected/28957d01-9e7d-4c20-be4a-917c030e61ca-kube-api-access-g2rqm\") pod \"keda-admission-6c4d864b86-p6wl7\" (UID: \"28957d01-9e7d-4c20-be4a-917c030e61ca\") " pod="openshift-keda/keda-admission-6c4d864b86-p6wl7"
Sep 19 08:26:09.959277 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:26:09.959109    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/28957d01-9e7d-4c20-be4a-917c030e61ca-certificates\") pod \"keda-admission-6c4d864b86-p6wl7\" (UID: \"28957d01-9e7d-4c20-be4a-917c030e61ca\") " pod="openshift-keda/keda-admission-6c4d864b86-p6wl7"
Sep 19 08:26:10.034645 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:26:10.034614    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-operator-579c8958f7-9q84f"]
Sep 19 08:26:10.037516 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:26:10.037484    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:26:10.037903 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:26:10.037880    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7548679_13cf_403e_ac2f_1331641074ca.slice/crio-cd3ae458f37dbf6c396dc41f278a20453249b7dba4b8c20be5b66a5605c1c73a WatchSource:0}: Error finding container cd3ae458f37dbf6c396dc41f278a20453249b7dba4b8c20be5b66a5605c1c73a: Status 404 returned error can't find the container with id cd3ae458f37dbf6c396dc41f278a20453249b7dba4b8c20be5b66a5605c1c73a
Sep 19 08:26:10.059630 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:26:10.059595    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g2rqm\" (UniqueName: \"kubernetes.io/projected/28957d01-9e7d-4c20-be4a-917c030e61ca-kube-api-access-g2rqm\") pod \"keda-admission-6c4d864b86-p6wl7\" (UID: \"28957d01-9e7d-4c20-be4a-917c030e61ca\") " pod="openshift-keda/keda-admission-6c4d864b86-p6wl7"
Sep 19 08:26:10.059630 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:26:10.059636    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/28957d01-9e7d-4c20-be4a-917c030e61ca-certificates\") pod \"keda-admission-6c4d864b86-p6wl7\" (UID: \"28957d01-9e7d-4c20-be4a-917c030e61ca\") " pod="openshift-keda/keda-admission-6c4d864b86-p6wl7"
Sep 19 08:26:10.062110 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:26:10.062090    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/28957d01-9e7d-4c20-be4a-917c030e61ca-certificates\") pod \"keda-admission-6c4d864b86-p6wl7\" (UID: \"28957d01-9e7d-4c20-be4a-917c030e61ca\") " pod="openshift-keda/keda-admission-6c4d864b86-p6wl7"
Sep 19 08:26:10.069383 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:26:10.069360    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g2rqm\" (UniqueName: \"kubernetes.io/projected/28957d01-9e7d-4c20-be4a-917c030e61ca-kube-api-access-g2rqm\") pod \"keda-admission-6c4d864b86-p6wl7\" (UID: \"28957d01-9e7d-4c20-be4a-917c030e61ca\") " pod="openshift-keda/keda-admission-6c4d864b86-p6wl7"
Sep 19 08:26:10.110397 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:26:10.110362    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:26:10.138541 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:26:10.138507    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-admission-6c4d864b86-p6wl7"
Sep 19 08:26:10.211256 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:26:10.211204    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:26:10.309703 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:26:10.309624    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:26:10.362196 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:26:10.362159    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/3be0da99-460a-4882-9323-0f141ffa3aed-certificates\") pod \"keda-metrics-apiserver-77857b7b98-kdjx4\" (UID: \"3be0da99-460a-4882-9323-0f141ffa3aed\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-kdjx4"
Sep 19 08:26:10.364791 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:26:10.364765    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/3be0da99-460a-4882-9323-0f141ffa3aed-certificates\") pod \"keda-metrics-apiserver-77857b7b98-kdjx4\" (UID: \"3be0da99-460a-4882-9323-0f141ffa3aed\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-kdjx4"
Sep 19 08:26:10.407629 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:26:10.407597    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-admission-6c4d864b86-p6wl7"]
Sep 19 08:26:10.410976 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:26:10.410945    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28957d01_9e7d_4c20_be4a_917c030e61ca.slice/crio-8e7c44651963a103f2bdd88b92dde618078db975c29eafe71f27fe3b03413d7a WatchSource:0}: Error finding container 8e7c44651963a103f2bdd88b92dde618078db975c29eafe71f27fe3b03413d7a: Status 404 returned error can't find the container with id 8e7c44651963a103f2bdd88b92dde618078db975c29eafe71f27fe3b03413d7a
Sep 19 08:26:10.510318 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:26:10.510272    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:26:10.553393 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:26:10.553357    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-admission-6c4d864b86-p6wl7" event={"ID":"28957d01-9e7d-4c20-be4a-917c030e61ca","Type":"ContainerStarted","Data":"8e7c44651963a103f2bdd88b92dde618078db975c29eafe71f27fe3b03413d7a"}
Sep 19 08:26:10.554629 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:26:10.554603    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-operator-579c8958f7-9q84f" event={"ID":"e7548679-13cf-403e-ac2f-1331641074ca","Type":"ContainerStarted","Data":"cd3ae458f37dbf6c396dc41f278a20453249b7dba4b8c20be5b66a5605c1c73a"}
Sep 19 08:26:10.575254 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:26:10.575174    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-metrics-apiserver-77857b7b98-kdjx4"
Sep 19 08:26:10.610683 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:26:10.610637    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:26:10.853292 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:26:10.853179    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-metrics-apiserver-77857b7b98-kdjx4"]
Sep 19 08:26:10.855588 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:26:10.855562    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3be0da99_460a_4882_9323_0f141ffa3aed.slice/crio-98616408bb31641225552f20c4f2ffa94110211c628e444a8eb101327f95e1f6 WatchSource:0}: Error finding container 98616408bb31641225552f20c4f2ffa94110211c628e444a8eb101327f95e1f6: Status 404 returned error can't find the container with id 98616408bb31641225552f20c4f2ffa94110211c628e444a8eb101327f95e1f6
Sep 19 08:26:11.560650 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:26:11.560610    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-metrics-apiserver-77857b7b98-kdjx4" event={"ID":"3be0da99-460a-4882-9323-0f141ffa3aed","Type":"ContainerStarted","Data":"98616408bb31641225552f20c4f2ffa94110211c628e444a8eb101327f95e1f6"}
Sep 19 08:26:12.566788 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:26:12.566747    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-admission-6c4d864b86-p6wl7" event={"ID":"28957d01-9e7d-4c20-be4a-917c030e61ca","Type":"ContainerStarted","Data":"b014e2bc26a6a6cd34594102239f3f78226ec53cd59e7ab3b5eb4e572f7cdad8"}
Sep 19 08:26:12.567230 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:26:12.566885    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-admission-6c4d864b86-p6wl7"
Sep 19 08:26:12.589049 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:26:12.588995    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/keda-admission-6c4d864b86-p6wl7" podStartSLOduration=2.172514771 podStartE2EDuration="3.588979694s" totalImagesPullingTime="1.416464923s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 08:26:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 08:26:12.587153594 +0000 UTC m=+356.963863457" watchObservedRunningTime="2026-09-19 08:26:12.588979694 +0000 UTC m=+356.965689598"
Sep 19 08:26:14.577527 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:26:14.577479    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-metrics-apiserver-77857b7b98-kdjx4" event={"ID":"3be0da99-460a-4882-9323-0f141ffa3aed","Type":"ContainerStarted","Data":"2bb2038b6905c9e92aa9a996490bd786debc392e50a4723e2b0a918527be5c85"}
Sep 19 08:26:14.577886 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:26:14.577553    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-metrics-apiserver-77857b7b98-kdjx4"
Sep 19 08:26:14.593615 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:26:14.593549    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/keda-metrics-apiserver-77857b7b98-kdjx4" podStartSLOduration=2.825875166 podStartE2EDuration="5.593527344s" totalImagesPullingTime="2.767652178s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 08:26:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 08:26:14.592928777 +0000 UTC m=+358.969638650" watchObservedRunningTime="2026-09-19 08:26:14.593527344 +0000 UTC m=+358.970237208"
Sep 19 08:26:18.602154 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:26:18.602118    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-operator-579c8958f7-9q84f" event={"ID":"e7548679-13cf-403e-ac2f-1331641074ca","Type":"ContainerStarted","Data":"174249995f0a8c4d6bdef2df69a548dba22d95070442213ccd96ec97e28230ef"}
Sep 19 08:26:18.602603 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:26:18.602262    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-operator-579c8958f7-9q84f"
Sep 19 08:26:18.618672 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:26:18.618623    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/keda-operator-579c8958f7-9q84f" podStartSLOduration=1.8317937560000002 podStartE2EDuration="9.618607211s" totalImagesPullingTime="7.786813455s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 08:26:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 08:26:18.617416488 +0000 UTC m=+362.994126351" watchObservedRunningTime="2026-09-19 08:26:18.618607211 +0000 UTC m=+362.995317073"
Sep 19 08:26:25.590896 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:26:25.590862    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-metrics-apiserver-77857b7b98-kdjx4"
Sep 19 08:26:30.557471 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:26:30.557436    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-628tz"
Sep 19 08:26:33.572769 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:26:33.572737    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-admission-6c4d864b86-p6wl7"
Sep 19 08:26:39.609409 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:26:39.609378    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-operator-579c8958f7-9q84f"
Sep 19 08:27:02.125692 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:27:02.125656    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:27:02.219768 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:27:02.219731    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:27:27.629143 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:27:27.629104    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:27:27.675987 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:27:27.675950    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/kserve-controller-manager-7dcf4bff6-vmxk7"]
Sep 19 08:27:27.678623 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:27:27.678606    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-7dcf4bff6-vmxk7"
Sep 19 08:27:27.681008 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:27:27.680983    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"kserve-webhook-server-cert\""
Sep 19 08:27:27.681127 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:27:27.681071    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"kube-root-ca.crt\""
Sep 19 08:27:27.681940 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:27:27.681924    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"openshift-service-ca.crt\""
Sep 19 08:27:27.682003 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:27:27.681925    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"kserve-controller-manager-dockercfg-t9vsh\""
Sep 19 08:27:27.688100 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:27:27.688074    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-7dcf4bff6-vmxk7"]
Sep 19 08:27:27.712547 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:27:27.712509    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:27:27.774504 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:27:27.774465    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:27:27.784030 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:27:27.783992    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jf2dh\" (UniqueName: \"kubernetes.io/projected/8b640099-9239-4cc6-8962-4a08d9f6aad8-kube-api-access-jf2dh\") pod \"kserve-controller-manager-7dcf4bff6-vmxk7\" (UID: \"8b640099-9239-4cc6-8962-4a08d9f6aad8\") " pod="kserve/kserve-controller-manager-7dcf4bff6-vmxk7"
Sep 19 08:27:27.784175 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:27:27.784125    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8b640099-9239-4cc6-8962-4a08d9f6aad8-cert\") pod \"kserve-controller-manager-7dcf4bff6-vmxk7\" (UID: \"8b640099-9239-4cc6-8962-4a08d9f6aad8\") " pod="kserve/kserve-controller-manager-7dcf4bff6-vmxk7"
Sep 19 08:27:27.875781 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:27:27.875743    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:27:27.884856 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:27:27.884802    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8b640099-9239-4cc6-8962-4a08d9f6aad8-cert\") pod \"kserve-controller-manager-7dcf4bff6-vmxk7\" (UID: \"8b640099-9239-4cc6-8962-4a08d9f6aad8\") " pod="kserve/kserve-controller-manager-7dcf4bff6-vmxk7"
Sep 19 08:27:27.884856 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:27:27.884851    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jf2dh\" (UniqueName: \"kubernetes.io/projected/8b640099-9239-4cc6-8962-4a08d9f6aad8-kube-api-access-jf2dh\") pod \"kserve-controller-manager-7dcf4bff6-vmxk7\" (UID: \"8b640099-9239-4cc6-8962-4a08d9f6aad8\") " pod="kserve/kserve-controller-manager-7dcf4bff6-vmxk7"
Sep 19 08:27:27.887330 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:27:27.887289    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8b640099-9239-4cc6-8962-4a08d9f6aad8-cert\") pod \"kserve-controller-manager-7dcf4bff6-vmxk7\" (UID: \"8b640099-9239-4cc6-8962-4a08d9f6aad8\") " pod="kserve/kserve-controller-manager-7dcf4bff6-vmxk7"
Sep 19 08:27:27.892312 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:27:27.892276    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jf2dh\" (UniqueName: \"kubernetes.io/projected/8b640099-9239-4cc6-8962-4a08d9f6aad8-kube-api-access-jf2dh\") pod \"kserve-controller-manager-7dcf4bff6-vmxk7\" (UID: \"8b640099-9239-4cc6-8962-4a08d9f6aad8\") " pod="kserve/kserve-controller-manager-7dcf4bff6-vmxk7"
Sep 19 08:27:27.976764 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:27:27.976734    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:27:27.989799 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:27:27.989768    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-7dcf4bff6-vmxk7"
Sep 19 08:27:28.184920 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:27:28.184829    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:27:28.255762 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:27:28.255738    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-7dcf4bff6-vmxk7"]
Sep 19 08:27:28.258130 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:27:28.258099    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b640099_9239_4cc6_8962_4a08d9f6aad8.slice/crio-d76c7e57d7c44a3af9bb3bae715825489bb181f908a8d75c49508900bceffc93 WatchSource:0}: Error finding container d76c7e57d7c44a3af9bb3bae715825489bb181f908a8d75c49508900bceffc93: Status 404 returned error can't find the container with id d76c7e57d7c44a3af9bb3bae715825489bb181f908a8d75c49508900bceffc93
Sep 19 08:27:28.258772 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:27:28.258744    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:27:28.902931 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:27:28.902897    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-7dcf4bff6-vmxk7" event={"ID":"8b640099-9239-4cc6-8962-4a08d9f6aad8","Type":"ContainerStarted","Data":"d76c7e57d7c44a3af9bb3bae715825489bb181f908a8d75c49508900bceffc93"}
Sep 19 08:27:31.917253 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:27:31.917213    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-7dcf4bff6-vmxk7" event={"ID":"8b640099-9239-4cc6-8962-4a08d9f6aad8","Type":"ContainerStarted","Data":"aa25e09ef06bbf61d8aeafe95ab50a35f1ad02a8918f73290499c3bc182e4083"}
Sep 19 08:27:31.917743 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:27:31.917279    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/kserve-controller-manager-7dcf4bff6-vmxk7"
Sep 19 08:27:31.935623 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:27:31.935578    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/kserve-controller-manager-7dcf4bff6-vmxk7" podStartSLOduration=2.321859225 podStartE2EDuration="4.935561814s" totalImagesPullingTime="2.613702589s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 08:27:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 08:27:31.934789143 +0000 UTC m=+436.311499016" watchObservedRunningTime="2026-09-19 08:27:31.935561814 +0000 UTC m=+436.312271675"
Sep 19 08:27:41.123894 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:27:41.123819    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-f94c86659-4mqfr"]
Sep 19 08:27:41.126816 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:27:41.126797    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-f94c86659-4mqfr"
Sep 19 08:27:41.137321 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:27:41.137274    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-f94c86659-4mqfr"]
Sep 19 08:27:41.178784 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:27:41.178740    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:27:41.279935 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:27:41.279889    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:27:41.292022 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:27:41.291984    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5cab1262-0ee5-473e-9077-af89e224ba45-service-ca\") pod \"console-f94c86659-4mqfr\" (UID: \"5cab1262-0ee5-473e-9077-af89e224ba45\") " pod="openshift-console/console-f94c86659-4mqfr"
Sep 19 08:27:41.292146 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:27:41.292038    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5cab1262-0ee5-473e-9077-af89e224ba45-oauth-serving-cert\") pod \"console-f94c86659-4mqfr\" (UID: \"5cab1262-0ee5-473e-9077-af89e224ba45\") " pod="openshift-console/console-f94c86659-4mqfr"
Sep 19 08:27:41.292146 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:27:41.292059    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5cab1262-0ee5-473e-9077-af89e224ba45-trusted-ca-bundle\") pod \"console-f94c86659-4mqfr\" (UID: \"5cab1262-0ee5-473e-9077-af89e224ba45\") " pod="openshift-console/console-f94c86659-4mqfr"
Sep 19 08:27:41.292146 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:27:41.292089    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5cab1262-0ee5-473e-9077-af89e224ba45-console-config\") pod \"console-f94c86659-4mqfr\" (UID: \"5cab1262-0ee5-473e-9077-af89e224ba45\") " pod="openshift-console/console-f94c86659-4mqfr"
Sep 19 08:27:41.292146 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:27:41.292107    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8r6qg\" (UniqueName: \"kubernetes.io/projected/5cab1262-0ee5-473e-9077-af89e224ba45-kube-api-access-8r6qg\") pod \"console-f94c86659-4mqfr\" (UID: \"5cab1262-0ee5-473e-9077-af89e224ba45\") " pod="openshift-console/console-f94c86659-4mqfr"
Sep 19 08:27:41.292146 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:27:41.292136    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5cab1262-0ee5-473e-9077-af89e224ba45-console-serving-cert\") pod \"console-f94c86659-4mqfr\" (UID: \"5cab1262-0ee5-473e-9077-af89e224ba45\") " pod="openshift-console/console-f94c86659-4mqfr"
Sep 19 08:27:41.292370 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:27:41.292150    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5cab1262-0ee5-473e-9077-af89e224ba45-console-oauth-config\") pod \"console-f94c86659-4mqfr\" (UID: \"5cab1262-0ee5-473e-9077-af89e224ba45\") " pod="openshift-console/console-f94c86659-4mqfr"
Sep 19 08:27:41.393129 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:27:41.393037    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5cab1262-0ee5-473e-9077-af89e224ba45-service-ca\") pod \"console-f94c86659-4mqfr\" (UID: \"5cab1262-0ee5-473e-9077-af89e224ba45\") " pod="openshift-console/console-f94c86659-4mqfr"
Sep 19 08:27:41.393129 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:27:41.393072    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5cab1262-0ee5-473e-9077-af89e224ba45-oauth-serving-cert\") pod \"console-f94c86659-4mqfr\" (UID: \"5cab1262-0ee5-473e-9077-af89e224ba45\") " pod="openshift-console/console-f94c86659-4mqfr"
Sep 19 08:27:41.393129 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:27:41.393091    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5cab1262-0ee5-473e-9077-af89e224ba45-trusted-ca-bundle\") pod \"console-f94c86659-4mqfr\" (UID: \"5cab1262-0ee5-473e-9077-af89e224ba45\") " pod="openshift-console/console-f94c86659-4mqfr"
Sep 19 08:27:41.393129 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:27:41.393109    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5cab1262-0ee5-473e-9077-af89e224ba45-console-config\") pod \"console-f94c86659-4mqfr\" (UID: \"5cab1262-0ee5-473e-9077-af89e224ba45\") " pod="openshift-console/console-f94c86659-4mqfr"
Sep 19 08:27:41.393129 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:27:41.393126    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8r6qg\" (UniqueName: \"kubernetes.io/projected/5cab1262-0ee5-473e-9077-af89e224ba45-kube-api-access-8r6qg\") pod \"console-f94c86659-4mqfr\" (UID: \"5cab1262-0ee5-473e-9077-af89e224ba45\") " pod="openshift-console/console-f94c86659-4mqfr"
Sep 19 08:27:41.393474 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:27:41.393166    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5cab1262-0ee5-473e-9077-af89e224ba45-console-serving-cert\") pod \"console-f94c86659-4mqfr\" (UID: \"5cab1262-0ee5-473e-9077-af89e224ba45\") " pod="openshift-console/console-f94c86659-4mqfr"
Sep 19 08:27:41.393474 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:27:41.393187    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5cab1262-0ee5-473e-9077-af89e224ba45-console-oauth-config\") pod \"console-f94c86659-4mqfr\" (UID: \"5cab1262-0ee5-473e-9077-af89e224ba45\") " pod="openshift-console/console-f94c86659-4mqfr"
Sep 19 08:27:41.393944 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:27:41.393918    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5cab1262-0ee5-473e-9077-af89e224ba45-oauth-serving-cert\") pod \"console-f94c86659-4mqfr\" (UID: \"5cab1262-0ee5-473e-9077-af89e224ba45\") " pod="openshift-console/console-f94c86659-4mqfr"
Sep 19 08:27:41.394073 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:27:41.393975    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5cab1262-0ee5-473e-9077-af89e224ba45-console-config\") pod \"console-f94c86659-4mqfr\" (UID: \"5cab1262-0ee5-473e-9077-af89e224ba45\") " pod="openshift-console/console-f94c86659-4mqfr"
Sep 19 08:27:41.394130 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:27:41.394007    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5cab1262-0ee5-473e-9077-af89e224ba45-service-ca\") pod \"console-f94c86659-4mqfr\" (UID: \"5cab1262-0ee5-473e-9077-af89e224ba45\") " pod="openshift-console/console-f94c86659-4mqfr"
Sep 19 08:27:41.394130 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:27:41.394109    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5cab1262-0ee5-473e-9077-af89e224ba45-trusted-ca-bundle\") pod \"console-f94c86659-4mqfr\" (UID: \"5cab1262-0ee5-473e-9077-af89e224ba45\") " pod="openshift-console/console-f94c86659-4mqfr"
Sep 19 08:27:41.395643 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:27:41.395622    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5cab1262-0ee5-473e-9077-af89e224ba45-console-oauth-config\") pod \"console-f94c86659-4mqfr\" (UID: \"5cab1262-0ee5-473e-9077-af89e224ba45\") " pod="openshift-console/console-f94c86659-4mqfr"
Sep 19 08:27:41.395889 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:27:41.395867    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5cab1262-0ee5-473e-9077-af89e224ba45-console-serving-cert\") pod \"console-f94c86659-4mqfr\" (UID: \"5cab1262-0ee5-473e-9077-af89e224ba45\") " pod="openshift-console/console-f94c86659-4mqfr"
Sep 19 08:27:41.401909 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:27:41.401887    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8r6qg\" (UniqueName: \"kubernetes.io/projected/5cab1262-0ee5-473e-9077-af89e224ba45-kube-api-access-8r6qg\") pod \"console-f94c86659-4mqfr\" (UID: \"5cab1262-0ee5-473e-9077-af89e224ba45\") " pod="openshift-console/console-f94c86659-4mqfr"
Sep 19 08:27:41.436158 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:27:41.436135    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-f94c86659-4mqfr"
Sep 19 08:27:41.706673 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:27:41.706645    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-f94c86659-4mqfr"]
Sep 19 08:27:41.709169 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:27:41.709142    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5cab1262_0ee5_473e_9077_af89e224ba45.slice/crio-5bb6e3956deb660b3c8a89b2c070f3276a65868274aa1f74f8797f57a7fb4fd0 WatchSource:0}: Error finding container 5bb6e3956deb660b3c8a89b2c070f3276a65868274aa1f74f8797f57a7fb4fd0: Status 404 returned error can't find the container with id 5bb6e3956deb660b3c8a89b2c070f3276a65868274aa1f74f8797f57a7fb4fd0
Sep 19 08:27:41.961281 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:27:41.961202    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-f94c86659-4mqfr" event={"ID":"5cab1262-0ee5-473e-9077-af89e224ba45","Type":"ContainerStarted","Data":"10a8d937110d1253f8c3904f8213a1ed70d3742739b017f966c0d2412bd0a8f8"}
Sep 19 08:27:41.961281 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:27:41.961250    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-f94c86659-4mqfr" event={"ID":"5cab1262-0ee5-473e-9077-af89e224ba45","Type":"ContainerStarted","Data":"5bb6e3956deb660b3c8a89b2c070f3276a65868274aa1f74f8797f57a7fb4fd0"}
Sep 19 08:27:41.978576 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:27:41.978528    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-f94c86659-4mqfr" podStartSLOduration=0.978512101 podStartE2EDuration="978.512101ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 08:27:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 08:27:41.977270971 +0000 UTC m=+446.353980834" watchObservedRunningTime="2026-09-19 08:27:41.978512101 +0000 UTC m=+446.355221957"
Sep 19 08:27:50.155471 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:27:50.155404    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:27:51.436881 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:27:51.436839    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-f94c86659-4mqfr"
Sep 19 08:27:51.437270 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:27:51.436899    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-f94c86659-4mqfr"
Sep 19 08:27:51.442103 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:27:51.442074    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-f94c86659-4mqfr"
Sep 19 08:27:52.007818 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:27:52.007784    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-f94c86659-4mqfr"
Sep 19 08:27:52.112949 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:27:52.112915    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-65b4797958-6c5jr"]
Sep 19 08:27:52.240144 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:27:52.240109    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:27:52.340864 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:27:52.340747    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:27:52.461708 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:27:52.461647    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:28:02.927061 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:28:02.927028    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/kserve-controller-manager-7dcf4bff6-vmxk7"
Sep 19 08:28:10.889313 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:28:10.889252    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve/kserve-controller-manager-7dcf4bff6-vmxk7"]
Sep 19 08:28:10.889741 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:28:10.889533    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve/kserve-controller-manager-7dcf4bff6-vmxk7" podUID="8b640099-9239-4cc6-8962-4a08d9f6aad8" containerName="manager" containerID="cri-o://aa25e09ef06bbf61d8aeafe95ab50a35f1ad02a8918f73290499c3bc182e4083" gracePeriod=10
Sep 19 08:28:10.922893 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:28:10.922843    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/kserve-controller-manager-7dcf4bff6-l2zf7"]
Sep 19 08:28:10.925107 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:28:10.925087    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-7dcf4bff6-l2zf7"
Sep 19 08:28:10.938283 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:28:10.938245    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-7dcf4bff6-l2zf7"]
Sep 19 08:28:10.994404 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:28:10.994369    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9wcf9\" (UniqueName: \"kubernetes.io/projected/60b42bfb-489c-49ce-bed0-e1751ba8ada8-kube-api-access-9wcf9\") pod \"kserve-controller-manager-7dcf4bff6-l2zf7\" (UID: \"60b42bfb-489c-49ce-bed0-e1751ba8ada8\") " pod="kserve/kserve-controller-manager-7dcf4bff6-l2zf7"
Sep 19 08:28:10.994533 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:28:10.994422    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/60b42bfb-489c-49ce-bed0-e1751ba8ada8-cert\") pod \"kserve-controller-manager-7dcf4bff6-l2zf7\" (UID: \"60b42bfb-489c-49ce-bed0-e1751ba8ada8\") " pod="kserve/kserve-controller-manager-7dcf4bff6-l2zf7"
Sep 19 08:28:11.085327 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:28:11.085277    2578 generic.go:356] "Generic (PLEG): container finished" podID="8b640099-9239-4cc6-8962-4a08d9f6aad8" containerID="aa25e09ef06bbf61d8aeafe95ab50a35f1ad02a8918f73290499c3bc182e4083" exitCode=0
Sep 19 08:28:11.085471 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:28:11.085328    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-7dcf4bff6-vmxk7" event={"ID":"8b640099-9239-4cc6-8962-4a08d9f6aad8","Type":"ContainerDied","Data":"aa25e09ef06bbf61d8aeafe95ab50a35f1ad02a8918f73290499c3bc182e4083"}
Sep 19 08:28:11.095611 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:28:11.095586    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9wcf9\" (UniqueName: \"kubernetes.io/projected/60b42bfb-489c-49ce-bed0-e1751ba8ada8-kube-api-access-9wcf9\") pod \"kserve-controller-manager-7dcf4bff6-l2zf7\" (UID: \"60b42bfb-489c-49ce-bed0-e1751ba8ada8\") " pod="kserve/kserve-controller-manager-7dcf4bff6-l2zf7"
Sep 19 08:28:11.095737 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:28:11.095638    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/60b42bfb-489c-49ce-bed0-e1751ba8ada8-cert\") pod \"kserve-controller-manager-7dcf4bff6-l2zf7\" (UID: \"60b42bfb-489c-49ce-bed0-e1751ba8ada8\") " pod="kserve/kserve-controller-manager-7dcf4bff6-l2zf7"
Sep 19 08:28:11.097938 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:28:11.097911    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/60b42bfb-489c-49ce-bed0-e1751ba8ada8-cert\") pod \"kserve-controller-manager-7dcf4bff6-l2zf7\" (UID: \"60b42bfb-489c-49ce-bed0-e1751ba8ada8\") " pod="kserve/kserve-controller-manager-7dcf4bff6-l2zf7"
Sep 19 08:28:11.103852 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:28:11.103828    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9wcf9\" (UniqueName: \"kubernetes.io/projected/60b42bfb-489c-49ce-bed0-e1751ba8ada8-kube-api-access-9wcf9\") pod \"kserve-controller-manager-7dcf4bff6-l2zf7\" (UID: \"60b42bfb-489c-49ce-bed0-e1751ba8ada8\") " pod="kserve/kserve-controller-manager-7dcf4bff6-l2zf7"
Sep 19 08:28:11.123357 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:28:11.123335    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-7dcf4bff6-vmxk7"
Sep 19 08:28:11.196450 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:28:11.196369    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8b640099-9239-4cc6-8962-4a08d9f6aad8-cert\") pod \"8b640099-9239-4cc6-8962-4a08d9f6aad8\" (UID: \"8b640099-9239-4cc6-8962-4a08d9f6aad8\") "
Sep 19 08:28:11.196450 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:28:11.196432    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jf2dh\" (UniqueName: \"kubernetes.io/projected/8b640099-9239-4cc6-8962-4a08d9f6aad8-kube-api-access-jf2dh\") pod \"8b640099-9239-4cc6-8962-4a08d9f6aad8\" (UID: \"8b640099-9239-4cc6-8962-4a08d9f6aad8\") "
Sep 19 08:28:11.198587 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:28:11.198557    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b640099-9239-4cc6-8962-4a08d9f6aad8-kube-api-access-jf2dh" (OuterVolumeSpecName: "kube-api-access-jf2dh") pod "8b640099-9239-4cc6-8962-4a08d9f6aad8" (UID: "8b640099-9239-4cc6-8962-4a08d9f6aad8"). InnerVolumeSpecName "kube-api-access-jf2dh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 08:28:11.198709 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:28:11.198560    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8b640099-9239-4cc6-8962-4a08d9f6aad8-cert" (OuterVolumeSpecName: "cert") pod "8b640099-9239-4cc6-8962-4a08d9f6aad8" (UID: "8b640099-9239-4cc6-8962-4a08d9f6aad8"). InnerVolumeSpecName "cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 08:28:11.279108 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:28:11.279065    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-7dcf4bff6-l2zf7"
Sep 19 08:28:11.297081 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:28:11.297054    2578 reconciler_common.go:299] "Volume detached for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8b640099-9239-4cc6-8962-4a08d9f6aad8-cert\") on node \"ip-10-0-139-157.ec2.internal\" DevicePath \"\""
Sep 19 08:28:11.297081 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:28:11.297081    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jf2dh\" (UniqueName: \"kubernetes.io/projected/8b640099-9239-4cc6-8962-4a08d9f6aad8-kube-api-access-jf2dh\") on node \"ip-10-0-139-157.ec2.internal\" DevicePath \"\""
Sep 19 08:28:11.545855 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:28:11.545828    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-7dcf4bff6-l2zf7"]
Sep 19 08:28:11.548210 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:28:11.548183    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60b42bfb_489c_49ce_bed0_e1751ba8ada8.slice/crio-e1651ba66f4f980d7b495aee7f6de75c3c7e5e8306608b7970577b63b00d5015 WatchSource:0}: Error finding container e1651ba66f4f980d7b495aee7f6de75c3c7e5e8306608b7970577b63b00d5015: Status 404 returned error can't find the container with id e1651ba66f4f980d7b495aee7f6de75c3c7e5e8306608b7970577b63b00d5015
Sep 19 08:28:12.091236 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:28:12.091210    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-7dcf4bff6-vmxk7"
Sep 19 08:28:12.091712 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:28:12.091209    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-7dcf4bff6-vmxk7" event={"ID":"8b640099-9239-4cc6-8962-4a08d9f6aad8","Type":"ContainerDied","Data":"d76c7e57d7c44a3af9bb3bae715825489bb181f908a8d75c49508900bceffc93"}
Sep 19 08:28:12.091712 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:28:12.091388    2578 scope.go:117] "RemoveContainer" containerID="aa25e09ef06bbf61d8aeafe95ab50a35f1ad02a8918f73290499c3bc182e4083"
Sep 19 08:28:12.093576 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:28:12.093530    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-7dcf4bff6-l2zf7" event={"ID":"60b42bfb-489c-49ce-bed0-e1751ba8ada8","Type":"ContainerStarted","Data":"0fcb6ddd8e233af830341aa0245203a1fc9649d4d00648d30a2817a8c3008e5f"}
Sep 19 08:28:12.093716 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:28:12.093588    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-7dcf4bff6-l2zf7" event={"ID":"60b42bfb-489c-49ce-bed0-e1751ba8ada8","Type":"ContainerStarted","Data":"e1651ba66f4f980d7b495aee7f6de75c3c7e5e8306608b7970577b63b00d5015"}
Sep 19 08:28:12.093716 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:28:12.093621    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/kserve-controller-manager-7dcf4bff6-l2zf7"
Sep 19 08:28:12.109447 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:28:12.109395    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/kserve-controller-manager-7dcf4bff6-l2zf7" podStartSLOduration=1.754710867 podStartE2EDuration="2.109379481s" totalImagesPullingTime="354.668614ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 08:28:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 08:28:12.10860895 +0000 UTC m=+476.485318815" watchObservedRunningTime="2026-09-19 08:28:12.109379481 +0000 UTC m=+476.486089343"
Sep 19 08:28:12.122352 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:28:12.122324    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve/kserve-controller-manager-7dcf4bff6-vmxk7"]
Sep 19 08:28:12.124573 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:28:12.124554    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve/kserve-controller-manager-7dcf4bff6-vmxk7"]
Sep 19 08:28:14.112789 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:28:14.112758    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8b640099-9239-4cc6-8962-4a08d9f6aad8" path="/var/lib/kubelet/pods/8b640099-9239-4cc6-8962-4a08d9f6aad8/volumes"
Sep 19 08:28:17.132984 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:28:17.132939    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-65b4797958-6c5jr" podUID="d6783502-5784-4288-93c8-e39e6c27a201" containerName="console" containerID="cri-o://aef69d11046d37433b04d75213470772cd8f7bc5a9d097be86ed47d3cef1d4d9" gracePeriod=15
Sep 19 08:28:17.387339 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:28:17.387261    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-65b4797958-6c5jr"
Sep 19 08:28:17.441807 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:28:17.441772    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d6783502-5784-4288-93c8-e39e6c27a201-service-ca\") pod \"d6783502-5784-4288-93c8-e39e6c27a201\" (UID: \"d6783502-5784-4288-93c8-e39e6c27a201\") "
Sep 19 08:28:17.441807 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:28:17.441816    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lg5x9\" (UniqueName: \"kubernetes.io/projected/d6783502-5784-4288-93c8-e39e6c27a201-kube-api-access-lg5x9\") pod \"d6783502-5784-4288-93c8-e39e6c27a201\" (UID: \"d6783502-5784-4288-93c8-e39e6c27a201\") "
Sep 19 08:28:17.442021 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:28:17.441872    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d6783502-5784-4288-93c8-e39e6c27a201-console-config\") pod \"d6783502-5784-4288-93c8-e39e6c27a201\" (UID: \"d6783502-5784-4288-93c8-e39e6c27a201\") "
Sep 19 08:28:17.442021 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:28:17.441897    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d6783502-5784-4288-93c8-e39e6c27a201-oauth-serving-cert\") pod \"d6783502-5784-4288-93c8-e39e6c27a201\" (UID: \"d6783502-5784-4288-93c8-e39e6c27a201\") "
Sep 19 08:28:17.442021 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:28:17.441924    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d6783502-5784-4288-93c8-e39e6c27a201-console-oauth-config\") pod \"d6783502-5784-4288-93c8-e39e6c27a201\" (UID: \"d6783502-5784-4288-93c8-e39e6c27a201\") "
Sep 19 08:28:17.442181 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:28:17.442016    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d6783502-5784-4288-93c8-e39e6c27a201-trusted-ca-bundle\") pod \"d6783502-5784-4288-93c8-e39e6c27a201\" (UID: \"d6783502-5784-4288-93c8-e39e6c27a201\") "
Sep 19 08:28:17.442181 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:28:17.442063    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d6783502-5784-4288-93c8-e39e6c27a201-console-serving-cert\") pod \"d6783502-5784-4288-93c8-e39e6c27a201\" (UID: \"d6783502-5784-4288-93c8-e39e6c27a201\") "
Sep 19 08:28:17.442328 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:28:17.442194    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d6783502-5784-4288-93c8-e39e6c27a201-service-ca" (OuterVolumeSpecName: "service-ca") pod "d6783502-5784-4288-93c8-e39e6c27a201" (UID: "d6783502-5784-4288-93c8-e39e6c27a201"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 08:28:17.442451 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:28:17.442353    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d6783502-5784-4288-93c8-e39e6c27a201-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "d6783502-5784-4288-93c8-e39e6c27a201" (UID: "d6783502-5784-4288-93c8-e39e6c27a201"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 08:28:17.442451 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:28:17.442367    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d6783502-5784-4288-93c8-e39e6c27a201-console-config" (OuterVolumeSpecName: "console-config") pod "d6783502-5784-4288-93c8-e39e6c27a201" (UID: "d6783502-5784-4288-93c8-e39e6c27a201"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 08:28:17.442534 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:28:17.442448    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d6783502-5784-4288-93c8-e39e6c27a201-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "d6783502-5784-4288-93c8-e39e6c27a201" (UID: "d6783502-5784-4288-93c8-e39e6c27a201"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 08:28:17.444142 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:28:17.444117    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d6783502-5784-4288-93c8-e39e6c27a201-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "d6783502-5784-4288-93c8-e39e6c27a201" (UID: "d6783502-5784-4288-93c8-e39e6c27a201"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 08:28:17.444215 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:28:17.444157    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d6783502-5784-4288-93c8-e39e6c27a201-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "d6783502-5784-4288-93c8-e39e6c27a201" (UID: "d6783502-5784-4288-93c8-e39e6c27a201"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 08:28:17.444215 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:28:17.444189    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d6783502-5784-4288-93c8-e39e6c27a201-kube-api-access-lg5x9" (OuterVolumeSpecName: "kube-api-access-lg5x9") pod "d6783502-5784-4288-93c8-e39e6c27a201" (UID: "d6783502-5784-4288-93c8-e39e6c27a201"). InnerVolumeSpecName "kube-api-access-lg5x9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 08:28:17.543426 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:28:17.543390    2578 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d6783502-5784-4288-93c8-e39e6c27a201-console-config\") on node \"ip-10-0-139-157.ec2.internal\" DevicePath \"\""
Sep 19 08:28:17.543426 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:28:17.543418    2578 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d6783502-5784-4288-93c8-e39e6c27a201-oauth-serving-cert\") on node \"ip-10-0-139-157.ec2.internal\" DevicePath \"\""
Sep 19 08:28:17.543426 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:28:17.543427    2578 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d6783502-5784-4288-93c8-e39e6c27a201-console-oauth-config\") on node \"ip-10-0-139-157.ec2.internal\" DevicePath \"\""
Sep 19 08:28:17.543426 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:28:17.543436    2578 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d6783502-5784-4288-93c8-e39e6c27a201-trusted-ca-bundle\") on node \"ip-10-0-139-157.ec2.internal\" DevicePath \"\""
Sep 19 08:28:17.543718 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:28:17.543444    2578 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d6783502-5784-4288-93c8-e39e6c27a201-console-serving-cert\") on node \"ip-10-0-139-157.ec2.internal\" DevicePath \"\""
Sep 19 08:28:17.543718 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:28:17.543453    2578 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d6783502-5784-4288-93c8-e39e6c27a201-service-ca\") on node \"ip-10-0-139-157.ec2.internal\" DevicePath \"\""
Sep 19 08:28:17.543718 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:28:17.543461    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lg5x9\" (UniqueName: \"kubernetes.io/projected/d6783502-5784-4288-93c8-e39e6c27a201-kube-api-access-lg5x9\") on node \"ip-10-0-139-157.ec2.internal\" DevicePath \"\""
Sep 19 08:28:18.121669 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:28:18.121642    2578 generic.go:356] "Generic (PLEG): container finished" podID="d6783502-5784-4288-93c8-e39e6c27a201" containerID="aef69d11046d37433b04d75213470772cd8f7bc5a9d097be86ed47d3cef1d4d9" exitCode=0
Sep 19 08:28:18.121812 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:28:18.121688    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-65b4797958-6c5jr"
Sep 19 08:28:18.121812 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:28:18.121723    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-65b4797958-6c5jr" event={"ID":"d6783502-5784-4288-93c8-e39e6c27a201","Type":"ContainerDied","Data":"aef69d11046d37433b04d75213470772cd8f7bc5a9d097be86ed47d3cef1d4d9"}
Sep 19 08:28:18.121812 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:28:18.121783    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-65b4797958-6c5jr" event={"ID":"d6783502-5784-4288-93c8-e39e6c27a201","Type":"ContainerDied","Data":"c97840ec0a8bb4663784b20e3fc9a9032b50cdb68bd55457d45e1efba9b27762"}
Sep 19 08:28:18.121812 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:28:18.121807    2578 scope.go:117] "RemoveContainer" containerID="aef69d11046d37433b04d75213470772cd8f7bc5a9d097be86ed47d3cef1d4d9"
Sep 19 08:28:18.129932 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:28:18.129911    2578 scope.go:117] "RemoveContainer" containerID="aef69d11046d37433b04d75213470772cd8f7bc5a9d097be86ed47d3cef1d4d9"
Sep 19 08:28:18.130252 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:28:18.130234    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aef69d11046d37433b04d75213470772cd8f7bc5a9d097be86ed47d3cef1d4d9\": container with ID starting with aef69d11046d37433b04d75213470772cd8f7bc5a9d097be86ed47d3cef1d4d9 not found: ID does not exist" containerID="aef69d11046d37433b04d75213470772cd8f7bc5a9d097be86ed47d3cef1d4d9"
Sep 19 08:28:18.130318 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:28:18.130262    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aef69d11046d37433b04d75213470772cd8f7bc5a9d097be86ed47d3cef1d4d9"} err="failed to get container status \"aef69d11046d37433b04d75213470772cd8f7bc5a9d097be86ed47d3cef1d4d9\": rpc error: code = NotFound desc = could not find container \"aef69d11046d37433b04d75213470772cd8f7bc5a9d097be86ed47d3cef1d4d9\": container with ID starting with aef69d11046d37433b04d75213470772cd8f7bc5a9d097be86ed47d3cef1d4d9 not found: ID does not exist"
Sep 19 08:28:18.137333 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:28:18.137294    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-65b4797958-6c5jr"]
Sep 19 08:28:18.143982 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:28:18.143953    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-65b4797958-6c5jr"]
Sep 19 08:28:20.112696 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:28:20.112663    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d6783502-5784-4288-93c8-e39e6c27a201" path="/var/lib/kubelet/pods/d6783502-5784-4288-93c8-e39e6c27a201/volumes"
Sep 19 08:28:43.103265 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:28:43.103232    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/kserve-controller-manager-7dcf4bff6-l2zf7"
Sep 19 08:28:44.290362 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:28:44.290325    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:28:44.325701 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:28:44.325666    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/odh-model-controller-7b966d7984-zlnrt"]
Sep 19 08:28:44.326147 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:28:44.326132    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b640099-9239-4cc6-8962-4a08d9f6aad8" containerName="manager"
Sep 19 08:28:44.326213 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:28:44.326150    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b640099-9239-4cc6-8962-4a08d9f6aad8" containerName="manager"
Sep 19 08:28:44.326213 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:28:44.326184    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d6783502-5784-4288-93c8-e39e6c27a201" containerName="console"
Sep 19 08:28:44.326213 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:28:44.326193    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="d6783502-5784-4288-93c8-e39e6c27a201" containerName="console"
Sep 19 08:28:44.326376 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:28:44.326260    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="8b640099-9239-4cc6-8962-4a08d9f6aad8" containerName="manager"
Sep 19 08:28:44.326376 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:28:44.326271    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="d6783502-5784-4288-93c8-e39e6c27a201" containerName="console"
Sep 19 08:28:44.328464 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:28:44.328443    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/odh-model-controller-7b966d7984-zlnrt"
Sep 19 08:28:44.330963 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:28:44.330944    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"odh-model-controller-metrics-tls\""
Sep 19 08:28:44.331259 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:28:44.331234    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"odh-model-controller-dockercfg-lgfhd\""
Sep 19 08:28:44.331498 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:28:44.331484    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"odh-model-controller-webhook-cert\""
Sep 19 08:28:44.339208 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:28:44.339169    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/odh-model-controller-7b966d7984-zlnrt"]
Sep 19 08:28:44.386767 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:28:44.386722    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:28:44.448094 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:28:44.448051    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0063b08e-aa2c-40e1-8973-5dad4ca2935f-cert\") pod \"odh-model-controller-7b966d7984-zlnrt\" (UID: \"0063b08e-aa2c-40e1-8973-5dad4ca2935f\") " pod="kserve/odh-model-controller-7b966d7984-zlnrt"
Sep 19 08:28:44.448262 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:28:44.448096    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0063b08e-aa2c-40e1-8973-5dad4ca2935f-metrics-certs\") pod \"odh-model-controller-7b966d7984-zlnrt\" (UID: \"0063b08e-aa2c-40e1-8973-5dad4ca2935f\") " pod="kserve/odh-model-controller-7b966d7984-zlnrt"
Sep 19 08:28:44.448262 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:28:44.448131    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wqb2f\" (UniqueName: \"kubernetes.io/projected/0063b08e-aa2c-40e1-8973-5dad4ca2935f-kube-api-access-wqb2f\") pod \"odh-model-controller-7b966d7984-zlnrt\" (UID: \"0063b08e-aa2c-40e1-8973-5dad4ca2935f\") " pod="kserve/odh-model-controller-7b966d7984-zlnrt"
Sep 19 08:28:44.486948 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:28:44.486914    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:28:44.548847 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:28:44.548759    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0063b08e-aa2c-40e1-8973-5dad4ca2935f-cert\") pod \"odh-model-controller-7b966d7984-zlnrt\" (UID: \"0063b08e-aa2c-40e1-8973-5dad4ca2935f\") " pod="kserve/odh-model-controller-7b966d7984-zlnrt"
Sep 19 08:28:44.548847 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:28:44.548804    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0063b08e-aa2c-40e1-8973-5dad4ca2935f-metrics-certs\") pod \"odh-model-controller-7b966d7984-zlnrt\" (UID: \"0063b08e-aa2c-40e1-8973-5dad4ca2935f\") " pod="kserve/odh-model-controller-7b966d7984-zlnrt"
Sep 19 08:28:44.548847 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:28:44.548832    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wqb2f\" (UniqueName: \"kubernetes.io/projected/0063b08e-aa2c-40e1-8973-5dad4ca2935f-kube-api-access-wqb2f\") pod \"odh-model-controller-7b966d7984-zlnrt\" (UID: \"0063b08e-aa2c-40e1-8973-5dad4ca2935f\") " pod="kserve/odh-model-controller-7b966d7984-zlnrt"
Sep 19 08:28:44.551442 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:28:44.551417    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0063b08e-aa2c-40e1-8973-5dad4ca2935f-cert\") pod \"odh-model-controller-7b966d7984-zlnrt\" (UID: \"0063b08e-aa2c-40e1-8973-5dad4ca2935f\") " pod="kserve/odh-model-controller-7b966d7984-zlnrt"
Sep 19 08:28:44.551610 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:28:44.551586    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0063b08e-aa2c-40e1-8973-5dad4ca2935f-metrics-certs\") pod \"odh-model-controller-7b966d7984-zlnrt\" (UID: \"0063b08e-aa2c-40e1-8973-5dad4ca2935f\") " pod="kserve/odh-model-controller-7b966d7984-zlnrt"
Sep 19 08:28:44.556460 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:28:44.556433    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wqb2f\" (UniqueName: \"kubernetes.io/projected/0063b08e-aa2c-40e1-8973-5dad4ca2935f-kube-api-access-wqb2f\") pod \"odh-model-controller-7b966d7984-zlnrt\" (UID: \"0063b08e-aa2c-40e1-8973-5dad4ca2935f\") " pod="kserve/odh-model-controller-7b966d7984-zlnrt"
Sep 19 08:28:44.623255 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:28:44.623221    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:28:44.643025 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:28:44.642995    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/odh-model-controller-7b966d7984-zlnrt"
Sep 19 08:28:44.716235 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:28:44.716206    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:28:44.911208 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:28:44.911134    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/odh-model-controller-7b966d7984-zlnrt"]
Sep 19 08:28:44.914506 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:28:44.914478    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0063b08e_aa2c_40e1_8973_5dad4ca2935f.slice/crio-256529af9076509b8bc27766c729ffd030761c5550c3d68433e6c6ab926cb22d WatchSource:0}: Error finding container 256529af9076509b8bc27766c729ffd030761c5550c3d68433e6c6ab926cb22d: Status 404 returned error can't find the container with id 256529af9076509b8bc27766c729ffd030761c5550c3d68433e6c6ab926cb22d
Sep 19 08:28:45.242752 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:28:45.242662    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/odh-model-controller-7b966d7984-zlnrt" event={"ID":"0063b08e-aa2c-40e1-8973-5dad4ca2935f","Type":"ContainerStarted","Data":"256529af9076509b8bc27766c729ffd030761c5550c3d68433e6c6ab926cb22d"}
Sep 19 08:28:45.336457 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:28:45.336413    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:28:45.390672 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:28:45.390637    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:28:48.259063 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:28:48.259021    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/odh-model-controller-7b966d7984-zlnrt" event={"ID":"0063b08e-aa2c-40e1-8973-5dad4ca2935f","Type":"ContainerStarted","Data":"2059420dcb6953216fe7e24cdb1bf00e53596a51b1281245f3e574b08b921c08"}
Sep 19 08:28:48.259436 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:28:48.259162    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/odh-model-controller-7b966d7984-zlnrt"
Sep 19 08:28:48.278657 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:28:48.278609    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/odh-model-controller-7b966d7984-zlnrt" podStartSLOduration=1.819929346 podStartE2EDuration="4.278595779s" totalImagesPullingTime="2.458666433s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 08:28:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 08:28:48.277294052 +0000 UTC m=+512.654003916" watchObservedRunningTime="2026-09-19 08:28:48.278595779 +0000 UTC m=+512.655305641"
Sep 19 08:28:59.265198 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:28:59.265163    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/odh-model-controller-7b966d7984-zlnrt"
Sep 19 08:29:19.245898 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:29:19.245852    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:29:19.318149 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:29:19.318116    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:29:49.997701 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:29:49.997663    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:29:50.053012 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:29:50.052985    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:30:16.012171 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:30:16.012142    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zd2zp_07ddac45-12a1-42e8-9566-55f08854c02e/ovn-acl-logging/0.log"
Sep 19 08:30:16.012938 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:30:16.012917    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zd2zp_07ddac45-12a1-42e8-9566-55f08854c02e/ovn-acl-logging/0.log"
Sep 19 08:30:17.620551 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:30:17.620513    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:30:17.721409 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:30:17.721372    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:30:31.350125 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:30:31.350080    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:30:31.641588 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:30:31.641496    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:30:31.744265 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:30:31.744225    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:30:31.817469 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:30:31.817439    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:30:31.876405 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:30:31.876367    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-83f4f-predictor-68d8b4ddbb-8tztn"]
Sep 19 08:30:31.879761 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:30:31.879736    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-83f4f-predictor-68d8b4ddbb-8tztn"
Sep 19 08:30:31.880139 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:30:31.879760    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:30:31.882426 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:30:31.882398    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Sep 19 08:30:31.882559 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:30:31.882438    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-83f4f-kube-rbac-proxy-sar-config\""
Sep 19 08:30:31.882690 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:30:31.882668    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-zd5sl\""
Sep 19 08:30:31.883244 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:30:31.883225    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-83f4f-predictor-serving-cert\""
Sep 19 08:30:31.883343 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:30:31.883231    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Sep 19 08:30:31.889643 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:30:31.889621    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-83f4f-predictor-68d8b4ddbb-8tztn"]
Sep 19 08:30:31.954617 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:30:31.954521    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/013d2e8c-8a98-4f0c-808a-86e4d3ae639e-proxy-tls\") pod \"error-404-isvc-83f4f-predictor-68d8b4ddbb-8tztn\" (UID: \"013d2e8c-8a98-4f0c-808a-86e4d3ae639e\") " pod="kserve-ci-e2e-test/error-404-isvc-83f4f-predictor-68d8b4ddbb-8tztn"
Sep 19 08:30:31.954617 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:30:31.954590    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/013d2e8c-8a98-4f0c-808a-86e4d3ae639e-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-83f4f-predictor-68d8b4ddbb-8tztn\" (UID: \"013d2e8c-8a98-4f0c-808a-86e4d3ae639e\") " pod="kserve-ci-e2e-test/error-404-isvc-83f4f-predictor-68d8b4ddbb-8tztn"
Sep 19 08:30:31.954828 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:30:31.954620    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jcdw9\" (UniqueName: \"kubernetes.io/projected/013d2e8c-8a98-4f0c-808a-86e4d3ae639e-kube-api-access-jcdw9\") pod \"error-404-isvc-83f4f-predictor-68d8b4ddbb-8tztn\" (UID: \"013d2e8c-8a98-4f0c-808a-86e4d3ae639e\") " pod="kserve-ci-e2e-test/error-404-isvc-83f4f-predictor-68d8b4ddbb-8tztn"
Sep 19 08:30:32.007733 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:30:32.007675    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:30:32.055476 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:30:32.055446    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/013d2e8c-8a98-4f0c-808a-86e4d3ae639e-proxy-tls\") pod \"error-404-isvc-83f4f-predictor-68d8b4ddbb-8tztn\" (UID: \"013d2e8c-8a98-4f0c-808a-86e4d3ae639e\") " pod="kserve-ci-e2e-test/error-404-isvc-83f4f-predictor-68d8b4ddbb-8tztn"
Sep 19 08:30:32.055668 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:30:32.055506    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/013d2e8c-8a98-4f0c-808a-86e4d3ae639e-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-83f4f-predictor-68d8b4ddbb-8tztn\" (UID: \"013d2e8c-8a98-4f0c-808a-86e4d3ae639e\") " pod="kserve-ci-e2e-test/error-404-isvc-83f4f-predictor-68d8b4ddbb-8tztn"
Sep 19 08:30:32.055668 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:30:32.055537    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jcdw9\" (UniqueName: \"kubernetes.io/projected/013d2e8c-8a98-4f0c-808a-86e4d3ae639e-kube-api-access-jcdw9\") pod \"error-404-isvc-83f4f-predictor-68d8b4ddbb-8tztn\" (UID: \"013d2e8c-8a98-4f0c-808a-86e4d3ae639e\") " pod="kserve-ci-e2e-test/error-404-isvc-83f4f-predictor-68d8b4ddbb-8tztn"
Sep 19 08:30:32.056284 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:30:32.056257    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/013d2e8c-8a98-4f0c-808a-86e4d3ae639e-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-83f4f-predictor-68d8b4ddbb-8tztn\" (UID: \"013d2e8c-8a98-4f0c-808a-86e4d3ae639e\") " pod="kserve-ci-e2e-test/error-404-isvc-83f4f-predictor-68d8b4ddbb-8tztn"
Sep 19 08:30:32.058098 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:30:32.058063    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/013d2e8c-8a98-4f0c-808a-86e4d3ae639e-proxy-tls\") pod \"error-404-isvc-83f4f-predictor-68d8b4ddbb-8tztn\" (UID: \"013d2e8c-8a98-4f0c-808a-86e4d3ae639e\") " pod="kserve-ci-e2e-test/error-404-isvc-83f4f-predictor-68d8b4ddbb-8tztn"
Sep 19 08:30:32.063135 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:30:32.063105    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jcdw9\" (UniqueName: \"kubernetes.io/projected/013d2e8c-8a98-4f0c-808a-86e4d3ae639e-kube-api-access-jcdw9\") pod \"error-404-isvc-83f4f-predictor-68d8b4ddbb-8tztn\" (UID: \"013d2e8c-8a98-4f0c-808a-86e4d3ae639e\") " pod="kserve-ci-e2e-test/error-404-isvc-83f4f-predictor-68d8b4ddbb-8tztn"
Sep 19 08:30:32.066506 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:30:32.066481    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:30:32.194321 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:30:32.194268    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-83f4f-predictor-68d8b4ddbb-8tztn"
Sep 19 08:30:32.467276 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:30:32.467251    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-83f4f-predictor-68d8b4ddbb-8tztn"]
Sep 19 08:30:32.469055 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:30:32.469025    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod013d2e8c_8a98_4f0c_808a_86e4d3ae639e.slice/crio-6e04928e30b09b32b22049fdcd525a4e900cc95be07338c91209b20f97e325ef WatchSource:0}: Error finding container 6e04928e30b09b32b22049fdcd525a4e900cc95be07338c91209b20f97e325ef: Status 404 returned error can't find the container with id 6e04928e30b09b32b22049fdcd525a4e900cc95be07338c91209b20f97e325ef
Sep 19 08:30:32.704038 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:30:32.704003    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-83f4f-predictor-68d8b4ddbb-8tztn" event={"ID":"013d2e8c-8a98-4f0c-808a-86e4d3ae639e","Type":"ContainerStarted","Data":"6e04928e30b09b32b22049fdcd525a4e900cc95be07338c91209b20f97e325ef"}
Sep 19 08:30:44.475545 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:30:44.475526    2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 19 08:30:44.773292 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:30:44.773260    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-83f4f-predictor-68d8b4ddbb-8tztn" event={"ID":"013d2e8c-8a98-4f0c-808a-86e4d3ae639e","Type":"ContainerStarted","Data":"48002e38fe40f0214fdfc204e383321134beabefa1e8959238de1f3dd40eafab"}
Sep 19 08:30:46.785633 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:30:46.785603    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-83f4f-predictor-68d8b4ddbb-8tztn" event={"ID":"013d2e8c-8a98-4f0c-808a-86e4d3ae639e","Type":"ContainerStarted","Data":"bb964e6fab63a847a698ef5ac1c092a5905ea51b8f60922456a2d690716e0e7b"}
Sep 19 08:30:46.786044 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:30:46.785662    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-83f4f-predictor-68d8b4ddbb-8tztn"
Sep 19 08:30:46.786044 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:30:46.785709    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-83f4f-predictor-68d8b4ddbb-8tztn"
Sep 19 08:30:46.787009 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:30:46.786983    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-83f4f-predictor-68d8b4ddbb-8tztn" podUID="013d2e8c-8a98-4f0c-808a-86e4d3ae639e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.33:8080: connect: connection refused"
Sep 19 08:30:46.810910 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:30:46.810868    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-83f4f-predictor-68d8b4ddbb-8tztn" podStartSLOduration=1.684113332 podStartE2EDuration="15.81085711s" totalImagesPullingTime="14.126743778s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 08:30:31 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 08:30:46.810572805 +0000 UTC m=+631.187282668" watchObservedRunningTime="2026-09-19 08:30:46.81085711 +0000 UTC m=+631.187566971"
Sep 19 08:30:47.790331 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:30:47.790274    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-83f4f-predictor-68d8b4ddbb-8tztn" podUID="013d2e8c-8a98-4f0c-808a-86e4d3ae639e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.33:8080: connect: connection refused"
Sep 19 08:30:49.865026 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:30:49.864987    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:30:52.795589 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:30:52.795562    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-83f4f-predictor-68d8b4ddbb-8tztn"
Sep 19 08:30:52.796054 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:30:52.796026    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-83f4f-predictor-68d8b4ddbb-8tztn" podUID="013d2e8c-8a98-4f0c-808a-86e4d3ae639e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.33:8080: connect: connection refused"
Sep 19 08:31:02.797043 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:31:02.797001    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-83f4f-predictor-68d8b4ddbb-8tztn" podUID="013d2e8c-8a98-4f0c-808a-86e4d3ae639e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.33:8080: connect: connection refused"
Sep 19 08:31:12.796158 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:31:12.796117    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-83f4f-predictor-68d8b4ddbb-8tztn" podUID="013d2e8c-8a98-4f0c-808a-86e4d3ae639e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.33:8080: connect: connection refused"
Sep 19 08:31:22.796376 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:31:22.796336    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-83f4f-predictor-68d8b4ddbb-8tztn" podUID="013d2e8c-8a98-4f0c-808a-86e4d3ae639e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.33:8080: connect: connection refused"
Sep 19 08:31:32.796940 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:31:32.796895    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-83f4f-predictor-68d8b4ddbb-8tztn" podUID="013d2e8c-8a98-4f0c-808a-86e4d3ae639e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.33:8080: connect: connection refused"
Sep 19 08:31:42.797231 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:31:42.797193    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-83f4f-predictor-68d8b4ddbb-8tztn"
Sep 19 08:31:59.038927 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:31:59.038889    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:31:59.152176 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:31:59.152139    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:32:01.477935 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:32:01.477895    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:32:01.849446 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:32:01.849410    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:32:12.482072 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:32:12.481987    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:32:12.604054 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:32:12.604020    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:32:12.612210 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:32:12.612179    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-83f4f-predictor-68d8b4ddbb-8tztn"]
Sep 19 08:32:12.612472 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:32:12.612439    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-83f4f-predictor-68d8b4ddbb-8tztn" podUID="013d2e8c-8a98-4f0c-808a-86e4d3ae639e" containerName="kserve-container" containerID="cri-o://48002e38fe40f0214fdfc204e383321134beabefa1e8959238de1f3dd40eafab" gracePeriod=30
Sep 19 08:32:12.612641 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:32:12.612465    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-83f4f-predictor-68d8b4ddbb-8tztn" podUID="013d2e8c-8a98-4f0c-808a-86e4d3ae639e" containerName="kube-rbac-proxy" containerID="cri-o://bb964e6fab63a847a698ef5ac1c092a5905ea51b8f60922456a2d690716e0e7b" gracePeriod=30
Sep 19 08:32:12.701992 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:32:12.701954    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:32:12.788944 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:32:12.788905    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:32:12.790979 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:32:12.790927    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-83f4f-predictor-68d8b4ddbb-8tztn" podUID="013d2e8c-8a98-4f0c-808a-86e4d3ae639e" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.33:8643/healthz\": dial tcp 10.132.0.33:8643: connect: connection refused"
Sep 19 08:32:12.792784 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:32:12.792760    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-96102-predictor-75c8898456-mwf4g"]
Sep 19 08:32:12.795217 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:32:12.795196    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-96102-predictor-75c8898456-mwf4g"
Sep 19 08:32:12.796868 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:32:12.796840    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-83f4f-predictor-68d8b4ddbb-8tztn" podUID="013d2e8c-8a98-4f0c-808a-86e4d3ae639e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.33:8080: connect: connection refused"
Sep 19 08:32:12.797537 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:32:12.797516    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-96102-predictor-serving-cert\""
Sep 19 08:32:12.797836 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:32:12.797817    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-96102-kube-rbac-proxy-sar-config\""
Sep 19 08:32:12.807067 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:32:12.807038    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-96102-predictor-75c8898456-mwf4g"]
Sep 19 08:32:12.851250 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:32:12.851212    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:32:12.861509 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:32:12.861477    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/46054877-aba8-4c16-b75d-a8a04f2454b2-proxy-tls\") pod \"error-404-isvc-96102-predictor-75c8898456-mwf4g\" (UID: \"46054877-aba8-4c16-b75d-a8a04f2454b2\") " pod="kserve-ci-e2e-test/error-404-isvc-96102-predictor-75c8898456-mwf4g"
Sep 19 08:32:12.861677 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:32:12.861538    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-49gsb\" (UniqueName: \"kubernetes.io/projected/46054877-aba8-4c16-b75d-a8a04f2454b2-kube-api-access-49gsb\") pod \"error-404-isvc-96102-predictor-75c8898456-mwf4g\" (UID: \"46054877-aba8-4c16-b75d-a8a04f2454b2\") " pod="kserve-ci-e2e-test/error-404-isvc-96102-predictor-75c8898456-mwf4g"
Sep 19 08:32:12.861677 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:32:12.861591    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/46054877-aba8-4c16-b75d-a8a04f2454b2-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-96102-predictor-75c8898456-mwf4g\" (UID: \"46054877-aba8-4c16-b75d-a8a04f2454b2\") " pod="kserve-ci-e2e-test/error-404-isvc-96102-predictor-75c8898456-mwf4g"
Sep 19 08:32:12.962034 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:32:12.961995    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/46054877-aba8-4c16-b75d-a8a04f2454b2-proxy-tls\") pod \"error-404-isvc-96102-predictor-75c8898456-mwf4g\" (UID: \"46054877-aba8-4c16-b75d-a8a04f2454b2\") " pod="kserve-ci-e2e-test/error-404-isvc-96102-predictor-75c8898456-mwf4g"
Sep 19 08:32:12.962034 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:32:12.962047    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-49gsb\" (UniqueName: \"kubernetes.io/projected/46054877-aba8-4c16-b75d-a8a04f2454b2-kube-api-access-49gsb\") pod \"error-404-isvc-96102-predictor-75c8898456-mwf4g\" (UID: \"46054877-aba8-4c16-b75d-a8a04f2454b2\") " pod="kserve-ci-e2e-test/error-404-isvc-96102-predictor-75c8898456-mwf4g"
Sep 19 08:32:12.962346 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:32:12.962082    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/46054877-aba8-4c16-b75d-a8a04f2454b2-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-96102-predictor-75c8898456-mwf4g\" (UID: \"46054877-aba8-4c16-b75d-a8a04f2454b2\") " pod="kserve-ci-e2e-test/error-404-isvc-96102-predictor-75c8898456-mwf4g"
Sep 19 08:32:12.962800 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:32:12.962776    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/46054877-aba8-4c16-b75d-a8a04f2454b2-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-96102-predictor-75c8898456-mwf4g\" (UID: \"46054877-aba8-4c16-b75d-a8a04f2454b2\") " pod="kserve-ci-e2e-test/error-404-isvc-96102-predictor-75c8898456-mwf4g"
Sep 19 08:32:12.965034 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:32:12.965006    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/46054877-aba8-4c16-b75d-a8a04f2454b2-proxy-tls\") pod \"error-404-isvc-96102-predictor-75c8898456-mwf4g\" (UID: \"46054877-aba8-4c16-b75d-a8a04f2454b2\") " pod="kserve-ci-e2e-test/error-404-isvc-96102-predictor-75c8898456-mwf4g"
Sep 19 08:32:12.970189 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:32:12.970161    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-49gsb\" (UniqueName: \"kubernetes.io/projected/46054877-aba8-4c16-b75d-a8a04f2454b2-kube-api-access-49gsb\") pod \"error-404-isvc-96102-predictor-75c8898456-mwf4g\" (UID: \"46054877-aba8-4c16-b75d-a8a04f2454b2\") " pod="kserve-ci-e2e-test/error-404-isvc-96102-predictor-75c8898456-mwf4g"
Sep 19 08:32:13.001981 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:32:13.001941    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:32:13.105366 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:32:13.105261    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-96102-predictor-75c8898456-mwf4g"
Sep 19 08:32:13.136354 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:32:13.136323    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:32:13.156460 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:32:13.156327    2578 generic.go:356] "Generic (PLEG): container finished" podID="013d2e8c-8a98-4f0c-808a-86e4d3ae639e" containerID="bb964e6fab63a847a698ef5ac1c092a5905ea51b8f60922456a2d690716e0e7b" exitCode=2
Sep 19 08:32:13.156460 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:32:13.156445    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-83f4f-predictor-68d8b4ddbb-8tztn" event={"ID":"013d2e8c-8a98-4f0c-808a-86e4d3ae639e","Type":"ContainerDied","Data":"bb964e6fab63a847a698ef5ac1c092a5905ea51b8f60922456a2d690716e0e7b"}
Sep 19 08:32:13.380610 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:32:13.380584    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-96102-predictor-75c8898456-mwf4g"]
Sep 19 08:32:13.383267 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:32:13.383232    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46054877_aba8_4c16_b75d_a8a04f2454b2.slice/crio-5815896f6fab8dcaabc2a1ab7435c60ecba92a0aad2414010b557bae1354d08a WatchSource:0}: Error finding container 5815896f6fab8dcaabc2a1ab7435c60ecba92a0aad2414010b557bae1354d08a: Status 404 returned error can't find the container with id 5815896f6fab8dcaabc2a1ab7435c60ecba92a0aad2414010b557bae1354d08a
Sep 19 08:32:14.164256 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:32:14.164222    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-96102-predictor-75c8898456-mwf4g" event={"ID":"46054877-aba8-4c16-b75d-a8a04f2454b2","Type":"ContainerStarted","Data":"77f0ca9c6792abd2332e0150d1bcd8ba361db1962db1206a499a4e44f5242bed"}
Sep 19 08:32:14.164658 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:32:14.164284    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-96102-predictor-75c8898456-mwf4g"
Sep 19 08:32:14.164658 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:32:14.164337    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-96102-predictor-75c8898456-mwf4g" event={"ID":"46054877-aba8-4c16-b75d-a8a04f2454b2","Type":"ContainerStarted","Data":"6a3a1d30c6517caf846adeb7f7cd1cbf91762d1e5b0290299dd5daa49da9c902"}
Sep 19 08:32:14.164658 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:32:14.164359    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-96102-predictor-75c8898456-mwf4g" event={"ID":"46054877-aba8-4c16-b75d-a8a04f2454b2","Type":"ContainerStarted","Data":"5815896f6fab8dcaabc2a1ab7435c60ecba92a0aad2414010b557bae1354d08a"}
Sep 19 08:32:14.180590 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:32:14.180532    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-96102-predictor-75c8898456-mwf4g" podStartSLOduration=2.180516279 podStartE2EDuration="2.180516279s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 08:32:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 08:32:14.179705852 +0000 UTC m=+718.556415718" watchObservedRunningTime="2026-09-19 08:32:14.180516279 +0000 UTC m=+718.557226141"
Sep 19 08:32:15.168187 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:32:15.168149    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-96102-predictor-75c8898456-mwf4g"
Sep 19 08:32:15.169681 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:32:15.169649    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-96102-predictor-75c8898456-mwf4g" podUID="46054877-aba8-4c16-b75d-a8a04f2454b2" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.34:8080: connect: connection refused"
Sep 19 08:32:16.172640 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:32:16.172595    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-96102-predictor-75c8898456-mwf4g" podUID="46054877-aba8-4c16-b75d-a8a04f2454b2" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.34:8080: connect: connection refused"
Sep 19 08:32:16.462249 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:32:16.462225    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-83f4f-predictor-68d8b4ddbb-8tztn"
Sep 19 08:32:16.487603 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:32:16.487573    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/013d2e8c-8a98-4f0c-808a-86e4d3ae639e-proxy-tls\") pod \"013d2e8c-8a98-4f0c-808a-86e4d3ae639e\" (UID: \"013d2e8c-8a98-4f0c-808a-86e4d3ae639e\") "
Sep 19 08:32:16.487763 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:32:16.487614    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jcdw9\" (UniqueName: \"kubernetes.io/projected/013d2e8c-8a98-4f0c-808a-86e4d3ae639e-kube-api-access-jcdw9\") pod \"013d2e8c-8a98-4f0c-808a-86e4d3ae639e\" (UID: \"013d2e8c-8a98-4f0c-808a-86e4d3ae639e\") "
Sep 19 08:32:16.487763 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:32:16.487664    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/013d2e8c-8a98-4f0c-808a-86e4d3ae639e-kube-rbac-proxy-sar-config\") pod \"013d2e8c-8a98-4f0c-808a-86e4d3ae639e\" (UID: \"013d2e8c-8a98-4f0c-808a-86e4d3ae639e\") "
Sep 19 08:32:16.488049 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:32:16.488026    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/013d2e8c-8a98-4f0c-808a-86e4d3ae639e-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "013d2e8c-8a98-4f0c-808a-86e4d3ae639e" (UID: "013d2e8c-8a98-4f0c-808a-86e4d3ae639e"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 08:32:16.489695 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:32:16.489666    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/013d2e8c-8a98-4f0c-808a-86e4d3ae639e-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "013d2e8c-8a98-4f0c-808a-86e4d3ae639e" (UID: "013d2e8c-8a98-4f0c-808a-86e4d3ae639e"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 08:32:16.489806 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:32:16.489780    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/013d2e8c-8a98-4f0c-808a-86e4d3ae639e-kube-api-access-jcdw9" (OuterVolumeSpecName: "kube-api-access-jcdw9") pod "013d2e8c-8a98-4f0c-808a-86e4d3ae639e" (UID: "013d2e8c-8a98-4f0c-808a-86e4d3ae639e"). InnerVolumeSpecName "kube-api-access-jcdw9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 08:32:16.588819 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:32:16.588784    2578 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/013d2e8c-8a98-4f0c-808a-86e4d3ae639e-proxy-tls\") on node \"ip-10-0-139-157.ec2.internal\" DevicePath \"\""
Sep 19 08:32:16.588819 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:32:16.588815    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jcdw9\" (UniqueName: \"kubernetes.io/projected/013d2e8c-8a98-4f0c-808a-86e4d3ae639e-kube-api-access-jcdw9\") on node \"ip-10-0-139-157.ec2.internal\" DevicePath \"\""
Sep 19 08:32:16.588819 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:32:16.588826    2578 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/013d2e8c-8a98-4f0c-808a-86e4d3ae639e-kube-rbac-proxy-sar-config\") on node \"ip-10-0-139-157.ec2.internal\" DevicePath \"\""
Sep 19 08:32:17.178038 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:32:17.178005    2578 generic.go:356] "Generic (PLEG): container finished" podID="013d2e8c-8a98-4f0c-808a-86e4d3ae639e" containerID="48002e38fe40f0214fdfc204e383321134beabefa1e8959238de1f3dd40eafab" exitCode=0
Sep 19 08:32:17.178479 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:32:17.178069    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-83f4f-predictor-68d8b4ddbb-8tztn"
Sep 19 08:32:17.178479 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:32:17.178096    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-83f4f-predictor-68d8b4ddbb-8tztn" event={"ID":"013d2e8c-8a98-4f0c-808a-86e4d3ae639e","Type":"ContainerDied","Data":"48002e38fe40f0214fdfc204e383321134beabefa1e8959238de1f3dd40eafab"}
Sep 19 08:32:17.178479 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:32:17.178158    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-83f4f-predictor-68d8b4ddbb-8tztn" event={"ID":"013d2e8c-8a98-4f0c-808a-86e4d3ae639e","Type":"ContainerDied","Data":"6e04928e30b09b32b22049fdcd525a4e900cc95be07338c91209b20f97e325ef"}
Sep 19 08:32:17.178479 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:32:17.178188    2578 scope.go:117] "RemoveContainer" containerID="bb964e6fab63a847a698ef5ac1c092a5905ea51b8f60922456a2d690716e0e7b"
Sep 19 08:32:17.187681 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:32:17.187663    2578 scope.go:117] "RemoveContainer" containerID="48002e38fe40f0214fdfc204e383321134beabefa1e8959238de1f3dd40eafab"
Sep 19 08:32:17.195293 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:32:17.195276    2578 scope.go:117] "RemoveContainer" containerID="bb964e6fab63a847a698ef5ac1c092a5905ea51b8f60922456a2d690716e0e7b"
Sep 19 08:32:17.195577 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:32:17.195556    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bb964e6fab63a847a698ef5ac1c092a5905ea51b8f60922456a2d690716e0e7b\": container with ID starting with bb964e6fab63a847a698ef5ac1c092a5905ea51b8f60922456a2d690716e0e7b not found: ID does not exist" containerID="bb964e6fab63a847a698ef5ac1c092a5905ea51b8f60922456a2d690716e0e7b"
Sep 19 08:32:17.195615 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:32:17.195587    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bb964e6fab63a847a698ef5ac1c092a5905ea51b8f60922456a2d690716e0e7b"} err="failed to get container status \"bb964e6fab63a847a698ef5ac1c092a5905ea51b8f60922456a2d690716e0e7b\": rpc error: code = NotFound desc = could not find container \"bb964e6fab63a847a698ef5ac1c092a5905ea51b8f60922456a2d690716e0e7b\": container with ID starting with bb964e6fab63a847a698ef5ac1c092a5905ea51b8f60922456a2d690716e0e7b not found: ID does not exist"
Sep 19 08:32:17.195615 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:32:17.195606    2578 scope.go:117] "RemoveContainer" containerID="48002e38fe40f0214fdfc204e383321134beabefa1e8959238de1f3dd40eafab"
Sep 19 08:32:17.195852 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:32:17.195836    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"48002e38fe40f0214fdfc204e383321134beabefa1e8959238de1f3dd40eafab\": container with ID starting with 48002e38fe40f0214fdfc204e383321134beabefa1e8959238de1f3dd40eafab not found: ID does not exist" containerID="48002e38fe40f0214fdfc204e383321134beabefa1e8959238de1f3dd40eafab"
Sep 19 08:32:17.195887 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:32:17.195859    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"48002e38fe40f0214fdfc204e383321134beabefa1e8959238de1f3dd40eafab"} err="failed to get container status \"48002e38fe40f0214fdfc204e383321134beabefa1e8959238de1f3dd40eafab\": rpc error: code = NotFound desc = could not find container \"48002e38fe40f0214fdfc204e383321134beabefa1e8959238de1f3dd40eafab\": container with ID starting with 48002e38fe40f0214fdfc204e383321134beabefa1e8959238de1f3dd40eafab not found: ID does not exist"
Sep 19 08:32:17.200048 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:32:17.200025    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-83f4f-predictor-68d8b4ddbb-8tztn"]
Sep 19 08:32:17.203904 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:32:17.203881    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-83f4f-predictor-68d8b4ddbb-8tztn"]
Sep 19 08:32:18.113574 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:32:18.113533    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="013d2e8c-8a98-4f0c-808a-86e4d3ae639e" path="/var/lib/kubelet/pods/013d2e8c-8a98-4f0c-808a-86e4d3ae639e/volumes"
Sep 19 08:32:21.176968 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:32:21.176936    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-96102-predictor-75c8898456-mwf4g"
Sep 19 08:32:21.177449 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:32:21.177423    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-96102-predictor-75c8898456-mwf4g" podUID="46054877-aba8-4c16-b75d-a8a04f2454b2" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.34:8080: connect: connection refused"
Sep 19 08:32:31.177897 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:32:31.177857    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-96102-predictor-75c8898456-mwf4g" podUID="46054877-aba8-4c16-b75d-a8a04f2454b2" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.34:8080: connect: connection refused"
Sep 19 08:32:33.591136 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:32:33.591099    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:32:33.652357 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:32:33.652322    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:32:41.177514 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:32:41.177473    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-96102-predictor-75c8898456-mwf4g" podUID="46054877-aba8-4c16-b75d-a8a04f2454b2" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.34:8080: connect: connection refused"
Sep 19 08:32:51.177405 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:32:51.177365    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-96102-predictor-75c8898456-mwf4g" podUID="46054877-aba8-4c16-b75d-a8a04f2454b2" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.34:8080: connect: connection refused"
Sep 19 08:32:51.451821 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:32:51.451732    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:32:52.106212 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:32:52.106169    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:32:52.205815 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:32:52.205778    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:33:01.177644 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:33:01.177607    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-96102-predictor-75c8898456-mwf4g" podUID="46054877-aba8-4c16-b75d-a8a04f2454b2" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.34:8080: connect: connection refused"
Sep 19 08:33:03.003251 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:33:03.003206    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:33:03.112949 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:33:03.112907    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:33:03.245699 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:33:03.245660    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:33:03.311570 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:33:03.311530    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:33:03.391745 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:33:03.391710    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:33:03.409888 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:33:03.409850    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-5cf0a-predictor-79d6c67446-q5bcc"]
Sep 19 08:33:03.410180 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:33:03.410169    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="013d2e8c-8a98-4f0c-808a-86e4d3ae639e" containerName="kube-rbac-proxy"
Sep 19 08:33:03.410223 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:33:03.410182    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="013d2e8c-8a98-4f0c-808a-86e4d3ae639e" containerName="kube-rbac-proxy"
Sep 19 08:33:03.410223 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:33:03.410198    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="013d2e8c-8a98-4f0c-808a-86e4d3ae639e" containerName="kserve-container"
Sep 19 08:33:03.410223 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:33:03.410203    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="013d2e8c-8a98-4f0c-808a-86e4d3ae639e" containerName="kserve-container"
Sep 19 08:33:03.410328 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:33:03.410259    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="013d2e8c-8a98-4f0c-808a-86e4d3ae639e" containerName="kube-rbac-proxy"
Sep 19 08:33:03.410328 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:33:03.410270    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="013d2e8c-8a98-4f0c-808a-86e4d3ae639e" containerName="kserve-container"
Sep 19 08:33:03.412844 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:33:03.412824    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-5cf0a-predictor-79d6c67446-q5bcc"
Sep 19 08:33:03.415506 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:33:03.415482    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-5cf0a-kube-rbac-proxy-sar-config\""
Sep 19 08:33:03.415647 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:33:03.415566    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-5cf0a-predictor-serving-cert\""
Sep 19 08:33:03.424463 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:33:03.424438    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-5cf0a-predictor-79d6c67446-q5bcc"]
Sep 19 08:33:03.457774 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:33:03.457727    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:33:03.522101 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:33:03.522068    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:33:03.566167 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:33:03.566079    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vfpbm\" (UniqueName: \"kubernetes.io/projected/b4de8d11-747c-4643-a67f-30f4f0e7bfe1-kube-api-access-vfpbm\") pod \"error-404-isvc-5cf0a-predictor-79d6c67446-q5bcc\" (UID: \"b4de8d11-747c-4643-a67f-30f4f0e7bfe1\") " pod="kserve-ci-e2e-test/error-404-isvc-5cf0a-predictor-79d6c67446-q5bcc"
Sep 19 08:33:03.566341 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:33:03.566170    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b4de8d11-747c-4643-a67f-30f4f0e7bfe1-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-5cf0a-predictor-79d6c67446-q5bcc\" (UID: \"b4de8d11-747c-4643-a67f-30f4f0e7bfe1\") " pod="kserve-ci-e2e-test/error-404-isvc-5cf0a-predictor-79d6c67446-q5bcc"
Sep 19 08:33:03.566341 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:33:03.566205    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b4de8d11-747c-4643-a67f-30f4f0e7bfe1-proxy-tls\") pod \"error-404-isvc-5cf0a-predictor-79d6c67446-q5bcc\" (UID: \"b4de8d11-747c-4643-a67f-30f4f0e7bfe1\") " pod="kserve-ci-e2e-test/error-404-isvc-5cf0a-predictor-79d6c67446-q5bcc"
Sep 19 08:33:03.667175 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:33:03.667133    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vfpbm\" (UniqueName: \"kubernetes.io/projected/b4de8d11-747c-4643-a67f-30f4f0e7bfe1-kube-api-access-vfpbm\") pod \"error-404-isvc-5cf0a-predictor-79d6c67446-q5bcc\" (UID: \"b4de8d11-747c-4643-a67f-30f4f0e7bfe1\") " pod="kserve-ci-e2e-test/error-404-isvc-5cf0a-predictor-79d6c67446-q5bcc"
Sep 19 08:33:03.667372 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:33:03.667208    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b4de8d11-747c-4643-a67f-30f4f0e7bfe1-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-5cf0a-predictor-79d6c67446-q5bcc\" (UID: \"b4de8d11-747c-4643-a67f-30f4f0e7bfe1\") " pod="kserve-ci-e2e-test/error-404-isvc-5cf0a-predictor-79d6c67446-q5bcc"
Sep 19 08:33:03.667372 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:33:03.667239    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b4de8d11-747c-4643-a67f-30f4f0e7bfe1-proxy-tls\") pod \"error-404-isvc-5cf0a-predictor-79d6c67446-q5bcc\" (UID: \"b4de8d11-747c-4643-a67f-30f4f0e7bfe1\") " pod="kserve-ci-e2e-test/error-404-isvc-5cf0a-predictor-79d6c67446-q5bcc"
Sep 19 08:33:03.667890 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:33:03.667870    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b4de8d11-747c-4643-a67f-30f4f0e7bfe1-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-5cf0a-predictor-79d6c67446-q5bcc\" (UID: \"b4de8d11-747c-4643-a67f-30f4f0e7bfe1\") " pod="kserve-ci-e2e-test/error-404-isvc-5cf0a-predictor-79d6c67446-q5bcc"
Sep 19 08:33:03.669938 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:33:03.669917    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b4de8d11-747c-4643-a67f-30f4f0e7bfe1-proxy-tls\") pod \"error-404-isvc-5cf0a-predictor-79d6c67446-q5bcc\" (UID: \"b4de8d11-747c-4643-a67f-30f4f0e7bfe1\") " pod="kserve-ci-e2e-test/error-404-isvc-5cf0a-predictor-79d6c67446-q5bcc"
Sep 19 08:33:03.675447 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:33:03.675427    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vfpbm\" (UniqueName: \"kubernetes.io/projected/b4de8d11-747c-4643-a67f-30f4f0e7bfe1-kube-api-access-vfpbm\") pod \"error-404-isvc-5cf0a-predictor-79d6c67446-q5bcc\" (UID: \"b4de8d11-747c-4643-a67f-30f4f0e7bfe1\") " pod="kserve-ci-e2e-test/error-404-isvc-5cf0a-predictor-79d6c67446-q5bcc"
Sep 19 08:33:03.723524 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:33:03.723489    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-5cf0a-predictor-79d6c67446-q5bcc"
Sep 19 08:33:03.996114 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:33:03.996087    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-5cf0a-predictor-79d6c67446-q5bcc"]
Sep 19 08:33:03.998570 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:33:03.998542    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4de8d11_747c_4643_a67f_30f4f0e7bfe1.slice/crio-ab28d181ddf19b2845c314205910b859a25486b6bd796fb2b9b411d280fe18c9 WatchSource:0}: Error finding container ab28d181ddf19b2845c314205910b859a25486b6bd796fb2b9b411d280fe18c9: Status 404 returned error can't find the container with id ab28d181ddf19b2845c314205910b859a25486b6bd796fb2b9b411d280fe18c9
Sep 19 08:33:04.387254 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:33:04.387214    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-5cf0a-predictor-79d6c67446-q5bcc" event={"ID":"b4de8d11-747c-4643-a67f-30f4f0e7bfe1","Type":"ContainerStarted","Data":"d32fde0c75c4e7ef72512a07ba7c40f740e95d92df3ea32774df3893f67334db"}
Sep 19 08:33:04.387755 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:33:04.387284    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-5cf0a-predictor-79d6c67446-q5bcc" event={"ID":"b4de8d11-747c-4643-a67f-30f4f0e7bfe1","Type":"ContainerStarted","Data":"a836ee23a2b811671e4ab6a2dd0d74672afe68c356fbb4ccd79062c84c8e4a79"}
Sep 19 08:33:04.387755 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:33:04.387340    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-5cf0a-predictor-79d6c67446-q5bcc" event={"ID":"b4de8d11-747c-4643-a67f-30f4f0e7bfe1","Type":"ContainerStarted","Data":"ab28d181ddf19b2845c314205910b859a25486b6bd796fb2b9b411d280fe18c9"}
Sep 19 08:33:04.387755 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:33:04.387377    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-5cf0a-predictor-79d6c67446-q5bcc"
Sep 19 08:33:04.405174 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:33:04.405122    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-5cf0a-predictor-79d6c67446-q5bcc" podStartSLOduration=1.405105602 podStartE2EDuration="1.405105602s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 08:33:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 08:33:04.402998408 +0000 UTC m=+768.779708281" watchObservedRunningTime="2026-09-19 08:33:04.405105602 +0000 UTC m=+768.781815463"
Sep 19 08:33:05.392719 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:33:05.392684    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-5cf0a-predictor-79d6c67446-q5bcc"
Sep 19 08:33:05.394007 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:33:05.393975    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-5cf0a-predictor-79d6c67446-q5bcc" podUID="b4de8d11-747c-4643-a67f-30f4f0e7bfe1" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.35:8080: connect: connection refused"
Sep 19 08:33:06.397217 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:33:06.397177    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-5cf0a-predictor-79d6c67446-q5bcc" podUID="b4de8d11-747c-4643-a67f-30f4f0e7bfe1" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.35:8080: connect: connection refused"
Sep 19 08:33:11.178284 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:33:11.178254    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-96102-predictor-75c8898456-mwf4g"
Sep 19 08:33:11.402326 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:33:11.402284    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-5cf0a-predictor-79d6c67446-q5bcc"
Sep 19 08:33:11.402935 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:33:11.402904    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-5cf0a-predictor-79d6c67446-q5bcc" podUID="b4de8d11-747c-4643-a67f-30f4f0e7bfe1" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.35:8080: connect: connection refused"
Sep 19 08:33:21.403333 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:33:21.403271    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-5cf0a-predictor-79d6c67446-q5bcc" podUID="b4de8d11-747c-4643-a67f-30f4f0e7bfe1" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.35:8080: connect: connection refused"
Sep 19 08:33:22.687508 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:33:22.687473    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:33:22.857408 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:33:22.857369    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:33:31.403356 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:33:31.403312    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-5cf0a-predictor-79d6c67446-q5bcc" podUID="b4de8d11-747c-4643-a67f-30f4f0e7bfe1" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.35:8080: connect: connection refused"
Sep 19 08:33:41.403329 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:33:41.403214    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-5cf0a-predictor-79d6c67446-q5bcc" podUID="b4de8d11-747c-4643-a67f-30f4f0e7bfe1" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.35:8080: connect: connection refused"
Sep 19 08:33:51.403923 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:33:51.403878    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-5cf0a-predictor-79d6c67446-q5bcc" podUID="b4de8d11-747c-4643-a67f-30f4f0e7bfe1" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.35:8080: connect: connection refused"
Sep 19 08:34:01.404384 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:34:01.404350    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-5cf0a-predictor-79d6c67446-q5bcc"
Sep 19 08:34:13.185567 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:34:13.185522    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:34:13.482926 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:34:13.482842    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:35:16.039590 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:35:16.039510    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zd2zp_07ddac45-12a1-42e8-9566-55f08854c02e/ovn-acl-logging/0.log"
Sep 19 08:35:16.040079 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:35:16.039595    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zd2zp_07ddac45-12a1-42e8-9566-55f08854c02e/ovn-acl-logging/0.log"
Sep 19 08:37:25.388542 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:37:25.388502    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:40:16.065973 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:40:16.065944    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zd2zp_07ddac45-12a1-42e8-9566-55f08854c02e/ovn-acl-logging/0.log"
Sep 19 08:40:16.067530 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:40:16.067505    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zd2zp_07ddac45-12a1-42e8-9566-55f08854c02e/ovn-acl-logging/0.log"
Sep 19 08:40:50.093942 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:40:50.093903    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:41:37.924579 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:41:37.924501    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:41:38.052823 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:41:38.052790    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:41:38.127569 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:41:38.127533    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-96102-predictor-75c8898456-mwf4g"]
Sep 19 08:41:38.127774 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:41:38.127753    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-96102-predictor-75c8898456-mwf4g" podUID="46054877-aba8-4c16-b75d-a8a04f2454b2" containerName="kserve-container" containerID="cri-o://6a3a1d30c6517caf846adeb7f7cd1cbf91762d1e5b0290299dd5daa49da9c902" gracePeriod=30
Sep 19 08:41:38.127838 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:41:38.127775    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-96102-predictor-75c8898456-mwf4g" podUID="46054877-aba8-4c16-b75d-a8a04f2454b2" containerName="kube-rbac-proxy" containerID="cri-o://77f0ca9c6792abd2332e0150d1bcd8ba361db1962db1206a499a4e44f5242bed" gracePeriod=30
Sep 19 08:41:38.131960 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:41:38.131934    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:41:38.191923 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:41:38.191848    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-88665-predictor-7c9cfdb5d6-xd4jw"]
Sep 19 08:41:38.195354 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:41:38.195333    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-88665-predictor-7c9cfdb5d6-xd4jw"
Sep 19 08:41:38.196978 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:41:38.196953    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:41:38.197772 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:41:38.197752    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-88665-predictor-serving-cert\""
Sep 19 08:41:38.197865 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:41:38.197755    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-88665-kube-rbac-proxy-sar-config\""
Sep 19 08:41:38.204932 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:41:38.204908    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-88665-predictor-7c9cfdb5d6-xd4jw"]
Sep 19 08:41:38.299863 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:41:38.299831    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:41:38.302963 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:41:38.302940    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/22a6f174-5523-4416-a8f3-4ee515d0f1d9-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-88665-predictor-7c9cfdb5d6-xd4jw\" (UID: \"22a6f174-5523-4416-a8f3-4ee515d0f1d9\") " pod="kserve-ci-e2e-test/error-404-isvc-88665-predictor-7c9cfdb5d6-xd4jw"
Sep 19 08:41:38.303096 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:41:38.302974    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/22a6f174-5523-4416-a8f3-4ee515d0f1d9-proxy-tls\") pod \"error-404-isvc-88665-predictor-7c9cfdb5d6-xd4jw\" (UID: \"22a6f174-5523-4416-a8f3-4ee515d0f1d9\") " pod="kserve-ci-e2e-test/error-404-isvc-88665-predictor-7c9cfdb5d6-xd4jw"
Sep 19 08:41:38.303096 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:41:38.302994    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qrf2n\" (UniqueName: \"kubernetes.io/projected/22a6f174-5523-4416-a8f3-4ee515d0f1d9-kube-api-access-qrf2n\") pod \"error-404-isvc-88665-predictor-7c9cfdb5d6-xd4jw\" (UID: \"22a6f174-5523-4416-a8f3-4ee515d0f1d9\") " pod="kserve-ci-e2e-test/error-404-isvc-88665-predictor-7c9cfdb5d6-xd4jw"
Sep 19 08:41:38.404395 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:41:38.404361    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/22a6f174-5523-4416-a8f3-4ee515d0f1d9-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-88665-predictor-7c9cfdb5d6-xd4jw\" (UID: \"22a6f174-5523-4416-a8f3-4ee515d0f1d9\") " pod="kserve-ci-e2e-test/error-404-isvc-88665-predictor-7c9cfdb5d6-xd4jw"
Sep 19 08:41:38.404395 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:41:38.404402    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/22a6f174-5523-4416-a8f3-4ee515d0f1d9-proxy-tls\") pod \"error-404-isvc-88665-predictor-7c9cfdb5d6-xd4jw\" (UID: \"22a6f174-5523-4416-a8f3-4ee515d0f1d9\") " pod="kserve-ci-e2e-test/error-404-isvc-88665-predictor-7c9cfdb5d6-xd4jw"
Sep 19 08:41:38.404639 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:41:38.404427    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qrf2n\" (UniqueName: \"kubernetes.io/projected/22a6f174-5523-4416-a8f3-4ee515d0f1d9-kube-api-access-qrf2n\") pod \"error-404-isvc-88665-predictor-7c9cfdb5d6-xd4jw\" (UID: \"22a6f174-5523-4416-a8f3-4ee515d0f1d9\") " pod="kserve-ci-e2e-test/error-404-isvc-88665-predictor-7c9cfdb5d6-xd4jw"
Sep 19 08:41:38.405209 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:41:38.405179    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/22a6f174-5523-4416-a8f3-4ee515d0f1d9-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-88665-predictor-7c9cfdb5d6-xd4jw\" (UID: \"22a6f174-5523-4416-a8f3-4ee515d0f1d9\") " pod="kserve-ci-e2e-test/error-404-isvc-88665-predictor-7c9cfdb5d6-xd4jw"
Sep 19 08:41:38.406998 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:41:38.406974    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/22a6f174-5523-4416-a8f3-4ee515d0f1d9-proxy-tls\") pod \"error-404-isvc-88665-predictor-7c9cfdb5d6-xd4jw\" (UID: \"22a6f174-5523-4416-a8f3-4ee515d0f1d9\") " pod="kserve-ci-e2e-test/error-404-isvc-88665-predictor-7c9cfdb5d6-xd4jw"
Sep 19 08:41:38.412714 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:41:38.412683    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qrf2n\" (UniqueName: \"kubernetes.io/projected/22a6f174-5523-4416-a8f3-4ee515d0f1d9-kube-api-access-qrf2n\") pod \"error-404-isvc-88665-predictor-7c9cfdb5d6-xd4jw\" (UID: \"22a6f174-5523-4416-a8f3-4ee515d0f1d9\") " pod="kserve-ci-e2e-test/error-404-isvc-88665-predictor-7c9cfdb5d6-xd4jw"
Sep 19 08:41:38.506569 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:41:38.506484    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-88665-predictor-7c9cfdb5d6-xd4jw"
Sep 19 08:41:38.656201 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:41:38.656163    2578 generic.go:356] "Generic (PLEG): container finished" podID="46054877-aba8-4c16-b75d-a8a04f2454b2" containerID="77f0ca9c6792abd2332e0150d1bcd8ba361db1962db1206a499a4e44f5242bed" exitCode=2
Sep 19 08:41:38.656395 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:41:38.656236    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-96102-predictor-75c8898456-mwf4g" event={"ID":"46054877-aba8-4c16-b75d-a8a04f2454b2","Type":"ContainerDied","Data":"77f0ca9c6792abd2332e0150d1bcd8ba361db1962db1206a499a4e44f5242bed"}
Sep 19 08:41:38.777764 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:41:38.777741    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-88665-predictor-7c9cfdb5d6-xd4jw"]
Sep 19 08:41:38.779624 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:41:38.779598    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22a6f174_5523_4416_a8f3_4ee515d0f1d9.slice/crio-c51b366177bc1a825e63b1e33fefac4a5f1fda50a612479245dee77ebfaf6fe5 WatchSource:0}: Error finding container c51b366177bc1a825e63b1e33fefac4a5f1fda50a612479245dee77ebfaf6fe5: Status 404 returned error can't find the container with id c51b366177bc1a825e63b1e33fefac4a5f1fda50a612479245dee77ebfaf6fe5
Sep 19 08:41:38.930805 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:41:38.930776    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:41:39.663284 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:41:39.663248    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-88665-predictor-7c9cfdb5d6-xd4jw" event={"ID":"22a6f174-5523-4416-a8f3-4ee515d0f1d9","Type":"ContainerStarted","Data":"f63a91a8bcb14db7e077a7c4c03b94f7bea3daf3837a06877ea0015fd4c7b368"}
Sep 19 08:41:39.663471 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:41:39.663317    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-88665-predictor-7c9cfdb5d6-xd4jw" event={"ID":"22a6f174-5523-4416-a8f3-4ee515d0f1d9","Type":"ContainerStarted","Data":"4d20d228fb738727a56421b5d672e84bc78faf2d07e274b385173ed8544419b0"}
Sep 19 08:41:39.663471 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:41:39.663339    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-88665-predictor-7c9cfdb5d6-xd4jw" event={"ID":"22a6f174-5523-4416-a8f3-4ee515d0f1d9","Type":"ContainerStarted","Data":"c51b366177bc1a825e63b1e33fefac4a5f1fda50a612479245dee77ebfaf6fe5"}
Sep 19 08:41:39.663471 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:41:39.663436    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-88665-predictor-7c9cfdb5d6-xd4jw"
Sep 19 08:41:39.682075 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:41:39.682026    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-88665-predictor-7c9cfdb5d6-xd4jw" podStartSLOduration=1.682006043 podStartE2EDuration="1.682006043s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 08:41:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 08:41:39.68195359 +0000 UTC m=+1284.058663465" watchObservedRunningTime="2026-09-19 08:41:39.682006043 +0000 UTC m=+1284.058715906"
Sep 19 08:41:40.667763 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:41:40.667732    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-88665-predictor-7c9cfdb5d6-xd4jw"
Sep 19 08:41:40.669018 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:41:40.668980    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-88665-predictor-7c9cfdb5d6-xd4jw" podUID="22a6f174-5523-4416-a8f3-4ee515d0f1d9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.36:8080: connect: connection refused"
Sep 19 08:41:41.173576 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:41:41.173535    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-96102-predictor-75c8898456-mwf4g" podUID="46054877-aba8-4c16-b75d-a8a04f2454b2" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.34:8643/healthz\": dial tcp 10.132.0.34:8643: connect: connection refused"
Sep 19 08:41:41.177680 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:41:41.177653    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-96102-predictor-75c8898456-mwf4g" podUID="46054877-aba8-4c16-b75d-a8a04f2454b2" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.34:8080: connect: connection refused"
Sep 19 08:41:41.675502 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:41:41.675461    2578 generic.go:356] "Generic (PLEG): container finished" podID="46054877-aba8-4c16-b75d-a8a04f2454b2" containerID="6a3a1d30c6517caf846adeb7f7cd1cbf91762d1e5b0290299dd5daa49da9c902" exitCode=0
Sep 19 08:41:41.675990 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:41:41.675908    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-96102-predictor-75c8898456-mwf4g" event={"ID":"46054877-aba8-4c16-b75d-a8a04f2454b2","Type":"ContainerDied","Data":"6a3a1d30c6517caf846adeb7f7cd1cbf91762d1e5b0290299dd5daa49da9c902"}
Sep 19 08:41:41.676545 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:41:41.676519    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-88665-predictor-7c9cfdb5d6-xd4jw" podUID="22a6f174-5523-4416-a8f3-4ee515d0f1d9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.36:8080: connect: connection refused"
Sep 19 08:41:41.784074 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:41:41.784052    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-96102-predictor-75c8898456-mwf4g"
Sep 19 08:41:41.932238 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:41:41.932144    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/46054877-aba8-4c16-b75d-a8a04f2454b2-kube-rbac-proxy-sar-config\") pod \"46054877-aba8-4c16-b75d-a8a04f2454b2\" (UID: \"46054877-aba8-4c16-b75d-a8a04f2454b2\") "
Sep 19 08:41:41.932430 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:41:41.932242    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-49gsb\" (UniqueName: \"kubernetes.io/projected/46054877-aba8-4c16-b75d-a8a04f2454b2-kube-api-access-49gsb\") pod \"46054877-aba8-4c16-b75d-a8a04f2454b2\" (UID: \"46054877-aba8-4c16-b75d-a8a04f2454b2\") "
Sep 19 08:41:41.932430 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:41:41.932352    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/46054877-aba8-4c16-b75d-a8a04f2454b2-proxy-tls\") pod \"46054877-aba8-4c16-b75d-a8a04f2454b2\" (UID: \"46054877-aba8-4c16-b75d-a8a04f2454b2\") "
Sep 19 08:41:41.932630 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:41:41.932603    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/46054877-aba8-4c16-b75d-a8a04f2454b2-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "46054877-aba8-4c16-b75d-a8a04f2454b2" (UID: "46054877-aba8-4c16-b75d-a8a04f2454b2"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 08:41:41.934375 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:41:41.934351    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/46054877-aba8-4c16-b75d-a8a04f2454b2-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "46054877-aba8-4c16-b75d-a8a04f2454b2" (UID: "46054877-aba8-4c16-b75d-a8a04f2454b2"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 08:41:41.934476 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:41:41.934394    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/46054877-aba8-4c16-b75d-a8a04f2454b2-kube-api-access-49gsb" (OuterVolumeSpecName: "kube-api-access-49gsb") pod "46054877-aba8-4c16-b75d-a8a04f2454b2" (UID: "46054877-aba8-4c16-b75d-a8a04f2454b2"). InnerVolumeSpecName "kube-api-access-49gsb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 08:41:42.032924 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:41:42.032891    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-49gsb\" (UniqueName: \"kubernetes.io/projected/46054877-aba8-4c16-b75d-a8a04f2454b2-kube-api-access-49gsb\") on node \"ip-10-0-139-157.ec2.internal\" DevicePath \"\""
Sep 19 08:41:42.032924 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:41:42.032922    2578 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/46054877-aba8-4c16-b75d-a8a04f2454b2-proxy-tls\") on node \"ip-10-0-139-157.ec2.internal\" DevicePath \"\""
Sep 19 08:41:42.032924 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:41:42.032932    2578 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/46054877-aba8-4c16-b75d-a8a04f2454b2-kube-rbac-proxy-sar-config\") on node \"ip-10-0-139-157.ec2.internal\" DevicePath \"\""
Sep 19 08:41:42.682435 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:41:42.682399    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-96102-predictor-75c8898456-mwf4g" event={"ID":"46054877-aba8-4c16-b75d-a8a04f2454b2","Type":"ContainerDied","Data":"5815896f6fab8dcaabc2a1ab7435c60ecba92a0aad2414010b557bae1354d08a"}
Sep 19 08:41:42.682830 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:41:42.682444    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-96102-predictor-75c8898456-mwf4g"
Sep 19 08:41:42.682830 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:41:42.682496    2578 scope.go:117] "RemoveContainer" containerID="77f0ca9c6792abd2332e0150d1bcd8ba361db1962db1206a499a4e44f5242bed"
Sep 19 08:41:42.691263 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:41:42.691239    2578 scope.go:117] "RemoveContainer" containerID="6a3a1d30c6517caf846adeb7f7cd1cbf91762d1e5b0290299dd5daa49da9c902"
Sep 19 08:41:42.700609 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:41:42.700580    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-96102-predictor-75c8898456-mwf4g"]
Sep 19 08:41:42.703997 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:41:42.703974    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-96102-predictor-75c8898456-mwf4g"]
Sep 19 08:41:44.113192 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:41:44.113160    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="46054877-aba8-4c16-b75d-a8a04f2454b2" path="/var/lib/kubelet/pods/46054877-aba8-4c16-b75d-a8a04f2454b2/volumes"
Sep 19 08:41:46.680530 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:41:46.680503    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-88665-predictor-7c9cfdb5d6-xd4jw"
Sep 19 08:41:46.681010 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:41:46.680987    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-88665-predictor-7c9cfdb5d6-xd4jw" podUID="22a6f174-5523-4416-a8f3-4ee515d0f1d9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.36:8080: connect: connection refused"
Sep 19 08:41:56.681381 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:41:56.681327    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-88665-predictor-7c9cfdb5d6-xd4jw" podUID="22a6f174-5523-4416-a8f3-4ee515d0f1d9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.36:8080: connect: connection refused"
Sep 19 08:42:06.681977 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:42:06.681934    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-88665-predictor-7c9cfdb5d6-xd4jw" podUID="22a6f174-5523-4416-a8f3-4ee515d0f1d9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.36:8080: connect: connection refused"
Sep 19 08:42:16.681152 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:42:16.681110    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-88665-predictor-7c9cfdb5d6-xd4jw" podUID="22a6f174-5523-4416-a8f3-4ee515d0f1d9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.36:8080: connect: connection refused"
Sep 19 08:42:26.681685 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:42:26.681647    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-88665-predictor-7c9cfdb5d6-xd4jw" podUID="22a6f174-5523-4416-a8f3-4ee515d0f1d9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.36:8080: connect: connection refused"
Sep 19 08:42:28.770175 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:42:28.770139    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:42:28.873036 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:42:28.873003    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-5cf0a-predictor-79d6c67446-q5bcc"]
Sep 19 08:42:28.873264 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:42:28.873235    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-5cf0a-predictor-79d6c67446-q5bcc" podUID="b4de8d11-747c-4643-a67f-30f4f0e7bfe1" containerName="kserve-container" containerID="cri-o://a836ee23a2b811671e4ab6a2dd0d74672afe68c356fbb4ccd79062c84c8e4a79" gracePeriod=30
Sep 19 08:42:28.873437 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:42:28.873276    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-5cf0a-predictor-79d6c67446-q5bcc" podUID="b4de8d11-747c-4643-a67f-30f4f0e7bfe1" containerName="kube-rbac-proxy" containerID="cri-o://d32fde0c75c4e7ef72512a07ba7c40f740e95d92df3ea32774df3893f67334db" gracePeriod=30
Sep 19 08:42:29.074405 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:42:29.074311    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-3bcba-predictor-8d6ff494-grfpb"]
Sep 19 08:42:29.074696 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:42:29.074682    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="46054877-aba8-4c16-b75d-a8a04f2454b2" containerName="kube-rbac-proxy"
Sep 19 08:42:29.074766 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:42:29.074697    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="46054877-aba8-4c16-b75d-a8a04f2454b2" containerName="kube-rbac-proxy"
Sep 19 08:42:29.074766 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:42:29.074710    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="46054877-aba8-4c16-b75d-a8a04f2454b2" containerName="kserve-container"
Sep 19 08:42:29.074766 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:42:29.074716    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="46054877-aba8-4c16-b75d-a8a04f2454b2" containerName="kserve-container"
Sep 19 08:42:29.074914 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:42:29.074780    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="46054877-aba8-4c16-b75d-a8a04f2454b2" containerName="kube-rbac-proxy"
Sep 19 08:42:29.074914 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:42:29.074788    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="46054877-aba8-4c16-b75d-a8a04f2454b2" containerName="kserve-container"
Sep 19 08:42:29.077822 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:42:29.077804    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-3bcba-predictor-8d6ff494-grfpb"
Sep 19 08:42:29.080193 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:42:29.080168    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-3bcba-kube-rbac-proxy-sar-config\""
Sep 19 08:42:29.080193 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:42:29.080185    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-3bcba-predictor-serving-cert\""
Sep 19 08:42:29.088495 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:42:29.088467    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-3bcba-predictor-8d6ff494-grfpb"]
Sep 19 08:42:29.211865 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:42:29.211818    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7fe275c2-758a-46d1-985f-572ab2fc0f59-proxy-tls\") pod \"error-404-isvc-3bcba-predictor-8d6ff494-grfpb\" (UID: \"7fe275c2-758a-46d1-985f-572ab2fc0f59\") " pod="kserve-ci-e2e-test/error-404-isvc-3bcba-predictor-8d6ff494-grfpb"
Sep 19 08:42:29.212035 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:42:29.211878    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/7fe275c2-758a-46d1-985f-572ab2fc0f59-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-3bcba-predictor-8d6ff494-grfpb\" (UID: \"7fe275c2-758a-46d1-985f-572ab2fc0f59\") " pod="kserve-ci-e2e-test/error-404-isvc-3bcba-predictor-8d6ff494-grfpb"
Sep 19 08:42:29.212088 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:42:29.212035    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r5698\" (UniqueName: \"kubernetes.io/projected/7fe275c2-758a-46d1-985f-572ab2fc0f59-kube-api-access-r5698\") pod \"error-404-isvc-3bcba-predictor-8d6ff494-grfpb\" (UID: \"7fe275c2-758a-46d1-985f-572ab2fc0f59\") " pod="kserve-ci-e2e-test/error-404-isvc-3bcba-predictor-8d6ff494-grfpb"
Sep 19 08:42:29.312621 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:42:29.312564    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r5698\" (UniqueName: \"kubernetes.io/projected/7fe275c2-758a-46d1-985f-572ab2fc0f59-kube-api-access-r5698\") pod \"error-404-isvc-3bcba-predictor-8d6ff494-grfpb\" (UID: \"7fe275c2-758a-46d1-985f-572ab2fc0f59\") " pod="kserve-ci-e2e-test/error-404-isvc-3bcba-predictor-8d6ff494-grfpb"
Sep 19 08:42:29.312831 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:42:29.312656    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7fe275c2-758a-46d1-985f-572ab2fc0f59-proxy-tls\") pod \"error-404-isvc-3bcba-predictor-8d6ff494-grfpb\" (UID: \"7fe275c2-758a-46d1-985f-572ab2fc0f59\") " pod="kserve-ci-e2e-test/error-404-isvc-3bcba-predictor-8d6ff494-grfpb"
Sep 19 08:42:29.312831 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:42:29.312695    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/7fe275c2-758a-46d1-985f-572ab2fc0f59-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-3bcba-predictor-8d6ff494-grfpb\" (UID: \"7fe275c2-758a-46d1-985f-572ab2fc0f59\") " pod="kserve-ci-e2e-test/error-404-isvc-3bcba-predictor-8d6ff494-grfpb"
Sep 19 08:42:29.313554 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:42:29.313527    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/7fe275c2-758a-46d1-985f-572ab2fc0f59-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-3bcba-predictor-8d6ff494-grfpb\" (UID: \"7fe275c2-758a-46d1-985f-572ab2fc0f59\") " pod="kserve-ci-e2e-test/error-404-isvc-3bcba-predictor-8d6ff494-grfpb"
Sep 19 08:42:29.315110 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:42:29.315084    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7fe275c2-758a-46d1-985f-572ab2fc0f59-proxy-tls\") pod \"error-404-isvc-3bcba-predictor-8d6ff494-grfpb\" (UID: \"7fe275c2-758a-46d1-985f-572ab2fc0f59\") " pod="kserve-ci-e2e-test/error-404-isvc-3bcba-predictor-8d6ff494-grfpb"
Sep 19 08:42:29.320469 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:42:29.320443    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r5698\" (UniqueName: \"kubernetes.io/projected/7fe275c2-758a-46d1-985f-572ab2fc0f59-kube-api-access-r5698\") pod \"error-404-isvc-3bcba-predictor-8d6ff494-grfpb\" (UID: \"7fe275c2-758a-46d1-985f-572ab2fc0f59\") " pod="kserve-ci-e2e-test/error-404-isvc-3bcba-predictor-8d6ff494-grfpb"
Sep 19 08:42:29.390367 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:42:29.390268    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-3bcba-predictor-8d6ff494-grfpb"
Sep 19 08:42:29.662873 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:42:29.662842    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-3bcba-predictor-8d6ff494-grfpb"]
Sep 19 08:42:29.665328 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:42:29.665271    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7fe275c2_758a_46d1_985f_572ab2fc0f59.slice/crio-7c5fe398934ea50d6db9f6cb1cd9b8219bf1363cab50f7b5812ad365947898dc WatchSource:0}: Error finding container 7c5fe398934ea50d6db9f6cb1cd9b8219bf1363cab50f7b5812ad365947898dc: Status 404 returned error can't find the container with id 7c5fe398934ea50d6db9f6cb1cd9b8219bf1363cab50f7b5812ad365947898dc
Sep 19 08:42:29.895395 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:42:29.895351    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-3bcba-predictor-8d6ff494-grfpb" event={"ID":"7fe275c2-758a-46d1-985f-572ab2fc0f59","Type":"ContainerStarted","Data":"e784d733602463eab71f36145c8a004337278812c98c356e6e9b2ae070c0b73d"}
Sep 19 08:42:29.895848 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:42:29.895457    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-3bcba-predictor-8d6ff494-grfpb"
Sep 19 08:42:29.895848 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:42:29.895490    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-3bcba-predictor-8d6ff494-grfpb" event={"ID":"7fe275c2-758a-46d1-985f-572ab2fc0f59","Type":"ContainerStarted","Data":"988615d3dfd6d435f3c167413bb307c5dc16e9102baa21247941d4f01bcc1615"}
Sep 19 08:42:29.895848 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:42:29.895517    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-3bcba-predictor-8d6ff494-grfpb" event={"ID":"7fe275c2-758a-46d1-985f-572ab2fc0f59","Type":"ContainerStarted","Data":"7c5fe398934ea50d6db9f6cb1cd9b8219bf1363cab50f7b5812ad365947898dc"}
Sep 19 08:42:29.895848 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:42:29.895549    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-3bcba-predictor-8d6ff494-grfpb"
Sep 19 08:42:29.897208 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:42:29.897168    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-3bcba-predictor-8d6ff494-grfpb" podUID="7fe275c2-758a-46d1-985f-572ab2fc0f59" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.37:8080: connect: connection refused"
Sep 19 08:42:29.897474 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:42:29.897447    2578 generic.go:356] "Generic (PLEG): container finished" podID="b4de8d11-747c-4643-a67f-30f4f0e7bfe1" containerID="d32fde0c75c4e7ef72512a07ba7c40f740e95d92df3ea32774df3893f67334db" exitCode=2
Sep 19 08:42:29.897564 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:42:29.897488    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-5cf0a-predictor-79d6c67446-q5bcc" event={"ID":"b4de8d11-747c-4643-a67f-30f4f0e7bfe1","Type":"ContainerDied","Data":"d32fde0c75c4e7ef72512a07ba7c40f740e95d92df3ea32774df3893f67334db"}
Sep 19 08:42:29.917162 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:42:29.917081    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-3bcba-predictor-8d6ff494-grfpb" podStartSLOduration=0.917060136 podStartE2EDuration="917.060136ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 08:42:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 08:42:29.91497002 +0000 UTC m=+1334.291679918" watchObservedRunningTime="2026-09-19 08:42:29.917060136 +0000 UTC m=+1334.293769999"
Sep 19 08:42:30.403380 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:42:30.403338    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:42:30.901978 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:42:30.901933    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-3bcba-predictor-8d6ff494-grfpb" podUID="7fe275c2-758a-46d1-985f-572ab2fc0f59" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.37:8080: connect: connection refused"
Sep 19 08:42:31.397529 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:42:31.397488    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-5cf0a-predictor-79d6c67446-q5bcc" podUID="b4de8d11-747c-4643-a67f-30f4f0e7bfe1" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.35:8643/healthz\": dial tcp 10.132.0.35:8643: connect: connection refused"
Sep 19 08:42:31.403585 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:42:31.403554    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-5cf0a-predictor-79d6c67446-q5bcc" podUID="b4de8d11-747c-4643-a67f-30f4f0e7bfe1" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.35:8080: connect: connection refused"
Sep 19 08:42:32.631025 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:42:32.631000    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-5cf0a-predictor-79d6c67446-q5bcc"
Sep 19 08:42:32.746470 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:42:32.746380    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b4de8d11-747c-4643-a67f-30f4f0e7bfe1-proxy-tls\") pod \"b4de8d11-747c-4643-a67f-30f4f0e7bfe1\" (UID: \"b4de8d11-747c-4643-a67f-30f4f0e7bfe1\") "
Sep 19 08:42:32.746470 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:42:32.746420    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vfpbm\" (UniqueName: \"kubernetes.io/projected/b4de8d11-747c-4643-a67f-30f4f0e7bfe1-kube-api-access-vfpbm\") pod \"b4de8d11-747c-4643-a67f-30f4f0e7bfe1\" (UID: \"b4de8d11-747c-4643-a67f-30f4f0e7bfe1\") "
Sep 19 08:42:32.746470 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:42:32.746467    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b4de8d11-747c-4643-a67f-30f4f0e7bfe1-kube-rbac-proxy-sar-config\") pod \"b4de8d11-747c-4643-a67f-30f4f0e7bfe1\" (UID: \"b4de8d11-747c-4643-a67f-30f4f0e7bfe1\") "
Sep 19 08:42:32.746893 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:42:32.746868    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b4de8d11-747c-4643-a67f-30f4f0e7bfe1-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "b4de8d11-747c-4643-a67f-30f4f0e7bfe1" (UID: "b4de8d11-747c-4643-a67f-30f4f0e7bfe1"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 08:42:32.748590 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:42:32.748569    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b4de8d11-747c-4643-a67f-30f4f0e7bfe1-kube-api-access-vfpbm" (OuterVolumeSpecName: "kube-api-access-vfpbm") pod "b4de8d11-747c-4643-a67f-30f4f0e7bfe1" (UID: "b4de8d11-747c-4643-a67f-30f4f0e7bfe1"). InnerVolumeSpecName "kube-api-access-vfpbm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 08:42:32.748693 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:42:32.748671    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b4de8d11-747c-4643-a67f-30f4f0e7bfe1-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "b4de8d11-747c-4643-a67f-30f4f0e7bfe1" (UID: "b4de8d11-747c-4643-a67f-30f4f0e7bfe1"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 08:42:32.847631 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:42:32.847584    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vfpbm\" (UniqueName: \"kubernetes.io/projected/b4de8d11-747c-4643-a67f-30f4f0e7bfe1-kube-api-access-vfpbm\") on node \"ip-10-0-139-157.ec2.internal\" DevicePath \"\""
Sep 19 08:42:32.847631 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:42:32.847620    2578 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b4de8d11-747c-4643-a67f-30f4f0e7bfe1-kube-rbac-proxy-sar-config\") on node \"ip-10-0-139-157.ec2.internal\" DevicePath \"\""
Sep 19 08:42:32.847631 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:42:32.847629    2578 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b4de8d11-747c-4643-a67f-30f4f0e7bfe1-proxy-tls\") on node \"ip-10-0-139-157.ec2.internal\" DevicePath \"\""
Sep 19 08:42:32.911761 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:42:32.911726    2578 generic.go:356] "Generic (PLEG): container finished" podID="b4de8d11-747c-4643-a67f-30f4f0e7bfe1" containerID="a836ee23a2b811671e4ab6a2dd0d74672afe68c356fbb4ccd79062c84c8e4a79" exitCode=0
Sep 19 08:42:32.911916 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:42:32.911774    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-5cf0a-predictor-79d6c67446-q5bcc"
Sep 19 08:42:32.911916 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:42:32.911807    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-5cf0a-predictor-79d6c67446-q5bcc" event={"ID":"b4de8d11-747c-4643-a67f-30f4f0e7bfe1","Type":"ContainerDied","Data":"a836ee23a2b811671e4ab6a2dd0d74672afe68c356fbb4ccd79062c84c8e4a79"}
Sep 19 08:42:32.911916 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:42:32.911869    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-5cf0a-predictor-79d6c67446-q5bcc" event={"ID":"b4de8d11-747c-4643-a67f-30f4f0e7bfe1","Type":"ContainerDied","Data":"ab28d181ddf19b2845c314205910b859a25486b6bd796fb2b9b411d280fe18c9"}
Sep 19 08:42:32.911916 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:42:32.911896    2578 scope.go:117] "RemoveContainer" containerID="d32fde0c75c4e7ef72512a07ba7c40f740e95d92df3ea32774df3893f67334db"
Sep 19 08:42:32.920477 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:42:32.920463    2578 scope.go:117] "RemoveContainer" containerID="a836ee23a2b811671e4ab6a2dd0d74672afe68c356fbb4ccd79062c84c8e4a79"
Sep 19 08:42:32.927518 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:42:32.927483    2578 scope.go:117] "RemoveContainer" containerID="d32fde0c75c4e7ef72512a07ba7c40f740e95d92df3ea32774df3893f67334db"
Sep 19 08:42:32.927758 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:42:32.927741    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d32fde0c75c4e7ef72512a07ba7c40f740e95d92df3ea32774df3893f67334db\": container with ID starting with d32fde0c75c4e7ef72512a07ba7c40f740e95d92df3ea32774df3893f67334db not found: ID does not exist" containerID="d32fde0c75c4e7ef72512a07ba7c40f740e95d92df3ea32774df3893f67334db"
Sep 19 08:42:32.927828 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:42:32.927771    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d32fde0c75c4e7ef72512a07ba7c40f740e95d92df3ea32774df3893f67334db"} err="failed to get container status \"d32fde0c75c4e7ef72512a07ba7c40f740e95d92df3ea32774df3893f67334db\": rpc error: code = NotFound desc = could not find container \"d32fde0c75c4e7ef72512a07ba7c40f740e95d92df3ea32774df3893f67334db\": container with ID starting with d32fde0c75c4e7ef72512a07ba7c40f740e95d92df3ea32774df3893f67334db not found: ID does not exist"
Sep 19 08:42:32.927828 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:42:32.927796    2578 scope.go:117] "RemoveContainer" containerID="a836ee23a2b811671e4ab6a2dd0d74672afe68c356fbb4ccd79062c84c8e4a79"
Sep 19 08:42:32.928052 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:42:32.928033    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a836ee23a2b811671e4ab6a2dd0d74672afe68c356fbb4ccd79062c84c8e4a79\": container with ID starting with a836ee23a2b811671e4ab6a2dd0d74672afe68c356fbb4ccd79062c84c8e4a79 not found: ID does not exist" containerID="a836ee23a2b811671e4ab6a2dd0d74672afe68c356fbb4ccd79062c84c8e4a79"
Sep 19 08:42:32.928091 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:42:32.928057    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a836ee23a2b811671e4ab6a2dd0d74672afe68c356fbb4ccd79062c84c8e4a79"} err="failed to get container status \"a836ee23a2b811671e4ab6a2dd0d74672afe68c356fbb4ccd79062c84c8e4a79\": rpc error: code = NotFound desc = could not find container \"a836ee23a2b811671e4ab6a2dd0d74672afe68c356fbb4ccd79062c84c8e4a79\": container with ID starting with a836ee23a2b811671e4ab6a2dd0d74672afe68c356fbb4ccd79062c84c8e4a79 not found: ID does not exist"
Sep 19 08:42:32.931956 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:42:32.931934    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-5cf0a-predictor-79d6c67446-q5bcc"]
Sep 19 08:42:32.935712 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:42:32.935693    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-5cf0a-predictor-79d6c67446-q5bcc"]
Sep 19 08:42:34.113736 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:42:34.113696    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b4de8d11-747c-4643-a67f-30f4f0e7bfe1" path="/var/lib/kubelet/pods/b4de8d11-747c-4643-a67f-30f4f0e7bfe1/volumes"
Sep 19 08:42:35.906580 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:42:35.906542    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-3bcba-predictor-8d6ff494-grfpb"
Sep 19 08:42:35.906974 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:42:35.906948    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-3bcba-predictor-8d6ff494-grfpb" podUID="7fe275c2-758a-46d1-985f-572ab2fc0f59" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.37:8080: connect: connection refused"
Sep 19 08:42:36.682502 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:42:36.682471    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-88665-predictor-7c9cfdb5d6-xd4jw"
Sep 19 08:42:45.907746 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:42:45.907661    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-3bcba-predictor-8d6ff494-grfpb" podUID="7fe275c2-758a-46d1-985f-572ab2fc0f59" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.37:8080: connect: connection refused"
Sep 19 08:42:55.907586 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:42:55.907534    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-3bcba-predictor-8d6ff494-grfpb" podUID="7fe275c2-758a-46d1-985f-572ab2fc0f59" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.37:8080: connect: connection refused"
Sep 19 08:42:58.205979 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:42:58.205940    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:42:58.434341 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:42:58.434291    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:42:58.752565 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:42:58.752535    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:43:05.907308 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:05.907256    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-3bcba-predictor-8d6ff494-grfpb" podUID="7fe275c2-758a-46d1-985f-572ab2fc0f59" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.37:8080: connect: connection refused"
Sep 19 08:43:08.764712 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:08.764671    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:43:08.880441 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:08.880397    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:43:08.886290 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:08.886263    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-88665-predictor-7c9cfdb5d6-xd4jw"]
Sep 19 08:43:08.886513 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:08.886488    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-88665-predictor-7c9cfdb5d6-xd4jw" podUID="22a6f174-5523-4416-a8f3-4ee515d0f1d9" containerName="kserve-container" containerID="cri-o://4d20d228fb738727a56421b5d672e84bc78faf2d07e274b385173ed8544419b0" gracePeriod=30
Sep 19 08:43:08.886585 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:08.886519    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-88665-predictor-7c9cfdb5d6-xd4jw" podUID="22a6f174-5523-4416-a8f3-4ee515d0f1d9" containerName="kube-rbac-proxy" containerID="cri-o://f63a91a8bcb14db7e077a7c4c03b94f7bea3daf3837a06877ea0015fd4c7b368" gracePeriod=30
Sep 19 08:43:08.979496 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:08.979457    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:43:09.079780 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:09.079548    2578 generic.go:356] "Generic (PLEG): container finished" podID="22a6f174-5523-4416-a8f3-4ee515d0f1d9" containerID="f63a91a8bcb14db7e077a7c4c03b94f7bea3daf3837a06877ea0015fd4c7b368" exitCode=2
Sep 19 08:43:09.079780 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:09.079657    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-88665-predictor-7c9cfdb5d6-xd4jw" event={"ID":"22a6f174-5523-4416-a8f3-4ee515d0f1d9","Type":"ContainerDied","Data":"f63a91a8bcb14db7e077a7c4c03b94f7bea3daf3837a06877ea0015fd4c7b368"}
Sep 19 08:43:09.101737 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:09.101686    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:43:09.125606 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:09.125569    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-fd725-predictor-5b7d599548-p7c8s"]
Sep 19 08:43:09.125906 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:09.125894    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b4de8d11-747c-4643-a67f-30f4f0e7bfe1" containerName="kube-rbac-proxy"
Sep 19 08:43:09.125969 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:09.125908    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="b4de8d11-747c-4643-a67f-30f4f0e7bfe1" containerName="kube-rbac-proxy"
Sep 19 08:43:09.125969 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:09.125921    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b4de8d11-747c-4643-a67f-30f4f0e7bfe1" containerName="kserve-container"
Sep 19 08:43:09.125969 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:09.125928    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="b4de8d11-747c-4643-a67f-30f4f0e7bfe1" containerName="kserve-container"
Sep 19 08:43:09.126127 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:09.125989    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="b4de8d11-747c-4643-a67f-30f4f0e7bfe1" containerName="kube-rbac-proxy"
Sep 19 08:43:09.126127 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:09.125998    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="b4de8d11-747c-4643-a67f-30f4f0e7bfe1" containerName="kserve-container"
Sep 19 08:43:09.129732 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:09.129713    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-fd725-predictor-5b7d599548-p7c8s"
Sep 19 08:43:09.136792 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:09.136767    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/21d4e7f8-82db-4272-9ad8-933f96783cde-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-fd725-predictor-5b7d599548-p7c8s\" (UID: \"21d4e7f8-82db-4272-9ad8-933f96783cde\") " pod="kserve-ci-e2e-test/error-404-isvc-fd725-predictor-5b7d599548-p7c8s"
Sep 19 08:43:09.136940 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:09.136816    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8nkmc\" (UniqueName: \"kubernetes.io/projected/21d4e7f8-82db-4272-9ad8-933f96783cde-kube-api-access-8nkmc\") pod \"error-404-isvc-fd725-predictor-5b7d599548-p7c8s\" (UID: \"21d4e7f8-82db-4272-9ad8-933f96783cde\") " pod="kserve-ci-e2e-test/error-404-isvc-fd725-predictor-5b7d599548-p7c8s"
Sep 19 08:43:09.136940 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:09.136857    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/21d4e7f8-82db-4272-9ad8-933f96783cde-proxy-tls\") pod \"error-404-isvc-fd725-predictor-5b7d599548-p7c8s\" (UID: \"21d4e7f8-82db-4272-9ad8-933f96783cde\") " pod="kserve-ci-e2e-test/error-404-isvc-fd725-predictor-5b7d599548-p7c8s"
Sep 19 08:43:09.137068 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:09.136966    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-fd725-predictor-serving-cert\""
Sep 19 08:43:09.137123 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:09.137098    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-fd725-kube-rbac-proxy-sar-config\""
Sep 19 08:43:09.145503 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:09.145473    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-fd725-predictor-5b7d599548-p7c8s"]
Sep 19 08:43:09.217008 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:09.216972    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:43:09.238231 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:09.238195    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8nkmc\" (UniqueName: \"kubernetes.io/projected/21d4e7f8-82db-4272-9ad8-933f96783cde-kube-api-access-8nkmc\") pod \"error-404-isvc-fd725-predictor-5b7d599548-p7c8s\" (UID: \"21d4e7f8-82db-4272-9ad8-933f96783cde\") " pod="kserve-ci-e2e-test/error-404-isvc-fd725-predictor-5b7d599548-p7c8s"
Sep 19 08:43:09.238435 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:09.238256    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/21d4e7f8-82db-4272-9ad8-933f96783cde-proxy-tls\") pod \"error-404-isvc-fd725-predictor-5b7d599548-p7c8s\" (UID: \"21d4e7f8-82db-4272-9ad8-933f96783cde\") " pod="kserve-ci-e2e-test/error-404-isvc-fd725-predictor-5b7d599548-p7c8s"
Sep 19 08:43:09.238435 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:09.238318    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/21d4e7f8-82db-4272-9ad8-933f96783cde-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-fd725-predictor-5b7d599548-p7c8s\" (UID: \"21d4e7f8-82db-4272-9ad8-933f96783cde\") " pod="kserve-ci-e2e-test/error-404-isvc-fd725-predictor-5b7d599548-p7c8s"
Sep 19 08:43:09.238586 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:43:09.238430    2578 secret.go:189] Couldn't get secret kserve-ci-e2e-test/error-404-isvc-fd725-predictor-serving-cert: secret "error-404-isvc-fd725-predictor-serving-cert" not found
Sep 19 08:43:09.238586 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:43:09.238501    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/21d4e7f8-82db-4272-9ad8-933f96783cde-proxy-tls podName:21d4e7f8-82db-4272-9ad8-933f96783cde nodeName:}" failed. No retries permitted until 2026-09-19 08:43:09.738481255 +0000 UTC m=+1374.115191095 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/21d4e7f8-82db-4272-9ad8-933f96783cde-proxy-tls") pod "error-404-isvc-fd725-predictor-5b7d599548-p7c8s" (UID: "21d4e7f8-82db-4272-9ad8-933f96783cde") : secret "error-404-isvc-fd725-predictor-serving-cert" not found
Sep 19 08:43:09.238964 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:09.238946    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/21d4e7f8-82db-4272-9ad8-933f96783cde-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-fd725-predictor-5b7d599548-p7c8s\" (UID: \"21d4e7f8-82db-4272-9ad8-933f96783cde\") " pod="kserve-ci-e2e-test/error-404-isvc-fd725-predictor-5b7d599548-p7c8s"
Sep 19 08:43:09.248337 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:09.248292    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8nkmc\" (UniqueName: \"kubernetes.io/projected/21d4e7f8-82db-4272-9ad8-933f96783cde-kube-api-access-8nkmc\") pod \"error-404-isvc-fd725-predictor-5b7d599548-p7c8s\" (UID: \"21d4e7f8-82db-4272-9ad8-933f96783cde\") " pod="kserve-ci-e2e-test/error-404-isvc-fd725-predictor-5b7d599548-p7c8s"
Sep 19 08:43:09.317017 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:09.316982    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:43:09.505019 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:09.504938    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:43:09.743316 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:09.743274    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/21d4e7f8-82db-4272-9ad8-933f96783cde-proxy-tls\") pod \"error-404-isvc-fd725-predictor-5b7d599548-p7c8s\" (UID: \"21d4e7f8-82db-4272-9ad8-933f96783cde\") " pod="kserve-ci-e2e-test/error-404-isvc-fd725-predictor-5b7d599548-p7c8s"
Sep 19 08:43:09.745855 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:09.745834    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/21d4e7f8-82db-4272-9ad8-933f96783cde-proxy-tls\") pod \"error-404-isvc-fd725-predictor-5b7d599548-p7c8s\" (UID: \"21d4e7f8-82db-4272-9ad8-933f96783cde\") " pod="kserve-ci-e2e-test/error-404-isvc-fd725-predictor-5b7d599548-p7c8s"
Sep 19 08:43:10.043034 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:10.043000    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-fd725-predictor-5b7d599548-p7c8s"
Sep 19 08:43:10.320475 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:10.320448    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-fd725-predictor-5b7d599548-p7c8s"]
Sep 19 08:43:10.322667 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:43:10.322639    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21d4e7f8_82db_4272_9ad8_933f96783cde.slice/crio-4b908f5494370c2e6c4a4d240dafd91b5d1bcc632a36f6634395cac01ff5e205 WatchSource:0}: Error finding container 4b908f5494370c2e6c4a4d240dafd91b5d1bcc632a36f6634395cac01ff5e205: Status 404 returned error can't find the container with id 4b908f5494370c2e6c4a4d240dafd91b5d1bcc632a36f6634395cac01ff5e205
Sep 19 08:43:11.089801 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:11.089765    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-fd725-predictor-5b7d599548-p7c8s" event={"ID":"21d4e7f8-82db-4272-9ad8-933f96783cde","Type":"ContainerStarted","Data":"b18d84d2e89cf7c5d20ecc1171648b30207f2729db43bcdfabbab75c1f5c5486"}
Sep 19 08:43:11.090200 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:11.089843    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-fd725-predictor-5b7d599548-p7c8s"
Sep 19 08:43:11.090200 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:11.089954    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-fd725-predictor-5b7d599548-p7c8s"
Sep 19 08:43:11.090200 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:11.089981    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-fd725-predictor-5b7d599548-p7c8s" event={"ID":"21d4e7f8-82db-4272-9ad8-933f96783cde","Type":"ContainerStarted","Data":"a27c98b4adb508a0cd1d8e0e98f6b4d806d5f139dbb4eccaa8b75cde08922bec"}
Sep 19 08:43:11.090200 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:11.090001    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-fd725-predictor-5b7d599548-p7c8s" event={"ID":"21d4e7f8-82db-4272-9ad8-933f96783cde","Type":"ContainerStarted","Data":"4b908f5494370c2e6c4a4d240dafd91b5d1bcc632a36f6634395cac01ff5e205"}
Sep 19 08:43:11.091428 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:11.091397    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-fd725-predictor-5b7d599548-p7c8s" podUID="21d4e7f8-82db-4272-9ad8-933f96783cde" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.38:8080: connect: connection refused"
Sep 19 08:43:11.106473 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:11.106415    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-fd725-predictor-5b7d599548-p7c8s" podStartSLOduration=2.106397614 podStartE2EDuration="2.106397614s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 08:43:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 08:43:11.105686552 +0000 UTC m=+1375.482396407" watchObservedRunningTime="2026-09-19 08:43:11.106397614 +0000 UTC m=+1375.483107469"
Sep 19 08:43:11.677484 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:11.677438    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-88665-predictor-7c9cfdb5d6-xd4jw" podUID="22a6f174-5523-4416-a8f3-4ee515d0f1d9" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.36:8643/healthz\": dial tcp 10.132.0.36:8643: connect: connection refused"
Sep 19 08:43:12.094347 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:12.094313    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-fd725-predictor-5b7d599548-p7c8s" podUID="21d4e7f8-82db-4272-9ad8-933f96783cde" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.38:8080: connect: connection refused"
Sep 19 08:43:12.835238 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:12.835210    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-88665-predictor-7c9cfdb5d6-xd4jw"
Sep 19 08:43:12.871019 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:12.870988    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/22a6f174-5523-4416-a8f3-4ee515d0f1d9-proxy-tls\") pod \"22a6f174-5523-4416-a8f3-4ee515d0f1d9\" (UID: \"22a6f174-5523-4416-a8f3-4ee515d0f1d9\") "
Sep 19 08:43:12.871222 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:12.871033    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/22a6f174-5523-4416-a8f3-4ee515d0f1d9-kube-rbac-proxy-sar-config\") pod \"22a6f174-5523-4416-a8f3-4ee515d0f1d9\" (UID: \"22a6f174-5523-4416-a8f3-4ee515d0f1d9\") "
Sep 19 08:43:12.871222 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:12.871083    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qrf2n\" (UniqueName: \"kubernetes.io/projected/22a6f174-5523-4416-a8f3-4ee515d0f1d9-kube-api-access-qrf2n\") pod \"22a6f174-5523-4416-a8f3-4ee515d0f1d9\" (UID: \"22a6f174-5523-4416-a8f3-4ee515d0f1d9\") "
Sep 19 08:43:12.871454 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:12.871424    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/22a6f174-5523-4416-a8f3-4ee515d0f1d9-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "22a6f174-5523-4416-a8f3-4ee515d0f1d9" (UID: "22a6f174-5523-4416-a8f3-4ee515d0f1d9"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 08:43:12.873268 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:12.873233    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/22a6f174-5523-4416-a8f3-4ee515d0f1d9-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "22a6f174-5523-4416-a8f3-4ee515d0f1d9" (UID: "22a6f174-5523-4416-a8f3-4ee515d0f1d9"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 08:43:12.873391 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:12.873241    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/22a6f174-5523-4416-a8f3-4ee515d0f1d9-kube-api-access-qrf2n" (OuterVolumeSpecName: "kube-api-access-qrf2n") pod "22a6f174-5523-4416-a8f3-4ee515d0f1d9" (UID: "22a6f174-5523-4416-a8f3-4ee515d0f1d9"). InnerVolumeSpecName "kube-api-access-qrf2n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 08:43:12.972166 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:12.972073    2578 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/22a6f174-5523-4416-a8f3-4ee515d0f1d9-proxy-tls\") on node \"ip-10-0-139-157.ec2.internal\" DevicePath \"\""
Sep 19 08:43:12.972166 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:12.972110    2578 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/22a6f174-5523-4416-a8f3-4ee515d0f1d9-kube-rbac-proxy-sar-config\") on node \"ip-10-0-139-157.ec2.internal\" DevicePath \"\""
Sep 19 08:43:12.972166 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:12.972121    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qrf2n\" (UniqueName: \"kubernetes.io/projected/22a6f174-5523-4416-a8f3-4ee515d0f1d9-kube-api-access-qrf2n\") on node \"ip-10-0-139-157.ec2.internal\" DevicePath \"\""
Sep 19 08:43:13.100542 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:13.100511    2578 generic.go:356] "Generic (PLEG): container finished" podID="22a6f174-5523-4416-a8f3-4ee515d0f1d9" containerID="4d20d228fb738727a56421b5d672e84bc78faf2d07e274b385173ed8544419b0" exitCode=0
Sep 19 08:43:13.100941 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:13.100565    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-88665-predictor-7c9cfdb5d6-xd4jw"
Sep 19 08:43:13.100941 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:13.100575    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-88665-predictor-7c9cfdb5d6-xd4jw" event={"ID":"22a6f174-5523-4416-a8f3-4ee515d0f1d9","Type":"ContainerDied","Data":"4d20d228fb738727a56421b5d672e84bc78faf2d07e274b385173ed8544419b0"}
Sep 19 08:43:13.100941 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:13.100655    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-88665-predictor-7c9cfdb5d6-xd4jw" event={"ID":"22a6f174-5523-4416-a8f3-4ee515d0f1d9","Type":"ContainerDied","Data":"c51b366177bc1a825e63b1e33fefac4a5f1fda50a612479245dee77ebfaf6fe5"}
Sep 19 08:43:13.100941 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:13.100682    2578 scope.go:117] "RemoveContainer" containerID="f63a91a8bcb14db7e077a7c4c03b94f7bea3daf3837a06877ea0015fd4c7b368"
Sep 19 08:43:13.109662 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:13.109643    2578 scope.go:117] "RemoveContainer" containerID="4d20d228fb738727a56421b5d672e84bc78faf2d07e274b385173ed8544419b0"
Sep 19 08:43:13.117211 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:13.117193    2578 scope.go:117] "RemoveContainer" containerID="f63a91a8bcb14db7e077a7c4c03b94f7bea3daf3837a06877ea0015fd4c7b368"
Sep 19 08:43:13.117509 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:43:13.117489    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f63a91a8bcb14db7e077a7c4c03b94f7bea3daf3837a06877ea0015fd4c7b368\": container with ID starting with f63a91a8bcb14db7e077a7c4c03b94f7bea3daf3837a06877ea0015fd4c7b368 not found: ID does not exist" containerID="f63a91a8bcb14db7e077a7c4c03b94f7bea3daf3837a06877ea0015fd4c7b368"
Sep 19 08:43:13.117593 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:13.117522    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f63a91a8bcb14db7e077a7c4c03b94f7bea3daf3837a06877ea0015fd4c7b368"} err="failed to get container status \"f63a91a8bcb14db7e077a7c4c03b94f7bea3daf3837a06877ea0015fd4c7b368\": rpc error: code = NotFound desc = could not find container \"f63a91a8bcb14db7e077a7c4c03b94f7bea3daf3837a06877ea0015fd4c7b368\": container with ID starting with f63a91a8bcb14db7e077a7c4c03b94f7bea3daf3837a06877ea0015fd4c7b368 not found: ID does not exist"
Sep 19 08:43:13.117593 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:13.117549    2578 scope.go:117] "RemoveContainer" containerID="4d20d228fb738727a56421b5d672e84bc78faf2d07e274b385173ed8544419b0"
Sep 19 08:43:13.117822 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:43:13.117805    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4d20d228fb738727a56421b5d672e84bc78faf2d07e274b385173ed8544419b0\": container with ID starting with 4d20d228fb738727a56421b5d672e84bc78faf2d07e274b385173ed8544419b0 not found: ID does not exist" containerID="4d20d228fb738727a56421b5d672e84bc78faf2d07e274b385173ed8544419b0"
Sep 19 08:43:13.117861 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:13.117827    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4d20d228fb738727a56421b5d672e84bc78faf2d07e274b385173ed8544419b0"} err="failed to get container status \"4d20d228fb738727a56421b5d672e84bc78faf2d07e274b385173ed8544419b0\": rpc error: code = NotFound desc = could not find container \"4d20d228fb738727a56421b5d672e84bc78faf2d07e274b385173ed8544419b0\": container with ID starting with 4d20d228fb738727a56421b5d672e84bc78faf2d07e274b385173ed8544419b0 not found: ID does not exist"
Sep 19 08:43:13.123087 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:13.123051    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-88665-predictor-7c9cfdb5d6-xd4jw"]
Sep 19 08:43:13.126756 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:13.126731    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-88665-predictor-7c9cfdb5d6-xd4jw"]
Sep 19 08:43:14.112504 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:14.112476    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="22a6f174-5523-4416-a8f3-4ee515d0f1d9" path="/var/lib/kubelet/pods/22a6f174-5523-4416-a8f3-4ee515d0f1d9/volumes"
Sep 19 08:43:15.907593 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:15.907556    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-3bcba-predictor-8d6ff494-grfpb" podUID="7fe275c2-758a-46d1-985f-572ab2fc0f59" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.37:8080: connect: connection refused"
Sep 19 08:43:17.099089 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:17.099058    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-fd725-predictor-5b7d599548-p7c8s"
Sep 19 08:43:17.099579 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:17.099554    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-fd725-predictor-5b7d599548-p7c8s" podUID="21d4e7f8-82db-4272-9ad8-933f96783cde" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.38:8080: connect: connection refused"
Sep 19 08:43:25.908474 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:25.908444    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-3bcba-predictor-8d6ff494-grfpb"
Sep 19 08:43:27.099995 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:27.099955    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-fd725-predictor-5b7d599548-p7c8s" podUID="21d4e7f8-82db-4272-9ad8-933f96783cde" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.38:8080: connect: connection refused"
Sep 19 08:43:37.100350 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:37.100282    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-fd725-predictor-5b7d599548-p7c8s" podUID="21d4e7f8-82db-4272-9ad8-933f96783cde" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.38:8080: connect: connection refused"
Sep 19 08:43:38.998973 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:38.998938    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:43:39.095558 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:39.095524    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:43:47.100458 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:47.100413    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-fd725-predictor-5b7d599548-p7c8s" podUID="21d4e7f8-82db-4272-9ad8-933f96783cde" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.38:8080: connect: connection refused"
Sep 19 08:43:49.968170 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:49.968133    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:43:50.045896 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:50.045858    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:43:50.054322 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:50.054272    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-3bcba-predictor-8d6ff494-grfpb"]
Sep 19 08:43:50.054730 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:50.054675    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-3bcba-predictor-8d6ff494-grfpb" podUID="7fe275c2-758a-46d1-985f-572ab2fc0f59" containerName="kserve-container" containerID="cri-o://988615d3dfd6d435f3c167413bb307c5dc16e9102baa21247941d4f01bcc1615" gracePeriod=30
Sep 19 08:43:50.054963 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:50.054701    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-3bcba-predictor-8d6ff494-grfpb" podUID="7fe275c2-758a-46d1-985f-572ab2fc0f59" containerName="kube-rbac-proxy" containerID="cri-o://e784d733602463eab71f36145c8a004337278812c98c356e6e9b2ae070c0b73d" gracePeriod=30
Sep 19 08:43:50.109629 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:50.109594    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:43:50.171802 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:50.171775    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:43:50.233575 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:50.233487    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:43:50.237812 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:50.237756    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-bdc39-predictor-5cb4dbc486-8vz74"]
Sep 19 08:43:50.238182 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:50.238165    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="22a6f174-5523-4416-a8f3-4ee515d0f1d9" containerName="kserve-container"
Sep 19 08:43:50.238269 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:50.238186    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="22a6f174-5523-4416-a8f3-4ee515d0f1d9" containerName="kserve-container"
Sep 19 08:43:50.238269 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:50.238215    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="22a6f174-5523-4416-a8f3-4ee515d0f1d9" containerName="kube-rbac-proxy"
Sep 19 08:43:50.238269 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:50.238225    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="22a6f174-5523-4416-a8f3-4ee515d0f1d9" containerName="kube-rbac-proxy"
Sep 19 08:43:50.238464 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:50.238321    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="22a6f174-5523-4416-a8f3-4ee515d0f1d9" containerName="kserve-container"
Sep 19 08:43:50.238464 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:50.238335    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="22a6f174-5523-4416-a8f3-4ee515d0f1d9" containerName="kube-rbac-proxy"
Sep 19 08:43:50.241376 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:50.241355    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-bdc39-predictor-5cb4dbc486-8vz74"
Sep 19 08:43:50.243519 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:50.243497    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-bdc39-predictor-serving-cert\""
Sep 19 08:43:50.243630 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:50.243612    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-bdc39-kube-rbac-proxy-sar-config\""
Sep 19 08:43:50.251367 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:50.251346    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-bdc39-predictor-5cb4dbc486-8vz74"]
Sep 19 08:43:50.267198 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:50.267163    2578 generic.go:356] "Generic (PLEG): container finished" podID="7fe275c2-758a-46d1-985f-572ab2fc0f59" containerID="e784d733602463eab71f36145c8a004337278812c98c356e6e9b2ae070c0b73d" exitCode=2
Sep 19 08:43:50.267356 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:50.267234    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-3bcba-predictor-8d6ff494-grfpb" event={"ID":"7fe275c2-758a-46d1-985f-572ab2fc0f59","Type":"ContainerDied","Data":"e784d733602463eab71f36145c8a004337278812c98c356e6e9b2ae070c0b73d"}
Sep 19 08:43:50.293466 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:50.293428    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:43:50.387668 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:50.387631    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b3a0e011-81cc-47b9-93bb-e3acb44417fd-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-bdc39-predictor-5cb4dbc486-8vz74\" (UID: \"b3a0e011-81cc-47b9-93bb-e3acb44417fd\") " pod="kserve-ci-e2e-test/error-404-isvc-bdc39-predictor-5cb4dbc486-8vz74"
Sep 19 08:43:50.387848 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:50.387679    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b3a0e011-81cc-47b9-93bb-e3acb44417fd-proxy-tls\") pod \"error-404-isvc-bdc39-predictor-5cb4dbc486-8vz74\" (UID: \"b3a0e011-81cc-47b9-93bb-e3acb44417fd\") " pod="kserve-ci-e2e-test/error-404-isvc-bdc39-predictor-5cb4dbc486-8vz74"
Sep 19 08:43:50.387848 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:50.387709    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4n9x8\" (UniqueName: \"kubernetes.io/projected/b3a0e011-81cc-47b9-93bb-e3acb44417fd-kube-api-access-4n9x8\") pod \"error-404-isvc-bdc39-predictor-5cb4dbc486-8vz74\" (UID: \"b3a0e011-81cc-47b9-93bb-e3acb44417fd\") " pod="kserve-ci-e2e-test/error-404-isvc-bdc39-predictor-5cb4dbc486-8vz74"
Sep 19 08:43:50.488722 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:50.488630    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b3a0e011-81cc-47b9-93bb-e3acb44417fd-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-bdc39-predictor-5cb4dbc486-8vz74\" (UID: \"b3a0e011-81cc-47b9-93bb-e3acb44417fd\") " pod="kserve-ci-e2e-test/error-404-isvc-bdc39-predictor-5cb4dbc486-8vz74"
Sep 19 08:43:50.488722 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:50.488674    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b3a0e011-81cc-47b9-93bb-e3acb44417fd-proxy-tls\") pod \"error-404-isvc-bdc39-predictor-5cb4dbc486-8vz74\" (UID: \"b3a0e011-81cc-47b9-93bb-e3acb44417fd\") " pod="kserve-ci-e2e-test/error-404-isvc-bdc39-predictor-5cb4dbc486-8vz74"
Sep 19 08:43:50.488722 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:50.488704    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4n9x8\" (UniqueName: \"kubernetes.io/projected/b3a0e011-81cc-47b9-93bb-e3acb44417fd-kube-api-access-4n9x8\") pod \"error-404-isvc-bdc39-predictor-5cb4dbc486-8vz74\" (UID: \"b3a0e011-81cc-47b9-93bb-e3acb44417fd\") " pod="kserve-ci-e2e-test/error-404-isvc-bdc39-predictor-5cb4dbc486-8vz74"
Sep 19 08:43:50.489017 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:43:50.488794    2578 secret.go:189] Couldn't get secret kserve-ci-e2e-test/error-404-isvc-bdc39-predictor-serving-cert: secret "error-404-isvc-bdc39-predictor-serving-cert" not found
Sep 19 08:43:50.489017 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:43:50.488867    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b3a0e011-81cc-47b9-93bb-e3acb44417fd-proxy-tls podName:b3a0e011-81cc-47b9-93bb-e3acb44417fd nodeName:}" failed. No retries permitted until 2026-09-19 08:43:50.988841888 +0000 UTC m=+1415.365551744 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/b3a0e011-81cc-47b9-93bb-e3acb44417fd-proxy-tls") pod "error-404-isvc-bdc39-predictor-5cb4dbc486-8vz74" (UID: "b3a0e011-81cc-47b9-93bb-e3acb44417fd") : secret "error-404-isvc-bdc39-predictor-serving-cert" not found
Sep 19 08:43:50.489401 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:50.489378    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b3a0e011-81cc-47b9-93bb-e3acb44417fd-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-bdc39-predictor-5cb4dbc486-8vz74\" (UID: \"b3a0e011-81cc-47b9-93bb-e3acb44417fd\") " pod="kserve-ci-e2e-test/error-404-isvc-bdc39-predictor-5cb4dbc486-8vz74"
Sep 19 08:43:50.502519 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:50.502493    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4n9x8\" (UniqueName: \"kubernetes.io/projected/b3a0e011-81cc-47b9-93bb-e3acb44417fd-kube-api-access-4n9x8\") pod \"error-404-isvc-bdc39-predictor-5cb4dbc486-8vz74\" (UID: \"b3a0e011-81cc-47b9-93bb-e3acb44417fd\") " pod="kserve-ci-e2e-test/error-404-isvc-bdc39-predictor-5cb4dbc486-8vz74"
Sep 19 08:43:50.553616 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:50.553575    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:43:50.902472 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:50.902428    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-3bcba-predictor-8d6ff494-grfpb" podUID="7fe275c2-758a-46d1-985f-572ab2fc0f59" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.37:8643/healthz\": dial tcp 10.132.0.37:8643: connect: connection refused"
Sep 19 08:43:50.994019 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:50.993961    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b3a0e011-81cc-47b9-93bb-e3acb44417fd-proxy-tls\") pod \"error-404-isvc-bdc39-predictor-5cb4dbc486-8vz74\" (UID: \"b3a0e011-81cc-47b9-93bb-e3acb44417fd\") " pod="kserve-ci-e2e-test/error-404-isvc-bdc39-predictor-5cb4dbc486-8vz74"
Sep 19 08:43:50.996548 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:50.996521    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b3a0e011-81cc-47b9-93bb-e3acb44417fd-proxy-tls\") pod \"error-404-isvc-bdc39-predictor-5cb4dbc486-8vz74\" (UID: \"b3a0e011-81cc-47b9-93bb-e3acb44417fd\") " pod="kserve-ci-e2e-test/error-404-isvc-bdc39-predictor-5cb4dbc486-8vz74"
Sep 19 08:43:51.151866 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:51.151830    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-bdc39-predictor-5cb4dbc486-8vz74"
Sep 19 08:43:51.426375 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:51.426348    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-bdc39-predictor-5cb4dbc486-8vz74"]
Sep 19 08:43:51.428830 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:43:51.428802    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3a0e011_81cc_47b9_93bb_e3acb44417fd.slice/crio-01e3226ecddb4a4e232f94e176a62280f236c4fce6e5c46a6462a8344d8031e6 WatchSource:0}: Error finding container 01e3226ecddb4a4e232f94e176a62280f236c4fce6e5c46a6462a8344d8031e6: Status 404 returned error can't find the container with id 01e3226ecddb4a4e232f94e176a62280f236c4fce6e5c46a6462a8344d8031e6
Sep 19 08:43:52.277546 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:52.277509    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-bdc39-predictor-5cb4dbc486-8vz74" event={"ID":"b3a0e011-81cc-47b9-93bb-e3acb44417fd","Type":"ContainerStarted","Data":"09adb5c8427e8c5f7404c376b05a18321d0eb72396f421fb230c3f6116adfce9"}
Sep 19 08:43:52.277929 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:52.277567    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-bdc39-predictor-5cb4dbc486-8vz74" event={"ID":"b3a0e011-81cc-47b9-93bb-e3acb44417fd","Type":"ContainerStarted","Data":"191624c9956702bd00b329faddf2074818c8215f1d3cc46c8d3cc5cdaa14001f"}
Sep 19 08:43:52.277929 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:52.277588    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-bdc39-predictor-5cb4dbc486-8vz74" event={"ID":"b3a0e011-81cc-47b9-93bb-e3acb44417fd","Type":"ContainerStarted","Data":"01e3226ecddb4a4e232f94e176a62280f236c4fce6e5c46a6462a8344d8031e6"}
Sep 19 08:43:52.277929 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:52.277610    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-bdc39-predictor-5cb4dbc486-8vz74"
Sep 19 08:43:52.294749 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:52.294697    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-bdc39-predictor-5cb4dbc486-8vz74" podStartSLOduration=2.294682645 podStartE2EDuration="2.294682645s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 08:43:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 08:43:52.293660528 +0000 UTC m=+1416.670370393" watchObservedRunningTime="2026-09-19 08:43:52.294682645 +0000 UTC m=+1416.671392508"
Sep 19 08:43:53.281919 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:53.281887    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-bdc39-predictor-5cb4dbc486-8vz74"
Sep 19 08:43:53.283185 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:53.283155    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-bdc39-predictor-5cb4dbc486-8vz74" podUID="b3a0e011-81cc-47b9-93bb-e3acb44417fd" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.39:8080: connect: connection refused"
Sep 19 08:43:53.812195 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:53.812169    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-3bcba-predictor-8d6ff494-grfpb"
Sep 19 08:43:53.920121 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:53.920029    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r5698\" (UniqueName: \"kubernetes.io/projected/7fe275c2-758a-46d1-985f-572ab2fc0f59-kube-api-access-r5698\") pod \"7fe275c2-758a-46d1-985f-572ab2fc0f59\" (UID: \"7fe275c2-758a-46d1-985f-572ab2fc0f59\") "
Sep 19 08:43:53.920121 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:53.920086    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/7fe275c2-758a-46d1-985f-572ab2fc0f59-kube-rbac-proxy-sar-config\") pod \"7fe275c2-758a-46d1-985f-572ab2fc0f59\" (UID: \"7fe275c2-758a-46d1-985f-572ab2fc0f59\") "
Sep 19 08:43:53.920121 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:53.920125    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7fe275c2-758a-46d1-985f-572ab2fc0f59-proxy-tls\") pod \"7fe275c2-758a-46d1-985f-572ab2fc0f59\" (UID: \"7fe275c2-758a-46d1-985f-572ab2fc0f59\") "
Sep 19 08:43:53.920556 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:53.920529    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7fe275c2-758a-46d1-985f-572ab2fc0f59-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "7fe275c2-758a-46d1-985f-572ab2fc0f59" (UID: "7fe275c2-758a-46d1-985f-572ab2fc0f59"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 08:43:53.922225 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:53.922199    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7fe275c2-758a-46d1-985f-572ab2fc0f59-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "7fe275c2-758a-46d1-985f-572ab2fc0f59" (UID: "7fe275c2-758a-46d1-985f-572ab2fc0f59"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 08:43:53.922225 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:53.922211    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7fe275c2-758a-46d1-985f-572ab2fc0f59-kube-api-access-r5698" (OuterVolumeSpecName: "kube-api-access-r5698") pod "7fe275c2-758a-46d1-985f-572ab2fc0f59" (UID: "7fe275c2-758a-46d1-985f-572ab2fc0f59"). InnerVolumeSpecName "kube-api-access-r5698". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 08:43:54.020956 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:54.020918    2578 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7fe275c2-758a-46d1-985f-572ab2fc0f59-proxy-tls\") on node \"ip-10-0-139-157.ec2.internal\" DevicePath \"\""
Sep 19 08:43:54.020956 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:54.020950    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r5698\" (UniqueName: \"kubernetes.io/projected/7fe275c2-758a-46d1-985f-572ab2fc0f59-kube-api-access-r5698\") on node \"ip-10-0-139-157.ec2.internal\" DevicePath \"\""
Sep 19 08:43:54.020956 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:54.020960    2578 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/7fe275c2-758a-46d1-985f-572ab2fc0f59-kube-rbac-proxy-sar-config\") on node \"ip-10-0-139-157.ec2.internal\" DevicePath \"\""
Sep 19 08:43:54.287587 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:54.287549    2578 generic.go:356] "Generic (PLEG): container finished" podID="7fe275c2-758a-46d1-985f-572ab2fc0f59" containerID="988615d3dfd6d435f3c167413bb307c5dc16e9102baa21247941d4f01bcc1615" exitCode=0
Sep 19 08:43:54.288031 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:54.287598    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-3bcba-predictor-8d6ff494-grfpb"
Sep 19 08:43:54.288031 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:54.287627    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-3bcba-predictor-8d6ff494-grfpb" event={"ID":"7fe275c2-758a-46d1-985f-572ab2fc0f59","Type":"ContainerDied","Data":"988615d3dfd6d435f3c167413bb307c5dc16e9102baa21247941d4f01bcc1615"}
Sep 19 08:43:54.288031 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:54.287692    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-3bcba-predictor-8d6ff494-grfpb" event={"ID":"7fe275c2-758a-46d1-985f-572ab2fc0f59","Type":"ContainerDied","Data":"7c5fe398934ea50d6db9f6cb1cd9b8219bf1363cab50f7b5812ad365947898dc"}
Sep 19 08:43:54.288031 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:54.287720    2578 scope.go:117] "RemoveContainer" containerID="e784d733602463eab71f36145c8a004337278812c98c356e6e9b2ae070c0b73d"
Sep 19 08:43:54.288257 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:54.288153    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-bdc39-predictor-5cb4dbc486-8vz74" podUID="b3a0e011-81cc-47b9-93bb-e3acb44417fd" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.39:8080: connect: connection refused"
Sep 19 08:43:54.296383 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:54.296366    2578 scope.go:117] "RemoveContainer" containerID="988615d3dfd6d435f3c167413bb307c5dc16e9102baa21247941d4f01bcc1615"
Sep 19 08:43:54.304736 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:54.304716    2578 scope.go:117] "RemoveContainer" containerID="e784d733602463eab71f36145c8a004337278812c98c356e6e9b2ae070c0b73d"
Sep 19 08:43:54.305043 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:43:54.305020    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e784d733602463eab71f36145c8a004337278812c98c356e6e9b2ae070c0b73d\": container with ID starting with e784d733602463eab71f36145c8a004337278812c98c356e6e9b2ae070c0b73d not found: ID does not exist" containerID="e784d733602463eab71f36145c8a004337278812c98c356e6e9b2ae070c0b73d"
Sep 19 08:43:54.305117 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:54.305042    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-3bcba-predictor-8d6ff494-grfpb"]
Sep 19 08:43:54.305117 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:54.305056    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e784d733602463eab71f36145c8a004337278812c98c356e6e9b2ae070c0b73d"} err="failed to get container status \"e784d733602463eab71f36145c8a004337278812c98c356e6e9b2ae070c0b73d\": rpc error: code = NotFound desc = could not find container \"e784d733602463eab71f36145c8a004337278812c98c356e6e9b2ae070c0b73d\": container with ID starting with e784d733602463eab71f36145c8a004337278812c98c356e6e9b2ae070c0b73d not found: ID does not exist"
Sep 19 08:43:54.305117 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:54.305080    2578 scope.go:117] "RemoveContainer" containerID="988615d3dfd6d435f3c167413bb307c5dc16e9102baa21247941d4f01bcc1615"
Sep 19 08:43:54.305435 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:43:54.305414    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"988615d3dfd6d435f3c167413bb307c5dc16e9102baa21247941d4f01bcc1615\": container with ID starting with 988615d3dfd6d435f3c167413bb307c5dc16e9102baa21247941d4f01bcc1615 not found: ID does not exist" containerID="988615d3dfd6d435f3c167413bb307c5dc16e9102baa21247941d4f01bcc1615"
Sep 19 08:43:54.305481 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:54.305442    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"988615d3dfd6d435f3c167413bb307c5dc16e9102baa21247941d4f01bcc1615"} err="failed to get container status \"988615d3dfd6d435f3c167413bb307c5dc16e9102baa21247941d4f01bcc1615\": rpc error: code = NotFound desc = could not find container \"988615d3dfd6d435f3c167413bb307c5dc16e9102baa21247941d4f01bcc1615\": container with ID starting with 988615d3dfd6d435f3c167413bb307c5dc16e9102baa21247941d4f01bcc1615 not found: ID does not exist"
Sep 19 08:43:54.310187 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:54.310166    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-3bcba-predictor-8d6ff494-grfpb"]
Sep 19 08:43:56.113178 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:56.113144    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7fe275c2-758a-46d1-985f-572ab2fc0f59" path="/var/lib/kubelet/pods/7fe275c2-758a-46d1-985f-572ab2fc0f59/volumes"
Sep 19 08:43:57.099985 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:57.099944    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-fd725-predictor-5b7d599548-p7c8s" podUID="21d4e7f8-82db-4272-9ad8-933f96783cde" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.38:8080: connect: connection refused"
Sep 19 08:43:59.292780 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:59.292750    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-bdc39-predictor-5cb4dbc486-8vz74"
Sep 19 08:43:59.293268 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:43:59.293238    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-bdc39-predictor-5cb4dbc486-8vz74" podUID="b3a0e011-81cc-47b9-93bb-e3acb44417fd" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.39:8080: connect: connection refused"
Sep 19 08:44:07.100485 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:44:07.100449    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-fd725-predictor-5b7d599548-p7c8s"
Sep 19 08:44:09.293892 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:44:09.293803    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-bdc39-predictor-5cb4dbc486-8vz74" podUID="b3a0e011-81cc-47b9-93bb-e3acb44417fd" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.39:8080: connect: connection refused"
Sep 19 08:44:19.003813 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:44:19.003778    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:44:19.147012 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:44:19.146976    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:44:19.294212 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:44:19.294172    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-bdc39-predictor-5cb4dbc486-8vz74" podUID="b3a0e011-81cc-47b9-93bb-e3acb44417fd" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.39:8080: connect: connection refused"
Sep 19 08:44:29.293426 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:44:29.293383    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-bdc39-predictor-5cb4dbc486-8vz74" podUID="b3a0e011-81cc-47b9-93bb-e3acb44417fd" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.39:8080: connect: connection refused"
Sep 19 08:44:39.293794 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:44:39.293752    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-bdc39-predictor-5cb4dbc486-8vz74" podUID="b3a0e011-81cc-47b9-93bb-e3acb44417fd" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.39:8080: connect: connection refused"
Sep 19 08:44:49.294150 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:44:49.294119    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-bdc39-predictor-5cb4dbc486-8vz74"
Sep 19 08:44:59.729704 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:44:59.729652    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:44:59.783608 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:44:59.783579    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:45:00.195073 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:45:00.195037    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:45:00.336839 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:45:00.336802    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:45:16.093116 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:45:16.093077    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zd2zp_07ddac45-12a1-42e8-9566-55f08854c02e/ovn-acl-logging/0.log"
Sep 19 08:45:16.095824 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:45:16.095802    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zd2zp_07ddac45-12a1-42e8-9566-55f08854c02e/ovn-acl-logging/0.log"
Sep 19 08:50:16.118942 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:50:16.118905    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zd2zp_07ddac45-12a1-42e8-9566-55f08854c02e/ovn-acl-logging/0.log"
Sep 19 08:50:16.124059 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:50:16.124039    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zd2zp_07ddac45-12a1-42e8-9566-55f08854c02e/ovn-acl-logging/0.log"
Sep 19 08:52:34.349016 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:52:34.348976    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-fd725-predictor-5b7d599548-p7c8s"]
Sep 19 08:52:34.349658 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:52:34.349367    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-fd725-predictor-5b7d599548-p7c8s" podUID="21d4e7f8-82db-4272-9ad8-933f96783cde" containerName="kserve-container" containerID="cri-o://a27c98b4adb508a0cd1d8e0e98f6b4d806d5f139dbb4eccaa8b75cde08922bec" gracePeriod=30
Sep 19 08:52:34.349658 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:52:34.349434    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-fd725-predictor-5b7d599548-p7c8s" podUID="21d4e7f8-82db-4272-9ad8-933f96783cde" containerName="kube-rbac-proxy" containerID="cri-o://b18d84d2e89cf7c5d20ecc1171648b30207f2729db43bcdfabbab75c1f5c5486" gracePeriod=30
Sep 19 08:52:34.544845 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:52:34.544809    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-f0ae0-predictor-f7dc887f6-54dbg"]
Sep 19 08:52:34.545329 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:52:34.545310    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7fe275c2-758a-46d1-985f-572ab2fc0f59" containerName="kube-rbac-proxy"
Sep 19 08:52:34.545329 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:52:34.545330    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="7fe275c2-758a-46d1-985f-572ab2fc0f59" containerName="kube-rbac-proxy"
Sep 19 08:52:34.545497 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:52:34.545346    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7fe275c2-758a-46d1-985f-572ab2fc0f59" containerName="kserve-container"
Sep 19 08:52:34.545497 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:52:34.545354    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="7fe275c2-758a-46d1-985f-572ab2fc0f59" containerName="kserve-container"
Sep 19 08:52:34.545497 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:52:34.545458    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="7fe275c2-758a-46d1-985f-572ab2fc0f59" containerName="kserve-container"
Sep 19 08:52:34.545497 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:52:34.545472    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="7fe275c2-758a-46d1-985f-572ab2fc0f59" containerName="kube-rbac-proxy"
Sep 19 08:52:34.548668 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:52:34.548649    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-f0ae0-predictor-f7dc887f6-54dbg"
Sep 19 08:52:34.550975 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:52:34.550952    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-f0ae0-kube-rbac-proxy-sar-config\""
Sep 19 08:52:34.551075 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:52:34.551031    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-f0ae0-predictor-serving-cert\""
Sep 19 08:52:34.558548 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:52:34.558521    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-f0ae0-predictor-f7dc887f6-54dbg"]
Sep 19 08:52:34.564017 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:52:34.563995    2578 generic.go:356] "Generic (PLEG): container finished" podID="21d4e7f8-82db-4272-9ad8-933f96783cde" containerID="b18d84d2e89cf7c5d20ecc1171648b30207f2729db43bcdfabbab75c1f5c5486" exitCode=2
Sep 19 08:52:34.564136 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:52:34.564059    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-fd725-predictor-5b7d599548-p7c8s" event={"ID":"21d4e7f8-82db-4272-9ad8-933f96783cde","Type":"ContainerDied","Data":"b18d84d2e89cf7c5d20ecc1171648b30207f2729db43bcdfabbab75c1f5c5486"}
Sep 19 08:52:34.675930 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:52:34.675854    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/14c987d2-df51-4bd2-af2e-7f96cd319b04-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-f0ae0-predictor-f7dc887f6-54dbg\" (UID: \"14c987d2-df51-4bd2-af2e-7f96cd319b04\") " pod="kserve-ci-e2e-test/error-404-isvc-f0ae0-predictor-f7dc887f6-54dbg"
Sep 19 08:52:34.675930 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:52:34.675895    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/14c987d2-df51-4bd2-af2e-7f96cd319b04-proxy-tls\") pod \"error-404-isvc-f0ae0-predictor-f7dc887f6-54dbg\" (UID: \"14c987d2-df51-4bd2-af2e-7f96cd319b04\") " pod="kserve-ci-e2e-test/error-404-isvc-f0ae0-predictor-f7dc887f6-54dbg"
Sep 19 08:52:34.676106 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:52:34.676027    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2477f\" (UniqueName: \"kubernetes.io/projected/14c987d2-df51-4bd2-af2e-7f96cd319b04-kube-api-access-2477f\") pod \"error-404-isvc-f0ae0-predictor-f7dc887f6-54dbg\" (UID: \"14c987d2-df51-4bd2-af2e-7f96cd319b04\") " pod="kserve-ci-e2e-test/error-404-isvc-f0ae0-predictor-f7dc887f6-54dbg"
Sep 19 08:52:34.776723 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:52:34.776685    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2477f\" (UniqueName: \"kubernetes.io/projected/14c987d2-df51-4bd2-af2e-7f96cd319b04-kube-api-access-2477f\") pod \"error-404-isvc-f0ae0-predictor-f7dc887f6-54dbg\" (UID: \"14c987d2-df51-4bd2-af2e-7f96cd319b04\") " pod="kserve-ci-e2e-test/error-404-isvc-f0ae0-predictor-f7dc887f6-54dbg"
Sep 19 08:52:34.776887 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:52:34.776736    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/14c987d2-df51-4bd2-af2e-7f96cd319b04-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-f0ae0-predictor-f7dc887f6-54dbg\" (UID: \"14c987d2-df51-4bd2-af2e-7f96cd319b04\") " pod="kserve-ci-e2e-test/error-404-isvc-f0ae0-predictor-f7dc887f6-54dbg"
Sep 19 08:52:34.776887 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:52:34.776757    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/14c987d2-df51-4bd2-af2e-7f96cd319b04-proxy-tls\") pod \"error-404-isvc-f0ae0-predictor-f7dc887f6-54dbg\" (UID: \"14c987d2-df51-4bd2-af2e-7f96cd319b04\") " pod="kserve-ci-e2e-test/error-404-isvc-f0ae0-predictor-f7dc887f6-54dbg"
Sep 19 08:52:34.777517 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:52:34.777489    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/14c987d2-df51-4bd2-af2e-7f96cd319b04-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-f0ae0-predictor-f7dc887f6-54dbg\" (UID: \"14c987d2-df51-4bd2-af2e-7f96cd319b04\") " pod="kserve-ci-e2e-test/error-404-isvc-f0ae0-predictor-f7dc887f6-54dbg"
Sep 19 08:52:34.779243 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:52:34.779223    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/14c987d2-df51-4bd2-af2e-7f96cd319b04-proxy-tls\") pod \"error-404-isvc-f0ae0-predictor-f7dc887f6-54dbg\" (UID: \"14c987d2-df51-4bd2-af2e-7f96cd319b04\") " pod="kserve-ci-e2e-test/error-404-isvc-f0ae0-predictor-f7dc887f6-54dbg"
Sep 19 08:52:34.784653 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:52:34.784632    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2477f\" (UniqueName: \"kubernetes.io/projected/14c987d2-df51-4bd2-af2e-7f96cd319b04-kube-api-access-2477f\") pod \"error-404-isvc-f0ae0-predictor-f7dc887f6-54dbg\" (UID: \"14c987d2-df51-4bd2-af2e-7f96cd319b04\") " pod="kserve-ci-e2e-test/error-404-isvc-f0ae0-predictor-f7dc887f6-54dbg"
Sep 19 08:52:34.861439 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:52:34.861410    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-f0ae0-predictor-f7dc887f6-54dbg"
Sep 19 08:52:35.078288 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:52:35.078250    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:52:35.136354 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:52:35.136266    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-f0ae0-predictor-f7dc887f6-54dbg"]
Sep 19 08:52:35.139006 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:52:35.138981    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod14c987d2_df51_4bd2_af2e_7f96cd319b04.slice/crio-a66245c6ca44ddb8e7992fef3679fd220ddf161ba4f281c4f84ec73cce166394 WatchSource:0}: Error finding container a66245c6ca44ddb8e7992fef3679fd220ddf161ba4f281c4f84ec73cce166394: Status 404 returned error can't find the container with id a66245c6ca44ddb8e7992fef3679fd220ddf161ba4f281c4f84ec73cce166394
Sep 19 08:52:35.570499 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:52:35.570465    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-f0ae0-predictor-f7dc887f6-54dbg" event={"ID":"14c987d2-df51-4bd2-af2e-7f96cd319b04","Type":"ContainerStarted","Data":"75204a90af7ffefae3b9279ec0ee751608d1e1af9a981cb375e6336c75fa655f"}
Sep 19 08:52:35.570928 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:52:35.570529    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-f0ae0-predictor-f7dc887f6-54dbg" event={"ID":"14c987d2-df51-4bd2-af2e-7f96cd319b04","Type":"ContainerStarted","Data":"6a23dc6009a4bce2315aae850220bb7c37b0e830136f0271d56a19a7fd64b4b8"}
Sep 19 08:52:35.570928 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:52:35.570668    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-f0ae0-predictor-f7dc887f6-54dbg"
Sep 19 08:52:35.570928 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:52:35.570699    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-f0ae0-predictor-f7dc887f6-54dbg"
Sep 19 08:52:35.570928 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:52:35.570715    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-f0ae0-predictor-f7dc887f6-54dbg" event={"ID":"14c987d2-df51-4bd2-af2e-7f96cd319b04","Type":"ContainerStarted","Data":"a66245c6ca44ddb8e7992fef3679fd220ddf161ba4f281c4f84ec73cce166394"}
Sep 19 08:52:35.572002 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:52:35.571978    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-f0ae0-predictor-f7dc887f6-54dbg" podUID="14c987d2-df51-4bd2-af2e-7f96cd319b04" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.40:8080: connect: connection refused"
Sep 19 08:52:35.593922 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:52:35.593866    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-f0ae0-predictor-f7dc887f6-54dbg" podStartSLOduration=1.593848521 podStartE2EDuration="1.593848521s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 08:52:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 08:52:35.590335054 +0000 UTC m=+1939.967044915" watchObservedRunningTime="2026-09-19 08:52:35.593848521 +0000 UTC m=+1939.970558383"
Sep 19 08:52:36.216883 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:52:36.216841    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:52:36.575413 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:52:36.575376    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-f0ae0-predictor-f7dc887f6-54dbg" podUID="14c987d2-df51-4bd2-af2e-7f96cd319b04" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.40:8080: connect: connection refused"
Sep 19 08:52:37.095325 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:52:37.095259    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-fd725-predictor-5b7d599548-p7c8s" podUID="21d4e7f8-82db-4272-9ad8-933f96783cde" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.38:8643/healthz\": dial tcp 10.132.0.38:8643: connect: connection refused"
Sep 19 08:52:37.099816 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:52:37.099784    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-fd725-predictor-5b7d599548-p7c8s" podUID="21d4e7f8-82db-4272-9ad8-933f96783cde" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.38:8080: connect: connection refused"
Sep 19 08:52:37.896204 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:52:37.896175    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-fd725-predictor-5b7d599548-p7c8s"
Sep 19 08:52:38.005630 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:52:38.005529    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/21d4e7f8-82db-4272-9ad8-933f96783cde-proxy-tls\") pod \"21d4e7f8-82db-4272-9ad8-933f96783cde\" (UID: \"21d4e7f8-82db-4272-9ad8-933f96783cde\") "
Sep 19 08:52:38.005630 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:52:38.005582    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8nkmc\" (UniqueName: \"kubernetes.io/projected/21d4e7f8-82db-4272-9ad8-933f96783cde-kube-api-access-8nkmc\") pod \"21d4e7f8-82db-4272-9ad8-933f96783cde\" (UID: \"21d4e7f8-82db-4272-9ad8-933f96783cde\") "
Sep 19 08:52:38.005863 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:52:38.005656    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/21d4e7f8-82db-4272-9ad8-933f96783cde-kube-rbac-proxy-sar-config\") pod \"21d4e7f8-82db-4272-9ad8-933f96783cde\" (UID: \"21d4e7f8-82db-4272-9ad8-933f96783cde\") "
Sep 19 08:52:38.006018 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:52:38.005986    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/21d4e7f8-82db-4272-9ad8-933f96783cde-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "21d4e7f8-82db-4272-9ad8-933f96783cde" (UID: "21d4e7f8-82db-4272-9ad8-933f96783cde"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 08:52:38.007690 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:52:38.007661    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/21d4e7f8-82db-4272-9ad8-933f96783cde-kube-api-access-8nkmc" (OuterVolumeSpecName: "kube-api-access-8nkmc") pod "21d4e7f8-82db-4272-9ad8-933f96783cde" (UID: "21d4e7f8-82db-4272-9ad8-933f96783cde"). InnerVolumeSpecName "kube-api-access-8nkmc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 08:52:38.007773 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:52:38.007748    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/21d4e7f8-82db-4272-9ad8-933f96783cde-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "21d4e7f8-82db-4272-9ad8-933f96783cde" (UID: "21d4e7f8-82db-4272-9ad8-933f96783cde"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 08:52:38.107233 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:52:38.107187    2578 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/21d4e7f8-82db-4272-9ad8-933f96783cde-kube-rbac-proxy-sar-config\") on node \"ip-10-0-139-157.ec2.internal\" DevicePath \"\""
Sep 19 08:52:38.107233 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:52:38.107227    2578 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/21d4e7f8-82db-4272-9ad8-933f96783cde-proxy-tls\") on node \"ip-10-0-139-157.ec2.internal\" DevicePath \"\""
Sep 19 08:52:38.107497 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:52:38.107243    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8nkmc\" (UniqueName: \"kubernetes.io/projected/21d4e7f8-82db-4272-9ad8-933f96783cde-kube-api-access-8nkmc\") on node \"ip-10-0-139-157.ec2.internal\" DevicePath \"\""
Sep 19 08:52:38.585609 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:52:38.585576    2578 generic.go:356] "Generic (PLEG): container finished" podID="21d4e7f8-82db-4272-9ad8-933f96783cde" containerID="a27c98b4adb508a0cd1d8e0e98f6b4d806d5f139dbb4eccaa8b75cde08922bec" exitCode=0
Sep 19 08:52:38.585774 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:52:38.585629    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-fd725-predictor-5b7d599548-p7c8s"
Sep 19 08:52:38.585774 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:52:38.585634    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-fd725-predictor-5b7d599548-p7c8s" event={"ID":"21d4e7f8-82db-4272-9ad8-933f96783cde","Type":"ContainerDied","Data":"a27c98b4adb508a0cd1d8e0e98f6b4d806d5f139dbb4eccaa8b75cde08922bec"}
Sep 19 08:52:38.585774 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:52:38.585696    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-fd725-predictor-5b7d599548-p7c8s" event={"ID":"21d4e7f8-82db-4272-9ad8-933f96783cde","Type":"ContainerDied","Data":"4b908f5494370c2e6c4a4d240dafd91b5d1bcc632a36f6634395cac01ff5e205"}
Sep 19 08:52:38.585774 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:52:38.585731    2578 scope.go:117] "RemoveContainer" containerID="b18d84d2e89cf7c5d20ecc1171648b30207f2729db43bcdfabbab75c1f5c5486"
Sep 19 08:52:38.593912 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:52:38.593895    2578 scope.go:117] "RemoveContainer" containerID="a27c98b4adb508a0cd1d8e0e98f6b4d806d5f139dbb4eccaa8b75cde08922bec"
Sep 19 08:52:38.601411 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:52:38.601387    2578 scope.go:117] "RemoveContainer" containerID="b18d84d2e89cf7c5d20ecc1171648b30207f2729db43bcdfabbab75c1f5c5486"
Sep 19 08:52:38.601696 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:52:38.601679    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b18d84d2e89cf7c5d20ecc1171648b30207f2729db43bcdfabbab75c1f5c5486\": container with ID starting with b18d84d2e89cf7c5d20ecc1171648b30207f2729db43bcdfabbab75c1f5c5486 not found: ID does not exist" containerID="b18d84d2e89cf7c5d20ecc1171648b30207f2729db43bcdfabbab75c1f5c5486"
Sep 19 08:52:38.601751 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:52:38.601706    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b18d84d2e89cf7c5d20ecc1171648b30207f2729db43bcdfabbab75c1f5c5486"} err="failed to get container status \"b18d84d2e89cf7c5d20ecc1171648b30207f2729db43bcdfabbab75c1f5c5486\": rpc error: code = NotFound desc = could not find container \"b18d84d2e89cf7c5d20ecc1171648b30207f2729db43bcdfabbab75c1f5c5486\": container with ID starting with b18d84d2e89cf7c5d20ecc1171648b30207f2729db43bcdfabbab75c1f5c5486 not found: ID does not exist"
Sep 19 08:52:38.601751 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:52:38.601723    2578 scope.go:117] "RemoveContainer" containerID="a27c98b4adb508a0cd1d8e0e98f6b4d806d5f139dbb4eccaa8b75cde08922bec"
Sep 19 08:52:38.601996 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:52:38.601972    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a27c98b4adb508a0cd1d8e0e98f6b4d806d5f139dbb4eccaa8b75cde08922bec\": container with ID starting with a27c98b4adb508a0cd1d8e0e98f6b4d806d5f139dbb4eccaa8b75cde08922bec not found: ID does not exist" containerID="a27c98b4adb508a0cd1d8e0e98f6b4d806d5f139dbb4eccaa8b75cde08922bec"
Sep 19 08:52:38.602059 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:52:38.601999    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a27c98b4adb508a0cd1d8e0e98f6b4d806d5f139dbb4eccaa8b75cde08922bec"} err="failed to get container status \"a27c98b4adb508a0cd1d8e0e98f6b4d806d5f139dbb4eccaa8b75cde08922bec\": rpc error: code = NotFound desc = could not find container \"a27c98b4adb508a0cd1d8e0e98f6b4d806d5f139dbb4eccaa8b75cde08922bec\": container with ID starting with a27c98b4adb508a0cd1d8e0e98f6b4d806d5f139dbb4eccaa8b75cde08922bec not found: ID does not exist"
Sep 19 08:52:38.602829 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:52:38.602804    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-fd725-predictor-5b7d599548-p7c8s"]
Sep 19 08:52:38.604956 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:52:38.604932    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-fd725-predictor-5b7d599548-p7c8s"]
Sep 19 08:52:40.112537 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:52:40.112502    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="21d4e7f8-82db-4272-9ad8-933f96783cde" path="/var/lib/kubelet/pods/21d4e7f8-82db-4272-9ad8-933f96783cde/volumes"
Sep 19 08:52:41.579967 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:52:41.579939    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-f0ae0-predictor-f7dc887f6-54dbg"
Sep 19 08:52:41.580392 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:52:41.580323    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-f0ae0-predictor-f7dc887f6-54dbg" podUID="14c987d2-df51-4bd2-af2e-7f96cd319b04" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.40:8080: connect: connection refused"
Sep 19 08:52:51.580924 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:52:51.580885    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-f0ae0-predictor-f7dc887f6-54dbg" podUID="14c987d2-df51-4bd2-af2e-7f96cd319b04" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.40:8080: connect: connection refused"
Sep 19 08:53:01.580747 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:53:01.580708    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-f0ae0-predictor-f7dc887f6-54dbg" podUID="14c987d2-df51-4bd2-af2e-7f96cd319b04" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.40:8080: connect: connection refused"
Sep 19 08:53:11.468064 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:53:11.467966    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:53:11.580419 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:53:11.580380    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-f0ae0-predictor-f7dc887f6-54dbg" podUID="14c987d2-df51-4bd2-af2e-7f96cd319b04" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.40:8080: connect: connection refused"
Sep 19 08:53:15.397630 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:53:15.397592    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:53:15.490580 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:53:15.490540    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:53:15.499814 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:53:15.499784    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-bdc39-predictor-5cb4dbc486-8vz74"]
Sep 19 08:53:15.500097 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:53:15.500072    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-bdc39-predictor-5cb4dbc486-8vz74" podUID="b3a0e011-81cc-47b9-93bb-e3acb44417fd" containerName="kserve-container" containerID="cri-o://191624c9956702bd00b329faddf2074818c8215f1d3cc46c8d3cc5cdaa14001f" gracePeriod=30
Sep 19 08:53:15.500182 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:53:15.500086    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-bdc39-predictor-5cb4dbc486-8vz74" podUID="b3a0e011-81cc-47b9-93bb-e3acb44417fd" containerName="kube-rbac-proxy" containerID="cri-o://09adb5c8427e8c5f7404c376b05a18321d0eb72396f421fb230c3f6116adfce9" gracePeriod=30
Sep 19 08:53:15.564667 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:53:15.564630    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:53:15.628503 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:53:15.628464    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:53:15.677980 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:53:15.677908    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-bde64-predictor-5dbbbb5dc4-z8dp7"]
Sep 19 08:53:15.678272 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:53:15.678259    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="21d4e7f8-82db-4272-9ad8-933f96783cde" containerName="kube-rbac-proxy"
Sep 19 08:53:15.678345 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:53:15.678275    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="21d4e7f8-82db-4272-9ad8-933f96783cde" containerName="kube-rbac-proxy"
Sep 19 08:53:15.678345 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:53:15.678288    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="21d4e7f8-82db-4272-9ad8-933f96783cde" containerName="kserve-container"
Sep 19 08:53:15.678345 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:53:15.678294    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="21d4e7f8-82db-4272-9ad8-933f96783cde" containerName="kserve-container"
Sep 19 08:53:15.678448 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:53:15.678368    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="21d4e7f8-82db-4272-9ad8-933f96783cde" containerName="kserve-container"
Sep 19 08:53:15.678448 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:53:15.678377    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="21d4e7f8-82db-4272-9ad8-933f96783cde" containerName="kube-rbac-proxy"
Sep 19 08:53:15.681278 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:53:15.681261    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-bde64-predictor-5dbbbb5dc4-z8dp7"
Sep 19 08:53:15.683699 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:53:15.683673    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-bde64-predictor-serving-cert\""
Sep 19 08:53:15.683829 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:53:15.683675    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-bde64-kube-rbac-proxy-sar-config\""
Sep 19 08:53:15.690567 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:53:15.690543    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-bde64-predictor-5dbbbb5dc4-z8dp7"]
Sep 19 08:53:15.692471 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:53:15.692336    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:53:15.744515 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:53:15.744478    2578 generic.go:356] "Generic (PLEG): container finished" podID="b3a0e011-81cc-47b9-93bb-e3acb44417fd" containerID="09adb5c8427e8c5f7404c376b05a18321d0eb72396f421fb230c3f6116adfce9" exitCode=2
Sep 19 08:53:15.744663 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:53:15.744553    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-bdc39-predictor-5cb4dbc486-8vz74" event={"ID":"b3a0e011-81cc-47b9-93bb-e3acb44417fd","Type":"ContainerDied","Data":"09adb5c8427e8c5f7404c376b05a18321d0eb72396f421fb230c3f6116adfce9"}
Sep 19 08:53:15.797637 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:53:15.797610    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1c5b9211-82b1-4882-a223-fcf439673a4a-proxy-tls\") pod \"error-404-isvc-bde64-predictor-5dbbbb5dc4-z8dp7\" (UID: \"1c5b9211-82b1-4882-a223-fcf439673a4a\") " pod="kserve-ci-e2e-test/error-404-isvc-bde64-predictor-5dbbbb5dc4-z8dp7"
Sep 19 08:53:15.797788 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:53:15.797651    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/1c5b9211-82b1-4882-a223-fcf439673a4a-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-bde64-predictor-5dbbbb5dc4-z8dp7\" (UID: \"1c5b9211-82b1-4882-a223-fcf439673a4a\") " pod="kserve-ci-e2e-test/error-404-isvc-bde64-predictor-5dbbbb5dc4-z8dp7"
Sep 19 08:53:15.797788 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:53:15.797701    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ccsvw\" (UniqueName: \"kubernetes.io/projected/1c5b9211-82b1-4882-a223-fcf439673a4a-kube-api-access-ccsvw\") pod \"error-404-isvc-bde64-predictor-5dbbbb5dc4-z8dp7\" (UID: \"1c5b9211-82b1-4882-a223-fcf439673a4a\") " pod="kserve-ci-e2e-test/error-404-isvc-bde64-predictor-5dbbbb5dc4-z8dp7"
Sep 19 08:53:15.899104 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:53:15.899065    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1c5b9211-82b1-4882-a223-fcf439673a4a-proxy-tls\") pod \"error-404-isvc-bde64-predictor-5dbbbb5dc4-z8dp7\" (UID: \"1c5b9211-82b1-4882-a223-fcf439673a4a\") " pod="kserve-ci-e2e-test/error-404-isvc-bde64-predictor-5dbbbb5dc4-z8dp7"
Sep 19 08:53:15.899292 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:53:15.899113    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/1c5b9211-82b1-4882-a223-fcf439673a4a-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-bde64-predictor-5dbbbb5dc4-z8dp7\" (UID: \"1c5b9211-82b1-4882-a223-fcf439673a4a\") " pod="kserve-ci-e2e-test/error-404-isvc-bde64-predictor-5dbbbb5dc4-z8dp7"
Sep 19 08:53:15.899292 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:53:15.899152    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ccsvw\" (UniqueName: \"kubernetes.io/projected/1c5b9211-82b1-4882-a223-fcf439673a4a-kube-api-access-ccsvw\") pod \"error-404-isvc-bde64-predictor-5dbbbb5dc4-z8dp7\" (UID: \"1c5b9211-82b1-4882-a223-fcf439673a4a\") " pod="kserve-ci-e2e-test/error-404-isvc-bde64-predictor-5dbbbb5dc4-z8dp7"
Sep 19 08:53:15.899292 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:53:15.899216    2578 secret.go:189] Couldn't get secret kserve-ci-e2e-test/error-404-isvc-bde64-predictor-serving-cert: secret "error-404-isvc-bde64-predictor-serving-cert" not found
Sep 19 08:53:15.899292 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:53:15.899317    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1c5b9211-82b1-4882-a223-fcf439673a4a-proxy-tls podName:1c5b9211-82b1-4882-a223-fcf439673a4a nodeName:}" failed. No retries permitted until 2026-09-19 08:53:16.399273967 +0000 UTC m=+1980.775983813 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/1c5b9211-82b1-4882-a223-fcf439673a4a-proxy-tls") pod "error-404-isvc-bde64-predictor-5dbbbb5dc4-z8dp7" (UID: "1c5b9211-82b1-4882-a223-fcf439673a4a") : secret "error-404-isvc-bde64-predictor-serving-cert" not found
Sep 19 08:53:15.899856 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:53:15.899837    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/1c5b9211-82b1-4882-a223-fcf439673a4a-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-bde64-predictor-5dbbbb5dc4-z8dp7\" (UID: \"1c5b9211-82b1-4882-a223-fcf439673a4a\") " pod="kserve-ci-e2e-test/error-404-isvc-bde64-predictor-5dbbbb5dc4-z8dp7"
Sep 19 08:53:15.910393 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:53:15.910364    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ccsvw\" (UniqueName: \"kubernetes.io/projected/1c5b9211-82b1-4882-a223-fcf439673a4a-kube-api-access-ccsvw\") pod \"error-404-isvc-bde64-predictor-5dbbbb5dc4-z8dp7\" (UID: \"1c5b9211-82b1-4882-a223-fcf439673a4a\") " pod="kserve-ci-e2e-test/error-404-isvc-bde64-predictor-5dbbbb5dc4-z8dp7"
Sep 19 08:53:16.127169 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:53:16.127130    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:53:16.228385 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:53:16.228357    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:53:16.403360 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:53:16.403277    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1c5b9211-82b1-4882-a223-fcf439673a4a-proxy-tls\") pod \"error-404-isvc-bde64-predictor-5dbbbb5dc4-z8dp7\" (UID: \"1c5b9211-82b1-4882-a223-fcf439673a4a\") " pod="kserve-ci-e2e-test/error-404-isvc-bde64-predictor-5dbbbb5dc4-z8dp7"
Sep 19 08:53:16.405716 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:53:16.405692    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1c5b9211-82b1-4882-a223-fcf439673a4a-proxy-tls\") pod \"error-404-isvc-bde64-predictor-5dbbbb5dc4-z8dp7\" (UID: \"1c5b9211-82b1-4882-a223-fcf439673a4a\") " pod="kserve-ci-e2e-test/error-404-isvc-bde64-predictor-5dbbbb5dc4-z8dp7"
Sep 19 08:53:16.593316 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:53:16.593273    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-bde64-predictor-5dbbbb5dc4-z8dp7"
Sep 19 08:53:16.864895 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:53:16.864869    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-bde64-predictor-5dbbbb5dc4-z8dp7"]
Sep 19 08:53:16.866842 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:53:16.866815    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c5b9211_82b1_4882_a223_fcf439673a4a.slice/crio-cd7021bc8f5307f0b5cf2745a94a0e63ebf17c867f3ea09d4b34ab78390db154 WatchSource:0}: Error finding container cd7021bc8f5307f0b5cf2745a94a0e63ebf17c867f3ea09d4b34ab78390db154: Status 404 returned error can't find the container with id cd7021bc8f5307f0b5cf2745a94a0e63ebf17c867f3ea09d4b34ab78390db154
Sep 19 08:53:17.754614 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:53:17.754579    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-bde64-predictor-5dbbbb5dc4-z8dp7" event={"ID":"1c5b9211-82b1-4882-a223-fcf439673a4a","Type":"ContainerStarted","Data":"93a9c753fb22ebf648bfd5240e74c3a1f0d6a27806fb0ddaa5ada3c6157c26c0"}
Sep 19 08:53:17.754995 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:53:17.754647    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-bde64-predictor-5dbbbb5dc4-z8dp7" event={"ID":"1c5b9211-82b1-4882-a223-fcf439673a4a","Type":"ContainerStarted","Data":"dda0af359fe02401b57f3be3892936e960090733f747ba1e9f737e79a8c305f9"}
Sep 19 08:53:17.754995 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:53:17.754675    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-bde64-predictor-5dbbbb5dc4-z8dp7" event={"ID":"1c5b9211-82b1-4882-a223-fcf439673a4a","Type":"ContainerStarted","Data":"cd7021bc8f5307f0b5cf2745a94a0e63ebf17c867f3ea09d4b34ab78390db154"}
Sep 19 08:53:17.754995 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:53:17.754705    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-bde64-predictor-5dbbbb5dc4-z8dp7"
Sep 19 08:53:17.773703 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:53:17.773655    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-bde64-predictor-5dbbbb5dc4-z8dp7" podStartSLOduration=2.773637967 podStartE2EDuration="2.773637967s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 08:53:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 08:53:17.771758638 +0000 UTC m=+1982.148468499" watchObservedRunningTime="2026-09-19 08:53:17.773637967 +0000 UTC m=+1982.150347830"
Sep 19 08:53:18.760370 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:53:18.760335    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-bde64-predictor-5dbbbb5dc4-z8dp7"
Sep 19 08:53:18.761765 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:53:18.761735    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-bde64-predictor-5dbbbb5dc4-z8dp7" podUID="1c5b9211-82b1-4882-a223-fcf439673a4a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.41:8080: connect: connection refused"
Sep 19 08:53:19.053824 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:53:19.053798    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-bdc39-predictor-5cb4dbc486-8vz74"
Sep 19 08:53:19.124215 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:53:19.124182    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4n9x8\" (UniqueName: \"kubernetes.io/projected/b3a0e011-81cc-47b9-93bb-e3acb44417fd-kube-api-access-4n9x8\") pod \"b3a0e011-81cc-47b9-93bb-e3acb44417fd\" (UID: \"b3a0e011-81cc-47b9-93bb-e3acb44417fd\") "
Sep 19 08:53:19.124425 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:53:19.124222    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b3a0e011-81cc-47b9-93bb-e3acb44417fd-kube-rbac-proxy-sar-config\") pod \"b3a0e011-81cc-47b9-93bb-e3acb44417fd\" (UID: \"b3a0e011-81cc-47b9-93bb-e3acb44417fd\") "
Sep 19 08:53:19.124425 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:53:19.124266    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b3a0e011-81cc-47b9-93bb-e3acb44417fd-proxy-tls\") pod \"b3a0e011-81cc-47b9-93bb-e3acb44417fd\" (UID: \"b3a0e011-81cc-47b9-93bb-e3acb44417fd\") "
Sep 19 08:53:19.124588 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:53:19.124565    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b3a0e011-81cc-47b9-93bb-e3acb44417fd-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "b3a0e011-81cc-47b9-93bb-e3acb44417fd" (UID: "b3a0e011-81cc-47b9-93bb-e3acb44417fd"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 08:53:19.126348 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:53:19.126320    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b3a0e011-81cc-47b9-93bb-e3acb44417fd-kube-api-access-4n9x8" (OuterVolumeSpecName: "kube-api-access-4n9x8") pod "b3a0e011-81cc-47b9-93bb-e3acb44417fd" (UID: "b3a0e011-81cc-47b9-93bb-e3acb44417fd"). InnerVolumeSpecName "kube-api-access-4n9x8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 08:53:19.126348 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:53:19.126336    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b3a0e011-81cc-47b9-93bb-e3acb44417fd-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "b3a0e011-81cc-47b9-93bb-e3acb44417fd" (UID: "b3a0e011-81cc-47b9-93bb-e3acb44417fd"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 08:53:19.225760 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:53:19.225721    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4n9x8\" (UniqueName: \"kubernetes.io/projected/b3a0e011-81cc-47b9-93bb-e3acb44417fd-kube-api-access-4n9x8\") on node \"ip-10-0-139-157.ec2.internal\" DevicePath \"\""
Sep 19 08:53:19.225760 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:53:19.225752    2578 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b3a0e011-81cc-47b9-93bb-e3acb44417fd-kube-rbac-proxy-sar-config\") on node \"ip-10-0-139-157.ec2.internal\" DevicePath \"\""
Sep 19 08:53:19.225760 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:53:19.225762    2578 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b3a0e011-81cc-47b9-93bb-e3acb44417fd-proxy-tls\") on node \"ip-10-0-139-157.ec2.internal\" DevicePath \"\""
Sep 19 08:53:19.766682 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:53:19.766651    2578 generic.go:356] "Generic (PLEG): container finished" podID="b3a0e011-81cc-47b9-93bb-e3acb44417fd" containerID="191624c9956702bd00b329faddf2074818c8215f1d3cc46c8d3cc5cdaa14001f" exitCode=0
Sep 19 08:53:19.767070 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:53:19.766703    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-bdc39-predictor-5cb4dbc486-8vz74"
Sep 19 08:53:19.767070 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:53:19.766740    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-bdc39-predictor-5cb4dbc486-8vz74" event={"ID":"b3a0e011-81cc-47b9-93bb-e3acb44417fd","Type":"ContainerDied","Data":"191624c9956702bd00b329faddf2074818c8215f1d3cc46c8d3cc5cdaa14001f"}
Sep 19 08:53:19.767070 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:53:19.766805    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-bdc39-predictor-5cb4dbc486-8vz74" event={"ID":"b3a0e011-81cc-47b9-93bb-e3acb44417fd","Type":"ContainerDied","Data":"01e3226ecddb4a4e232f94e176a62280f236c4fce6e5c46a6462a8344d8031e6"}
Sep 19 08:53:19.767070 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:53:19.766834    2578 scope.go:117] "RemoveContainer" containerID="09adb5c8427e8c5f7404c376b05a18321d0eb72396f421fb230c3f6116adfce9"
Sep 19 08:53:19.767446 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:53:19.767422    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-bde64-predictor-5dbbbb5dc4-z8dp7" podUID="1c5b9211-82b1-4882-a223-fcf439673a4a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.41:8080: connect: connection refused"
Sep 19 08:53:19.775757 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:53:19.775743    2578 scope.go:117] "RemoveContainer" containerID="191624c9956702bd00b329faddf2074818c8215f1d3cc46c8d3cc5cdaa14001f"
Sep 19 08:53:19.782982 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:53:19.782963    2578 scope.go:117] "RemoveContainer" containerID="09adb5c8427e8c5f7404c376b05a18321d0eb72396f421fb230c3f6116adfce9"
Sep 19 08:53:19.783252 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:53:19.783235    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"09adb5c8427e8c5f7404c376b05a18321d0eb72396f421fb230c3f6116adfce9\": container with ID starting with 09adb5c8427e8c5f7404c376b05a18321d0eb72396f421fb230c3f6116adfce9 not found: ID does not exist" containerID="09adb5c8427e8c5f7404c376b05a18321d0eb72396f421fb230c3f6116adfce9"
Sep 19 08:53:19.783292 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:53:19.783261    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"09adb5c8427e8c5f7404c376b05a18321d0eb72396f421fb230c3f6116adfce9"} err="failed to get container status \"09adb5c8427e8c5f7404c376b05a18321d0eb72396f421fb230c3f6116adfce9\": rpc error: code = NotFound desc = could not find container \"09adb5c8427e8c5f7404c376b05a18321d0eb72396f421fb230c3f6116adfce9\": container with ID starting with 09adb5c8427e8c5f7404c376b05a18321d0eb72396f421fb230c3f6116adfce9 not found: ID does not exist"
Sep 19 08:53:19.783292 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:53:19.783279    2578 scope.go:117] "RemoveContainer" containerID="191624c9956702bd00b329faddf2074818c8215f1d3cc46c8d3cc5cdaa14001f"
Sep 19 08:53:19.783556 ip-10-0-139-157 kubenswrapper[2578]: E0919 08:53:19.783535    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"191624c9956702bd00b329faddf2074818c8215f1d3cc46c8d3cc5cdaa14001f\": container with ID starting with 191624c9956702bd00b329faddf2074818c8215f1d3cc46c8d3cc5cdaa14001f not found: ID does not exist" containerID="191624c9956702bd00b329faddf2074818c8215f1d3cc46c8d3cc5cdaa14001f"
Sep 19 08:53:19.783592 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:53:19.783562    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"191624c9956702bd00b329faddf2074818c8215f1d3cc46c8d3cc5cdaa14001f"} err="failed to get container status \"191624c9956702bd00b329faddf2074818c8215f1d3cc46c8d3cc5cdaa14001f\": rpc error: code = NotFound desc = could not find container \"191624c9956702bd00b329faddf2074818c8215f1d3cc46c8d3cc5cdaa14001f\": container with ID starting with 191624c9956702bd00b329faddf2074818c8215f1d3cc46c8d3cc5cdaa14001f not found: ID does not exist"
Sep 19 08:53:19.788317 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:53:19.788279    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-bdc39-predictor-5cb4dbc486-8vz74"]
Sep 19 08:53:19.793968 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:53:19.793946    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-bdc39-predictor-5cb4dbc486-8vz74"]
Sep 19 08:53:20.113062 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:53:20.112983    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b3a0e011-81cc-47b9-93bb-e3acb44417fd" path="/var/lib/kubelet/pods/b3a0e011-81cc-47b9-93bb-e3acb44417fd/volumes"
Sep 19 08:53:21.581163 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:53:21.581121    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-f0ae0-predictor-f7dc887f6-54dbg" podUID="14c987d2-df51-4bd2-af2e-7f96cd319b04" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.40:8080: connect: connection refused"
Sep 19 08:53:24.771318 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:53:24.771277    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-bde64-predictor-5dbbbb5dc4-z8dp7"
Sep 19 08:53:24.771866 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:53:24.771838    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-bde64-predictor-5dbbbb5dc4-z8dp7" podUID="1c5b9211-82b1-4882-a223-fcf439673a4a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.41:8080: connect: connection refused"
Sep 19 08:53:31.581529 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:53:31.581486    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-f0ae0-predictor-f7dc887f6-54dbg"
Sep 19 08:53:34.771991 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:53:34.771953    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-bde64-predictor-5dbbbb5dc4-z8dp7" podUID="1c5b9211-82b1-4882-a223-fcf439673a4a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.41:8080: connect: connection refused"
Sep 19 08:53:44.439863 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:53:44.439775    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:53:44.534100 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:53:44.534059    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:53:44.635097 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:53:44.635061    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:53:44.772235 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:53:44.772204    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-bde64-predictor-5dbbbb5dc4-z8dp7" podUID="1c5b9211-82b1-4882-a223-fcf439673a4a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.41:8080: connect: connection refused"
Sep 19 08:53:54.771741 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:53:54.771700    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-bde64-predictor-5dbbbb5dc4-z8dp7" podUID="1c5b9211-82b1-4882-a223-fcf439673a4a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.41:8080: connect: connection refused"
Sep 19 08:53:55.372517 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:53:55.372477    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:53:55.459854 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:53:55.459819    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:53:55.471658 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:53:55.471623    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-f0ae0-predictor-f7dc887f6-54dbg"]
Sep 19 08:53:55.471896 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:53:55.471859    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-f0ae0-predictor-f7dc887f6-54dbg" podUID="14c987d2-df51-4bd2-af2e-7f96cd319b04" containerName="kserve-container" containerID="cri-o://6a23dc6009a4bce2315aae850220bb7c37b0e830136f0271d56a19a7fd64b4b8" gracePeriod=30
Sep 19 08:53:55.471985 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:53:55.471914    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-f0ae0-predictor-f7dc887f6-54dbg" podUID="14c987d2-df51-4bd2-af2e-7f96cd319b04" containerName="kube-rbac-proxy" containerID="cri-o://75204a90af7ffefae3b9279ec0ee751608d1e1af9a981cb375e6336c75fa655f" gracePeriod=30
Sep 19 08:53:55.527822 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:53:55.527786    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:53:55.629633 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:53:55.629548    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:53:55.644196 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:53:55.644167    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-de19a-predictor-69c5f5cb44-v96vd"]
Sep 19 08:53:55.644510 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:53:55.644497    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b3a0e011-81cc-47b9-93bb-e3acb44417fd" containerName="kserve-container"
Sep 19 08:53:55.644510 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:53:55.644511    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="b3a0e011-81cc-47b9-93bb-e3acb44417fd" containerName="kserve-container"
Sep 19 08:53:55.644654 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:53:55.644520    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b3a0e011-81cc-47b9-93bb-e3acb44417fd" containerName="kube-rbac-proxy"
Sep 19 08:53:55.644654 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:53:55.644534    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="b3a0e011-81cc-47b9-93bb-e3acb44417fd" containerName="kube-rbac-proxy"
Sep 19 08:53:55.644654 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:53:55.644604    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="b3a0e011-81cc-47b9-93bb-e3acb44417fd" containerName="kserve-container"
Sep 19 08:53:55.644654 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:53:55.644612    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="b3a0e011-81cc-47b9-93bb-e3acb44417fd" containerName="kube-rbac-proxy"
Sep 19 08:53:55.648358 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:53:55.648339    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-de19a-predictor-69c5f5cb44-v96vd"
Sep 19 08:53:55.650597 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:53:55.650580    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-de19a-predictor-serving-cert\""
Sep 19 08:53:55.650783 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:53:55.650761    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-de19a-kube-rbac-proxy-sar-config\""
Sep 19 08:53:55.659760 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:53:55.659737    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-de19a-predictor-69c5f5cb44-v96vd"]
Sep 19 08:53:55.724840 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:53:55.724809    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/fa828c7e-3029-4864-92bc-a53d7fad2505-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-de19a-predictor-69c5f5cb44-v96vd\" (UID: \"fa828c7e-3029-4864-92bc-a53d7fad2505\") " pod="kserve-ci-e2e-test/error-404-isvc-de19a-predictor-69c5f5cb44-v96vd"
Sep 19 08:53:55.725024 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:53:55.724863    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/fa828c7e-3029-4864-92bc-a53d7fad2505-proxy-tls\") pod \"error-404-isvc-de19a-predictor-69c5f5cb44-v96vd\" (UID: \"fa828c7e-3029-4864-92bc-a53d7fad2505\") " pod="kserve-ci-e2e-test/error-404-isvc-de19a-predictor-69c5f5cb44-v96vd"
Sep 19 08:53:55.725024 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:53:55.724926    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b8whx\" (UniqueName: \"kubernetes.io/projected/fa828c7e-3029-4864-92bc-a53d7fad2505-kube-api-access-b8whx\") pod \"error-404-isvc-de19a-predictor-69c5f5cb44-v96vd\" (UID: \"fa828c7e-3029-4864-92bc-a53d7fad2505\") " pod="kserve-ci-e2e-test/error-404-isvc-de19a-predictor-69c5f5cb44-v96vd"
Sep 19 08:53:55.726561 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:53:55.726527    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:53:55.825911 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:53:55.825872    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/fa828c7e-3029-4864-92bc-a53d7fad2505-proxy-tls\") pod \"error-404-isvc-de19a-predictor-69c5f5cb44-v96vd\" (UID: \"fa828c7e-3029-4864-92bc-a53d7fad2505\") " pod="kserve-ci-e2e-test/error-404-isvc-de19a-predictor-69c5f5cb44-v96vd"
Sep 19 08:53:55.826410 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:53:55.825919    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b8whx\" (UniqueName: \"kubernetes.io/projected/fa828c7e-3029-4864-92bc-a53d7fad2505-kube-api-access-b8whx\") pod \"error-404-isvc-de19a-predictor-69c5f5cb44-v96vd\" (UID: \"fa828c7e-3029-4864-92bc-a53d7fad2505\") " pod="kserve-ci-e2e-test/error-404-isvc-de19a-predictor-69c5f5cb44-v96vd"
Sep 19 08:53:55.826410 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:53:55.825993    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/fa828c7e-3029-4864-92bc-a53d7fad2505-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-de19a-predictor-69c5f5cb44-v96vd\" (UID: \"fa828c7e-3029-4864-92bc-a53d7fad2505\") " pod="kserve-ci-e2e-test/error-404-isvc-de19a-predictor-69c5f5cb44-v96vd"
Sep 19 08:53:55.826759 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:53:55.826735    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/fa828c7e-3029-4864-92bc-a53d7fad2505-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-de19a-predictor-69c5f5cb44-v96vd\" (UID: \"fa828c7e-3029-4864-92bc-a53d7fad2505\") " pod="kserve-ci-e2e-test/error-404-isvc-de19a-predictor-69c5f5cb44-v96vd"
Sep 19 08:53:55.827596 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:53:55.827562    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:53:55.828534 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:53:55.828507    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/fa828c7e-3029-4864-92bc-a53d7fad2505-proxy-tls\") pod \"error-404-isvc-de19a-predictor-69c5f5cb44-v96vd\" (UID: \"fa828c7e-3029-4864-92bc-a53d7fad2505\") " pod="kserve-ci-e2e-test/error-404-isvc-de19a-predictor-69c5f5cb44-v96vd"
Sep 19 08:53:55.833105 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:53:55.833084    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b8whx\" (UniqueName: \"kubernetes.io/projected/fa828c7e-3029-4864-92bc-a53d7fad2505-kube-api-access-b8whx\") pod \"error-404-isvc-de19a-predictor-69c5f5cb44-v96vd\" (UID: \"fa828c7e-3029-4864-92bc-a53d7fad2505\") " pod="kserve-ci-e2e-test/error-404-isvc-de19a-predictor-69c5f5cb44-v96vd"
Sep 19 08:53:55.886889 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:53:55.886821    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:53:55.936413 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:53:55.936387    2578 generic.go:356] "Generic (PLEG): container finished" podID="14c987d2-df51-4bd2-af2e-7f96cd319b04" containerID="75204a90af7ffefae3b9279ec0ee751608d1e1af9a981cb375e6336c75fa655f" exitCode=2
Sep 19 08:53:55.936522 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:53:55.936438    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-f0ae0-predictor-f7dc887f6-54dbg" event={"ID":"14c987d2-df51-4bd2-af2e-7f96cd319b04","Type":"ContainerDied","Data":"75204a90af7ffefae3b9279ec0ee751608d1e1af9a981cb375e6336c75fa655f"}
Sep 19 08:53:55.959880 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:53:55.959848    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-de19a-predictor-69c5f5cb44-v96vd"
Sep 19 08:53:56.230220 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:53:56.230183    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-de19a-predictor-69c5f5cb44-v96vd"]
Sep 19 08:53:56.232549 ip-10-0-139-157 kubenswrapper[2578]: W0919 08:53:56.232516    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa828c7e_3029_4864_92bc_a53d7fad2505.slice/crio-31cf185f9b9a3f5a24a1a9071f2bdfcfb8b2c647371a33f7341711c3788e504f WatchSource:0}: Error finding container 31cf185f9b9a3f5a24a1a9071f2bdfcfb8b2c647371a33f7341711c3788e504f: Status 404 returned error can't find the container with id 31cf185f9b9a3f5a24a1a9071f2bdfcfb8b2c647371a33f7341711c3788e504f
Sep 19 08:53:56.576412 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:53:56.576373    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-f0ae0-predictor-f7dc887f6-54dbg" podUID="14c987d2-df51-4bd2-af2e-7f96cd319b04" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.40:8643/healthz\": dial tcp 10.132.0.40:8643: connect: connection refused"
Sep 19 08:53:56.942612 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:53:56.942525    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-de19a-predictor-69c5f5cb44-v96vd" event={"ID":"fa828c7e-3029-4864-92bc-a53d7fad2505","Type":"ContainerStarted","Data":"57c412a01ae4caaaa145b8193758f274dc2252646eb6affe0791bfed1943dc20"}
Sep 19 08:53:56.942612 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:53:56.942580    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-de19a-predictor-69c5f5cb44-v96vd" event={"ID":"fa828c7e-3029-4864-92bc-a53d7fad2505","Type":"ContainerStarted","Data":"787deec1609cc80d5a35a3e4137cbb870f0e3c3ff4552fe17241a886c99be917"}
Sep 19 08:53:56.942612 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:53:56.942599    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-de19a-predictor-69c5f5cb44-v96vd" event={"ID":"fa828c7e-3029-4864-92bc-a53d7fad2505","Type":"ContainerStarted","Data":"31cf185f9b9a3f5a24a1a9071f2bdfcfb8b2c647371a33f7341711c3788e504f"}
Sep 19 08:53:56.943052 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:53:56.942675    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-de19a-predictor-69c5f5cb44-v96vd"
Sep 19 08:53:56.962002 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:53:56.961944    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-de19a-predictor-69c5f5cb44-v96vd" podStartSLOduration=1.961928157 podStartE2EDuration="1.961928157s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 08:53:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 08:53:56.96010589 +0000 UTC m=+2021.336815746" watchObservedRunningTime="2026-09-19 08:53:56.961928157 +0000 UTC m=+2021.338638018"
Sep 19 08:53:57.947233 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:53:57.947203    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-de19a-predictor-69c5f5cb44-v96vd"
Sep 19 08:53:57.948399 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:53:57.948367    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-de19a-predictor-69c5f5cb44-v96vd" podUID="fa828c7e-3029-4864-92bc-a53d7fad2505" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.42:8080: connect: connection refused"
Sep 19 08:53:58.955514 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:53:58.955473    2578 generic.go:356] "Generic (PLEG): container finished" podID="14c987d2-df51-4bd2-af2e-7f96cd319b04" containerID="6a23dc6009a4bce2315aae850220bb7c37b0e830136f0271d56a19a7fd64b4b8" exitCode=0
Sep 19 08:53:58.955890 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:53:58.955560    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-f0ae0-predictor-f7dc887f6-54dbg" event={"ID":"14c987d2-df51-4bd2-af2e-7f96cd319b04","Type":"ContainerDied","Data":"6a23dc6009a4bce2315aae850220bb7c37b0e830136f0271d56a19a7fd64b4b8"}
Sep 19 08:53:58.955986 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:53:58.955961    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-de19a-predictor-69c5f5cb44-v96vd" podUID="fa828c7e-3029-4864-92bc-a53d7fad2505" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.42:8080: connect: connection refused"
Sep 19 08:53:59.012749 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:53:59.012725    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-f0ae0-predictor-f7dc887f6-54dbg"
Sep 19 08:53:59.048831 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:53:59.048803    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/14c987d2-df51-4bd2-af2e-7f96cd319b04-kube-rbac-proxy-sar-config\") pod \"14c987d2-df51-4bd2-af2e-7f96cd319b04\" (UID: \"14c987d2-df51-4bd2-af2e-7f96cd319b04\") "
Sep 19 08:53:59.048971 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:53:59.048846    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/14c987d2-df51-4bd2-af2e-7f96cd319b04-proxy-tls\") pod \"14c987d2-df51-4bd2-af2e-7f96cd319b04\" (UID: \"14c987d2-df51-4bd2-af2e-7f96cd319b04\") "
Sep 19 08:53:59.048971 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:53:59.048884    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2477f\" (UniqueName: \"kubernetes.io/projected/14c987d2-df51-4bd2-af2e-7f96cd319b04-kube-api-access-2477f\") pod \"14c987d2-df51-4bd2-af2e-7f96cd319b04\" (UID: \"14c987d2-df51-4bd2-af2e-7f96cd319b04\") "
Sep 19 08:53:59.049211 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:53:59.049189    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/14c987d2-df51-4bd2-af2e-7f96cd319b04-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "14c987d2-df51-4bd2-af2e-7f96cd319b04" (UID: "14c987d2-df51-4bd2-af2e-7f96cd319b04"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 08:53:59.051034 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:53:59.051011    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/14c987d2-df51-4bd2-af2e-7f96cd319b04-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "14c987d2-df51-4bd2-af2e-7f96cd319b04" (UID: "14c987d2-df51-4bd2-af2e-7f96cd319b04"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 08:53:59.051119 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:53:59.051096    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/14c987d2-df51-4bd2-af2e-7f96cd319b04-kube-api-access-2477f" (OuterVolumeSpecName: "kube-api-access-2477f") pod "14c987d2-df51-4bd2-af2e-7f96cd319b04" (UID: "14c987d2-df51-4bd2-af2e-7f96cd319b04"). InnerVolumeSpecName "kube-api-access-2477f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 08:53:59.149950 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:53:59.149867    2578 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/14c987d2-df51-4bd2-af2e-7f96cd319b04-kube-rbac-proxy-sar-config\") on node \"ip-10-0-139-157.ec2.internal\" DevicePath \"\""
Sep 19 08:53:59.149950 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:53:59.149892    2578 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/14c987d2-df51-4bd2-af2e-7f96cd319b04-proxy-tls\") on node \"ip-10-0-139-157.ec2.internal\" DevicePath \"\""
Sep 19 08:53:59.149950 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:53:59.149902    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2477f\" (UniqueName: \"kubernetes.io/projected/14c987d2-df51-4bd2-af2e-7f96cd319b04-kube-api-access-2477f\") on node \"ip-10-0-139-157.ec2.internal\" DevicePath \"\""
Sep 19 08:53:59.961900 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:53:59.961856    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-f0ae0-predictor-f7dc887f6-54dbg" event={"ID":"14c987d2-df51-4bd2-af2e-7f96cd319b04","Type":"ContainerDied","Data":"a66245c6ca44ddb8e7992fef3679fd220ddf161ba4f281c4f84ec73cce166394"}
Sep 19 08:53:59.961900 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:53:59.961888    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-f0ae0-predictor-f7dc887f6-54dbg"
Sep 19 08:53:59.962379 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:53:59.961946    2578 scope.go:117] "RemoveContainer" containerID="75204a90af7ffefae3b9279ec0ee751608d1e1af9a981cb375e6336c75fa655f"
Sep 19 08:53:59.970691 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:53:59.970676    2578 scope.go:117] "RemoveContainer" containerID="6a23dc6009a4bce2315aae850220bb7c37b0e830136f0271d56a19a7fd64b4b8"
Sep 19 08:53:59.982733 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:53:59.982710    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-f0ae0-predictor-f7dc887f6-54dbg"]
Sep 19 08:53:59.986481 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:53:59.986461    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-f0ae0-predictor-f7dc887f6-54dbg"]
Sep 19 08:54:00.113595 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:54:00.113569    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="14c987d2-df51-4bd2-af2e-7f96cd319b04" path="/var/lib/kubelet/pods/14c987d2-df51-4bd2-af2e-7f96cd319b04/volumes"
Sep 19 08:54:03.360966 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:54:03.360920    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:54:03.961139 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:54:03.961107    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-de19a-predictor-69c5f5cb44-v96vd"
Sep 19 08:54:03.961651 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:54:03.961621    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-de19a-predictor-69c5f5cb44-v96vd" podUID="fa828c7e-3029-4864-92bc-a53d7fad2505" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.42:8080: connect: connection refused"
Sep 19 08:54:04.772546 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:54:04.772501    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-bde64-predictor-5dbbbb5dc4-z8dp7" podUID="1c5b9211-82b1-4882-a223-fcf439673a4a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.41:8080: connect: connection refused"
Sep 19 08:54:13.962183 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:54:13.962144    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-de19a-predictor-69c5f5cb44-v96vd" podUID="fa828c7e-3029-4864-92bc-a53d7fad2505" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.42:8080: connect: connection refused"
Sep 19 08:54:14.772417 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:54:14.772388    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-bde64-predictor-5dbbbb5dc4-z8dp7"
Sep 19 08:54:23.962333 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:54:23.962280    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-de19a-predictor-69c5f5cb44-v96vd" podUID="fa828c7e-3029-4864-92bc-a53d7fad2505" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.42:8080: connect: connection refused"
Sep 19 08:54:25.620054 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:54:25.620021    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:54:26.539686 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:54:26.539647    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:54:33.961938 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:54:33.961896    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-de19a-predictor-69c5f5cb44-v96vd" podUID="fa828c7e-3029-4864-92bc-a53d7fad2505" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.42:8080: connect: connection refused"
Sep 19 08:54:43.962613 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:54:43.962525    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-de19a-predictor-69c5f5cb44-v96vd" podUID="fa828c7e-3029-4864-92bc-a53d7fad2505" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.42:8080: connect: connection refused"
Sep 19 08:54:53.963032 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:54:53.963000    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-de19a-predictor-69c5f5cb44-v96vd"
Sep 19 08:55:05.589645 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:55:05.589585    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:55:05.853997 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:55:05.853902    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:55:16.143708 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:55:16.143678    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zd2zp_07ddac45-12a1-42e8-9566-55f08854c02e/ovn-acl-logging/0.log"
Sep 19 08:55:16.149730 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:55:16.149710    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zd2zp_07ddac45-12a1-42e8-9566-55f08854c02e/ovn-acl-logging/0.log"
Sep 19 08:55:23.639557 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:55:23.639523    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 08:59:54.514723 ip-10-0-139-157 kubenswrapper[2578]: I0919 08:59:54.514678    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 09:00:16.168410 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:00:16.168381    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zd2zp_07ddac45-12a1-42e8-9566-55f08854c02e/ovn-acl-logging/0.log"
Sep 19 09:00:16.176451 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:00:16.176430    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zd2zp_07ddac45-12a1-42e8-9566-55f08854c02e/ovn-acl-logging/0.log"
Sep 19 09:03:20.786403 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:03:20.786361    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 09:03:20.877469 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:03:20.877432    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 09:03:20.883576 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:03:20.883548    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-de19a-predictor-69c5f5cb44-v96vd"]
Sep 19 09:03:20.883843 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:03:20.883798    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-de19a-predictor-69c5f5cb44-v96vd" podUID="fa828c7e-3029-4864-92bc-a53d7fad2505" containerName="kserve-container" containerID="cri-o://787deec1609cc80d5a35a3e4137cbb870f0e3c3ff4552fe17241a886c99be917" gracePeriod=30
Sep 19 09:03:20.883939 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:03:20.883798    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-de19a-predictor-69c5f5cb44-v96vd" podUID="fa828c7e-3029-4864-92bc-a53d7fad2505" containerName="kube-rbac-proxy" containerID="cri-o://57c412a01ae4caaaa145b8193758f274dc2252646eb6affe0791bfed1943dc20" gracePeriod=30
Sep 19 09:03:20.938568 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:03:20.938529    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 09:03:21.400556 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:03:21.400521    2578 generic.go:356] "Generic (PLEG): container finished" podID="fa828c7e-3029-4864-92bc-a53d7fad2505" containerID="57c412a01ae4caaaa145b8193758f274dc2252646eb6affe0791bfed1943dc20" exitCode=2
Sep 19 09:03:21.400759 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:03:21.400573    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-de19a-predictor-69c5f5cb44-v96vd" event={"ID":"fa828c7e-3029-4864-92bc-a53d7fad2505","Type":"ContainerDied","Data":"57c412a01ae4caaaa145b8193758f274dc2252646eb6affe0791bfed1943dc20"}
Sep 19 09:03:23.957326 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:03:23.957252    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-de19a-predictor-69c5f5cb44-v96vd" podUID="fa828c7e-3029-4864-92bc-a53d7fad2505" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.42:8643/healthz\": dial tcp 10.132.0.42:8643: connect: connection refused"
Sep 19 09:03:23.961967 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:03:23.961724    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-de19a-predictor-69c5f5cb44-v96vd" podUID="fa828c7e-3029-4864-92bc-a53d7fad2505" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.42:8080: connect: connection refused"
Sep 19 09:03:24.129749 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:03:24.129726    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-de19a-predictor-69c5f5cb44-v96vd"
Sep 19 09:03:24.243816 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:03:24.243736    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/fa828c7e-3029-4864-92bc-a53d7fad2505-kube-rbac-proxy-sar-config\") pod \"fa828c7e-3029-4864-92bc-a53d7fad2505\" (UID: \"fa828c7e-3029-4864-92bc-a53d7fad2505\") "
Sep 19 09:03:24.243816 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:03:24.243799    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b8whx\" (UniqueName: \"kubernetes.io/projected/fa828c7e-3029-4864-92bc-a53d7fad2505-kube-api-access-b8whx\") pod \"fa828c7e-3029-4864-92bc-a53d7fad2505\" (UID: \"fa828c7e-3029-4864-92bc-a53d7fad2505\") "
Sep 19 09:03:24.244030 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:03:24.243823    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/fa828c7e-3029-4864-92bc-a53d7fad2505-proxy-tls\") pod \"fa828c7e-3029-4864-92bc-a53d7fad2505\" (UID: \"fa828c7e-3029-4864-92bc-a53d7fad2505\") "
Sep 19 09:03:24.244155 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:03:24.244133    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fa828c7e-3029-4864-92bc-a53d7fad2505-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "fa828c7e-3029-4864-92bc-a53d7fad2505" (UID: "fa828c7e-3029-4864-92bc-a53d7fad2505"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 09:03:24.245884 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:03:24.245841    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fa828c7e-3029-4864-92bc-a53d7fad2505-kube-api-access-b8whx" (OuterVolumeSpecName: "kube-api-access-b8whx") pod "fa828c7e-3029-4864-92bc-a53d7fad2505" (UID: "fa828c7e-3029-4864-92bc-a53d7fad2505"). InnerVolumeSpecName "kube-api-access-b8whx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 09:03:24.245884 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:03:24.245847    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fa828c7e-3029-4864-92bc-a53d7fad2505-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "fa828c7e-3029-4864-92bc-a53d7fad2505" (UID: "fa828c7e-3029-4864-92bc-a53d7fad2505"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 09:03:24.344401 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:03:24.344371    2578 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/fa828c7e-3029-4864-92bc-a53d7fad2505-kube-rbac-proxy-sar-config\") on node \"ip-10-0-139-157.ec2.internal\" DevicePath \"\""
Sep 19 09:03:24.344401 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:03:24.344395    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b8whx\" (UniqueName: \"kubernetes.io/projected/fa828c7e-3029-4864-92bc-a53d7fad2505-kube-api-access-b8whx\") on node \"ip-10-0-139-157.ec2.internal\" DevicePath \"\""
Sep 19 09:03:24.344401 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:03:24.344406    2578 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/fa828c7e-3029-4864-92bc-a53d7fad2505-proxy-tls\") on node \"ip-10-0-139-157.ec2.internal\" DevicePath \"\""
Sep 19 09:03:24.415525 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:03:24.415494    2578 generic.go:356] "Generic (PLEG): container finished" podID="fa828c7e-3029-4864-92bc-a53d7fad2505" containerID="787deec1609cc80d5a35a3e4137cbb870f0e3c3ff4552fe17241a886c99be917" exitCode=0
Sep 19 09:03:24.415667 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:03:24.415544    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-de19a-predictor-69c5f5cb44-v96vd"
Sep 19 09:03:24.415667 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:03:24.415582    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-de19a-predictor-69c5f5cb44-v96vd" event={"ID":"fa828c7e-3029-4864-92bc-a53d7fad2505","Type":"ContainerDied","Data":"787deec1609cc80d5a35a3e4137cbb870f0e3c3ff4552fe17241a886c99be917"}
Sep 19 09:03:24.415667 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:03:24.415649    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-de19a-predictor-69c5f5cb44-v96vd" event={"ID":"fa828c7e-3029-4864-92bc-a53d7fad2505","Type":"ContainerDied","Data":"31cf185f9b9a3f5a24a1a9071f2bdfcfb8b2c647371a33f7341711c3788e504f"}
Sep 19 09:03:24.415768 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:03:24.415686    2578 scope.go:117] "RemoveContainer" containerID="57c412a01ae4caaaa145b8193758f274dc2252646eb6affe0791bfed1943dc20"
Sep 19 09:03:24.423954 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:03:24.423935    2578 scope.go:117] "RemoveContainer" containerID="787deec1609cc80d5a35a3e4137cbb870f0e3c3ff4552fe17241a886c99be917"
Sep 19 09:03:24.430813 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:03:24.430795    2578 scope.go:117] "RemoveContainer" containerID="57c412a01ae4caaaa145b8193758f274dc2252646eb6affe0791bfed1943dc20"
Sep 19 09:03:24.431053 ip-10-0-139-157 kubenswrapper[2578]: E0919 09:03:24.431035    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"57c412a01ae4caaaa145b8193758f274dc2252646eb6affe0791bfed1943dc20\": container with ID starting with 57c412a01ae4caaaa145b8193758f274dc2252646eb6affe0791bfed1943dc20 not found: ID does not exist" containerID="57c412a01ae4caaaa145b8193758f274dc2252646eb6affe0791bfed1943dc20"
Sep 19 09:03:24.431098 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:03:24.431062    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"57c412a01ae4caaaa145b8193758f274dc2252646eb6affe0791bfed1943dc20"} err="failed to get container status \"57c412a01ae4caaaa145b8193758f274dc2252646eb6affe0791bfed1943dc20\": rpc error: code = NotFound desc = could not find container \"57c412a01ae4caaaa145b8193758f274dc2252646eb6affe0791bfed1943dc20\": container with ID starting with 57c412a01ae4caaaa145b8193758f274dc2252646eb6affe0791bfed1943dc20 not found: ID does not exist"
Sep 19 09:03:24.431098 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:03:24.431077    2578 scope.go:117] "RemoveContainer" containerID="787deec1609cc80d5a35a3e4137cbb870f0e3c3ff4552fe17241a886c99be917"
Sep 19 09:03:24.431341 ip-10-0-139-157 kubenswrapper[2578]: E0919 09:03:24.431323    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"787deec1609cc80d5a35a3e4137cbb870f0e3c3ff4552fe17241a886c99be917\": container with ID starting with 787deec1609cc80d5a35a3e4137cbb870f0e3c3ff4552fe17241a886c99be917 not found: ID does not exist" containerID="787deec1609cc80d5a35a3e4137cbb870f0e3c3ff4552fe17241a886c99be917"
Sep 19 09:03:24.431408 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:03:24.431346    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"787deec1609cc80d5a35a3e4137cbb870f0e3c3ff4552fe17241a886c99be917"} err="failed to get container status \"787deec1609cc80d5a35a3e4137cbb870f0e3c3ff4552fe17241a886c99be917\": rpc error: code = NotFound desc = could not find container \"787deec1609cc80d5a35a3e4137cbb870f0e3c3ff4552fe17241a886c99be917\": container with ID starting with 787deec1609cc80d5a35a3e4137cbb870f0e3c3ff4552fe17241a886c99be917 not found: ID does not exist"
Sep 19 09:03:24.436995 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:03:24.436974    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-de19a-predictor-69c5f5cb44-v96vd"]
Sep 19 09:03:24.440546 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:03:24.440522    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-de19a-predictor-69c5f5cb44-v96vd"]
Sep 19 09:03:26.113062 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:03:26.113029    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fa828c7e-3029-4864-92bc-a53d7fad2505" path="/var/lib/kubelet/pods/fa828c7e-3029-4864-92bc-a53d7fad2505/volumes"
Sep 19 09:03:37.532533 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:03:37.532498    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 09:05:16.193100 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:05:16.193022    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zd2zp_07ddac45-12a1-42e8-9566-55f08854c02e/ovn-acl-logging/0.log"
Sep 19 09:05:16.202092 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:05:16.202072    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zd2zp_07ddac45-12a1-42e8-9566-55f08854c02e/ovn-acl-logging/0.log"
Sep 19 09:08:37.533748 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:08:37.533668    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 09:09:04.692844 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:09:04.692782    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 09:10:16.216430 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:10:16.216403    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zd2zp_07ddac45-12a1-42e8-9566-55f08854c02e/ovn-acl-logging/0.log"
Sep 19 09:10:16.227251 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:10:16.227231    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zd2zp_07ddac45-12a1-42e8-9566-55f08854c02e/ovn-acl-logging/0.log"
Sep 19 09:10:45.647756 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:10:45.647701    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 09:10:45.756849 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:10:45.756815    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 09:10:45.763158 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:10:45.763132    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-bde64-predictor-5dbbbb5dc4-z8dp7"]
Sep 19 09:10:45.763394 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:10:45.763372    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-bde64-predictor-5dbbbb5dc4-z8dp7" podUID="1c5b9211-82b1-4882-a223-fcf439673a4a" containerName="kserve-container" containerID="cri-o://dda0af359fe02401b57f3be3892936e960090733f747ba1e9f737e79a8c305f9" gracePeriod=30
Sep 19 09:10:45.763465 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:10:45.763412    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-bde64-predictor-5dbbbb5dc4-z8dp7" podUID="1c5b9211-82b1-4882-a223-fcf439673a4a" containerName="kube-rbac-proxy" containerID="cri-o://93a9c753fb22ebf648bfd5240e74c3a1f0d6a27806fb0ddaa5ada3c6157c26c0" gracePeriod=30
Sep 19 09:10:45.856268 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:10:45.856232    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 09:10:46.305628 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:10:46.305597    2578 generic.go:356] "Generic (PLEG): container finished" podID="1c5b9211-82b1-4882-a223-fcf439673a4a" containerID="93a9c753fb22ebf648bfd5240e74c3a1f0d6a27806fb0ddaa5ada3c6157c26c0" exitCode=2
Sep 19 09:10:46.305819 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:10:46.305670    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-bde64-predictor-5dbbbb5dc4-z8dp7" event={"ID":"1c5b9211-82b1-4882-a223-fcf439673a4a","Type":"ContainerDied","Data":"93a9c753fb22ebf648bfd5240e74c3a1f0d6a27806fb0ddaa5ada3c6157c26c0"}
Sep 19 09:10:46.586358 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:10:46.586263    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 09:10:46.662464 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:10:46.662431    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 09:10:49.002838 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:10:49.002812    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-bde64-predictor-5dbbbb5dc4-z8dp7"
Sep 19 09:10:49.139866 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:10:49.139771    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ccsvw\" (UniqueName: \"kubernetes.io/projected/1c5b9211-82b1-4882-a223-fcf439673a4a-kube-api-access-ccsvw\") pod \"1c5b9211-82b1-4882-a223-fcf439673a4a\" (UID: \"1c5b9211-82b1-4882-a223-fcf439673a4a\") "
Sep 19 09:10:49.139866 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:10:49.139825    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1c5b9211-82b1-4882-a223-fcf439673a4a-proxy-tls\") pod \"1c5b9211-82b1-4882-a223-fcf439673a4a\" (UID: \"1c5b9211-82b1-4882-a223-fcf439673a4a\") "
Sep 19 09:10:49.139866 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:10:49.139864    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/1c5b9211-82b1-4882-a223-fcf439673a4a-kube-rbac-proxy-sar-config\") pod \"1c5b9211-82b1-4882-a223-fcf439673a4a\" (UID: \"1c5b9211-82b1-4882-a223-fcf439673a4a\") "
Sep 19 09:10:49.140242 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:10:49.140219    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1c5b9211-82b1-4882-a223-fcf439673a4a-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "1c5b9211-82b1-4882-a223-fcf439673a4a" (UID: "1c5b9211-82b1-4882-a223-fcf439673a4a"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 09:10:49.141893 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:10:49.141862    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1c5b9211-82b1-4882-a223-fcf439673a4a-kube-api-access-ccsvw" (OuterVolumeSpecName: "kube-api-access-ccsvw") pod "1c5b9211-82b1-4882-a223-fcf439673a4a" (UID: "1c5b9211-82b1-4882-a223-fcf439673a4a"). InnerVolumeSpecName "kube-api-access-ccsvw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 09:10:49.141993 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:10:49.141926    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1c5b9211-82b1-4882-a223-fcf439673a4a-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "1c5b9211-82b1-4882-a223-fcf439673a4a" (UID: "1c5b9211-82b1-4882-a223-fcf439673a4a"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 09:10:49.241185 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:10:49.241156    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ccsvw\" (UniqueName: \"kubernetes.io/projected/1c5b9211-82b1-4882-a223-fcf439673a4a-kube-api-access-ccsvw\") on node \"ip-10-0-139-157.ec2.internal\" DevicePath \"\""
Sep 19 09:10:49.241185 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:10:49.241182    2578 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1c5b9211-82b1-4882-a223-fcf439673a4a-proxy-tls\") on node \"ip-10-0-139-157.ec2.internal\" DevicePath \"\""
Sep 19 09:10:49.241371 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:10:49.241194    2578 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/1c5b9211-82b1-4882-a223-fcf439673a4a-kube-rbac-proxy-sar-config\") on node \"ip-10-0-139-157.ec2.internal\" DevicePath \"\""
Sep 19 09:10:49.320248 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:10:49.320219    2578 generic.go:356] "Generic (PLEG): container finished" podID="1c5b9211-82b1-4882-a223-fcf439673a4a" containerID="dda0af359fe02401b57f3be3892936e960090733f747ba1e9f737e79a8c305f9" exitCode=0
Sep 19 09:10:49.320419 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:10:49.320280    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-bde64-predictor-5dbbbb5dc4-z8dp7"
Sep 19 09:10:49.320419 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:10:49.320321    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-bde64-predictor-5dbbbb5dc4-z8dp7" event={"ID":"1c5b9211-82b1-4882-a223-fcf439673a4a","Type":"ContainerDied","Data":"dda0af359fe02401b57f3be3892936e960090733f747ba1e9f737e79a8c305f9"}
Sep 19 09:10:49.320419 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:10:49.320381    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-bde64-predictor-5dbbbb5dc4-z8dp7" event={"ID":"1c5b9211-82b1-4882-a223-fcf439673a4a","Type":"ContainerDied","Data":"cd7021bc8f5307f0b5cf2745a94a0e63ebf17c867f3ea09d4b34ab78390db154"}
Sep 19 09:10:49.320419 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:10:49.320409    2578 scope.go:117] "RemoveContainer" containerID="93a9c753fb22ebf648bfd5240e74c3a1f0d6a27806fb0ddaa5ada3c6157c26c0"
Sep 19 09:10:49.328498 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:10:49.328480    2578 scope.go:117] "RemoveContainer" containerID="dda0af359fe02401b57f3be3892936e960090733f747ba1e9f737e79a8c305f9"
Sep 19 09:10:49.335675 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:10:49.335630    2578 scope.go:117] "RemoveContainer" containerID="93a9c753fb22ebf648bfd5240e74c3a1f0d6a27806fb0ddaa5ada3c6157c26c0"
Sep 19 09:10:49.335888 ip-10-0-139-157 kubenswrapper[2578]: E0919 09:10:49.335867    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"93a9c753fb22ebf648bfd5240e74c3a1f0d6a27806fb0ddaa5ada3c6157c26c0\": container with ID starting with 93a9c753fb22ebf648bfd5240e74c3a1f0d6a27806fb0ddaa5ada3c6157c26c0 not found: ID does not exist" containerID="93a9c753fb22ebf648bfd5240e74c3a1f0d6a27806fb0ddaa5ada3c6157c26c0"
Sep 19 09:10:49.335974 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:10:49.335891    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"93a9c753fb22ebf648bfd5240e74c3a1f0d6a27806fb0ddaa5ada3c6157c26c0"} err="failed to get container status \"93a9c753fb22ebf648bfd5240e74c3a1f0d6a27806fb0ddaa5ada3c6157c26c0\": rpc error: code = NotFound desc = could not find container \"93a9c753fb22ebf648bfd5240e74c3a1f0d6a27806fb0ddaa5ada3c6157c26c0\": container with ID starting with 93a9c753fb22ebf648bfd5240e74c3a1f0d6a27806fb0ddaa5ada3c6157c26c0 not found: ID does not exist"
Sep 19 09:10:49.335974 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:10:49.335914    2578 scope.go:117] "RemoveContainer" containerID="dda0af359fe02401b57f3be3892936e960090733f747ba1e9f737e79a8c305f9"
Sep 19 09:10:49.336141 ip-10-0-139-157 kubenswrapper[2578]: E0919 09:10:49.336127    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dda0af359fe02401b57f3be3892936e960090733f747ba1e9f737e79a8c305f9\": container with ID starting with dda0af359fe02401b57f3be3892936e960090733f747ba1e9f737e79a8c305f9 not found: ID does not exist" containerID="dda0af359fe02401b57f3be3892936e960090733f747ba1e9f737e79a8c305f9"
Sep 19 09:10:49.336176 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:10:49.336145    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dda0af359fe02401b57f3be3892936e960090733f747ba1e9f737e79a8c305f9"} err="failed to get container status \"dda0af359fe02401b57f3be3892936e960090733f747ba1e9f737e79a8c305f9\": rpc error: code = NotFound desc = could not find container \"dda0af359fe02401b57f3be3892936e960090733f747ba1e9f737e79a8c305f9\": container with ID starting with dda0af359fe02401b57f3be3892936e960090733f747ba1e9f737e79a8c305f9 not found: ID does not exist"
Sep 19 09:10:49.342354 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:10:49.342332    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-bde64-predictor-5dbbbb5dc4-z8dp7"]
Sep 19 09:10:49.344850 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:10:49.344829    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-bde64-predictor-5dbbbb5dc4-z8dp7"]
Sep 19 09:10:50.112585 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:10:50.112554    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1c5b9211-82b1-4882-a223-fcf439673a4a" path="/var/lib/kubelet/pods/1c5b9211-82b1-4882-a223-fcf439673a4a/volumes"
Sep 19 09:11:18.172286 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:11:18.172225    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 09:11:18.337466 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:11:18.337435    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 09:11:18.447544 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:11:18.447468    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 09:11:21.157684 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:11:21.157651    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-xq8d8_19642aea-2c2e-4208-8329-09f03b1e5d5b/global-pull-secret-syncer/0.log"
Sep 19 09:11:21.213742 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:11:21.213711    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log"
Sep 19 09:11:21.325769 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:11:21.325740    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-139-157.ec2.internal_ecf1d89d174b946c499fe0ba49cfe70a/haproxy/0.log"
Sep 19 09:11:23.915217 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:11:23.915186    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 09:11:24.014368 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:11:24.014329    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tt6m_0904bd86-22aa-4c03-af38-ee3e765096be/konnectivity-agent/0.log" limit=1024
Sep 19 09:11:25.349171 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:11:25.349139    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-847b8bd8d7-mdm6c_6f5aff65-5f00-4cbe-bb6f-f6139743adac/cluster-monitoring-operator/0.log"
Sep 19 09:11:25.635532 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:11:25.635461    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-9thpj_b5e8a425-558b-4e10-82b8-764bdd1ad9cc/node-exporter/0.log"
Sep 19 09:11:25.658278 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:11:25.658239    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-9thpj_b5e8a425-558b-4e10-82b8-764bdd1ad9cc/kube-rbac-proxy/0.log"
Sep 19 09:11:25.679888 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:11:25.679864    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-9thpj_b5e8a425-558b-4e10-82b8-764bdd1ad9cc/init-textfile/0.log"
Sep 19 09:11:26.220167 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:11:26.220138    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-5c6747dc7b-gbgd2_cf1e52ee-e10d-4155-ad44-3e0fddb87ab7/prometheus-operator-admission-webhook/0.log"
Sep 19 09:11:27.774055 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:11:27.774025    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6446c8d75d-tbw4d_cedc64f2-407a-4344-96be-399fb3c5d499/networking-console-plugin/0.log"
Sep 19 09:11:28.780198 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:11:28.780172    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-f94c86659-4mqfr_5cab1262-0ee5-473e-9077-af89e224ba45/console/0.log"
Sep 19 09:11:28.817314 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:11:28.817274    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-6594f4c75-49nzk_5fc3e86d-41cb-41fa-abce-4c8f9688d573/download-server/0.log"
Sep 19 09:11:29.319625 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:11:29.319592    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-wb8bw/perf-node-gather-daemonset-l5f2v"]
Sep 19 09:11:29.319896 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:11:29.319884    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1c5b9211-82b1-4882-a223-fcf439673a4a" containerName="kserve-container"
Sep 19 09:11:29.319941 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:11:29.319897    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="1c5b9211-82b1-4882-a223-fcf439673a4a" containerName="kserve-container"
Sep 19 09:11:29.319941 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:11:29.319906    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1c5b9211-82b1-4882-a223-fcf439673a4a" containerName="kube-rbac-proxy"
Sep 19 09:11:29.319941 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:11:29.319912    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="1c5b9211-82b1-4882-a223-fcf439673a4a" containerName="kube-rbac-proxy"
Sep 19 09:11:29.319941 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:11:29.319926    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fa828c7e-3029-4864-92bc-a53d7fad2505" containerName="kserve-container"
Sep 19 09:11:29.319941 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:11:29.319932    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="fa828c7e-3029-4864-92bc-a53d7fad2505" containerName="kserve-container"
Sep 19 09:11:29.319941 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:11:29.319942    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fa828c7e-3029-4864-92bc-a53d7fad2505" containerName="kube-rbac-proxy"
Sep 19 09:11:29.320126 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:11:29.319948    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="fa828c7e-3029-4864-92bc-a53d7fad2505" containerName="kube-rbac-proxy"
Sep 19 09:11:29.320126 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:11:29.319954    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="14c987d2-df51-4bd2-af2e-7f96cd319b04" containerName="kserve-container"
Sep 19 09:11:29.320126 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:11:29.319960    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="14c987d2-df51-4bd2-af2e-7f96cd319b04" containerName="kserve-container"
Sep 19 09:11:29.320126 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:11:29.319972    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="14c987d2-df51-4bd2-af2e-7f96cd319b04" containerName="kube-rbac-proxy"
Sep 19 09:11:29.320126 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:11:29.319977    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="14c987d2-df51-4bd2-af2e-7f96cd319b04" containerName="kube-rbac-proxy"
Sep 19 09:11:29.320126 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:11:29.320022    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="14c987d2-df51-4bd2-af2e-7f96cd319b04" containerName="kserve-container"
Sep 19 09:11:29.320126 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:11:29.320028    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="fa828c7e-3029-4864-92bc-a53d7fad2505" containerName="kserve-container"
Sep 19 09:11:29.320126 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:11:29.320035    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="14c987d2-df51-4bd2-af2e-7f96cd319b04" containerName="kube-rbac-proxy"
Sep 19 09:11:29.320126 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:11:29.320040    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="1c5b9211-82b1-4882-a223-fcf439673a4a" containerName="kserve-container"
Sep 19 09:11:29.320126 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:11:29.320049    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="1c5b9211-82b1-4882-a223-fcf439673a4a" containerName="kube-rbac-proxy"
Sep 19 09:11:29.320126 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:11:29.320054    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="fa828c7e-3029-4864-92bc-a53d7fad2505" containerName="kube-rbac-proxy"
Sep 19 09:11:29.322888 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:11:29.322866    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-wb8bw/perf-node-gather-daemonset-l5f2v"
Sep 19 09:11:29.325622 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:11:29.325603    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-wb8bw\"/\"kube-root-ca.crt\""
Sep 19 09:11:29.325848 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:11:29.325829    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-wb8bw\"/\"openshift-service-ca.crt\""
Sep 19 09:11:29.326669 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:11:29.326653    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-wb8bw\"/\"default-dockercfg-b4dzd\""
Sep 19 09:11:29.334439 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:11:29.334417    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-wb8bw/perf-node-gather-daemonset-l5f2v"]
Sep 19 09:11:29.449332 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:11:29.449266    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/3dbe85bb-d384-49b1-bef5-23839ffaa793-proc\") pod \"perf-node-gather-daemonset-l5f2v\" (UID: \"3dbe85bb-d384-49b1-bef5-23839ffaa793\") " pod="openshift-must-gather-wb8bw/perf-node-gather-daemonset-l5f2v"
Sep 19 09:11:29.449332 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:11:29.449323    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-srf52\" (UniqueName: \"kubernetes.io/projected/3dbe85bb-d384-49b1-bef5-23839ffaa793-kube-api-access-srf52\") pod \"perf-node-gather-daemonset-l5f2v\" (UID: \"3dbe85bb-d384-49b1-bef5-23839ffaa793\") " pod="openshift-must-gather-wb8bw/perf-node-gather-daemonset-l5f2v"
Sep 19 09:11:29.449553 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:11:29.449350    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3dbe85bb-d384-49b1-bef5-23839ffaa793-lib-modules\") pod \"perf-node-gather-daemonset-l5f2v\" (UID: \"3dbe85bb-d384-49b1-bef5-23839ffaa793\") " pod="openshift-must-gather-wb8bw/perf-node-gather-daemonset-l5f2v"
Sep 19 09:11:29.449553 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:11:29.449403    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3dbe85bb-d384-49b1-bef5-23839ffaa793-sys\") pod \"perf-node-gather-daemonset-l5f2v\" (UID: \"3dbe85bb-d384-49b1-bef5-23839ffaa793\") " pod="openshift-must-gather-wb8bw/perf-node-gather-daemonset-l5f2v"
Sep 19 09:11:29.449553 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:11:29.449431    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/3dbe85bb-d384-49b1-bef5-23839ffaa793-podres\") pod \"perf-node-gather-daemonset-l5f2v\" (UID: \"3dbe85bb-d384-49b1-bef5-23839ffaa793\") " pod="openshift-must-gather-wb8bw/perf-node-gather-daemonset-l5f2v"
Sep 19 09:11:29.550282 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:11:29.550246    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/3dbe85bb-d384-49b1-bef5-23839ffaa793-proc\") pod \"perf-node-gather-daemonset-l5f2v\" (UID: \"3dbe85bb-d384-49b1-bef5-23839ffaa793\") " pod="openshift-must-gather-wb8bw/perf-node-gather-daemonset-l5f2v"
Sep 19 09:11:29.550282 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:11:29.550282    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-srf52\" (UniqueName: \"kubernetes.io/projected/3dbe85bb-d384-49b1-bef5-23839ffaa793-kube-api-access-srf52\") pod \"perf-node-gather-daemonset-l5f2v\" (UID: \"3dbe85bb-d384-49b1-bef5-23839ffaa793\") " pod="openshift-must-gather-wb8bw/perf-node-gather-daemonset-l5f2v"
Sep 19 09:11:29.550511 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:11:29.550325    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3dbe85bb-d384-49b1-bef5-23839ffaa793-lib-modules\") pod \"perf-node-gather-daemonset-l5f2v\" (UID: \"3dbe85bb-d384-49b1-bef5-23839ffaa793\") " pod="openshift-must-gather-wb8bw/perf-node-gather-daemonset-l5f2v"
Sep 19 09:11:29.550511 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:11:29.550360    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3dbe85bb-d384-49b1-bef5-23839ffaa793-sys\") pod \"perf-node-gather-daemonset-l5f2v\" (UID: \"3dbe85bb-d384-49b1-bef5-23839ffaa793\") " pod="openshift-must-gather-wb8bw/perf-node-gather-daemonset-l5f2v"
Sep 19 09:11:29.550511 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:11:29.550377    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/3dbe85bb-d384-49b1-bef5-23839ffaa793-proc\") pod \"perf-node-gather-daemonset-l5f2v\" (UID: \"3dbe85bb-d384-49b1-bef5-23839ffaa793\") " pod="openshift-must-gather-wb8bw/perf-node-gather-daemonset-l5f2v"
Sep 19 09:11:29.550511 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:11:29.550431    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3dbe85bb-d384-49b1-bef5-23839ffaa793-sys\") pod \"perf-node-gather-daemonset-l5f2v\" (UID: \"3dbe85bb-d384-49b1-bef5-23839ffaa793\") " pod="openshift-must-gather-wb8bw/perf-node-gather-daemonset-l5f2v"
Sep 19 09:11:29.550511 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:11:29.550483    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/3dbe85bb-d384-49b1-bef5-23839ffaa793-podres\") pod \"perf-node-gather-daemonset-l5f2v\" (UID: \"3dbe85bb-d384-49b1-bef5-23839ffaa793\") " pod="openshift-must-gather-wb8bw/perf-node-gather-daemonset-l5f2v"
Sep 19 09:11:29.550511 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:11:29.550491    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3dbe85bb-d384-49b1-bef5-23839ffaa793-lib-modules\") pod \"perf-node-gather-daemonset-l5f2v\" (UID: \"3dbe85bb-d384-49b1-bef5-23839ffaa793\") " pod="openshift-must-gather-wb8bw/perf-node-gather-daemonset-l5f2v"
Sep 19 09:11:29.550727 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:11:29.550585    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/3dbe85bb-d384-49b1-bef5-23839ffaa793-podres\") pod \"perf-node-gather-daemonset-l5f2v\" (UID: \"3dbe85bb-d384-49b1-bef5-23839ffaa793\") " pod="openshift-must-gather-wb8bw/perf-node-gather-daemonset-l5f2v"
Sep 19 09:11:29.559517 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:11:29.559497    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-srf52\" (UniqueName: \"kubernetes.io/projected/3dbe85bb-d384-49b1-bef5-23839ffaa793-kube-api-access-srf52\") pod \"perf-node-gather-daemonset-l5f2v\" (UID: \"3dbe85bb-d384-49b1-bef5-23839ffaa793\") " pod="openshift-must-gather-wb8bw/perf-node-gather-daemonset-l5f2v"
Sep 19 09:11:29.632953 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:11:29.632873    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-wb8bw/perf-node-gather-daemonset-l5f2v"
Sep 19 09:11:29.906365 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:11:29.906342    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-wb8bw/perf-node-gather-daemonset-l5f2v"]
Sep 19 09:11:29.908572 ip-10-0-139-157 kubenswrapper[2578]: W0919 09:11:29.908542    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod3dbe85bb_d384_49b1_bef5_23839ffaa793.slice/crio-0b3efbb08cebffda6a240f49795a1bb38b4758f6f8632853fe47b429b416bf9a WatchSource:0}: Error finding container 0b3efbb08cebffda6a240f49795a1bb38b4758f6f8632853fe47b429b416bf9a: Status 404 returned error can't find the container with id 0b3efbb08cebffda6a240f49795a1bb38b4758f6f8632853fe47b429b416bf9a
Sep 19 09:11:30.444352 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:11:30.444326    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-ldghd_61a85fdc-d1d2-4a56-a247-968786b293a5/dns/0.log"
Sep 19 09:11:30.484772 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:11:30.484747    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-ldghd_61a85fdc-d1d2-4a56-a247-968786b293a5/kube-rbac-proxy/0.log"
Sep 19 09:11:30.492264 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:11:30.492233    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-wb8bw/perf-node-gather-daemonset-l5f2v" event={"ID":"3dbe85bb-d384-49b1-bef5-23839ffaa793","Type":"ContainerStarted","Data":"68005b935d8a4a551e4bb7f30321d1efde2842d9d463388a4745b57e221a81d8"}
Sep 19 09:11:30.492420 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:11:30.492279    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-wb8bw/perf-node-gather-daemonset-l5f2v" event={"ID":"3dbe85bb-d384-49b1-bef5-23839ffaa793","Type":"ContainerStarted","Data":"0b3efbb08cebffda6a240f49795a1bb38b4758f6f8632853fe47b429b416bf9a"}
Sep 19 09:11:30.492420 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:11:30.492337    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-wb8bw/perf-node-gather-daemonset-l5f2v"
Sep 19 09:11:30.515875 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:11:30.515825    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-wb8bw/perf-node-gather-daemonset-l5f2v" podStartSLOduration=1.51580979 podStartE2EDuration="1.51580979s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 09:11:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:11:30.514519412 +0000 UTC m=+3074.891229274" watchObservedRunningTime="2026-09-19 09:11:30.51580979 +0000 UTC m=+3074.892519652"
Sep 19 09:11:30.636285 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:11:30.636257    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-lqdbb_637bbfb9-45ba-481d-b9fe-87f60051712c/dns-node-resolver/0.log"
Sep 19 09:11:31.132853 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:11:31.132819    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-hb9xj_65dc9622-6c75-457d-ad22-6841b2673aae/node-ca/0.log"
Sep 19 09:11:32.290685 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:11:32.290656    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-6thbg_3135bdcd-43c1-4e88-9ec1-0a9a5d2ecdb0/serve-healthcheck-canary/0.log"
Sep 19 09:11:32.684782 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:11:32.684703    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-dm5sf_220642c2-d6c2-4681-a58b-087ba432f53a/kube-rbac-proxy/0.log"
Sep 19 09:11:32.706051 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:11:32.706023    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-dm5sf_220642c2-d6c2-4681-a58b-087ba432f53a/exporter/0.log"
Sep 19 09:11:32.777695 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:11:32.777668    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-dm5sf_220642c2-d6c2-4681-a58b-087ba432f53a/extractor/0.log"
Sep 19 09:11:35.250408 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:11:35.250376    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_kserve-controller-manager-7dcf4bff6-l2zf7_60b42bfb-489c-49ce-bed0-e1751ba8ada8/manager/0.log"
Sep 19 09:11:35.400103 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:11:35.400075    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_odh-model-controller-7b966d7984-zlnrt_0063b08e-aa2c-40e1-8973-5dad4ca2935f/manager/0.log"
Sep 19 09:11:36.505720 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:11:36.505692    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-wb8bw/perf-node-gather-daemonset-l5f2v"
Sep 19 09:11:40.022131 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:11:40.022102    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-55889f7b47-tjr4s_ebaf628e-811b-4dbc-a814-f257d77db389/kube-storage-version-migrator-operator/1.log"
Sep 19 09:11:40.022891 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:11:40.022874    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-55889f7b47-tjr4s_ebaf628e-811b-4dbc-a814-f257d77db389/kube-storage-version-migrator-operator/0.log"
Sep 19 09:11:41.069205 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:11:41.069178    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9frbh_647eb10e-629e-4cbc-bad4-234bbb862d6e/kube-multus-additional-cni-plugins/0.log"
Sep 19 09:11:41.091541 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:11:41.091475    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9frbh_647eb10e-629e-4cbc-bad4-234bbb862d6e/egress-router-binary-copy/0.log"
Sep 19 09:11:41.114990 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:11:41.114970    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9frbh_647eb10e-629e-4cbc-bad4-234bbb862d6e/cni-plugins/0.log"
Sep 19 09:11:41.138354 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:11:41.138331    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9frbh_647eb10e-629e-4cbc-bad4-234bbb862d6e/bond-cni-plugin/0.log"
Sep 19 09:11:41.159970 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:11:41.159942    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9frbh_647eb10e-629e-4cbc-bad4-234bbb862d6e/routeoverride-cni/0.log"
Sep 19 09:11:41.181075 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:11:41.181045    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9frbh_647eb10e-629e-4cbc-bad4-234bbb862d6e/whereabouts-cni-bincopy/0.log"
Sep 19 09:11:41.205335 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:11:41.205293    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9frbh_647eb10e-629e-4cbc-bad4-234bbb862d6e/whereabouts-cni/0.log"
Sep 19 09:11:41.608928 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:11:41.608897    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-wln5n_c3a67fda-81e8-4d20-83d4-a37e277382b4/kube-multus/0.log"
Sep 19 09:11:41.762225 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:11:41.762196    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-md9ln_79f6ab54-a988-4015-81c2-da35716eab52/network-metrics-daemon/0.log"
Sep 19 09:11:41.780829 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:11:41.780806    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-md9ln_79f6ab54-a988-4015-81c2-da35716eab52/kube-rbac-proxy/0.log"
Sep 19 09:11:42.959557 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:11:42.959527    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zd2zp_07ddac45-12a1-42e8-9566-55f08854c02e/ovn-controller/0.log"
Sep 19 09:11:42.974525 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:11:42.974502    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zd2zp_07ddac45-12a1-42e8-9566-55f08854c02e/ovn-acl-logging/0.log"
Sep 19 09:11:42.974525 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:11:42.974525    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zd2zp_07ddac45-12a1-42e8-9566-55f08854c02e/ovn-acl-logging/1.log"
Sep 19 09:11:42.994952 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:11:42.993057    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zd2zp_07ddac45-12a1-42e8-9566-55f08854c02e/kube-rbac-proxy-node/0.log"
Sep 19 09:11:43.013136 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:11:43.013108    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zd2zp_07ddac45-12a1-42e8-9566-55f08854c02e/kube-rbac-proxy-ovn-metrics/0.log"
Sep 19 09:11:43.032761 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:11:43.032726    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zd2zp_07ddac45-12a1-42e8-9566-55f08854c02e/northd/0.log"
Sep 19 09:11:43.052603 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:11:43.052581    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zd2zp_07ddac45-12a1-42e8-9566-55f08854c02e/nbdb/0.log"
Sep 19 09:11:43.076939 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:11:43.076913    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zd2zp_07ddac45-12a1-42e8-9566-55f08854c02e/sbdb/0.log"
Sep 19 09:11:43.116017 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:11:43.115976    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zd2zp_07ddac45-12a1-42e8-9566-55f08854c02e/ovnkube-controller/0.log"
Sep 19 09:11:44.077143 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:11:44.077108    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-source-7cbc45fddc-2bdxn_54012075-1fb8-4ac5-9b11-74d546b4216f/check-endpoints/0.log"
Sep 19 09:11:44.129949 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:11:44.129916    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-l6m8x_8324d8db-b082-475a-8bb7-8c578404b02e/network-check-target-container/0.log"
Sep 19 09:11:45.036909 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:11:45.036881    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-jfkwf_dcc90f06-1e07-4b6b-add7-46f0cdd95ecf/iptables-alerter/0.log"
Sep 19 09:11:45.690100 ip-10-0-139-157 kubenswrapper[2578]: I0919 09:11:45.690075    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-node-tuning-operator_tuned-5x6rt_90d27f61-9e28-4cc5-bc29-12f69fda90a0/tuned/0.log"