Jul 28 16:21:21.359537 ip-10-0-140-154 systemd[1]: Starting Kubernetes Kubelet...
Jul 28 16:21:21.759336 ip-10-0-140-154 kubenswrapper[2557]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 28 16:21:21.759336 ip-10-0-140-154 kubenswrapper[2557]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 28 16:21:21.759336 ip-10-0-140-154 kubenswrapper[2557]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 28 16:21:21.759336 ip-10-0-140-154 kubenswrapper[2557]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 28 16:21:21.759336 ip-10-0-140-154 kubenswrapper[2557]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 28 16:21:21.761129 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.761050    2557 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 28 16:21:21.765234 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.765194    2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 16:21:21.765234 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.765220    2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 16:21:21.765234 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.765224    2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 16:21:21.765234 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.765228    2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 16:21:21.765234 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.765230    2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 16:21:21.765234 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.765233    2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 16:21:21.765234 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.765236    2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 16:21:21.765234 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.765239    2557 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 16:21:21.765466 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.765241    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 16:21:21.765466 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.765244    2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 16:21:21.765466 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.765246    2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 16:21:21.765466 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.765249    2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 16:21:21.765466 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.765251    2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 16:21:21.765466 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.765254    2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 16:21:21.765466 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.765257    2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 16:21:21.765466 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.765260    2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 16:21:21.765466 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.765262    2557 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 16:21:21.765466 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.765265    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 16:21:21.765466 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.765267    2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 16:21:21.765466 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.765270    2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 16:21:21.765466 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.765272    2557 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 16:21:21.765466 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.765282    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 16:21:21.765466 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.765285    2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 16:21:21.765466 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.765287    2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 16:21:21.765466 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.765290    2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 16:21:21.765466 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.765292    2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 16:21:21.765466 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.765295    2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 16:21:21.765956 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.765297    2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 16:21:21.765956 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.765300    2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 16:21:21.765956 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.765302    2557 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 16:21:21.765956 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.765304    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 16:21:21.765956 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.765307    2557 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 16:21:21.765956 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.765309    2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 16:21:21.765956 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.765312    2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 16:21:21.765956 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.765314    2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 16:21:21.765956 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.765317    2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 16:21:21.765956 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.765319    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 16:21:21.765956 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.765322    2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 16:21:21.765956 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.765324    2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 16:21:21.765956 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.765326    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 16:21:21.765956 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.765329    2557 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 16:21:21.765956 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.765332    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 16:21:21.765956 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.765335    2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 16:21:21.765956 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.765337    2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 16:21:21.765956 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.765340    2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 16:21:21.765956 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.765345    2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 16:21:21.765956 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.765349    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 16:21:21.766458 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.765351    2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 16:21:21.766458 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.765354    2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 16:21:21.766458 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.765356    2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 16:21:21.766458 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.765359    2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 16:21:21.766458 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.765361    2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 16:21:21.766458 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.765364    2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 16:21:21.766458 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.765367    2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 16:21:21.766458 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.765369    2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 16:21:21.766458 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.765372    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 16:21:21.766458 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.765374    2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 16:21:21.766458 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.765377    2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 16:21:21.766458 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.765379    2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 16:21:21.766458 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.765381    2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 16:21:21.766458 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.765384    2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 16:21:21.766458 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.765386    2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 16:21:21.766458 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.765389    2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 16:21:21.766458 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.765392    2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 16:21:21.766458 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.765394    2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 16:21:21.766458 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.765396    2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 16:21:21.766458 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.765399    2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 16:21:21.767155 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.765401    2557 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 16:21:21.767155 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.765403    2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 16:21:21.767155 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.765406    2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 16:21:21.767155 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.765408    2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 16:21:21.767155 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.765411    2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 16:21:21.767155 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.765413    2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 16:21:21.767155 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.765416    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 16:21:21.767155 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.765419    2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 16:21:21.767155 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.765421    2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 16:21:21.767155 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.765424    2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 16:21:21.767155 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.765426    2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 16:21:21.767155 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.765428    2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 16:21:21.767155 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.765431    2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 16:21:21.767155 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.765433    2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 16:21:21.767155 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.765436    2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 16:21:21.767155 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.765438    2557 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 16:21:21.767155 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.765441    2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 16:21:21.767155 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.765443    2557 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 16:21:21.767155 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.765445    2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 16:21:21.767155 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.765455    2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 16:21:21.767942 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.765464    2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 16:21:21.767942 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.765468    2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 16:21:21.767942 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.765473    2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 16:21:21.767942 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.765476    2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 16:21:21.767942 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.765479    2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 16:21:21.767942 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.765482    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 16:21:21.767942 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.765485    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 16:21:21.767942 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.765487    2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 16:21:21.767942 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.765490    2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 16:21:21.767942 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.765492    2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 16:21:21.767942 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.765494    2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 16:21:21.767942 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.765497    2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 16:21:21.767942 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.766868    2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 16:21:21.767942 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.766886    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 16:21:21.767942 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.766893    2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 16:21:21.767942 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.766897    2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 16:21:21.767942 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.766902    2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 16:21:21.767942 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.766907    2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 16:21:21.767942 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.766913    2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 16:21:21.768437 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.766918    2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 16:21:21.768437 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.766923    2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 16:21:21.768437 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.766927    2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 16:21:21.768437 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.766932    2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 16:21:21.768437 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.766936    2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 16:21:21.768437 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.766940    2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 16:21:21.768437 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.766944    2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 16:21:21.768437 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.766953    2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 16:21:21.768437 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.766957    2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 16:21:21.768437 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.766962    2557 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 16:21:21.768437 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.766966    2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 16:21:21.768437 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.766970    2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 16:21:21.768437 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.766974    2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 16:21:21.768437 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.766979    2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 16:21:21.768437 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.766984    2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 16:21:21.768437 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.766989    2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 16:21:21.768437 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.766993    2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 16:21:21.768437 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.766997    2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 16:21:21.768437 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.767001    2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 16:21:21.768437 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.767009    2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 16:21:21.768956 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.767013    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 16:21:21.768956 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.767017    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 16:21:21.768956 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.767021    2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 16:21:21.768956 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.767025    2557 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 16:21:21.768956 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.767030    2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 16:21:21.768956 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.767034    2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 16:21:21.768956 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.767038    2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 16:21:21.768956 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.767043    2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 16:21:21.768956 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.767047    2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 16:21:21.768956 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.767051    2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 16:21:21.768956 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.767056    2557 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 16:21:21.768956 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.767061    2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 16:21:21.768956 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.767069    2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 16:21:21.768956 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.767074    2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 16:21:21.768956 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.767078    2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 16:21:21.768956 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.767082    2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 16:21:21.768956 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.767086    2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 16:21:21.768956 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.767090    2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 16:21:21.768956 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.767094    2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 16:21:21.768956 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.767098    2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 16:21:21.769483 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.767105    2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 16:21:21.769483 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.767109    2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 16:21:21.769483 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.767113    2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 16:21:21.769483 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.767117    2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 16:21:21.769483 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.767125    2557 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 16:21:21.769483 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.767129    2557 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 16:21:21.769483 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.767133    2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 16:21:21.769483 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.767138    2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 16:21:21.769483 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.767143    2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 16:21:21.769483 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.767147    2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 16:21:21.769483 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.767151    2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 16:21:21.769483 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.767155    2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 16:21:21.769483 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.767162    2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 16:21:21.769483 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.767168    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 16:21:21.769483 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.767172    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 16:21:21.769483 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.767177    2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 16:21:21.769483 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.767187    2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 16:21:21.769483 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.767191    2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 16:21:21.769483 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.767195    2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 16:21:21.769951 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.767200    2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 16:21:21.769951 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.767221    2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 16:21:21.769951 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.767226    2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 16:21:21.769951 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.767231    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 16:21:21.769951 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.767235    2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 16:21:21.769951 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.767239    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 16:21:21.769951 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.767244    2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 16:21:21.769951 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.767248    2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 16:21:21.769951 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.767252    2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 16:21:21.769951 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.767256    2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 16:21:21.769951 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.767265    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 16:21:21.769951 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.767269    2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 16:21:21.769951 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.767277    2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 16:21:21.769951 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.767282    2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 16:21:21.769951 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.767287    2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 16:21:21.769951 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.767291    2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 16:21:21.769951 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.767295    2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 16:21:21.769951 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.767299    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 16:21:21.769951 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.767303    2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 16:21:21.770445 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.767308    2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 16:21:21.770445 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.767312    2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 16:21:21.770445 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.767322    2557 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 16:21:21.770445 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.767326    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 16:21:21.770445 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.767331    2557 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 16:21:21.770445 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.767335    2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 16:21:21.770445 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.767339    2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 16:21:21.770445 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.767343    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 16:21:21.770445 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.767348    2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 16:21:21.770445 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.767352    2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 16:21:21.770445 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.767387    2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 16:21:21.770445 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.767411    2557 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 16:21:21.770445 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.767418    2557 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 16:21:21.770445 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.767424    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 16:21:21.770445 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.768105    2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 16:21:21.770445 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.768116    2557 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 16:21:21.770445 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.768120    2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 16:21:21.770445 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.768124    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 16:21:21.770445 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.768127    2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 16:21:21.770445 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.768130    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 16:21:21.770932 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.768133    2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 16:21:21.770932 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.768135    2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 16:21:21.770932 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.768138    2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 16:21:21.770932 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.768141    2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 16:21:21.770932 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.768144    2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 16:21:21.770932 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.768147    2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 16:21:21.770932 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.768149    2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 16:21:21.770932 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.768152    2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 16:21:21.770932 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.768155    2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 16:21:21.770932 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.768159    2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 16:21:21.770932 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.768164    2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 16:21:21.770932 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.768168    2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 16:21:21.770932 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.768171    2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 16:21:21.770932 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.768173    2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 16:21:21.770932 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.768176    2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 16:21:21.770932 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.768178    2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 16:21:21.770932 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.768181    2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 16:21:21.770932 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.768183    2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 16:21:21.770932 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.768186    2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 16:21:21.771512 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.768188    2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 16:21:21.771512 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.768191    2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 16:21:21.771512 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.768193    2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 16:21:21.771512 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.768196    2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 16:21:21.771512 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.768199    2557 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 16:21:21.771512 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.768201    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 16:21:21.771512 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.768222    2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 16:21:21.771512 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.768230    2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 16:21:21.771512 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.768234    2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 16:21:21.771512 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.768236    2557 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 16:21:21.771512 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.768239    2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 16:21:21.771512 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.768242    2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 16:21:21.771512 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.768244    2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 16:21:21.771512 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.768247    2557 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 16:21:21.771512 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.768250    2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 16:21:21.771512 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.768252    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 16:21:21.771512 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.768255    2557 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 16:21:21.771512 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.768258    2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 16:21:21.771512 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.768261    2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 16:21:21.771512 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.768263    2557 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 16:21:21.772060 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.768265    2557 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 16:21:21.772060 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.768268    2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 16:21:21.772060 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.768270    2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 16:21:21.772060 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.768272    2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 16:21:21.772060 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.768275    2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 16:21:21.772060 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.768278    2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 16:21:21.772060 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.768280    2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 16:21:21.772060 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.768283    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 16:21:21.772060 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.768285    2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 16:21:21.772060 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.768288    2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 16:21:21.772060 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.768290    2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 16:21:21.772060 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.768292    2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 16:21:21.772060 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.768295    2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 16:21:21.772060 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.768297    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 16:21:21.772060 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.768300    2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 16:21:21.772060 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.768302    2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 16:21:21.772060 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.768306    2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 16:21:21.772060 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.768309    2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 16:21:21.772060 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.768312    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 16:21:21.772559 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.768315    2557 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 16:21:21.772559 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.768318    2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 16:21:21.772559 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.768321    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 16:21:21.772559 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.768323    2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 16:21:21.772559 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.768326    2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 16:21:21.772559 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.768328    2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 16:21:21.772559 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.768331    2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 16:21:21.772559 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.768333    2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 16:21:21.772559 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.768336    2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 16:21:21.772559 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.768338    2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 16:21:21.772559 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.768341    2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 16:21:21.772559 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.768344    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 16:21:21.772559 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.768346    2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 16:21:21.772559 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.768349    2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 16:21:21.772559 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.768351    2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 16:21:21.772559 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.768354    2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 16:21:21.772559 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.768356    2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 16:21:21.772559 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.768358    2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 16:21:21.772559 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.768362    2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 16:21:21.772559 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.768364    2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 16:21:21.773083 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.768367    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 16:21:21.773083 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.768369    2557 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 16:21:21.773083 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.768371    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 16:21:21.773083 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.768374    2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 16:21:21.773083 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.768376    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 16:21:21.773083 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.768379    2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 16:21:21.773083 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.768381    2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 16:21:21.773083 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.768384    2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 16:21:21.773083 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.768386    2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 16:21:21.773083 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.768388    2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 16:21:21.773083 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.768391    2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 16:21:21.773083 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.768393    2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 16:21:21.773083 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.768395    2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 16:21:21.773083 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.768399    2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 16:21:21.773083 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.768402    2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 16:21:21.773083 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768470    2557 flags.go:64] FLAG: --address="0.0.0.0"
Jul 28 16:21:21.773083 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768477    2557 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 28 16:21:21.773083 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768482    2557 flags.go:64] FLAG: --anonymous-auth="true"
Jul 28 16:21:21.773083 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768488    2557 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 28 16:21:21.773083 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768495    2557 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 28 16:21:21.773083 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768501    2557 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 28 16:21:21.773621 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768507    2557 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 28 16:21:21.773621 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768512    2557 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 28 16:21:21.773621 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768516    2557 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 28 16:21:21.773621 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768518    2557 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 28 16:21:21.773621 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768522    2557 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 28 16:21:21.773621 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768525    2557 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 28 16:21:21.773621 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768528    2557 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 28 16:21:21.773621 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768531    2557 flags.go:64] FLAG: --cgroup-root=""
Jul 28 16:21:21.773621 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768533    2557 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 28 16:21:21.773621 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768536    2557 flags.go:64] FLAG: --client-ca-file=""
Jul 28 16:21:21.773621 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768540    2557 flags.go:64] FLAG: --cloud-provider="external"
Jul 28 16:21:21.773621 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768543    2557 flags.go:64] FLAG: --cluster-dns="[]"
Jul 28 16:21:21.773621 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768547    2557 flags.go:64] FLAG: --cluster-domain=""
Jul 28 16:21:21.773621 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768550    2557 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 28 16:21:21.773621 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768553    2557 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 28 16:21:21.773621 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768556    2557 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 28 16:21:21.773621 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768559    2557 flags.go:64] FLAG: --container-log-max-files="5"
Jul 28 16:21:21.773621 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768563    2557 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 28 16:21:21.773621 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768565    2557 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 28 16:21:21.773621 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768568    2557 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 28 16:21:21.773621 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768571    2557 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 28 16:21:21.773621 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768574    2557 flags.go:64] FLAG: --contention-profiling="false"
Jul 28 16:21:21.773621 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768577    2557 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 28 16:21:21.773621 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768579    2557 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 28 16:21:21.774230 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768586    2557 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 28 16:21:21.774230 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768589    2557 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 28 16:21:21.774230 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768593    2557 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 28 16:21:21.774230 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768596    2557 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 28 16:21:21.774230 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768598    2557 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 28 16:21:21.774230 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768601    2557 flags.go:64] FLAG: --enable-load-reader="false"
Jul 28 16:21:21.774230 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768604    2557 flags.go:64] FLAG: --enable-server="true"
Jul 28 16:21:21.774230 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768606    2557 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 28 16:21:21.774230 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768610    2557 flags.go:64] FLAG: --event-burst="100"
Jul 28 16:21:21.774230 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768614    2557 flags.go:64] FLAG: --event-qps="50"
Jul 28 16:21:21.774230 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768616    2557 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 28 16:21:21.774230 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768619    2557 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 28 16:21:21.774230 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768622    2557 flags.go:64] FLAG: --eviction-hard=""
Jul 28 16:21:21.774230 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768626    2557 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 28 16:21:21.774230 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768628    2557 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 28 16:21:21.774230 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768631    2557 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 28 16:21:21.774230 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768634    2557 flags.go:64] FLAG: --eviction-soft=""
Jul 28 16:21:21.774230 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768637    2557 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 28 16:21:21.774230 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768639    2557 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 28 16:21:21.774230 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768643    2557 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 28 16:21:21.774230 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768645    2557 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 28 16:21:21.774230 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768649    2557 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 28 16:21:21.774230 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768651    2557 flags.go:64] FLAG: --fail-swap-on="true"
Jul 28 16:21:21.774230 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768654    2557 flags.go:64] FLAG: --feature-gates=""
Jul 28 16:21:21.774230 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768662    2557 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 28 16:21:21.774857 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768665    2557 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 28 16:21:21.774857 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768668    2557 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 28 16:21:21.774857 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768670    2557 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 28 16:21:21.774857 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768675    2557 flags.go:64] FLAG: --healthz-port="10248"
Jul 28 16:21:21.774857 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768678    2557 flags.go:64] FLAG: --help="false"
Jul 28 16:21:21.774857 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768680    2557 flags.go:64] FLAG: --hostname-override="ip-10-0-140-154.ec2.internal"
Jul 28 16:21:21.774857 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768683    2557 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 28 16:21:21.774857 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768688    2557 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 28 16:21:21.774857 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768690    2557 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 28 16:21:21.774857 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768694    2557 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 28 16:21:21.774857 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768697    2557 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 28 16:21:21.774857 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768700    2557 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 28 16:21:21.774857 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768702    2557 flags.go:64] FLAG: --image-service-endpoint=""
Jul 28 16:21:21.774857 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768705    2557 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 28 16:21:21.774857 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768708    2557 flags.go:64] FLAG: --kube-api-burst="100"
Jul 28 16:21:21.774857 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768711    2557 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 28 16:21:21.774857 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768713    2557 flags.go:64] FLAG: --kube-api-qps="50"
Jul 28 16:21:21.774857 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768716    2557 flags.go:64] FLAG: --kube-reserved=""
Jul 28 16:21:21.774857 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768719    2557 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 28 16:21:21.774857 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768721    2557 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 28 16:21:21.774857 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768724    2557 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 28 16:21:21.774857 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768727    2557 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 28 16:21:21.774857 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768729    2557 flags.go:64] FLAG: --lock-file=""
Jul 28 16:21:21.774857 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768732    2557 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 28 16:21:21.775463 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768735    2557 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 28 16:21:21.775463 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768737    2557 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 28 16:21:21.775463 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768742    2557 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 28 16:21:21.775463 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768746    2557 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 28 16:21:21.775463 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768749    2557 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 28 16:21:21.775463 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768752    2557 flags.go:64] FLAG: --logging-format="text"
Jul 28 16:21:21.775463 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768755    2557 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 28 16:21:21.775463 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768758    2557 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 28 16:21:21.775463 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768761    2557 flags.go:64] FLAG: --manifest-url=""
Jul 28 16:21:21.775463 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768763    2557 flags.go:64] FLAG: --manifest-url-header=""
Jul 28 16:21:21.775463 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768767    2557 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 28 16:21:21.775463 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768771    2557 flags.go:64] FLAG: --max-open-files="1000000"
Jul 28 16:21:21.775463 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768778    2557 flags.go:64] FLAG: --max-pods="110"
Jul 28 16:21:21.775463 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768780    2557 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 28 16:21:21.775463 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768783    2557 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 28 16:21:21.775463 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768787    2557 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 28 16:21:21.775463 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768790    2557 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 28 16:21:21.775463 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768793    2557 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 28 16:21:21.775463 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768796    2557 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 28 16:21:21.775463 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768798    2557 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 28 16:21:21.775463 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768805    2557 flags.go:64] FLAG: --node-status-max-images="50"
Jul 28 16:21:21.775463 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768808    2557 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 28 16:21:21.775463 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768811    2557 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 28 16:21:21.775463 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768814    2557 flags.go:64] FLAG: --pod-cidr=""
Jul 28 16:21:21.776051 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768816    2557 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb"
Jul 28 16:21:21.776051 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768821    2557 flags.go:64] FLAG: --pod-manifest-path=""
Jul 28 16:21:21.776051 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768824    2557 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 28 16:21:21.776051 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768827    2557 flags.go:64] FLAG: --pods-per-core="0"
Jul 28 16:21:21.776051 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768829    2557 flags.go:64] FLAG: --port="10250"
Jul 28 16:21:21.776051 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768832    2557 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 28 16:21:21.776051 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768835    2557 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0114f2a9d9db673e1"
Jul 28 16:21:21.776051 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768838    2557 flags.go:64] FLAG: --qos-reserved=""
Jul 28 16:21:21.776051 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768841    2557 flags.go:64] FLAG: --read-only-port="10255"
Jul 28 16:21:21.776051 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768844    2557 flags.go:64] FLAG: --register-node="true"
Jul 28 16:21:21.776051 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768846    2557 flags.go:64] FLAG: --register-with-taints=""
Jul 28 16:21:21.776051 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768850    2557 flags.go:64] FLAG: --registry-burst="10"
Jul 28 16:21:21.776051 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768853    2557 flags.go:64] FLAG: --registry-qps="5"
Jul 28 16:21:21.776051 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768856    2557 flags.go:64] FLAG: --reserved-cpus=""
Jul 28 16:21:21.776051 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768859    2557 flags.go:64] FLAG: --reserved-memory=""
Jul 28 16:21:21.776051 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768862    2557 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 28 16:21:21.776051 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768865    2557 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 28 16:21:21.776051 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768867    2557 flags.go:64] FLAG: --rotate-certificates="false"
Jul 28 16:21:21.776051 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768870    2557 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 28 16:21:21.776051 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768873    2557 flags.go:64] FLAG: --runonce="false"
Jul 28 16:21:21.776051 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768876    2557 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 28 16:21:21.776051 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768880    2557 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 28 16:21:21.776051 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768882    2557 flags.go:64] FLAG: --seccomp-default="false"
Jul 28 16:21:21.776051 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768886    2557 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 28 16:21:21.776051 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768889    2557 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 28 16:21:21.776706 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768892    2557 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 28 16:21:21.776706 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768895    2557 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 28 16:21:21.776706 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768898    2557 flags.go:64] FLAG: --storage-driver-password="root"
Jul 28 16:21:21.776706 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768900    2557 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 28 16:21:21.776706 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768903    2557 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 28 16:21:21.776706 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768905    2557 flags.go:64] FLAG: --storage-driver-user="root"
Jul 28 16:21:21.776706 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768908    2557 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 28 16:21:21.776706 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768911    2557 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 28 16:21:21.776706 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768914    2557 flags.go:64] FLAG: --system-cgroups=""
Jul 28 16:21:21.776706 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768916    2557 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 28 16:21:21.776706 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768922    2557 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 28 16:21:21.776706 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768924    2557 flags.go:64] FLAG: --tls-cert-file=""
Jul 28 16:21:21.776706 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768927    2557 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 28 16:21:21.776706 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768930    2557 flags.go:64] FLAG: --tls-min-version=""
Jul 28 16:21:21.776706 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768933    2557 flags.go:64] FLAG: --tls-private-key-file=""
Jul 28 16:21:21.776706 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768936    2557 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 28 16:21:21.776706 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768938    2557 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 28 16:21:21.776706 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768941    2557 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 28 16:21:21.776706 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768944    2557 flags.go:64] FLAG: --v="2"
Jul 28 16:21:21.776706 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768948    2557 flags.go:64] FLAG: --version="false"
Jul 28 16:21:21.776706 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768952    2557 flags.go:64] FLAG: --vmodule=""
Jul 28 16:21:21.776706 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768955    2557 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 28 16:21:21.776706 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.768959    2557 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 28 16:21:21.776706 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.769068    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 16:21:21.776706 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.769072    2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 16:21:21.777341 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.769075    2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 16:21:21.777341 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.769078    2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 16:21:21.777341 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.769082    2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 16:21:21.777341 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.769085    2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 16:21:21.777341 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.769087    2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 16:21:21.777341 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.769092    2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 16:21:21.777341 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.769094    2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 16:21:21.777341 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.769097    2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 16:21:21.777341 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.769099    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 16:21:21.777341 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.769102    2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 16:21:21.777341 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.769104    2557 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 16:21:21.777341 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.769106    2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 16:21:21.777341 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.769109    2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 16:21:21.777341 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.769111    2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 16:21:21.777341 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.769114    2557 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 16:21:21.777341 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.769116    2557 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 16:21:21.777341 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.769119    2557 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 16:21:21.777341 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.769123    2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 16:21:21.777341 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.769127    2557 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 16:21:21.777341 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.769129    2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 16:21:21.777844 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.769132    2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 16:21:21.777844 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.769135    2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 16:21:21.777844 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.769137    2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 16:21:21.777844 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.769139    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 16:21:21.777844 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.769142    2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 16:21:21.777844 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.769144    2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 16:21:21.777844 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.769146    2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 16:21:21.777844 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.769150    2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 16:21:21.777844 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.769156    2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 16:21:21.777844 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.769159    2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 16:21:21.777844 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.769162    2557 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 16:21:21.777844 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.769165    2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 16:21:21.777844 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.769167    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 16:21:21.777844 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.769170    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 16:21:21.777844 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.769172    2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 16:21:21.777844 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.769175    2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 16:21:21.777844 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.769177    2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 16:21:21.777844 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.769181    2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 16:21:21.777844 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.769184    2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 16:21:21.778368 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.769186    2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 16:21:21.778368 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.769189    2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 16:21:21.778368 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.769191    2557 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 16:21:21.778368 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.769194    2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 16:21:21.778368 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.769196    2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 16:21:21.778368 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.769199    2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 16:21:21.778368 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.769201    2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 16:21:21.778368 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.769225    2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 16:21:21.778368 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.769230    2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 16:21:21.778368 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.769232    2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 16:21:21.778368 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.769235    2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 16:21:21.778368 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.769237    2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 16:21:21.778368 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.769240    2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 16:21:21.778368 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.769242    2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 16:21:21.778368 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.769244    2557 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 16:21:21.778368 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.769247    2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 16:21:21.778368 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.769251    2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 16:21:21.778368 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.769254    2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 16:21:21.778368 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.769257    2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 16:21:21.778832 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.769259    2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 16:21:21.778832 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.769262    2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 16:21:21.778832 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.769264    2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 16:21:21.778832 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.769268    2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 16:21:21.778832 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.769270    2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 16:21:21.778832 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.769273    2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 16:21:21.778832 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.769275    2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 16:21:21.778832 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.769278    2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 16:21:21.778832 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.769280    2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 16:21:21.778832 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.769282    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 16:21:21.778832 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.769285    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 16:21:21.778832 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.769289    2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 16:21:21.778832 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.769291    2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 16:21:21.778832 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.769294    2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 16:21:21.778832 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.769296    2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 16:21:21.778832 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.769299    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 16:21:21.778832 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.769301    2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 16:21:21.778832 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.769304    2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 16:21:21.778832 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.769306    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 16:21:21.779313 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.769309    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 16:21:21.779313 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.769311    2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 16:21:21.779313 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.769314    2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 16:21:21.779313 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.769316    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 16:21:21.779313 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.769318    2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 16:21:21.779313 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.769321    2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 16:21:21.779313 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.769323    2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 16:21:21.779313 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.769326    2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 16:21:21.779313 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.769328    2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 16:21:21.779313 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.769330    2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 16:21:21.779313 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.769332    2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 16:21:21.779313 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.769335    2557 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 16:21:21.779313 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.769337    2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 16:21:21.779313 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.769340    2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 16:21:21.779313 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.769342    2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 16:21:21.779313 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.769345    2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 16:21:21.779313 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.769347    2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 16:21:21.779313 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.769351    2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 16:21:21.779313 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.769353    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 16:21:21.779313 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.769355    2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 16:21:21.779784 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.769361    2557 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 28 16:21:21.779784 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.776407    2557 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 28 16:21:21.779784 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.776423    2557 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 28 16:21:21.779784 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.777265    2557 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 28 16:21:21.779784 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.777273    2557 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 28 16:21:21.779784 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777331    2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 16:21:21.779784 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777336    2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 16:21:21.779784 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777340    2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 16:21:21.779784 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777343    2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 16:21:21.779784 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777346    2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 16:21:21.779784 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777349    2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 16:21:21.779784 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777352    2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 16:21:21.779784 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777355    2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 16:21:21.779784 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777359    2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 16:21:21.780138 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777363    2557 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 16:21:21.780138 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777367    2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 16:21:21.780138 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777370    2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 16:21:21.780138 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777372    2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 16:21:21.780138 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777375    2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 16:21:21.780138 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777377    2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 16:21:21.780138 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777380    2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 16:21:21.780138 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777383    2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 16:21:21.780138 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777386    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 16:21:21.780138 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777389    2557 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 16:21:21.780138 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777391    2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 16:21:21.780138 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777394    2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 16:21:21.780138 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777397    2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 16:21:21.780138 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777399    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 16:21:21.780138 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777402    2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 16:21:21.780138 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777404    2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 16:21:21.780138 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777407    2557 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 16:21:21.780138 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777409    2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 16:21:21.780138 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777412    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 16:21:21.780138 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777414    2557 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 16:21:21.780642 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777419    2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 16:21:21.780642 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777421    2557 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 16:21:21.780642 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777424    2557 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 16:21:21.780642 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777426    2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 16:21:21.780642 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777429    2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 16:21:21.780642 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777431    2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 16:21:21.780642 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777434    2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 16:21:21.780642 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777436    2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 16:21:21.780642 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777439    2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 16:21:21.780642 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777441    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 16:21:21.780642 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777443    2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 16:21:21.780642 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777446    2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 16:21:21.780642 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777457    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 16:21:21.780642 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777460    2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 16:21:21.780642 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777463    2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 16:21:21.780642 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777465    2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 16:21:21.780642 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777468    2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 16:21:21.780642 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777471    2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 16:21:21.780642 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777473    2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 16:21:21.780642 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777476    2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 16:21:21.781127 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777478    2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 16:21:21.781127 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777481    2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 16:21:21.781127 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777483    2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 16:21:21.781127 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777486    2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 16:21:21.781127 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777488    2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 16:21:21.781127 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777491    2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 16:21:21.781127 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777493    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 16:21:21.781127 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777495    2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 16:21:21.781127 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777498    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 16:21:21.781127 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777501    2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 16:21:21.781127 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777503    2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 16:21:21.781127 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777506    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 16:21:21.781127 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777509    2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 16:21:21.781127 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777513    2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 16:21:21.781127 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777515    2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 16:21:21.781127 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777518    2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 16:21:21.781127 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777521    2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 16:21:21.781127 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777523    2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 16:21:21.781127 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777526    2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 16:21:21.781127 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777528    2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 16:21:21.781642 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777531    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 16:21:21.781642 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777534    2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 16:21:21.781642 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777536    2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 16:21:21.781642 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777539    2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 16:21:21.781642 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777542    2557 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 16:21:21.781642 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777544    2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 16:21:21.781642 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777546    2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 16:21:21.781642 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777549    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 16:21:21.781642 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777551    2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 16:21:21.781642 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777554    2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 16:21:21.781642 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777557    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 16:21:21.781642 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777559    2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 16:21:21.781642 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777561    2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 16:21:21.781642 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777566    2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 16:21:21.781642 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777569    2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 16:21:21.781642 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777573    2557 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 16:21:21.781642 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777576    2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 16:21:21.781642 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777579    2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 16:21:21.781642 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777582    2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 16:21:21.782101 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777585    2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 16:21:21.782101 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777587    2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 16:21:21.782101 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777590    2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 16:21:21.782101 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777592    2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 16:21:21.782101 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777594    2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 16:21:21.782101 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777597    2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 16:21:21.782101 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777600    2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 16:21:21.782101 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777604    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 16:21:21.782101 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777606    2557 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 16:21:21.782101 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777609    2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 16:21:21.782101 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777611    2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 16:21:21.782101 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.777616    2557 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 28 16:21:21.782101 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777728    2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 16:21:21.782101 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777732    2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 16:21:21.782540 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777735    2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 16:21:21.782540 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777738    2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 16:21:21.782540 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777741    2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 16:21:21.782540 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777744    2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 16:21:21.782540 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777746    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 16:21:21.782540 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777749    2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 16:21:21.782540 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777752    2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 16:21:21.782540 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777754    2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 16:21:21.782540 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777757    2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 16:21:21.782540 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777759    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 16:21:21.782540 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777762    2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 16:21:21.782540 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777764    2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 16:21:21.782540 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777767    2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 16:21:21.782540 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777770    2557 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 16:21:21.782540 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777772    2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 16:21:21.782540 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777775    2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 16:21:21.782540 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777777    2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 16:21:21.782540 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777779    2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 16:21:21.782540 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777782    2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 16:21:21.782994 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777785    2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 16:21:21.782994 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777787    2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 16:21:21.782994 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777790    2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 16:21:21.782994 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777792    2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 16:21:21.782994 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777795    2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 16:21:21.782994 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777797    2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 16:21:21.782994 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777800    2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 16:21:21.782994 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777806    2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 16:21:21.782994 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777808    2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 16:21:21.782994 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777811    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 16:21:21.782994 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777813    2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 16:21:21.782994 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777816    2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 16:21:21.782994 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777818    2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 16:21:21.782994 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777820    2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 16:21:21.782994 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777823    2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 16:21:21.782994 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777825    2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 16:21:21.782994 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777827    2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 16:21:21.782994 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777830    2557 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 16:21:21.782994 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777832    2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 16:21:21.782994 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777835    2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 16:21:21.783474 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777837    2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 16:21:21.783474 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777840    2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 16:21:21.783474 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777843    2557 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 16:21:21.783474 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777845    2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 16:21:21.783474 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777848    2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 16:21:21.783474 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777851    2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 16:21:21.783474 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777854    2557 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 16:21:21.783474 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777856    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 16:21:21.783474 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777859    2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 16:21:21.783474 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777862    2557 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 16:21:21.783474 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777864    2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 16:21:21.783474 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777866    2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 16:21:21.783474 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777869    2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 16:21:21.783474 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777871    2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 16:21:21.783474 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777874    2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 16:21:21.783474 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777876    2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 16:21:21.783474 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777879    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 16:21:21.783474 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777881    2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 16:21:21.783474 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777884    2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 16:21:21.783474 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777886    2557 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 16:21:21.783953 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777890    2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 16:21:21.783953 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777893    2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 16:21:21.783953 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777896    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 16:21:21.783953 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777898    2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 16:21:21.783953 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777901    2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 16:21:21.783953 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777903    2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 16:21:21.783953 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777906    2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 16:21:21.783953 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777908    2557 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 16:21:21.783953 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777910    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 16:21:21.783953 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777913    2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 16:21:21.783953 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777915    2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 16:21:21.783953 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777918    2557 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 16:21:21.783953 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777920    2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 16:21:21.783953 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777922    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 16:21:21.783953 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777925    2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 16:21:21.783953 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777927    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 16:21:21.783953 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777930    2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 16:21:21.783953 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777933    2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 16:21:21.783953 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777935    2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 16:21:21.783953 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777937    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 16:21:21.784436 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777939    2557 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 16:21:21.784436 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777942    2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 16:21:21.784436 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777944    2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 16:21:21.784436 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777963    2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 16:21:21.784436 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777966    2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 16:21:21.784436 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777969    2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 16:21:21.784436 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777972    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 16:21:21.784436 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777975    2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 16:21:21.784436 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777978    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 16:21:21.784436 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777982    2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 16:21:21.784436 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777986    2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 16:21:21.784436 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777989    2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 16:21:21.784436 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777992    2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 16:21:21.784436 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777995    2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 16:21:21.784436 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.777997    2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 16:21:21.784436 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.778000    2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 16:21:21.784436 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.778003    2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 16:21:21.784436 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:21.778006    2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 16:21:21.784865 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.778010    2557 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 28 16:21:21.784865 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.778118    2557 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 28 16:21:21.784865 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.781104    2557 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 28 16:21:21.784865 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.784301    2557 server.go:1019] "Starting client certificate rotation"
Jul 28 16:21:21.784865 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.784407    2557 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 28 16:21:21.784865 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.784439    2557 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 28 16:21:21.786955 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.786942    2557 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 28 16:21:21.789630 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.789614    2557 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 28 16:21:21.798596 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.798575    2557 log.go:25] "Validated CRI v1 runtime API"
Jul 28 16:21:21.802509 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.802493    2557 log.go:25] "Validated CRI v1 image API"
Jul 28 16:21:21.803350 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.803336    2557 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 28 16:21:21.805370 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.805352    2557 fs.go:135] Filesystem UUIDs: map[3e3d86b0-82a6-4d85-ba5b-abb67f77b697:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 fbf179fc-5b80-4f6f-913b-33c47c648a50:/dev/nvme0n1p4]
Jul 28 16:21:21.805423 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.805371    2557 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 28 16:21:21.811148 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.811044    2557 manager.go:216] Machine: {Timestamp:2026-07-28 16:21:21.808897459 +0000 UTC m=+0.333654651 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3098761 MemoryCapacity:32812175360 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2f9ca325dd6d7633134a9f8b2d70e1 SystemUUID:ec2f9ca3-25dd-6d76-3313-4a9f8b2d70e1 BootID:8a36701e-f601-4ef8-9e4e-241a2bf8d9d6 Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:cd:ff:60:2f:0b Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:cd:ff:60:2f:0b Speed:0 Mtu:9001} {Name:ovs-system MacAddress:9e:c6:c0:df:e4:a0 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812175360 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 28 16:21:21.811148 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.811137    2557 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 28 16:21:21.811285 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.811233    2557 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 28 16:21:21.812842 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.812826    2557 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 28 16:21:21.815174 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.815150    2557 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 28 16:21:21.815318 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.815177    2557 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-140-154.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 28 16:21:21.815916 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.815907    2557 topology_manager.go:138] "Creating topology manager with none policy"
Jul 28 16:21:21.815945 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.815920    2557 container_manager_linux.go:309] "Creating device plugin manager"
Jul 28 16:21:21.815945 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.815928    2557 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 28 16:21:21.816563 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.816553    2557 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 28 16:21:21.817304 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.817294    2557 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 28 16:21:21.819149 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.819138    2557 state_mem.go:36] "Initialized new in-memory state store"
Jul 28 16:21:21.819281 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.819272    2557 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 28 16:21:21.819315 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.819302    2557 kubelet.go:486] "Attempting to sync node with API server"
Jul 28 16:21:21.819315 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.819312    2557 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 28 16:21:21.819371 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.819327    2557 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 28 16:21:21.819371 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.819336    2557 kubelet.go:398] "Adding apiserver pod source"
Jul 28 16:21:21.819371 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.819343    2557 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 28 16:21:21.819671 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.819654    2557 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 28 16:21:21.819768 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.819680    2557 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 28 16:21:21.821842 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.821825    2557 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 28 16:21:21.823344 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.823332    2557 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 28 16:21:21.823396 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.823361    2557 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 28 16:21:21.824727 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.824715    2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 28 16:21:21.824778 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.824731    2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 28 16:21:21.824778 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.824738    2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 28 16:21:21.824778 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.824746    2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 28 16:21:21.824778 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.824757    2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 28 16:21:21.824778 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.824763    2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 28 16:21:21.824778 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.824770    2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 28 16:21:21.824778 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.824775    2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 28 16:21:21.824956 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.824817    2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 28 16:21:21.824956 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.824823    2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 28 16:21:21.824956 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.824831    2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 28 16:21:21.824956 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.824836    2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 28 16:21:21.828367 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.828354    2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 28 16:21:21.828367 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.828367    2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 28 16:21:21.829960 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:21.829939    2557 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 28 16:21:21.829960 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:21.829948    2557 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-140-154.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 28 16:21:21.831547 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.831522    2557 server.go:1268] "Started kubelet"
Jul 28 16:21:21.831645 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.831557    2557 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 28 16:21:21.831706 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.831631    2557 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 28 16:21:21.831706 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.831701    2557 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 28 16:21:21.832245 ip-10-0-140-154 systemd[1]: Started Kubernetes Kubelet.
Jul 28 16:21:21.837421 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.837371    2557 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 28 16:21:21.838392 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:21.838370    2557 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 28 16:21:21.838747 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.838730    2557 server.go:310] "Adding debug handlers to kubelet server"
Jul 28 16:21:21.842100 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.842079    2557 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-140-154.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 28 16:21:21.842730 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:21.841925    2557 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-154.ec2.internal.18c680957cce70f6  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-154.ec2.internal,UID:ip-10-0-140-154.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-140-154.ec2.internal,},FirstTimestamp:2026-07-28 16:21:21.831366902 +0000 UTC m=+0.356124094,LastTimestamp:2026-07-28 16:21:21.831366902 +0000 UTC m=+0.356124094,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-154.ec2.internal,}"
Jul 28 16:21:21.844871 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.844856    2557 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 28 16:21:21.845143 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.844872    2557 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 28 16:21:21.845493 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:21.845468    2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-154.ec2.internal\" not found"
Jul 28 16:21:21.845592 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.845518    2557 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 28 16:21:21.845592 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.845530    2557 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 28 16:21:21.845684 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.845596    2557 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 28 16:21:21.845684 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.845655    2557 reconstruct.go:97] "Volume reconstruction finished"
Jul 28 16:21:21.845684 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.845664    2557 reconciler.go:29] "Reconciler: start to sync state"
Jul 28 16:21:21.846657 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.846634    2557 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 28 16:21:21.846657 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.846653    2557 factory.go:55] Registering systemd factory
Jul 28 16:21:21.846657 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.846660    2557 factory.go:223] Registration of the systemd container factory successfully
Jul 28 16:21:21.846889 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.846870    2557 factory.go:153] Registering CRI-O factory
Jul 28 16:21:21.846889 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.846890    2557 factory.go:223] Registration of the crio container factory successfully
Jul 28 16:21:21.846992 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.846907    2557 factory.go:103] Registering Raw factory
Jul 28 16:21:21.846992 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.846918    2557 manager.go:1196] Started watching for new ooms in manager
Jul 28 16:21:21.847524 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.847509    2557 manager.go:349] Starting recovery of all containers
Jul 28 16:21:21.850297 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:21.850272    2557 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-140-154.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 28 16:21:21.850374 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:21.850357    2557 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 28 16:21:21.853952 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.853814    2557 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 28 16:21:21.857007 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.856912    2557 manager.go:354] Recovery completed
Jul 28 16:21:21.857128 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.857112    2557 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-7f6nb"
Jul 28 16:21:21.862286 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.862271    2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 16:21:21.864095 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.864078    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-154.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 16:21:21.864166 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.864113    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-154.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 16:21:21.864166 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.864126    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-154.ec2.internal" event="NodeHasSufficientPID"
Jul 28 16:21:21.864474 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.864459    2557 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-7f6nb"
Jul 28 16:21:21.864637 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.864623    2557 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 28 16:21:21.864670 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.864637    2557 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 28 16:21:21.864670 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.864650    2557 state_mem.go:36] "Initialized new in-memory state store"
Jul 28 16:21:21.864733 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.864685    2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 16:21:21.865512 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.865492    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-154.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 16:21:21.865580 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.865525    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-154.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 16:21:21.865580 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.865536    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-154.ec2.internal" event="NodeHasSufficientPID"
Jul 28 16:21:21.866196 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:21.866131    2557 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-154.ec2.internal.18c680957ec1dd74  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-154.ec2.internal,UID:ip-10-0-140-154.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-140-154.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-140-154.ec2.internal,},FirstTimestamp:2026-07-28 16:21:21.86409714 +0000 UTC m=+0.388854332,LastTimestamp:2026-07-28 16:21:21.86409714 +0000 UTC m=+0.388854332,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-154.ec2.internal,}"
Jul 28 16:21:21.867265 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.867254    2557 policy_none.go:49] "None policy: Start"
Jul 28 16:21:21.867316 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.867269    2557 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 28 16:21:21.867316 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.867280    2557 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 28 16:21:21.868367 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.868356    2557 policy_none.go:47] "Start"
Jul 28 16:21:21.900741 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.900727    2557 manager.go:337] "Starting Device Plugin manager"
Jul 28 16:21:21.900803 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:21.900763    2557 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 28 16:21:21.900803 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.900773    2557 server.go:87] "Starting device plugin registration server"
Jul 28 16:21:21.901005 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.900992    2557 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 28 16:21:21.901063 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.901008    2557 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 28 16:21:21.901116 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.901084    2557 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 28 16:21:21.901240 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.901170    2557 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 28 16:21:21.901240 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:21.901178    2557 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 28 16:21:21.901706 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:21.901690    2557 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 28 16:21:21.901791 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:21.901737    2557 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-140-154.ec2.internal\" not found"
Jul 28 16:21:22.001664 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:22.001647    2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 16:21:22.002941 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:22.002688    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-154.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 16:21:22.002941 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:22.002717    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-154.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 16:21:22.002941 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:22.002727    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-154.ec2.internal" event="NodeHasSufficientPID"
Jul 28 16:21:22.002941 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:22.002755    2557 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-140-154.ec2.internal"
Jul 28 16:21:22.008689 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:22.008675    2557 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-140-154.ec2.internal"
Jul 28 16:21:22.008766 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:22.008698    2557 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-140-154.ec2.internal\": node \"ip-10-0-140-154.ec2.internal\" not found"
Jul 28 16:21:22.024577 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:22.024527    2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-154.ec2.internal\" not found"
Jul 28 16:21:22.041098 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:22.041083    2557 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 28 16:21:22.041161 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:22.041123    2557 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 28 16:21:22.041161 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:22.041141    2557 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 28 16:21:22.041248 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:22.041176    2557 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 28 16:21:22.045761 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:22.045746    2557 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 16:21:22.124780 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:22.124752    2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-154.ec2.internal\" not found"
Jul 28 16:21:22.141922 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:22.141900    2557 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-140-154.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-154.ec2.internal"]
Jul 28 16:21:22.141980 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:22.141952    2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 16:21:22.143285 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:22.143271    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-154.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 16:21:22.143335 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:22.143298    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-154.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 16:21:22.143335 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:22.143308    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-154.ec2.internal" event="NodeHasSufficientPID"
Jul 28 16:21:22.144456 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:22.144444    2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 16:21:22.144569 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:22.144552    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-154.ec2.internal"
Jul 28 16:21:22.144608 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:22.144581    2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 16:21:22.145123 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:22.145107    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-154.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 16:21:22.145202 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:22.145129    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-154.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 16:21:22.145202 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:22.145155    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-154.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 16:21:22.145202 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:22.145172    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-154.ec2.internal" event="NodeHasSufficientPID"
Jul 28 16:21:22.145202 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:22.145147    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-154.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 16:21:22.145411 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:22.145233    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-154.ec2.internal" event="NodeHasSufficientPID"
Jul 28 16:21:22.146410 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:22.146395    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-154.ec2.internal"
Jul 28 16:21:22.146458 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:22.146422    2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 16:21:22.146970 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:22.146952    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/49d5be5d46952d796bae73c6a91a14e0-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-154.ec2.internal\" (UID: \"49d5be5d46952d796bae73c6a91a14e0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-154.ec2.internal"
Jul 28 16:21:22.147033 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:22.146986    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/851ad20362c20c22501be689ae7a68aa-config\") pod \"kube-apiserver-proxy-ip-10-0-140-154.ec2.internal\" (UID: \"851ad20362c20c22501be689ae7a68aa\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-154.ec2.internal"
Jul 28 16:21:22.147033 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:22.147013    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-154.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 16:21:22.147033 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:22.147023    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/49d5be5d46952d796bae73c6a91a14e0-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-154.ec2.internal\" (UID: \"49d5be5d46952d796bae73c6a91a14e0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-154.ec2.internal"
Jul 28 16:21:22.147132 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:22.147037    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-154.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 16:21:22.147132 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:22.147050    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-154.ec2.internal" event="NodeHasSufficientPID"
Jul 28 16:21:22.165762 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:22.165738    2557 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-154.ec2.internal\" not found" node="ip-10-0-140-154.ec2.internal"
Jul 28 16:21:22.169477 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:22.169464    2557 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-154.ec2.internal\" not found" node="ip-10-0-140-154.ec2.internal"
Jul 28 16:21:22.225233 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:22.225194    2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-154.ec2.internal\" not found"
Jul 28 16:21:22.247703 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:22.247681    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/851ad20362c20c22501be689ae7a68aa-config\") pod \"kube-apiserver-proxy-ip-10-0-140-154.ec2.internal\" (UID: \"851ad20362c20c22501be689ae7a68aa\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-154.ec2.internal"
Jul 28 16:21:22.247758 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:22.247708    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/49d5be5d46952d796bae73c6a91a14e0-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-154.ec2.internal\" (UID: \"49d5be5d46952d796bae73c6a91a14e0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-154.ec2.internal"
Jul 28 16:21:22.247758 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:22.247735    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/49d5be5d46952d796bae73c6a91a14e0-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-154.ec2.internal\" (UID: \"49d5be5d46952d796bae73c6a91a14e0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-154.ec2.internal"
Jul 28 16:21:22.247828 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:22.247786    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/49d5be5d46952d796bae73c6a91a14e0-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-154.ec2.internal\" (UID: \"49d5be5d46952d796bae73c6a91a14e0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-154.ec2.internal"
Jul 28 16:21:22.247828 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:22.247796    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/49d5be5d46952d796bae73c6a91a14e0-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-154.ec2.internal\" (UID: \"49d5be5d46952d796bae73c6a91a14e0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-154.ec2.internal"
Jul 28 16:21:22.247828 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:22.247786    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/851ad20362c20c22501be689ae7a68aa-config\") pod \"kube-apiserver-proxy-ip-10-0-140-154.ec2.internal\" (UID: \"851ad20362c20c22501be689ae7a68aa\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-154.ec2.internal"
Jul 28 16:21:22.326050 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:22.325994    2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-154.ec2.internal\" not found"
Jul 28 16:21:22.426467 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:22.426441    2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-154.ec2.internal\" not found"
Jul 28 16:21:22.467988 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:22.467970    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-154.ec2.internal"
Jul 28 16:21:22.472512 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:22.472495    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-154.ec2.internal"
Jul 28 16:21:22.526654 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:22.526634    2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-154.ec2.internal\" not found"
Jul 28 16:21:22.627053 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:22.627014    2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-154.ec2.internal\" not found"
Jul 28 16:21:22.727506 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:22.727483    2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-154.ec2.internal\" not found"
Jul 28 16:21:22.781942 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:22.781920    2557 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 28 16:21:22.782366 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:22.782043    2557 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 28 16:21:22.828408 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:22.828377    2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-154.ec2.internal\" not found"
Jul 28 16:21:22.869956 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:22.869923    2557 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-27 16:16:23 +0000 UTC" deadline="2028-04-01 12:20:08.153065585 +0000 UTC"
Jul 28 16:21:22.870067 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:22.869956    2557 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14707h58m45.283114079s"
Jul 28 16:21:22.871744 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:22.871729    2557 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 16:21:22.928679 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:22.928611    2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-154.ec2.internal\" not found"
Jul 28 16:21:22.943774 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:22.943748    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49d5be5d46952d796bae73c6a91a14e0.slice/crio-33318e80680e9deb8530e2bc9da11a15bfe21dde2b07f894de909355978e7429 WatchSource:0}: Error finding container 33318e80680e9deb8530e2bc9da11a15bfe21dde2b07f894de909355978e7429: Status 404 returned error can't find the container with id 33318e80680e9deb8530e2bc9da11a15bfe21dde2b07f894de909355978e7429
Jul 28 16:21:22.943990 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:22.943972    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod851ad20362c20c22501be689ae7a68aa.slice/crio-00143488b3cefb2fd78ea1c33a39ea18dbf50b5ef9d76eec8ab8fdf7ab2541cc WatchSource:0}: Error finding container 00143488b3cefb2fd78ea1c33a39ea18dbf50b5ef9d76eec8ab8fdf7ab2541cc: Status 404 returned error can't find the container with id 00143488b3cefb2fd78ea1c33a39ea18dbf50b5ef9d76eec8ab8fdf7ab2541cc
Jul 28 16:21:24.335003 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:24.334990    2557 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 16:21:24.415561 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:24.415528    2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-154.ec2.internal\" not found"
Jul 28 16:21:24.430850 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:24.430808    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-154.ec2.internal" event={"ID":"49d5be5d46952d796bae73c6a91a14e0","Type":"ContainerStarted","Data":"33318e80680e9deb8530e2bc9da11a15bfe21dde2b07f894de909355978e7429"}
Jul 28 16:21:24.431946 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:24.431922    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-154.ec2.internal" event={"ID":"851ad20362c20c22501be689ae7a68aa","Type":"ContainerStarted","Data":"00143488b3cefb2fd78ea1c33a39ea18dbf50b5ef9d76eec8ab8fdf7ab2541cc"}
Jul 28 16:21:24.516276 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:24.516239    2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-154.ec2.internal\" not found"
Jul 28 16:21:24.525333 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:24.525309    2557 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 16:21:24.557665 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:24.557642    2557 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 16:21:24.632341 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:24.632285    2557 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-154.ec2.internal"
Jul 28 16:21:24.643195 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:24.643178    2557 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 28 16:21:24.643928 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:24.643916    2557 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-154.ec2.internal"
Jul 28 16:21:24.650218 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:24.650197    2557 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 28 16:21:25.207247 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.207215    2557 apiserver.go:52] "Watching apiserver"
Jul 28 16:21:25.214937 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.214916    2557 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 28 16:21:25.216449 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.216104    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ovn-kubernetes/ovnkube-node-sbf8h","kube-system/konnectivity-agent-6rk8j","kube-system/kube-apiserver-proxy-ip-10-0-140-154.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nlb66","openshift-cluster-node-tuning-operator/tuned-nmwkn","openshift-dns/node-resolver-rn2z4","openshift-image-registry/node-ca-f8l6z","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-154.ec2.internal","openshift-multus/multus-6mmzq","kube-system/global-pull-secret-syncer-9vtwz","openshift-multus/multus-additional-cni-plugins-x2pv7","openshift-multus/network-metrics-daemon-qm9v7","openshift-network-diagnostics/network-check-target-qzkm5","openshift-network-operator/iptables-alerter-2lpp7"]
Jul 28 16:21:25.217582 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.217561    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-f8l6z"
Jul 28 16:21:25.218894 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.218867    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qzkm5"
Jul 28 16:21:25.219000 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:25.218937    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qzkm5" podUID="c687045e-f5d0-4a22-b71e-9c01463677e6"
Jul 28 16:21:25.219915 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.219895    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-zscdj\""
Jul 28 16:21:25.220149 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.220135    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 28 16:21:25.220319 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.220305    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 28 16:21:25.220507 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.220476    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 28 16:21:25.221012 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.220997    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-2lpp7"
Jul 28 16:21:25.221926 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.221909    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nlb66"
Jul 28 16:21:25.222760 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.222741    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-nmwkn"
Jul 28 16:21:25.224340 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.223374    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 28 16:21:25.224340 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.223399    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 28 16:21:25.224340 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.223531    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-7bx46\""
Jul 28 16:21:25.224340 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.223639    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 28 16:21:25.224340 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.223812    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-rn2z4"
Jul 28 16:21:25.224340 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.223857    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 28 16:21:25.224632 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.224415    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 28 16:21:25.224632 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.224427    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 28 16:21:25.224632 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.224417    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-jwpxp\""
Jul 28 16:21:25.224807 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.224743    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 28 16:21:25.225334 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.224894    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-rs8jt\""
Jul 28 16:21:25.225334 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.224914    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 28 16:21:25.225334 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.225129    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-6mmzq"
Jul 28 16:21:25.225847 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.225823    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 28 16:21:25.225847 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.225843    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 28 16:21:25.226211 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.226182    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-8jrq5\""
Jul 28 16:21:25.227030 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.226913    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 28 16:21:25.227096 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.227088    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 28 16:21:25.227396 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.227378    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 28 16:21:25.227470 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.227377    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 28 16:21:25.227564 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.227528    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-c89mm\""
Jul 28 16:21:25.227801 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.227782    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-sbf8h"
Jul 28 16:21:25.229261 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.229244    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-6rk8j"
Jul 28 16:21:25.230197 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.229652    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 28 16:21:25.230197 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.229891    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 28 16:21:25.230197 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.230039    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 28 16:21:25.230197 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.230042    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 28 16:21:25.232245 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.231783    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-qt455\""
Jul 28 16:21:25.232245 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.232018    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 28 16:21:25.232797 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.232772    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 28 16:21:25.232904 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.232888    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-b5d9w\""
Jul 28 16:21:25.235618 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.233525    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9vtwz"
Jul 28 16:21:25.235618 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.233573    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 28 16:21:25.235618 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:25.233718    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9vtwz" podUID="31992b1b-4643-467f-a2b1-674af56e8c0e"
Jul 28 16:21:25.235618 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.234078    2557 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 28 16:21:25.235618 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.234549    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-x2pv7"
Jul 28 16:21:25.235618 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.234678    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 28 16:21:25.236992 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.236976    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qm9v7"
Jul 28 16:21:25.237072 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:25.237037    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qm9v7" podUID="92ef4728-9e89-4494-8bdf-72dcfe9b3ff4"
Jul 28 16:21:25.237626 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.237605    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 28 16:21:25.237776 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.237760    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 28 16:21:25.238001 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.237800    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-bv6h7\""
Jul 28 16:21:25.240684 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.240665    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/da6efa62-02e3-4cac-8881-e62018734a17-agent-certs\") pod \"konnectivity-agent-6rk8j\" (UID: \"da6efa62-02e3-4cac-8881-e62018734a17\") " pod="kube-system/konnectivity-agent-6rk8j"
Jul 28 16:21:25.240847 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.240827    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/695dbc5b-d1b4-42f3-921a-cc62010ddc14-cni-binary-copy\") pod \"multus-additional-cni-plugins-x2pv7\" (UID: \"695dbc5b-d1b4-42f3-921a-cc62010ddc14\") " pod="openshift-multus/multus-additional-cni-plugins-x2pv7"
Jul 28 16:21:25.240931 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.240870    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/38125fdc-3616-4f16-8683-aabde8a95527-etc-selinux\") pod \"aws-ebs-csi-driver-node-nlb66\" (UID: \"38125fdc-3616-4f16-8683-aabde8a95527\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nlb66"
Jul 28 16:21:25.240983 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.240927    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9af11dde-7687-459a-8857-0d41fd189315-etc-kubernetes\") pod \"tuned-nmwkn\" (UID: \"9af11dde-7687-459a-8857-0d41fd189315\") " pod="openshift-cluster-node-tuning-operator/tuned-nmwkn"
Jul 28 16:21:25.240983 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.240953    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-64snv\" (UniqueName: \"kubernetes.io/projected/9af11dde-7687-459a-8857-0d41fd189315-kube-api-access-64snv\") pod \"tuned-nmwkn\" (UID: \"9af11dde-7687-459a-8857-0d41fd189315\") " pod="openshift-cluster-node-tuning-operator/tuned-nmwkn"
Jul 28 16:21:25.241092 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.240988    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/396c56d3-19a2-49b4-bc4a-c3dabeb144b4-run-systemd\") pod \"ovnkube-node-sbf8h\" (UID: \"396c56d3-19a2-49b4-bc4a-c3dabeb144b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-sbf8h"
Jul 28 16:21:25.241092 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.241011    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/396c56d3-19a2-49b4-bc4a-c3dabeb144b4-env-overrides\") pod \"ovnkube-node-sbf8h\" (UID: \"396c56d3-19a2-49b4-bc4a-c3dabeb144b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-sbf8h"
Jul 28 16:21:25.241092 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.241034    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/92ef4728-9e89-4494-8bdf-72dcfe9b3ff4-metrics-certs\") pod \"network-metrics-daemon-qm9v7\" (UID: \"92ef4728-9e89-4494-8bdf-72dcfe9b3ff4\") " pod="openshift-multus/network-metrics-daemon-qm9v7"
Jul 28 16:21:25.241092 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.241069    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/695dbc5b-d1b4-42f3-921a-cc62010ddc14-cnibin\") pod \"multus-additional-cni-plugins-x2pv7\" (UID: \"695dbc5b-d1b4-42f3-921a-cc62010ddc14\") " pod="openshift-multus/multus-additional-cni-plugins-x2pv7"
Jul 28 16:21:25.241287 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.241091    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/695dbc5b-d1b4-42f3-921a-cc62010ddc14-os-release\") pod \"multus-additional-cni-plugins-x2pv7\" (UID: \"695dbc5b-d1b4-42f3-921a-cc62010ddc14\") " pod="openshift-multus/multus-additional-cni-plugins-x2pv7"
Jul 28 16:21:25.241287 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.241116    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/38125fdc-3616-4f16-8683-aabde8a95527-device-dir\") pod \"aws-ebs-csi-driver-node-nlb66\" (UID: \"38125fdc-3616-4f16-8683-aabde8a95527\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nlb66"
Jul 28 16:21:25.241287 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.241139    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9af11dde-7687-459a-8857-0d41fd189315-etc-sysctl-conf\") pod \"tuned-nmwkn\" (UID: \"9af11dde-7687-459a-8857-0d41fd189315\") " pod="openshift-cluster-node-tuning-operator/tuned-nmwkn"
Jul 28 16:21:25.241287 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.241220    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mhddf\" (UniqueName: \"kubernetes.io/projected/228cc225-7549-41ea-928b-23d353595cd3-kube-api-access-mhddf\") pod \"multus-6mmzq\" (UID: \"228cc225-7549-41ea-928b-23d353595cd3\") " pod="openshift-multus/multus-6mmzq"
Jul 28 16:21:25.241287 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.241243    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/396c56d3-19a2-49b4-bc4a-c3dabeb144b4-systemd-units\") pod \"ovnkube-node-sbf8h\" (UID: \"396c56d3-19a2-49b4-bc4a-c3dabeb144b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-sbf8h"
Jul 28 16:21:25.241287 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.241265    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/396c56d3-19a2-49b4-bc4a-c3dabeb144b4-node-log\") pod \"ovnkube-node-sbf8h\" (UID: \"396c56d3-19a2-49b4-bc4a-c3dabeb144b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-sbf8h"
Jul 28 16:21:25.241818 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.241289    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/31992b1b-4643-467f-a2b1-674af56e8c0e-dbus\") pod \"global-pull-secret-syncer-9vtwz\" (UID: \"31992b1b-4643-467f-a2b1-674af56e8c0e\") " pod="kube-system/global-pull-secret-syncer-9vtwz"
Jul 28 16:21:25.241818 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.241312    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7xzg4\" (UniqueName: \"kubernetes.io/projected/ecff0248-22af-459d-ae9a-083ed7508975-kube-api-access-7xzg4\") pod \"iptables-alerter-2lpp7\" (UID: \"ecff0248-22af-459d-ae9a-083ed7508975\") " pod="openshift-network-operator/iptables-alerter-2lpp7"
Jul 28 16:21:25.241818 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.241335    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/38125fdc-3616-4f16-8683-aabde8a95527-socket-dir\") pod \"aws-ebs-csi-driver-node-nlb66\" (UID: \"38125fdc-3616-4f16-8683-aabde8a95527\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nlb66"
Jul 28 16:21:25.241818 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.241355    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/228cc225-7549-41ea-928b-23d353595cd3-hostroot\") pod \"multus-6mmzq\" (UID: \"228cc225-7549-41ea-928b-23d353595cd3\") " pod="openshift-multus/multus-6mmzq"
Jul 28 16:21:25.241818 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.241410    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/396c56d3-19a2-49b4-bc4a-c3dabeb144b4-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-sbf8h\" (UID: \"396c56d3-19a2-49b4-bc4a-c3dabeb144b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-sbf8h"
Jul 28 16:21:25.241818 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.241484    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/38125fdc-3616-4f16-8683-aabde8a95527-kubelet-dir\") pod \"aws-ebs-csi-driver-node-nlb66\" (UID: \"38125fdc-3616-4f16-8683-aabde8a95527\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nlb66"
Jul 28 16:21:25.241818 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.241524    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9af11dde-7687-459a-8857-0d41fd189315-var-lib-kubelet\") pod \"tuned-nmwkn\" (UID: \"9af11dde-7687-459a-8857-0d41fd189315\") " pod="openshift-cluster-node-tuning-operator/tuned-nmwkn"
Jul 28 16:21:25.241818 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.241546    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/228cc225-7549-41ea-928b-23d353595cd3-host-run-multus-certs\") pod \"multus-6mmzq\" (UID: \"228cc225-7549-41ea-928b-23d353595cd3\") " pod="openshift-multus/multus-6mmzq"
Jul 28 16:21:25.241818 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.241576    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/228cc225-7549-41ea-928b-23d353595cd3-etc-kubernetes\") pod \"multus-6mmzq\" (UID: \"228cc225-7549-41ea-928b-23d353595cd3\") " pod="openshift-multus/multus-6mmzq"
Jul 28 16:21:25.241818 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.241610    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/695dbc5b-d1b4-42f3-921a-cc62010ddc14-system-cni-dir\") pod \"multus-additional-cni-plugins-x2pv7\" (UID: \"695dbc5b-d1b4-42f3-921a-cc62010ddc14\") " pod="openshift-multus/multus-additional-cni-plugins-x2pv7"
Jul 28 16:21:25.241818 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.241632    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9af11dde-7687-459a-8857-0d41fd189315-etc-modprobe-d\") pod \"tuned-nmwkn\" (UID: \"9af11dde-7687-459a-8857-0d41fd189315\") " pod="openshift-cluster-node-tuning-operator/tuned-nmwkn"
Jul 28 16:21:25.241818 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.241655    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/228cc225-7549-41ea-928b-23d353595cd3-cni-binary-copy\") pod \"multus-6mmzq\" (UID: \"228cc225-7549-41ea-928b-23d353595cd3\") " pod="openshift-multus/multus-6mmzq"
Jul 28 16:21:25.241818 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.241680    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2zdzz\" (UniqueName: \"kubernetes.io/projected/92ef4728-9e89-4494-8bdf-72dcfe9b3ff4-kube-api-access-2zdzz\") pod \"network-metrics-daemon-qm9v7\" (UID: \"92ef4728-9e89-4494-8bdf-72dcfe9b3ff4\") " pod="openshift-multus/network-metrics-daemon-qm9v7"
Jul 28 16:21:25.241818 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.241705    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/695dbc5b-d1b4-42f3-921a-cc62010ddc14-tuning-conf-dir\") pod \"multus-additional-cni-plugins-x2pv7\" (UID: \"695dbc5b-d1b4-42f3-921a-cc62010ddc14\") " pod="openshift-multus/multus-additional-cni-plugins-x2pv7"
Jul 28 16:21:25.241818 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.241774    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9af11dde-7687-459a-8857-0d41fd189315-etc-sysctl-d\") pod \"tuned-nmwkn\" (UID: \"9af11dde-7687-459a-8857-0d41fd189315\") " pod="openshift-cluster-node-tuning-operator/tuned-nmwkn"
Jul 28 16:21:25.241818 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.241795    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/396c56d3-19a2-49b4-bc4a-c3dabeb144b4-host-kubelet\") pod \"ovnkube-node-sbf8h\" (UID: \"396c56d3-19a2-49b4-bc4a-c3dabeb144b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-sbf8h"
Jul 28 16:21:25.242532 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.241837    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/31992b1b-4643-467f-a2b1-674af56e8c0e-original-pull-secret\") pod \"global-pull-secret-syncer-9vtwz\" (UID: \"31992b1b-4643-467f-a2b1-674af56e8c0e\") " pod="kube-system/global-pull-secret-syncer-9vtwz"
Jul 28 16:21:25.242532 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.241861    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qbqd4\" (UniqueName: \"kubernetes.io/projected/695dbc5b-d1b4-42f3-921a-cc62010ddc14-kube-api-access-qbqd4\") pod \"multus-additional-cni-plugins-x2pv7\" (UID: \"695dbc5b-d1b4-42f3-921a-cc62010ddc14\") " pod="openshift-multus/multus-additional-cni-plugins-x2pv7"
Jul 28 16:21:25.242532 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.241883    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9af11dde-7687-459a-8857-0d41fd189315-run\") pod \"tuned-nmwkn\" (UID: \"9af11dde-7687-459a-8857-0d41fd189315\") " pod="openshift-cluster-node-tuning-operator/tuned-nmwkn"
Jul 28 16:21:25.242532 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.241904    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9af11dde-7687-459a-8857-0d41fd189315-sys\") pod \"tuned-nmwkn\" (UID: \"9af11dde-7687-459a-8857-0d41fd189315\") " pod="openshift-cluster-node-tuning-operator/tuned-nmwkn"
Jul 28 16:21:25.242532 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.241928    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/228cc225-7549-41ea-928b-23d353595cd3-host-run-netns\") pod \"multus-6mmzq\" (UID: \"228cc225-7549-41ea-928b-23d353595cd3\") " pod="openshift-multus/multus-6mmzq"
Jul 28 16:21:25.242532 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.242004    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/396c56d3-19a2-49b4-bc4a-c3dabeb144b4-host-run-ovn-kubernetes\") pod \"ovnkube-node-sbf8h\" (UID: \"396c56d3-19a2-49b4-bc4a-c3dabeb144b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-sbf8h"
Jul 28 16:21:25.242532 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.242026    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/396c56d3-19a2-49b4-bc4a-c3dabeb144b4-ovnkube-script-lib\") pod \"ovnkube-node-sbf8h\" (UID: \"396c56d3-19a2-49b4-bc4a-c3dabeb144b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-sbf8h"
Jul 28 16:21:25.242532 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.242065    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ff81c208-be53-4aff-9993-47b1965201b3-serviceca\") pod \"node-ca-f8l6z\" (UID: \"ff81c208-be53-4aff-9993-47b1965201b3\") " pod="openshift-image-registry/node-ca-f8l6z"
Jul 28 16:21:25.242532 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.242099    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hq7l9\" (UniqueName: \"kubernetes.io/projected/ff81c208-be53-4aff-9993-47b1965201b3-kube-api-access-hq7l9\") pod \"node-ca-f8l6z\" (UID: \"ff81c208-be53-4aff-9993-47b1965201b3\") " pod="openshift-image-registry/node-ca-f8l6z"
Jul 28 16:21:25.242532 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.242122    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9af11dde-7687-459a-8857-0d41fd189315-tmp\") pod \"tuned-nmwkn\" (UID: \"9af11dde-7687-459a-8857-0d41fd189315\") " pod="openshift-cluster-node-tuning-operator/tuned-nmwkn"
Jul 28 16:21:25.242532 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.242146    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/228cc225-7549-41ea-928b-23d353595cd3-host-var-lib-kubelet\") pod \"multus-6mmzq\" (UID: \"228cc225-7549-41ea-928b-23d353595cd3\") " pod="openshift-multus/multus-6mmzq"
Jul 28 16:21:25.242532 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.242179    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/396c56d3-19a2-49b4-bc4a-c3dabeb144b4-run-ovn\") pod \"ovnkube-node-sbf8h\" (UID: \"396c56d3-19a2-49b4-bc4a-c3dabeb144b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-sbf8h"
Jul 28 16:21:25.242532 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.242217    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pdnhd\" (UniqueName: \"kubernetes.io/projected/396c56d3-19a2-49b4-bc4a-c3dabeb144b4-kube-api-access-pdnhd\") pod \"ovnkube-node-sbf8h\" (UID: \"396c56d3-19a2-49b4-bc4a-c3dabeb144b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-sbf8h"
Jul 28 16:21:25.242532 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.242251    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/228cc225-7549-41ea-928b-23d353595cd3-multus-cni-dir\") pod \"multus-6mmzq\" (UID: \"228cc225-7549-41ea-928b-23d353595cd3\") " pod="openshift-multus/multus-6mmzq"
Jul 28 16:21:25.242532 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.242294    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/228cc225-7549-41ea-928b-23d353595cd3-multus-socket-dir-parent\") pod \"multus-6mmzq\" (UID: \"228cc225-7549-41ea-928b-23d353595cd3\") " pod="openshift-multus/multus-6mmzq"
Jul 28 16:21:25.242532 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.242327    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ecff0248-22af-459d-ae9a-083ed7508975-iptables-alerter-script\") pod \"iptables-alerter-2lpp7\" (UID: \"ecff0248-22af-459d-ae9a-083ed7508975\") " pod="openshift-network-operator/iptables-alerter-2lpp7"
Jul 28 16:21:25.243315 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.242360    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b6c82e3b-ae45-41a0-b919-d4fbd8c3fc64-tmp-dir\") pod \"node-resolver-rn2z4\" (UID: \"b6c82e3b-ae45-41a0-b919-d4fbd8c3fc64\") " pod="openshift-dns/node-resolver-rn2z4"
Jul 28 16:21:25.243315 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.242389    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/da6efa62-02e3-4cac-8881-e62018734a17-konnectivity-ca\") pod \"konnectivity-agent-6rk8j\" (UID: \"da6efa62-02e3-4cac-8881-e62018734a17\") " pod="kube-system/konnectivity-agent-6rk8j"
Jul 28 16:21:25.243315 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.242424    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/396c56d3-19a2-49b4-bc4a-c3dabeb144b4-etc-openvswitch\") pod \"ovnkube-node-sbf8h\" (UID: \"396c56d3-19a2-49b4-bc4a-c3dabeb144b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-sbf8h"
Jul 28 16:21:25.243315 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.242448    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/396c56d3-19a2-49b4-bc4a-c3dabeb144b4-log-socket\") pod \"ovnkube-node-sbf8h\" (UID: \"396c56d3-19a2-49b4-bc4a-c3dabeb144b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-sbf8h"
Jul 28 16:21:25.243315 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.242471    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/228cc225-7549-41ea-928b-23d353595cd3-cnibin\") pod \"multus-6mmzq\" (UID: \"228cc225-7549-41ea-928b-23d353595cd3\") " pod="openshift-multus/multus-6mmzq"
Jul 28 16:21:25.243315 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.242510    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/31992b1b-4643-467f-a2b1-674af56e8c0e-kubelet-config\") pod \"global-pull-secret-syncer-9vtwz\" (UID: \"31992b1b-4643-467f-a2b1-674af56e8c0e\") " pod="kube-system/global-pull-secret-syncer-9vtwz"
Jul 28 16:21:25.243315 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.242562    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/38125fdc-3616-4f16-8683-aabde8a95527-sys-fs\") pod \"aws-ebs-csi-driver-node-nlb66\" (UID: \"38125fdc-3616-4f16-8683-aabde8a95527\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nlb66"
Jul 28 16:21:25.243315 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.242588    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/228cc225-7549-41ea-928b-23d353595cd3-system-cni-dir\") pod \"multus-6mmzq\" (UID: \"228cc225-7549-41ea-928b-23d353595cd3\") " pod="openshift-multus/multus-6mmzq"
Jul 28 16:21:25.243315 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.242615    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ecff0248-22af-459d-ae9a-083ed7508975-host-slash\") pod \"iptables-alerter-2lpp7\" (UID: \"ecff0248-22af-459d-ae9a-083ed7508975\") " pod="openshift-network-operator/iptables-alerter-2lpp7"
Jul 28 16:21:25.243315 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.242640    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bzfpk\" (UniqueName: \"kubernetes.io/projected/38125fdc-3616-4f16-8683-aabde8a95527-kube-api-access-bzfpk\") pod \"aws-ebs-csi-driver-node-nlb66\" (UID: \"38125fdc-3616-4f16-8683-aabde8a95527\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nlb66"
Jul 28 16:21:25.243315 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.242668    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/228cc225-7549-41ea-928b-23d353595cd3-os-release\") pod \"multus-6mmzq\" (UID: \"228cc225-7549-41ea-928b-23d353595cd3\") " pod="openshift-multus/multus-6mmzq"
Jul 28 16:21:25.243315 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.242692    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/396c56d3-19a2-49b4-bc4a-c3dabeb144b4-ovn-node-metrics-cert\") pod \"ovnkube-node-sbf8h\" (UID: \"396c56d3-19a2-49b4-bc4a-c3dabeb144b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-sbf8h"
Jul 28 16:21:25.243315 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.242804    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ff81c208-be53-4aff-9993-47b1965201b3-host\") pod \"node-ca-f8l6z\" (UID: \"ff81c208-be53-4aff-9993-47b1965201b3\") " pod="openshift-image-registry/node-ca-f8l6z"
Jul 28 16:21:25.243315 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.242836    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9af11dde-7687-459a-8857-0d41fd189315-lib-modules\") pod \"tuned-nmwkn\" (UID: \"9af11dde-7687-459a-8857-0d41fd189315\") " pod="openshift-cluster-node-tuning-operator/tuned-nmwkn"
Jul 28 16:21:25.243315 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.242860    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/228cc225-7549-41ea-928b-23d353595cd3-host-var-lib-cni-multus\") pod \"multus-6mmzq\" (UID: \"228cc225-7549-41ea-928b-23d353595cd3\") " pod="openshift-multus/multus-6mmzq"
Jul 28 16:21:25.243315 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.242889    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/228cc225-7549-41ea-928b-23d353595cd3-multus-daemon-config\") pod \"multus-6mmzq\" (UID: \"228cc225-7549-41ea-928b-23d353595cd3\") " pod="openshift-multus/multus-6mmzq"
Jul 28 16:21:25.243315 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.242919    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/396c56d3-19a2-49b4-bc4a-c3dabeb144b4-ovnkube-config\") pod \"ovnkube-node-sbf8h\" (UID: \"396c56d3-19a2-49b4-bc4a-c3dabeb144b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-sbf8h"
Jul 28 16:21:25.244009 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.242943    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xl2sp\" (UniqueName: \"kubernetes.io/projected/c687045e-f5d0-4a22-b71e-9c01463677e6-kube-api-access-xl2sp\") pod \"network-check-target-qzkm5\" (UID: \"c687045e-f5d0-4a22-b71e-9c01463677e6\") " pod="openshift-network-diagnostics/network-check-target-qzkm5"
Jul 28 16:21:25.244009 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.242977    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b6c82e3b-ae45-41a0-b919-d4fbd8c3fc64-hosts-file\") pod \"node-resolver-rn2z4\" (UID: \"b6c82e3b-ae45-41a0-b919-d4fbd8c3fc64\") " pod="openshift-dns/node-resolver-rn2z4"
Jul 28 16:21:25.244009 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.243017    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/38125fdc-3616-4f16-8683-aabde8a95527-registration-dir\") pod \"aws-ebs-csi-driver-node-nlb66\" (UID: \"38125fdc-3616-4f16-8683-aabde8a95527\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nlb66"
Jul 28 16:21:25.244009 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.243041    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9af11dde-7687-459a-8857-0d41fd189315-etc-sysconfig\") pod \"tuned-nmwkn\" (UID: \"9af11dde-7687-459a-8857-0d41fd189315\") " pod="openshift-cluster-node-tuning-operator/tuned-nmwkn"
Jul 28 16:21:25.244009 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.243065    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/228cc225-7549-41ea-928b-23d353595cd3-host-run-k8s-cni-cncf-io\") pod \"multus-6mmzq\" (UID: \"228cc225-7549-41ea-928b-23d353595cd3\") " pod="openshift-multus/multus-6mmzq"
Jul 28 16:21:25.244009 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.243091    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/228cc225-7549-41ea-928b-23d353595cd3-multus-conf-dir\") pod \"multus-6mmzq\" (UID: \"228cc225-7549-41ea-928b-23d353595cd3\") " pod="openshift-multus/multus-6mmzq"
Jul 28 16:21:25.244009 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.243125    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/396c56d3-19a2-49b4-bc4a-c3dabeb144b4-host-cni-netd\") pod \"ovnkube-node-sbf8h\" (UID: \"396c56d3-19a2-49b4-bc4a-c3dabeb144b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-sbf8h"
Jul 28 16:21:25.244009 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.243167    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-79wql\" (UniqueName: \"kubernetes.io/projected/b6c82e3b-ae45-41a0-b919-d4fbd8c3fc64-kube-api-access-79wql\") pod \"node-resolver-rn2z4\" (UID: \"b6c82e3b-ae45-41a0-b919-d4fbd8c3fc64\") " pod="openshift-dns/node-resolver-rn2z4"
Jul 28 16:21:25.244009 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.243214    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/695dbc5b-d1b4-42f3-921a-cc62010ddc14-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-x2pv7\" (UID: \"695dbc5b-d1b4-42f3-921a-cc62010ddc14\") " pod="openshift-multus/multus-additional-cni-plugins-x2pv7"
Jul 28 16:21:25.244009 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.243267    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/695dbc5b-d1b4-42f3-921a-cc62010ddc14-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-x2pv7\" (UID: \"695dbc5b-d1b4-42f3-921a-cc62010ddc14\") " pod="openshift-multus/multus-additional-cni-plugins-x2pv7"
Jul 28 16:21:25.244009 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.243387    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9af11dde-7687-459a-8857-0d41fd189315-host\") pod \"tuned-nmwkn\" (UID: \"9af11dde-7687-459a-8857-0d41fd189315\") " pod="openshift-cluster-node-tuning-operator/tuned-nmwkn"
Jul 28 16:21:25.244009 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.243416    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/396c56d3-19a2-49b4-bc4a-c3dabeb144b4-host-run-netns\") pod \"ovnkube-node-sbf8h\" (UID: \"396c56d3-19a2-49b4-bc4a-c3dabeb144b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-sbf8h"
Jul 28 16:21:25.244009 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.243451    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/396c56d3-19a2-49b4-bc4a-c3dabeb144b4-host-cni-bin\") pod \"ovnkube-node-sbf8h\" (UID: \"396c56d3-19a2-49b4-bc4a-c3dabeb144b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-sbf8h"
Jul 28 16:21:25.244009 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.243481    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9af11dde-7687-459a-8857-0d41fd189315-etc-systemd\") pod \"tuned-nmwkn\" (UID: \"9af11dde-7687-459a-8857-0d41fd189315\") " pod="openshift-cluster-node-tuning-operator/tuned-nmwkn"
Jul 28 16:21:25.244009 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.243525    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9af11dde-7687-459a-8857-0d41fd189315-etc-tuned\") pod \"tuned-nmwkn\" (UID: \"9af11dde-7687-459a-8857-0d41fd189315\") " pod="openshift-cluster-node-tuning-operator/tuned-nmwkn"
Jul 28 16:21:25.244009 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.243562    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/228cc225-7549-41ea-928b-23d353595cd3-host-var-lib-cni-bin\") pod \"multus-6mmzq\" (UID: \"228cc225-7549-41ea-928b-23d353595cd3\") " pod="openshift-multus/multus-6mmzq"
Jul 28 16:21:25.244707 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.243609    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/396c56d3-19a2-49b4-bc4a-c3dabeb144b4-host-slash\") pod \"ovnkube-node-sbf8h\" (UID: \"396c56d3-19a2-49b4-bc4a-c3dabeb144b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-sbf8h"
Jul 28 16:21:25.244707 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.243651    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/396c56d3-19a2-49b4-bc4a-c3dabeb144b4-var-lib-openvswitch\") pod \"ovnkube-node-sbf8h\" (UID: \"396c56d3-19a2-49b4-bc4a-c3dabeb144b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-sbf8h"
Jul 28 16:21:25.244707 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.243680    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/396c56d3-19a2-49b4-bc4a-c3dabeb144b4-run-openvswitch\") pod \"ovnkube-node-sbf8h\" (UID: \"396c56d3-19a2-49b4-bc4a-c3dabeb144b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-sbf8h"
Jul 28 16:21:25.248876 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.248855    2557 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 28 16:21:25.274896 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.274878    2557 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-m5pjk"
Jul 28 16:21:25.279968 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.279948    2557 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-m5pjk"
Jul 28 16:21:25.333151 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.333126    2557 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 28 16:21:25.344873 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.344850    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/695dbc5b-d1b4-42f3-921a-cc62010ddc14-system-cni-dir\") pod \"multus-additional-cni-plugins-x2pv7\" (UID: \"695dbc5b-d1b4-42f3-921a-cc62010ddc14\") " pod="openshift-multus/multus-additional-cni-plugins-x2pv7"
Jul 28 16:21:25.344977 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.344884    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9af11dde-7687-459a-8857-0d41fd189315-etc-modprobe-d\") pod \"tuned-nmwkn\" (UID: \"9af11dde-7687-459a-8857-0d41fd189315\") " pod="openshift-cluster-node-tuning-operator/tuned-nmwkn"
Jul 28 16:21:25.344977 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.344911    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/228cc225-7549-41ea-928b-23d353595cd3-cni-binary-copy\") pod \"multus-6mmzq\" (UID: \"228cc225-7549-41ea-928b-23d353595cd3\") " pod="openshift-multus/multus-6mmzq"
Jul 28 16:21:25.344977 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.344930    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2zdzz\" (UniqueName: \"kubernetes.io/projected/92ef4728-9e89-4494-8bdf-72dcfe9b3ff4-kube-api-access-2zdzz\") pod \"network-metrics-daemon-qm9v7\" (UID: \"92ef4728-9e89-4494-8bdf-72dcfe9b3ff4\") " pod="openshift-multus/network-metrics-daemon-qm9v7"
Jul 28 16:21:25.344977 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.344948    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/695dbc5b-d1b4-42f3-921a-cc62010ddc14-tuning-conf-dir\") pod \"multus-additional-cni-plugins-x2pv7\" (UID: \"695dbc5b-d1b4-42f3-921a-cc62010ddc14\") " pod="openshift-multus/multus-additional-cni-plugins-x2pv7"
Jul 28 16:21:25.344977 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.344967    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/695dbc5b-d1b4-42f3-921a-cc62010ddc14-system-cni-dir\") pod \"multus-additional-cni-plugins-x2pv7\" (UID: \"695dbc5b-d1b4-42f3-921a-cc62010ddc14\") " pod="openshift-multus/multus-additional-cni-plugins-x2pv7"
Jul 28 16:21:25.345221 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.344980    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9af11dde-7687-459a-8857-0d41fd189315-etc-sysctl-d\") pod \"tuned-nmwkn\" (UID: \"9af11dde-7687-459a-8857-0d41fd189315\") " pod="openshift-cluster-node-tuning-operator/tuned-nmwkn"
Jul 28 16:21:25.345221 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.345018    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/396c56d3-19a2-49b4-bc4a-c3dabeb144b4-host-kubelet\") pod \"ovnkube-node-sbf8h\" (UID: \"396c56d3-19a2-49b4-bc4a-c3dabeb144b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-sbf8h"
Jul 28 16:21:25.345221 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.345061    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/31992b1b-4643-467f-a2b1-674af56e8c0e-original-pull-secret\") pod \"global-pull-secret-syncer-9vtwz\" (UID: \"31992b1b-4643-467f-a2b1-674af56e8c0e\") " pod="kube-system/global-pull-secret-syncer-9vtwz"
Jul 28 16:21:25.345221 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.345122    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9af11dde-7687-459a-8857-0d41fd189315-etc-modprobe-d\") pod \"tuned-nmwkn\" (UID: \"9af11dde-7687-459a-8857-0d41fd189315\") " pod="openshift-cluster-node-tuning-operator/tuned-nmwkn"
Jul 28 16:21:25.345221 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.345165    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/695dbc5b-d1b4-42f3-921a-cc62010ddc14-tuning-conf-dir\") pod \"multus-additional-cni-plugins-x2pv7\" (UID: \"695dbc5b-d1b4-42f3-921a-cc62010ddc14\") " pod="openshift-multus/multus-additional-cni-plugins-x2pv7"
Jul 28 16:21:25.345221 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.345187    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/396c56d3-19a2-49b4-bc4a-c3dabeb144b4-host-kubelet\") pod \"ovnkube-node-sbf8h\" (UID: \"396c56d3-19a2-49b4-bc4a-c3dabeb144b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-sbf8h"
Jul 28 16:21:25.345221 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.345198    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qbqd4\" (UniqueName: \"kubernetes.io/projected/695dbc5b-d1b4-42f3-921a-cc62010ddc14-kube-api-access-qbqd4\") pod \"multus-additional-cni-plugins-x2pv7\" (UID: \"695dbc5b-d1b4-42f3-921a-cc62010ddc14\") " pod="openshift-multus/multus-additional-cni-plugins-x2pv7"
Jul 28 16:21:25.345221 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.345224    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9af11dde-7687-459a-8857-0d41fd189315-run\") pod \"tuned-nmwkn\" (UID: \"9af11dde-7687-459a-8857-0d41fd189315\") " pod="openshift-cluster-node-tuning-operator/tuned-nmwkn"
Jul 28 16:21:25.345475 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.345247    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9af11dde-7687-459a-8857-0d41fd189315-sys\") pod \"tuned-nmwkn\" (UID: \"9af11dde-7687-459a-8857-0d41fd189315\") " pod="openshift-cluster-node-tuning-operator/tuned-nmwkn"
Jul 28 16:21:25.345475 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:25.345261    2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 16:21:25.345475 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.345268    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/228cc225-7549-41ea-928b-23d353595cd3-host-run-netns\") pod \"multus-6mmzq\" (UID: \"228cc225-7549-41ea-928b-23d353595cd3\") " pod="openshift-multus/multus-6mmzq"
Jul 28 16:21:25.345475 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.345304    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9af11dde-7687-459a-8857-0d41fd189315-run\") pod \"tuned-nmwkn\" (UID: \"9af11dde-7687-459a-8857-0d41fd189315\") " pod="openshift-cluster-node-tuning-operator/tuned-nmwkn"
Jul 28 16:21:25.345475 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:25.345333    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31992b1b-4643-467f-a2b1-674af56e8c0e-original-pull-secret podName:31992b1b-4643-467f-a2b1-674af56e8c0e nodeName:}" failed. No retries permitted until 2026-07-28 16:21:25.84529535 +0000 UTC m=+2.983764743 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/31992b1b-4643-467f-a2b1-674af56e8c0e-original-pull-secret") pod "global-pull-secret-syncer-9vtwz" (UID: "31992b1b-4643-467f-a2b1-674af56e8c0e") : object "kube-system"/"original-pull-secret" not registered
Jul 28 16:21:25.345475 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.345353    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/396c56d3-19a2-49b4-bc4a-c3dabeb144b4-host-run-ovn-kubernetes\") pod \"ovnkube-node-sbf8h\" (UID: \"396c56d3-19a2-49b4-bc4a-c3dabeb144b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-sbf8h"
Jul 28 16:21:25.345475 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.345360    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/228cc225-7549-41ea-928b-23d353595cd3-host-run-netns\") pod \"multus-6mmzq\" (UID: \"228cc225-7549-41ea-928b-23d353595cd3\") " pod="openshift-multus/multus-6mmzq"
Jul 28 16:21:25.345475 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.345375    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/396c56d3-19a2-49b4-bc4a-c3dabeb144b4-ovnkube-script-lib\") pod \"ovnkube-node-sbf8h\" (UID: \"396c56d3-19a2-49b4-bc4a-c3dabeb144b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-sbf8h"
Jul 28 16:21:25.345475 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.345379    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9af11dde-7687-459a-8857-0d41fd189315-sys\") pod \"tuned-nmwkn\" (UID: \"9af11dde-7687-459a-8857-0d41fd189315\") " pod="openshift-cluster-node-tuning-operator/tuned-nmwkn"
Jul 28 16:21:25.345475 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.345395    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ff81c208-be53-4aff-9993-47b1965201b3-serviceca\") pod \"node-ca-f8l6z\" (UID: \"ff81c208-be53-4aff-9993-47b1965201b3\") " pod="openshift-image-registry/node-ca-f8l6z"
Jul 28 16:21:25.345475 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.345428    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/396c56d3-19a2-49b4-bc4a-c3dabeb144b4-host-run-ovn-kubernetes\") pod \"ovnkube-node-sbf8h\" (UID: \"396c56d3-19a2-49b4-bc4a-c3dabeb144b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-sbf8h"
Jul 28 16:21:25.345475 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.345435    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hq7l9\" (UniqueName: \"kubernetes.io/projected/ff81c208-be53-4aff-9993-47b1965201b3-kube-api-access-hq7l9\") pod \"node-ca-f8l6z\" (UID: \"ff81c208-be53-4aff-9993-47b1965201b3\") " pod="openshift-image-registry/node-ca-f8l6z"
Jul 28 16:21:25.345475 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.345454    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9af11dde-7687-459a-8857-0d41fd189315-tmp\") pod \"tuned-nmwkn\" (UID: \"9af11dde-7687-459a-8857-0d41fd189315\") " pod="openshift-cluster-node-tuning-operator/tuned-nmwkn"
Jul 28 16:21:25.345475 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.345468    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/228cc225-7549-41ea-928b-23d353595cd3-host-var-lib-kubelet\") pod \"multus-6mmzq\" (UID: \"228cc225-7549-41ea-928b-23d353595cd3\") " pod="openshift-multus/multus-6mmzq"
Jul 28 16:21:25.345955 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.345510    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/396c56d3-19a2-49b4-bc4a-c3dabeb144b4-run-ovn\") pod \"ovnkube-node-sbf8h\" (UID: \"396c56d3-19a2-49b4-bc4a-c3dabeb144b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-sbf8h"
Jul 28 16:21:25.345955 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.345525    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pdnhd\" (UniqueName: \"kubernetes.io/projected/396c56d3-19a2-49b4-bc4a-c3dabeb144b4-kube-api-access-pdnhd\") pod \"ovnkube-node-sbf8h\" (UID: \"396c56d3-19a2-49b4-bc4a-c3dabeb144b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-sbf8h"
Jul 28 16:21:25.345955 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.345544    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/228cc225-7549-41ea-928b-23d353595cd3-multus-cni-dir\") pod \"multus-6mmzq\" (UID: \"228cc225-7549-41ea-928b-23d353595cd3\") " pod="openshift-multus/multus-6mmzq"
Jul 28 16:21:25.345955 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.345566    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/228cc225-7549-41ea-928b-23d353595cd3-multus-socket-dir-parent\") pod \"multus-6mmzq\" (UID: \"228cc225-7549-41ea-928b-23d353595cd3\") " pod="openshift-multus/multus-6mmzq"
Jul 28 16:21:25.345955 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.345586    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ecff0248-22af-459d-ae9a-083ed7508975-iptables-alerter-script\") pod \"iptables-alerter-2lpp7\" (UID: \"ecff0248-22af-459d-ae9a-083ed7508975\") " pod="openshift-network-operator/iptables-alerter-2lpp7"
Jul 28 16:21:25.345955 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.345600    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b6c82e3b-ae45-41a0-b919-d4fbd8c3fc64-tmp-dir\") pod \"node-resolver-rn2z4\" (UID: \"b6c82e3b-ae45-41a0-b919-d4fbd8c3fc64\") " pod="openshift-dns/node-resolver-rn2z4"
Jul 28 16:21:25.345955 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.345616    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/da6efa62-02e3-4cac-8881-e62018734a17-konnectivity-ca\") pod \"konnectivity-agent-6rk8j\" (UID: \"da6efa62-02e3-4cac-8881-e62018734a17\") " pod="kube-system/konnectivity-agent-6rk8j"
Jul 28 16:21:25.345955 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.345625    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/228cc225-7549-41ea-928b-23d353595cd3-host-var-lib-kubelet\") pod \"multus-6mmzq\" (UID: \"228cc225-7549-41ea-928b-23d353595cd3\") " pod="openshift-multus/multus-6mmzq"
Jul 28 16:21:25.345955 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.345631    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/396c56d3-19a2-49b4-bc4a-c3dabeb144b4-etc-openvswitch\") pod \"ovnkube-node-sbf8h\" (UID: \"396c56d3-19a2-49b4-bc4a-c3dabeb144b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-sbf8h"
Jul 28 16:21:25.345955 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.345668    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/396c56d3-19a2-49b4-bc4a-c3dabeb144b4-log-socket\") pod \"ovnkube-node-sbf8h\" (UID: \"396c56d3-19a2-49b4-bc4a-c3dabeb144b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-sbf8h"
Jul 28 16:21:25.345955 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.345675    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/228cc225-7549-41ea-928b-23d353595cd3-cni-binary-copy\") pod \"multus-6mmzq\" (UID: \"228cc225-7549-41ea-928b-23d353595cd3\") " pod="openshift-multus/multus-6mmzq"
Jul 28 16:21:25.345955 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.345683    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/228cc225-7549-41ea-928b-23d353595cd3-cnibin\") pod \"multus-6mmzq\" (UID: \"228cc225-7549-41ea-928b-23d353595cd3\") " pod="openshift-multus/multus-6mmzq"
Jul 28 16:21:25.345955 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.345707    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/31992b1b-4643-467f-a2b1-674af56e8c0e-kubelet-config\") pod \"global-pull-secret-syncer-9vtwz\" (UID: \"31992b1b-4643-467f-a2b1-674af56e8c0e\") " pod="kube-system/global-pull-secret-syncer-9vtwz"
Jul 28 16:21:25.345955 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.345729    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/38125fdc-3616-4f16-8683-aabde8a95527-sys-fs\") pod \"aws-ebs-csi-driver-node-nlb66\" (UID: \"38125fdc-3616-4f16-8683-aabde8a95527\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nlb66"
Jul 28 16:21:25.345955 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.345747    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/228cc225-7549-41ea-928b-23d353595cd3-system-cni-dir\") pod \"multus-6mmzq\" (UID: \"228cc225-7549-41ea-928b-23d353595cd3\") " pod="openshift-multus/multus-6mmzq"
Jul 28 16:21:25.345955 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.345761    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ecff0248-22af-459d-ae9a-083ed7508975-host-slash\") pod \"iptables-alerter-2lpp7\" (UID: \"ecff0248-22af-459d-ae9a-083ed7508975\") " pod="openshift-network-operator/iptables-alerter-2lpp7"
Jul 28 16:21:25.345955 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.345775    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bzfpk\" (UniqueName: \"kubernetes.io/projected/38125fdc-3616-4f16-8683-aabde8a95527-kube-api-access-bzfpk\") pod \"aws-ebs-csi-driver-node-nlb66\" (UID: \"38125fdc-3616-4f16-8683-aabde8a95527\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nlb66"
Jul 28 16:21:25.345955 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.345789    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/228cc225-7549-41ea-928b-23d353595cd3-os-release\") pod \"multus-6mmzq\" (UID: \"228cc225-7549-41ea-928b-23d353595cd3\") " pod="openshift-multus/multus-6mmzq"
Jul 28 16:21:25.346836 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.345810    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/396c56d3-19a2-49b4-bc4a-c3dabeb144b4-ovn-node-metrics-cert\") pod \"ovnkube-node-sbf8h\" (UID: \"396c56d3-19a2-49b4-bc4a-c3dabeb144b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-sbf8h"
Jul 28 16:21:25.346836 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.345829    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ff81c208-be53-4aff-9993-47b1965201b3-host\") pod \"node-ca-f8l6z\" (UID: \"ff81c208-be53-4aff-9993-47b1965201b3\") " pod="openshift-image-registry/node-ca-f8l6z"
Jul 28 16:21:25.346836 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.345856    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9af11dde-7687-459a-8857-0d41fd189315-lib-modules\") pod \"tuned-nmwkn\" (UID: \"9af11dde-7687-459a-8857-0d41fd189315\") " pod="openshift-cluster-node-tuning-operator/tuned-nmwkn"
Jul 28 16:21:25.346836 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.345870    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/228cc225-7549-41ea-928b-23d353595cd3-host-var-lib-cni-multus\") pod \"multus-6mmzq\" (UID: \"228cc225-7549-41ea-928b-23d353595cd3\") " pod="openshift-multus/multus-6mmzq"
Jul 28 16:21:25.346836 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.345890    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/396c56d3-19a2-49b4-bc4a-c3dabeb144b4-log-socket\") pod \"ovnkube-node-sbf8h\" (UID: \"396c56d3-19a2-49b4-bc4a-c3dabeb144b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-sbf8h"
Jul 28 16:21:25.346836 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.345938    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/31992b1b-4643-467f-a2b1-674af56e8c0e-kubelet-config\") pod \"global-pull-secret-syncer-9vtwz\" (UID: \"31992b1b-4643-467f-a2b1-674af56e8c0e\") " pod="kube-system/global-pull-secret-syncer-9vtwz"
Jul 28 16:21:25.346836 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.345983    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/38125fdc-3616-4f16-8683-aabde8a95527-sys-fs\") pod \"aws-ebs-csi-driver-node-nlb66\" (UID: \"38125fdc-3616-4f16-8683-aabde8a95527\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nlb66"
Jul 28 16:21:25.346836 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.346027    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/228cc225-7549-41ea-928b-23d353595cd3-system-cni-dir\") pod \"multus-6mmzq\" (UID: \"228cc225-7549-41ea-928b-23d353595cd3\") " pod="openshift-multus/multus-6mmzq"
Jul 28 16:21:25.346836 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.346054    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ecff0248-22af-459d-ae9a-083ed7508975-host-slash\") pod \"iptables-alerter-2lpp7\" (UID: \"ecff0248-22af-459d-ae9a-083ed7508975\") " pod="openshift-network-operator/iptables-alerter-2lpp7"
Jul 28 16:21:25.346836 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.346092    2557 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 28 16:21:25.346836 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.346268    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/228cc225-7549-41ea-928b-23d353595cd3-os-release\") pod \"multus-6mmzq\" (UID: \"228cc225-7549-41ea-928b-23d353595cd3\") " pod="openshift-multus/multus-6mmzq"
Jul 28 16:21:25.346836 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.346342    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ff81c208-be53-4aff-9993-47b1965201b3-serviceca\") pod \"node-ca-f8l6z\" (UID: \"ff81c208-be53-4aff-9993-47b1965201b3\") " pod="openshift-image-registry/node-ca-f8l6z"
Jul 28 16:21:25.346836 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.346403    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/228cc225-7549-41ea-928b-23d353595cd3-multus-daemon-config\") pod \"multus-6mmzq\" (UID: \"228cc225-7549-41ea-928b-23d353595cd3\") " pod="openshift-multus/multus-6mmzq"
Jul 28 16:21:25.346836 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.346460    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/228cc225-7549-41ea-928b-23d353595cd3-cnibin\") pod \"multus-6mmzq\" (UID: \"228cc225-7549-41ea-928b-23d353595cd3\") " pod="openshift-multus/multus-6mmzq"
Jul 28 16:21:25.346836 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.346553    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/396c56d3-19a2-49b4-bc4a-c3dabeb144b4-ovnkube-script-lib\") pod \"ovnkube-node-sbf8h\" (UID: \"396c56d3-19a2-49b4-bc4a-c3dabeb144b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-sbf8h"
Jul 28 16:21:25.346836 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.346596    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/396c56d3-19a2-49b4-bc4a-c3dabeb144b4-run-ovn\") pod \"ovnkube-node-sbf8h\" (UID: \"396c56d3-19a2-49b4-bc4a-c3dabeb144b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-sbf8h"
Jul 28 16:21:25.346836 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.345850    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9af11dde-7687-459a-8857-0d41fd189315-etc-sysctl-d\") pod \"tuned-nmwkn\" (UID: \"9af11dde-7687-459a-8857-0d41fd189315\") " pod="openshift-cluster-node-tuning-operator/tuned-nmwkn"
Jul 28 16:21:25.346836 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.346651    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/228cc225-7549-41ea-928b-23d353595cd3-host-var-lib-cni-multus\") pod \"multus-6mmzq\" (UID: \"228cc225-7549-41ea-928b-23d353595cd3\") " pod="openshift-multus/multus-6mmzq"
Jul 28 16:21:25.347699 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.346680    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9af11dde-7687-459a-8857-0d41fd189315-lib-modules\") pod \"tuned-nmwkn\" (UID: \"9af11dde-7687-459a-8857-0d41fd189315\") " pod="openshift-cluster-node-tuning-operator/tuned-nmwkn"
Jul 28 16:21:25.347699 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.346690    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/396c56d3-19a2-49b4-bc4a-c3dabeb144b4-etc-openvswitch\") pod \"ovnkube-node-sbf8h\" (UID: \"396c56d3-19a2-49b4-bc4a-c3dabeb144b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-sbf8h"
Jul 28 16:21:25.347699 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.346717    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ff81c208-be53-4aff-9993-47b1965201b3-host\") pod \"node-ca-f8l6z\" (UID: \"ff81c208-be53-4aff-9993-47b1965201b3\") " pod="openshift-image-registry/node-ca-f8l6z"
Jul 28 16:21:25.347699 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.345893    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/228cc225-7549-41ea-928b-23d353595cd3-multus-daemon-config\") pod \"multus-6mmzq\" (UID: \"228cc225-7549-41ea-928b-23d353595cd3\") " pod="openshift-multus/multus-6mmzq"
Jul 28 16:21:25.347699 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.346768    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/396c56d3-19a2-49b4-bc4a-c3dabeb144b4-ovnkube-config\") pod \"ovnkube-node-sbf8h\" (UID: \"396c56d3-19a2-49b4-bc4a-c3dabeb144b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-sbf8h"
Jul 28 16:21:25.347699 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.346783    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b6c82e3b-ae45-41a0-b919-d4fbd8c3fc64-tmp-dir\") pod \"node-resolver-rn2z4\" (UID: \"b6c82e3b-ae45-41a0-b919-d4fbd8c3fc64\") " pod="openshift-dns/node-resolver-rn2z4"
Jul 28 16:21:25.347699 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.346795    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xl2sp\" (UniqueName: \"kubernetes.io/projected/c687045e-f5d0-4a22-b71e-9c01463677e6-kube-api-access-xl2sp\") pod \"network-check-target-qzkm5\" (UID: \"c687045e-f5d0-4a22-b71e-9c01463677e6\") " pod="openshift-network-diagnostics/network-check-target-qzkm5"
Jul 28 16:21:25.347699 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.346819    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b6c82e3b-ae45-41a0-b919-d4fbd8c3fc64-hosts-file\") pod \"node-resolver-rn2z4\" (UID: \"b6c82e3b-ae45-41a0-b919-d4fbd8c3fc64\") " pod="openshift-dns/node-resolver-rn2z4"
Jul 28 16:21:25.347699 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.346839    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/38125fdc-3616-4f16-8683-aabde8a95527-registration-dir\") pod \"aws-ebs-csi-driver-node-nlb66\" (UID: \"38125fdc-3616-4f16-8683-aabde8a95527\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nlb66"
Jul 28 16:21:25.347699 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.346861    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9af11dde-7687-459a-8857-0d41fd189315-etc-sysconfig\") pod \"tuned-nmwkn\" (UID: \"9af11dde-7687-459a-8857-0d41fd189315\") " pod="openshift-cluster-node-tuning-operator/tuned-nmwkn"
Jul 28 16:21:25.347699 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.346898    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/228cc225-7549-41ea-928b-23d353595cd3-host-run-k8s-cni-cncf-io\") pod \"multus-6mmzq\" (UID: \"228cc225-7549-41ea-928b-23d353595cd3\") " pod="openshift-multus/multus-6mmzq"
Jul 28 16:21:25.347699 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.346924    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/228cc225-7549-41ea-928b-23d353595cd3-multus-conf-dir\") pod \"multus-6mmzq\" (UID: \"228cc225-7549-41ea-928b-23d353595cd3\") " pod="openshift-multus/multus-6mmzq"
Jul 28 16:21:25.347699 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.346936    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/228cc225-7549-41ea-928b-23d353595cd3-multus-cni-dir\") pod \"multus-6mmzq\" (UID: \"228cc225-7549-41ea-928b-23d353595cd3\") " pod="openshift-multus/multus-6mmzq"
Jul 28 16:21:25.347699 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.346950    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/396c56d3-19a2-49b4-bc4a-c3dabeb144b4-host-cni-netd\") pod \"ovnkube-node-sbf8h\" (UID: \"396c56d3-19a2-49b4-bc4a-c3dabeb144b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-sbf8h"
Jul 28 16:21:25.347699 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.346975    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-79wql\" (UniqueName: \"kubernetes.io/projected/b6c82e3b-ae45-41a0-b919-d4fbd8c3fc64-kube-api-access-79wql\") pod \"node-resolver-rn2z4\" (UID: \"b6c82e3b-ae45-41a0-b919-d4fbd8c3fc64\") " pod="openshift-dns/node-resolver-rn2z4"
Jul 28 16:21:25.347699 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.346985    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/38125fdc-3616-4f16-8683-aabde8a95527-registration-dir\") pod \"aws-ebs-csi-driver-node-nlb66\" (UID: \"38125fdc-3616-4f16-8683-aabde8a95527\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nlb66"
Jul 28 16:21:25.347699 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.346998    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/695dbc5b-d1b4-42f3-921a-cc62010ddc14-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-x2pv7\" (UID: \"695dbc5b-d1b4-42f3-921a-cc62010ddc14\") " pod="openshift-multus/multus-additional-cni-plugins-x2pv7"
Jul 28 16:21:25.348486 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.347021    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/695dbc5b-d1b4-42f3-921a-cc62010ddc14-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-x2pv7\" (UID: \"695dbc5b-d1b4-42f3-921a-cc62010ddc14\") " pod="openshift-multus/multus-additional-cni-plugins-x2pv7"
Jul 28 16:21:25.348486 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.347043    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9af11dde-7687-459a-8857-0d41fd189315-host\") pod \"tuned-nmwkn\" (UID: \"9af11dde-7687-459a-8857-0d41fd189315\") " pod="openshift-cluster-node-tuning-operator/tuned-nmwkn"
Jul 28 16:21:25.348486 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.347065    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/396c56d3-19a2-49b4-bc4a-c3dabeb144b4-host-run-netns\") pod \"ovnkube-node-sbf8h\" (UID: \"396c56d3-19a2-49b4-bc4a-c3dabeb144b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-sbf8h"
Jul 28 16:21:25.348486 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.347087    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/396c56d3-19a2-49b4-bc4a-c3dabeb144b4-host-cni-bin\") pod \"ovnkube-node-sbf8h\" (UID: \"396c56d3-19a2-49b4-bc4a-c3dabeb144b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-sbf8h"
Jul 28 16:21:25.348486 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.347111    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9af11dde-7687-459a-8857-0d41fd189315-etc-systemd\") pod \"tuned-nmwkn\" (UID: \"9af11dde-7687-459a-8857-0d41fd189315\") " pod="openshift-cluster-node-tuning-operator/tuned-nmwkn"
Jul 28 16:21:25.348486 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.347131    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9af11dde-7687-459a-8857-0d41fd189315-etc-tuned\") pod \"tuned-nmwkn\" (UID: \"9af11dde-7687-459a-8857-0d41fd189315\") " pod="openshift-cluster-node-tuning-operator/tuned-nmwkn"
Jul 28 16:21:25.348486 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.347149    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/228cc225-7549-41ea-928b-23d353595cd3-host-var-lib-cni-bin\") pod \"multus-6mmzq\" (UID: \"228cc225-7549-41ea-928b-23d353595cd3\") " pod="openshift-multus/multus-6mmzq"
Jul 28 16:21:25.348486 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.347172    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/396c56d3-19a2-49b4-bc4a-c3dabeb144b4-host-slash\") pod \"ovnkube-node-sbf8h\" (UID: \"396c56d3-19a2-49b4-bc4a-c3dabeb144b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-sbf8h"
Jul 28 16:21:25.348486 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.347194    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/396c56d3-19a2-49b4-bc4a-c3dabeb144b4-var-lib-openvswitch\") pod \"ovnkube-node-sbf8h\" (UID: \"396c56d3-19a2-49b4-bc4a-c3dabeb144b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-sbf8h"
Jul 28 16:21:25.348486 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.347217    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/396c56d3-19a2-49b4-bc4a-c3dabeb144b4-run-openvswitch\") pod \"ovnkube-node-sbf8h\" (UID: \"396c56d3-19a2-49b4-bc4a-c3dabeb144b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-sbf8h"
Jul 28 16:21:25.348486 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.347239    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/da6efa62-02e3-4cac-8881-e62018734a17-agent-certs\") pod \"konnectivity-agent-6rk8j\" (UID: \"da6efa62-02e3-4cac-8881-e62018734a17\") " pod="kube-system/konnectivity-agent-6rk8j"
Jul 28 16:21:25.348486 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.347263    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/695dbc5b-d1b4-42f3-921a-cc62010ddc14-cni-binary-copy\") pod \"multus-additional-cni-plugins-x2pv7\" (UID: \"695dbc5b-d1b4-42f3-921a-cc62010ddc14\") " pod="openshift-multus/multus-additional-cni-plugins-x2pv7"
Jul 28 16:21:25.348486 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.347292    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b6c82e3b-ae45-41a0-b919-d4fbd8c3fc64-hosts-file\") pod \"node-resolver-rn2z4\" (UID: \"b6c82e3b-ae45-41a0-b919-d4fbd8c3fc64\") " pod="openshift-dns/node-resolver-rn2z4"
Jul 28 16:21:25.348486 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.347305    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/da6efa62-02e3-4cac-8881-e62018734a17-konnectivity-ca\") pod \"konnectivity-agent-6rk8j\" (UID: \"da6efa62-02e3-4cac-8881-e62018734a17\") " pod="kube-system/konnectivity-agent-6rk8j"
Jul 28 16:21:25.348486 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.347320    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/228cc225-7549-41ea-928b-23d353595cd3-multus-socket-dir-parent\") pod \"multus-6mmzq\" (UID: \"228cc225-7549-41ea-928b-23d353595cd3\") " pod="openshift-multus/multus-6mmzq"
Jul 28 16:21:25.348486 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.347289    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/38125fdc-3616-4f16-8683-aabde8a95527-etc-selinux\") pod \"aws-ebs-csi-driver-node-nlb66\" (UID: \"38125fdc-3616-4f16-8683-aabde8a95527\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nlb66"
Jul 28 16:21:25.348486 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.347353    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/228cc225-7549-41ea-928b-23d353595cd3-host-run-k8s-cni-cncf-io\") pod \"multus-6mmzq\" (UID: \"228cc225-7549-41ea-928b-23d353595cd3\") " pod="openshift-multus/multus-6mmzq"
Jul 28 16:21:25.349229 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.347384    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9af11dde-7687-459a-8857-0d41fd189315-etc-kubernetes\") pod \"tuned-nmwkn\" (UID: \"9af11dde-7687-459a-8857-0d41fd189315\") " pod="openshift-cluster-node-tuning-operator/tuned-nmwkn"
Jul 28 16:21:25.349229 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.347399    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/38125fdc-3616-4f16-8683-aabde8a95527-etc-selinux\") pod \"aws-ebs-csi-driver-node-nlb66\" (UID: \"38125fdc-3616-4f16-8683-aabde8a95527\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nlb66"
Jul 28 16:21:25.349229 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.347431    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-64snv\" (UniqueName: \"kubernetes.io/projected/9af11dde-7687-459a-8857-0d41fd189315-kube-api-access-64snv\") pod \"tuned-nmwkn\" (UID: \"9af11dde-7687-459a-8857-0d41fd189315\") " pod="openshift-cluster-node-tuning-operator/tuned-nmwkn"
Jul 28 16:21:25.349229 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.347443    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9af11dde-7687-459a-8857-0d41fd189315-etc-systemd\") pod \"tuned-nmwkn\" (UID: \"9af11dde-7687-459a-8857-0d41fd189315\") " pod="openshift-cluster-node-tuning-operator/tuned-nmwkn"
Jul 28 16:21:25.349229 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.347455    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/396c56d3-19a2-49b4-bc4a-c3dabeb144b4-run-systemd\") pod \"ovnkube-node-sbf8h\" (UID: \"396c56d3-19a2-49b4-bc4a-c3dabeb144b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-sbf8h"
Jul 28 16:21:25.349229 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.347465    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/396c56d3-19a2-49b4-bc4a-c3dabeb144b4-host-cni-bin\") pod \"ovnkube-node-sbf8h\" (UID: \"396c56d3-19a2-49b4-bc4a-c3dabeb144b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-sbf8h"
Jul 28 16:21:25.349229 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.347476    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/396c56d3-19a2-49b4-bc4a-c3dabeb144b4-env-overrides\") pod \"ovnkube-node-sbf8h\" (UID: \"396c56d3-19a2-49b4-bc4a-c3dabeb144b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-sbf8h"
Jul 28 16:21:25.349229 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.347527    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/92ef4728-9e89-4494-8bdf-72dcfe9b3ff4-metrics-certs\") pod \"network-metrics-daemon-qm9v7\" (UID: \"92ef4728-9e89-4494-8bdf-72dcfe9b3ff4\") " pod="openshift-multus/network-metrics-daemon-qm9v7"
Jul 28 16:21:25.349229 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.347555    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/695dbc5b-d1b4-42f3-921a-cc62010ddc14-cnibin\") pod \"multus-additional-cni-plugins-x2pv7\" (UID: \"695dbc5b-d1b4-42f3-921a-cc62010ddc14\") " pod="openshift-multus/multus-additional-cni-plugins-x2pv7"
Jul 28 16:21:25.349229 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.347580    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/695dbc5b-d1b4-42f3-921a-cc62010ddc14-os-release\") pod \"multus-additional-cni-plugins-x2pv7\" (UID: \"695dbc5b-d1b4-42f3-921a-cc62010ddc14\") " pod="openshift-multus/multus-additional-cni-plugins-x2pv7"
Jul 28 16:21:25.349229 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.347604    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/38125fdc-3616-4f16-8683-aabde8a95527-device-dir\") pod \"aws-ebs-csi-driver-node-nlb66\" (UID: \"38125fdc-3616-4f16-8683-aabde8a95527\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nlb66"
Jul 28 16:21:25.349229 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.347639    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9af11dde-7687-459a-8857-0d41fd189315-etc-sysctl-conf\") pod \"tuned-nmwkn\" (UID: \"9af11dde-7687-459a-8857-0d41fd189315\") " pod="openshift-cluster-node-tuning-operator/tuned-nmwkn"
Jul 28 16:21:25.349229 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.347666    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mhddf\" (UniqueName: \"kubernetes.io/projected/228cc225-7549-41ea-928b-23d353595cd3-kube-api-access-mhddf\") pod \"multus-6mmzq\" (UID: \"228cc225-7549-41ea-928b-23d353595cd3\") " pod="openshift-multus/multus-6mmzq"
Jul 28 16:21:25.349229 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:25.347695    2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 16:21:25.349229 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.347716    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/396c56d3-19a2-49b4-bc4a-c3dabeb144b4-systemd-units\") pod \"ovnkube-node-sbf8h\" (UID: \"396c56d3-19a2-49b4-bc4a-c3dabeb144b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-sbf8h"
Jul 28 16:21:25.349229 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:25.347758    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92ef4728-9e89-4494-8bdf-72dcfe9b3ff4-metrics-certs podName:92ef4728-9e89-4494-8bdf-72dcfe9b3ff4 nodeName:}" failed. No retries permitted until 2026-07-28 16:21:25.847732579 +0000 UTC m=+2.986201988 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/92ef4728-9e89-4494-8bdf-72dcfe9b3ff4-metrics-certs") pod "network-metrics-daemon-qm9v7" (UID: "92ef4728-9e89-4494-8bdf-72dcfe9b3ff4") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 16:21:25.349229 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.347779    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/396c56d3-19a2-49b4-bc4a-c3dabeb144b4-systemd-units\") pod \"ovnkube-node-sbf8h\" (UID: \"396c56d3-19a2-49b4-bc4a-c3dabeb144b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-sbf8h"
Jul 28 16:21:25.350038 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.347850    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/695dbc5b-d1b4-42f3-921a-cc62010ddc14-cnibin\") pod \"multus-additional-cni-plugins-x2pv7\" (UID: \"695dbc5b-d1b4-42f3-921a-cc62010ddc14\") " pod="openshift-multus/multus-additional-cni-plugins-x2pv7"
Jul 28 16:21:25.350038 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.347880    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/396c56d3-19a2-49b4-bc4a-c3dabeb144b4-env-overrides\") pod \"ovnkube-node-sbf8h\" (UID: \"396c56d3-19a2-49b4-bc4a-c3dabeb144b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-sbf8h"
Jul 28 16:21:25.350038 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.347908    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/695dbc5b-d1b4-42f3-921a-cc62010ddc14-os-release\") pod \"multus-additional-cni-plugins-x2pv7\" (UID: \"695dbc5b-d1b4-42f3-921a-cc62010ddc14\") " pod="openshift-multus/multus-additional-cni-plugins-x2pv7"
Jul 28 16:21:25.350038 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.347926    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/695dbc5b-d1b4-42f3-921a-cc62010ddc14-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-x2pv7\" (UID: \"695dbc5b-d1b4-42f3-921a-cc62010ddc14\") " pod="openshift-multus/multus-additional-cni-plugins-x2pv7"
Jul 28 16:21:25.350038 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.347947    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9af11dde-7687-459a-8857-0d41fd189315-etc-sysconfig\") pod \"tuned-nmwkn\" (UID: \"9af11dde-7687-459a-8857-0d41fd189315\") " pod="openshift-cluster-node-tuning-operator/tuned-nmwkn"
Jul 28 16:21:25.350038 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.347955    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/38125fdc-3616-4f16-8683-aabde8a95527-device-dir\") pod \"aws-ebs-csi-driver-node-nlb66\" (UID: \"38125fdc-3616-4f16-8683-aabde8a95527\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nlb66"
Jul 28 16:21:25.350038 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.347988    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/396c56d3-19a2-49b4-bc4a-c3dabeb144b4-host-cni-netd\") pod \"ovnkube-node-sbf8h\" (UID: \"396c56d3-19a2-49b4-bc4a-c3dabeb144b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-sbf8h"
Jul 28 16:21:25.350038 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.348002    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9af11dde-7687-459a-8857-0d41fd189315-host\") pod \"tuned-nmwkn\" (UID: \"9af11dde-7687-459a-8857-0d41fd189315\") " pod="openshift-cluster-node-tuning-operator/tuned-nmwkn"
Jul 28 16:21:25.350038 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.348044    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/396c56d3-19a2-49b4-bc4a-c3dabeb144b4-host-run-netns\") pod \"ovnkube-node-sbf8h\" (UID: \"396c56d3-19a2-49b4-bc4a-c3dabeb144b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-sbf8h"
Jul 28 16:21:25.350038 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.348061    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9af11dde-7687-459a-8857-0d41fd189315-etc-sysctl-conf\") pod \"tuned-nmwkn\" (UID: \"9af11dde-7687-459a-8857-0d41fd189315\") " pod="openshift-cluster-node-tuning-operator/tuned-nmwkn"
Jul 28 16:21:25.350038 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.347791    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/396c56d3-19a2-49b4-bc4a-c3dabeb144b4-node-log\") pod \"ovnkube-node-sbf8h\" (UID: \"396c56d3-19a2-49b4-bc4a-c3dabeb144b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-sbf8h"
Jul 28 16:21:25.350038 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.348226    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/31992b1b-4643-467f-a2b1-674af56e8c0e-dbus\") pod \"global-pull-secret-syncer-9vtwz\" (UID: \"31992b1b-4643-467f-a2b1-674af56e8c0e\") " pod="kube-system/global-pull-secret-syncer-9vtwz"
Jul 28 16:21:25.350038 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.348244    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7xzg4\" (UniqueName: \"kubernetes.io/projected/ecff0248-22af-459d-ae9a-083ed7508975-kube-api-access-7xzg4\") pod \"iptables-alerter-2lpp7\" (UID: \"ecff0248-22af-459d-ae9a-083ed7508975\") " pod="openshift-network-operator/iptables-alerter-2lpp7"
Jul 28 16:21:25.350038 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.348263    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/38125fdc-3616-4f16-8683-aabde8a95527-socket-dir\") pod \"aws-ebs-csi-driver-node-nlb66\" (UID: \"38125fdc-3616-4f16-8683-aabde8a95527\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nlb66"
Jul 28 16:21:25.350038 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.348249    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9af11dde-7687-459a-8857-0d41fd189315-etc-kubernetes\") pod \"tuned-nmwkn\" (UID: \"9af11dde-7687-459a-8857-0d41fd189315\") " pod="openshift-cluster-node-tuning-operator/tuned-nmwkn"
Jul 28 16:21:25.350038 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.348279    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/228cc225-7549-41ea-928b-23d353595cd3-hostroot\") pod \"multus-6mmzq\" (UID: \"228cc225-7549-41ea-928b-23d353595cd3\") " pod="openshift-multus/multus-6mmzq"
Jul 28 16:21:25.350038 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.348307    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/396c56d3-19a2-49b4-bc4a-c3dabeb144b4-run-systemd\") pod \"ovnkube-node-sbf8h\" (UID: \"396c56d3-19a2-49b4-bc4a-c3dabeb144b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-sbf8h"
Jul 28 16:21:25.350038 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.348317    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/228cc225-7549-41ea-928b-23d353595cd3-hostroot\") pod \"multus-6mmzq\" (UID: \"228cc225-7549-41ea-928b-23d353595cd3\") " pod="openshift-multus/multus-6mmzq"
Jul 28 16:21:25.350775 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.348322    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/396c56d3-19a2-49b4-bc4a-c3dabeb144b4-run-openvswitch\") pod \"ovnkube-node-sbf8h\" (UID: \"396c56d3-19a2-49b4-bc4a-c3dabeb144b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-sbf8h"
Jul 28 16:21:25.350775 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.348382    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/396c56d3-19a2-49b4-bc4a-c3dabeb144b4-node-log\") pod \"ovnkube-node-sbf8h\" (UID: \"396c56d3-19a2-49b4-bc4a-c3dabeb144b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-sbf8h"
Jul 28 16:21:25.350775 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.348402    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/695dbc5b-d1b4-42f3-921a-cc62010ddc14-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-x2pv7\" (UID: \"695dbc5b-d1b4-42f3-921a-cc62010ddc14\") " pod="openshift-multus/multus-additional-cni-plugins-x2pv7"
Jul 28 16:21:25.350775 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.348411    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/38125fdc-3616-4f16-8683-aabde8a95527-socket-dir\") pod \"aws-ebs-csi-driver-node-nlb66\" (UID: \"38125fdc-3616-4f16-8683-aabde8a95527\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nlb66"
Jul 28 16:21:25.350775 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.347235    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ecff0248-22af-459d-ae9a-083ed7508975-iptables-alerter-script\") pod \"iptables-alerter-2lpp7\" (UID: \"ecff0248-22af-459d-ae9a-083ed7508975\") " pod="openshift-network-operator/iptables-alerter-2lpp7"
Jul 28 16:21:25.350775 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.348457    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/228cc225-7549-41ea-928b-23d353595cd3-multus-conf-dir\") pod \"multus-6mmzq\" (UID: \"228cc225-7549-41ea-928b-23d353595cd3\") " pod="openshift-multus/multus-6mmzq"
Jul 28 16:21:25.350775 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.348530    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/396c56d3-19a2-49b4-bc4a-c3dabeb144b4-host-slash\") pod \"ovnkube-node-sbf8h\" (UID: \"396c56d3-19a2-49b4-bc4a-c3dabeb144b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-sbf8h"
Jul 28 16:21:25.350775 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.348573    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/31992b1b-4643-467f-a2b1-674af56e8c0e-dbus\") pod \"global-pull-secret-syncer-9vtwz\" (UID: \"31992b1b-4643-467f-a2b1-674af56e8c0e\") " pod="kube-system/global-pull-secret-syncer-9vtwz"
Jul 28 16:21:25.350775 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.348577    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/228cc225-7549-41ea-928b-23d353595cd3-host-var-lib-cni-bin\") pod \"multus-6mmzq\" (UID: \"228cc225-7549-41ea-928b-23d353595cd3\") " pod="openshift-multus/multus-6mmzq"
Jul 28 16:21:25.350775 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.348614    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/396c56d3-19a2-49b4-bc4a-c3dabeb144b4-var-lib-openvswitch\") pod \"ovnkube-node-sbf8h\" (UID: \"396c56d3-19a2-49b4-bc4a-c3dabeb144b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-sbf8h"
Jul 28 16:21:25.350775 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.348621    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/396c56d3-19a2-49b4-bc4a-c3dabeb144b4-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-sbf8h\" (UID: \"396c56d3-19a2-49b4-bc4a-c3dabeb144b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-sbf8h"
Jul 28 16:21:25.350775 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.348653    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/38125fdc-3616-4f16-8683-aabde8a95527-kubelet-dir\") pod \"aws-ebs-csi-driver-node-nlb66\" (UID: \"38125fdc-3616-4f16-8683-aabde8a95527\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nlb66"
Jul 28 16:21:25.350775 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.348658    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/396c56d3-19a2-49b4-bc4a-c3dabeb144b4-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-sbf8h\" (UID: \"396c56d3-19a2-49b4-bc4a-c3dabeb144b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-sbf8h"
Jul 28 16:21:25.350775 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.348676    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9af11dde-7687-459a-8857-0d41fd189315-var-lib-kubelet\") pod \"tuned-nmwkn\" (UID: \"9af11dde-7687-459a-8857-0d41fd189315\") " pod="openshift-cluster-node-tuning-operator/tuned-nmwkn"
Jul 28 16:21:25.350775 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.348700    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/228cc225-7549-41ea-928b-23d353595cd3-host-run-multus-certs\") pod \"multus-6mmzq\" (UID: \"228cc225-7549-41ea-928b-23d353595cd3\") " pod="openshift-multus/multus-6mmzq"
Jul 28 16:21:25.350775 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.348708    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/38125fdc-3616-4f16-8683-aabde8a95527-kubelet-dir\") pod \"aws-ebs-csi-driver-node-nlb66\" (UID: \"38125fdc-3616-4f16-8683-aabde8a95527\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nlb66"
Jul 28 16:21:25.350775 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.348724    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/228cc225-7549-41ea-928b-23d353595cd3-etc-kubernetes\") pod \"multus-6mmzq\" (UID: \"228cc225-7549-41ea-928b-23d353595cd3\") " pod="openshift-multus/multus-6mmzq"
Jul 28 16:21:25.351545 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.348753    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9af11dde-7687-459a-8857-0d41fd189315-var-lib-kubelet\") pod \"tuned-nmwkn\" (UID: \"9af11dde-7687-459a-8857-0d41fd189315\") " pod="openshift-cluster-node-tuning-operator/tuned-nmwkn"
Jul 28 16:21:25.351545 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.348782    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/228cc225-7549-41ea-928b-23d353595cd3-etc-kubernetes\") pod \"multus-6mmzq\" (UID: \"228cc225-7549-41ea-928b-23d353595cd3\") " pod="openshift-multus/multus-6mmzq"
Jul 28 16:21:25.351545 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.348829    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/228cc225-7549-41ea-928b-23d353595cd3-host-run-multus-certs\") pod \"multus-6mmzq\" (UID: \"228cc225-7549-41ea-928b-23d353595cd3\") " pod="openshift-multus/multus-6mmzq"
Jul 28 16:21:25.351545 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.349052    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/396c56d3-19a2-49b4-bc4a-c3dabeb144b4-ovnkube-config\") pod \"ovnkube-node-sbf8h\" (UID: \"396c56d3-19a2-49b4-bc4a-c3dabeb144b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-sbf8h"
Jul 28 16:21:25.351545 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.349155    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/695dbc5b-d1b4-42f3-921a-cc62010ddc14-cni-binary-copy\") pod \"multus-additional-cni-plugins-x2pv7\" (UID: \"695dbc5b-d1b4-42f3-921a-cc62010ddc14\") " pod="openshift-multus/multus-additional-cni-plugins-x2pv7"
Jul 28 16:21:25.351545 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.350008    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9af11dde-7687-459a-8857-0d41fd189315-tmp\") pod \"tuned-nmwkn\" (UID: \"9af11dde-7687-459a-8857-0d41fd189315\") " pod="openshift-cluster-node-tuning-operator/tuned-nmwkn"
Jul 28 16:21:25.351545 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.350848    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9af11dde-7687-459a-8857-0d41fd189315-etc-tuned\") pod \"tuned-nmwkn\" (UID: \"9af11dde-7687-459a-8857-0d41fd189315\") " pod="openshift-cluster-node-tuning-operator/tuned-nmwkn"
Jul 28 16:21:25.353749 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.353427    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/da6efa62-02e3-4cac-8881-e62018734a17-agent-certs\") pod \"konnectivity-agent-6rk8j\" (UID: \"da6efa62-02e3-4cac-8881-e62018734a17\") " pod="kube-system/konnectivity-agent-6rk8j"
Jul 28 16:21:25.353935 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.353824    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/396c56d3-19a2-49b4-bc4a-c3dabeb144b4-ovn-node-metrics-cert\") pod \"ovnkube-node-sbf8h\" (UID: \"396c56d3-19a2-49b4-bc4a-c3dabeb144b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-sbf8h"
Jul 28 16:21:25.353935 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:25.353877    2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 16:21:25.353935 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:25.353898    2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 16:21:25.353935 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:25.353911    2557 projected.go:196] Error preparing data for projected volume kube-api-access-xl2sp for pod openshift-network-diagnostics/network-check-target-qzkm5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 16:21:25.354065 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:25.353979    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c687045e-f5d0-4a22-b71e-9c01463677e6-kube-api-access-xl2sp podName:c687045e-f5d0-4a22-b71e-9c01463677e6 nodeName:}" failed. No retries permitted until 2026-07-28 16:21:25.853957973 +0000 UTC m=+2.992427379 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-xl2sp" (UniqueName: "kubernetes.io/projected/c687045e-f5d0-4a22-b71e-9c01463677e6-kube-api-access-xl2sp") pod "network-check-target-qzkm5" (UID: "c687045e-f5d0-4a22-b71e-9c01463677e6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 16:21:25.355541 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.355515    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hq7l9\" (UniqueName: \"kubernetes.io/projected/ff81c208-be53-4aff-9993-47b1965201b3-kube-api-access-hq7l9\") pod \"node-ca-f8l6z\" (UID: \"ff81c208-be53-4aff-9993-47b1965201b3\") " pod="openshift-image-registry/node-ca-f8l6z"
Jul 28 16:21:25.356240 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.356217    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qbqd4\" (UniqueName: \"kubernetes.io/projected/695dbc5b-d1b4-42f3-921a-cc62010ddc14-kube-api-access-qbqd4\") pod \"multus-additional-cni-plugins-x2pv7\" (UID: \"695dbc5b-d1b4-42f3-921a-cc62010ddc14\") " pod="openshift-multus/multus-additional-cni-plugins-x2pv7"
Jul 28 16:21:25.357339 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.357316    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-64snv\" (UniqueName: \"kubernetes.io/projected/9af11dde-7687-459a-8857-0d41fd189315-kube-api-access-64snv\") pod \"tuned-nmwkn\" (UID: \"9af11dde-7687-459a-8857-0d41fd189315\") " pod="openshift-cluster-node-tuning-operator/tuned-nmwkn"
Jul 28 16:21:25.357744 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.357717    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pdnhd\" (UniqueName: \"kubernetes.io/projected/396c56d3-19a2-49b4-bc4a-c3dabeb144b4-kube-api-access-pdnhd\") pod \"ovnkube-node-sbf8h\" (UID: \"396c56d3-19a2-49b4-bc4a-c3dabeb144b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-sbf8h"
Jul 28 16:21:25.360859 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.360841    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2zdzz\" (UniqueName: \"kubernetes.io/projected/92ef4728-9e89-4494-8bdf-72dcfe9b3ff4-kube-api-access-2zdzz\") pod \"network-metrics-daemon-qm9v7\" (UID: \"92ef4728-9e89-4494-8bdf-72dcfe9b3ff4\") " pod="openshift-multus/network-metrics-daemon-qm9v7"
Jul 28 16:21:25.364874 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.364854    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7xzg4\" (UniqueName: \"kubernetes.io/projected/ecff0248-22af-459d-ae9a-083ed7508975-kube-api-access-7xzg4\") pod \"iptables-alerter-2lpp7\" (UID: \"ecff0248-22af-459d-ae9a-083ed7508975\") " pod="openshift-network-operator/iptables-alerter-2lpp7"
Jul 28 16:21:25.364952 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.364904    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-79wql\" (UniqueName: \"kubernetes.io/projected/b6c82e3b-ae45-41a0-b919-d4fbd8c3fc64-kube-api-access-79wql\") pod \"node-resolver-rn2z4\" (UID: \"b6c82e3b-ae45-41a0-b919-d4fbd8c3fc64\") " pod="openshift-dns/node-resolver-rn2z4"
Jul 28 16:21:25.365325 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.365298    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bzfpk\" (UniqueName: \"kubernetes.io/projected/38125fdc-3616-4f16-8683-aabde8a95527-kube-api-access-bzfpk\") pod \"aws-ebs-csi-driver-node-nlb66\" (UID: \"38125fdc-3616-4f16-8683-aabde8a95527\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nlb66"
Jul 28 16:21:25.366739 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.366720    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mhddf\" (UniqueName: \"kubernetes.io/projected/228cc225-7549-41ea-928b-23d353595cd3-kube-api-access-mhddf\") pod \"multus-6mmzq\" (UID: \"228cc225-7549-41ea-928b-23d353595cd3\") " pod="openshift-multus/multus-6mmzq"
Jul 28 16:21:25.537068 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.537005    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-f8l6z"
Jul 28 16:21:25.545885 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.545851    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-nmwkn"
Jul 28 16:21:25.554644 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.554626    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-2lpp7"
Jul 28 16:21:25.559276 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.559259    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nlb66"
Jul 28 16:21:25.564746 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.564731    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-rn2z4"
Jul 28 16:21:25.571220 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.571197    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-6mmzq"
Jul 28 16:21:25.577694 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.577675    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-sbf8h"
Jul 28 16:21:25.583149 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.583128    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-6rk8j"
Jul 28 16:21:25.586649 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.586628    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-x2pv7"
Jul 28 16:21:25.616009 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.615990    2557 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 16:21:25.852701 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.852676    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/31992b1b-4643-467f-a2b1-674af56e8c0e-original-pull-secret\") pod \"global-pull-secret-syncer-9vtwz\" (UID: \"31992b1b-4643-467f-a2b1-674af56e8c0e\") " pod="kube-system/global-pull-secret-syncer-9vtwz"
Jul 28 16:21:25.852894 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.852758    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/92ef4728-9e89-4494-8bdf-72dcfe9b3ff4-metrics-certs\") pod \"network-metrics-daemon-qm9v7\" (UID: \"92ef4728-9e89-4494-8bdf-72dcfe9b3ff4\") " pod="openshift-multus/network-metrics-daemon-qm9v7"
Jul 28 16:21:25.852894 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:25.852814    2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 16:21:25.852894 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:25.852862    2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 16:21:25.852894 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:25.852882    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31992b1b-4643-467f-a2b1-674af56e8c0e-original-pull-secret podName:31992b1b-4643-467f-a2b1-674af56e8c0e nodeName:}" failed. No retries permitted until 2026-07-28 16:21:26.852864776 +0000 UTC m=+3.991334169 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/31992b1b-4643-467f-a2b1-674af56e8c0e-original-pull-secret") pod "global-pull-secret-syncer-9vtwz" (UID: "31992b1b-4643-467f-a2b1-674af56e8c0e") : object "kube-system"/"original-pull-secret" not registered
Jul 28 16:21:25.853103 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:25.852909    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92ef4728-9e89-4494-8bdf-72dcfe9b3ff4-metrics-certs podName:92ef4728-9e89-4494-8bdf-72dcfe9b3ff4 nodeName:}" failed. No retries permitted until 2026-07-28 16:21:26.852892574 +0000 UTC m=+3.991361968 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/92ef4728-9e89-4494-8bdf-72dcfe9b3ff4-metrics-certs") pod "network-metrics-daemon-qm9v7" (UID: "92ef4728-9e89-4494-8bdf-72dcfe9b3ff4") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 16:21:25.953923 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:25.953892    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xl2sp\" (UniqueName: \"kubernetes.io/projected/c687045e-f5d0-4a22-b71e-9c01463677e6-kube-api-access-xl2sp\") pod \"network-check-target-qzkm5\" (UID: \"c687045e-f5d0-4a22-b71e-9c01463677e6\") " pod="openshift-network-diagnostics/network-check-target-qzkm5"
Jul 28 16:21:25.954060 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:25.954045    2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 16:21:25.954124 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:25.954064    2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 16:21:25.954124 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:25.954073    2557 projected.go:196] Error preparing data for projected volume kube-api-access-xl2sp for pod openshift-network-diagnostics/network-check-target-qzkm5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 16:21:25.954124 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:25.954123    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c687045e-f5d0-4a22-b71e-9c01463677e6-kube-api-access-xl2sp podName:c687045e-f5d0-4a22-b71e-9c01463677e6 nodeName:}" failed. No retries permitted until 2026-07-28 16:21:26.954105183 +0000 UTC m=+4.092574593 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-xl2sp" (UniqueName: "kubernetes.io/projected/c687045e-f5d0-4a22-b71e-9c01463677e6-kube-api-access-xl2sp") pod "network-check-target-qzkm5" (UID: "c687045e-f5d0-4a22-b71e-9c01463677e6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 16:21:26.145513 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:26.145478    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod396c56d3_19a2_49b4_bc4a_c3dabeb144b4.slice/crio-e1c2cf2389318c8c0f557d2b949230c9d1d13aaf25334baa99fffcea0e48560c WatchSource:0}: Error finding container e1c2cf2389318c8c0f557d2b949230c9d1d13aaf25334baa99fffcea0e48560c: Status 404 returned error can't find the container with id e1c2cf2389318c8c0f557d2b949230c9d1d13aaf25334baa99fffcea0e48560c
Jul 28 16:21:26.147121 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:26.147095    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38125fdc_3616_4f16_8683_aabde8a95527.slice/crio-4b6593478ff14a3eba466b90a042ae52f1d9e8b05b3c0b1f2ee2e27c2c8b457f WatchSource:0}: Error finding container 4b6593478ff14a3eba466b90a042ae52f1d9e8b05b3c0b1f2ee2e27c2c8b457f: Status 404 returned error can't find the container with id 4b6593478ff14a3eba466b90a042ae52f1d9e8b05b3c0b1f2ee2e27c2c8b457f
Jul 28 16:21:26.150809 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:26.150782    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9af11dde_7687_459a_8857_0d41fd189315.slice/crio-82aebaaa2413324344edbf5a45510f6d802a1e672b334b38508c84ee59076925 WatchSource:0}: Error finding container 82aebaaa2413324344edbf5a45510f6d802a1e672b334b38508c84ee59076925: Status 404 returned error can't find the container with id 82aebaaa2413324344edbf5a45510f6d802a1e672b334b38508c84ee59076925
Jul 28 16:21:26.152689 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:26.152668    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff81c208_be53_4aff_9993_47b1965201b3.slice/crio-36433c1609c65fc9516da31782ef92d1873e0750d11ee89bfe0d067120d83077 WatchSource:0}: Error finding container 36433c1609c65fc9516da31782ef92d1873e0750d11ee89bfe0d067120d83077: Status 404 returned error can't find the container with id 36433c1609c65fc9516da31782ef92d1873e0750d11ee89bfe0d067120d83077
Jul 28 16:21:26.153278 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:26.153260    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podecff0248_22af_459d_ae9a_083ed7508975.slice/crio-7f85e4a9c4e9e77ea624e80bc1ce64b01abd823f799669e85709a3b915fe1eb3 WatchSource:0}: Error finding container 7f85e4a9c4e9e77ea624e80bc1ce64b01abd823f799669e85709a3b915fe1eb3: Status 404 returned error can't find the container with id 7f85e4a9c4e9e77ea624e80bc1ce64b01abd823f799669e85709a3b915fe1eb3
Jul 28 16:21:26.154165 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:26.154053    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod228cc225_7549_41ea_928b_23d353595cd3.slice/crio-f6dbfa8f69a65809c045ccddce8c0888e0fcd3be38341f92c6d40eb277a42613 WatchSource:0}: Error finding container f6dbfa8f69a65809c045ccddce8c0888e0fcd3be38341f92c6d40eb277a42613: Status 404 returned error can't find the container with id f6dbfa8f69a65809c045ccddce8c0888e0fcd3be38341f92c6d40eb277a42613
Jul 28 16:21:26.282326 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:26.282194    2557 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-27 16:16:25 +0000 UTC" deadline="2028-03-03 19:57:16.566726003 +0000 UTC"
Jul 28 16:21:26.282326 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:26.282327    2557 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14019h35m50.284404734s"
Jul 28 16:21:26.430904 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:26.428727    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qzkm5"
Jul 28 16:21:26.430904 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:26.428901    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qzkm5" podUID="c687045e-f5d0-4a22-b71e-9c01463677e6"
Jul 28 16:21:26.437201 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:26.437177    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-154.ec2.internal" event={"ID":"851ad20362c20c22501be689ae7a68aa","Type":"ContainerStarted","Data":"62546833aca22fcd20735238d1dc6100770f99aad6b10181a0b5b103397f5d37"}
Jul 28 16:21:26.438965 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:26.438945    2557 generic.go:356] "Generic (PLEG): container finished" podID="49d5be5d46952d796bae73c6a91a14e0" containerID="56dc5b2417e49bae0408c60c909577b99aecbd267b7c0791cffec8ab0065b2d7" exitCode=0
Jul 28 16:21:26.439040 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:26.439005    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-154.ec2.internal" event={"ID":"49d5be5d46952d796bae73c6a91a14e0","Type":"ContainerDied","Data":"56dc5b2417e49bae0408c60c909577b99aecbd267b7c0791cffec8ab0065b2d7"}
Jul 28 16:21:26.441135 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:26.441115    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-rn2z4" event={"ID":"b6c82e3b-ae45-41a0-b919-d4fbd8c3fc64","Type":"ContainerStarted","Data":"733d543007611eb2cdd1b8cbb9da08aef6ff9d6c6a886b1f3842be82d84caaa0"}
Jul 28 16:21:26.442137 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:26.442119    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-2lpp7" event={"ID":"ecff0248-22af-459d-ae9a-083ed7508975","Type":"ContainerStarted","Data":"7f85e4a9c4e9e77ea624e80bc1ce64b01abd823f799669e85709a3b915fe1eb3"}
Jul 28 16:21:26.443192 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:26.443171    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-x2pv7" event={"ID":"695dbc5b-d1b4-42f3-921a-cc62010ddc14","Type":"ContainerStarted","Data":"2b36190b8e2a929de8ba9352d7c6bdd6f7253b20652a02bf1613ce95c43fe50f"}
Jul 28 16:21:26.444197 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:26.444170    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-6rk8j" event={"ID":"da6efa62-02e3-4cac-8881-e62018734a17","Type":"ContainerStarted","Data":"50af9d9e046f82f0d51af5babd7412c49765eeb924e4314517951d257e08620d"}
Jul 28 16:21:26.445165 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:26.445148    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-nmwkn" event={"ID":"9af11dde-7687-459a-8857-0d41fd189315","Type":"ContainerStarted","Data":"82aebaaa2413324344edbf5a45510f6d802a1e672b334b38508c84ee59076925"}
Jul 28 16:21:26.446197 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:26.446178    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-6mmzq" event={"ID":"228cc225-7549-41ea-928b-23d353595cd3","Type":"ContainerStarted","Data":"f6dbfa8f69a65809c045ccddce8c0888e0fcd3be38341f92c6d40eb277a42613"}
Jul 28 16:21:26.447237 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:26.447220    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-f8l6z" event={"ID":"ff81c208-be53-4aff-9993-47b1965201b3","Type":"ContainerStarted","Data":"36433c1609c65fc9516da31782ef92d1873e0750d11ee89bfe0d067120d83077"}
Jul 28 16:21:26.448304 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:26.448282    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nlb66" event={"ID":"38125fdc-3616-4f16-8683-aabde8a95527","Type":"ContainerStarted","Data":"4b6593478ff14a3eba466b90a042ae52f1d9e8b05b3c0b1f2ee2e27c2c8b457f"}
Jul 28 16:21:26.450720 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:26.450697    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sbf8h" event={"ID":"396c56d3-19a2-49b4-bc4a-c3dabeb144b4","Type":"ContainerStarted","Data":"e1c2cf2389318c8c0f557d2b949230c9d1d13aaf25334baa99fffcea0e48560c"}
Jul 28 16:21:26.452937 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:26.452903    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-154.ec2.internal" podStartSLOduration=2.452893096 podStartE2EDuration="2.452893096s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 16:21:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:21:26.452480557 +0000 UTC m=+3.590949970" watchObservedRunningTime="2026-07-28 16:21:26.452893096 +0000 UTC m=+3.591362510"
Jul 28 16:21:26.861360 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:26.861281    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/92ef4728-9e89-4494-8bdf-72dcfe9b3ff4-metrics-certs\") pod \"network-metrics-daemon-qm9v7\" (UID: \"92ef4728-9e89-4494-8bdf-72dcfe9b3ff4\") " pod="openshift-multus/network-metrics-daemon-qm9v7"
Jul 28 16:21:26.861360 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:26.861335    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/31992b1b-4643-467f-a2b1-674af56e8c0e-original-pull-secret\") pod \"global-pull-secret-syncer-9vtwz\" (UID: \"31992b1b-4643-467f-a2b1-674af56e8c0e\") " pod="kube-system/global-pull-secret-syncer-9vtwz"
Jul 28 16:21:26.861591 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:26.861472    2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 16:21:26.861591 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:26.861544    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31992b1b-4643-467f-a2b1-674af56e8c0e-original-pull-secret podName:31992b1b-4643-467f-a2b1-674af56e8c0e nodeName:}" failed. No retries permitted until 2026-07-28 16:21:28.861526157 +0000 UTC m=+5.999995551 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/31992b1b-4643-467f-a2b1-674af56e8c0e-original-pull-secret") pod "global-pull-secret-syncer-9vtwz" (UID: "31992b1b-4643-467f-a2b1-674af56e8c0e") : object "kube-system"/"original-pull-secret" not registered
Jul 28 16:21:26.862012 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:26.861991    2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 16:21:26.862118 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:26.862046    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92ef4728-9e89-4494-8bdf-72dcfe9b3ff4-metrics-certs podName:92ef4728-9e89-4494-8bdf-72dcfe9b3ff4 nodeName:}" failed. No retries permitted until 2026-07-28 16:21:28.862031537 +0000 UTC m=+6.000500944 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/92ef4728-9e89-4494-8bdf-72dcfe9b3ff4-metrics-certs") pod "network-metrics-daemon-qm9v7" (UID: "92ef4728-9e89-4494-8bdf-72dcfe9b3ff4") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 16:21:26.961730 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:26.961687    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xl2sp\" (UniqueName: \"kubernetes.io/projected/c687045e-f5d0-4a22-b71e-9c01463677e6-kube-api-access-xl2sp\") pod \"network-check-target-qzkm5\" (UID: \"c687045e-f5d0-4a22-b71e-9c01463677e6\") " pod="openshift-network-diagnostics/network-check-target-qzkm5"
Jul 28 16:21:26.961891 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:26.961878    2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 16:21:26.961966 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:26.961902    2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 16:21:26.961966 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:26.961916    2557 projected.go:196] Error preparing data for projected volume kube-api-access-xl2sp for pod openshift-network-diagnostics/network-check-target-qzkm5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 16:21:26.962070 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:26.961967    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c687045e-f5d0-4a22-b71e-9c01463677e6-kube-api-access-xl2sp podName:c687045e-f5d0-4a22-b71e-9c01463677e6 nodeName:}" failed. No retries permitted until 2026-07-28 16:21:28.961949005 +0000 UTC m=+6.100418403 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-xl2sp" (UniqueName: "kubernetes.io/projected/c687045e-f5d0-4a22-b71e-9c01463677e6-kube-api-access-xl2sp") pod "network-check-target-qzkm5" (UID: "c687045e-f5d0-4a22-b71e-9c01463677e6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 16:21:27.283150 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:27.283041    2557 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-27 16:16:25 +0000 UTC" deadline="2028-03-27 11:34:17.060227217 +0000 UTC"
Jul 28 16:21:27.283150 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:27.283105    2557 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14587h12m49.777126142s"
Jul 28 16:21:27.434383 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:27.433641    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9vtwz"
Jul 28 16:21:27.434383 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:27.433766    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9vtwz" podUID="31992b1b-4643-467f-a2b1-674af56e8c0e"
Jul 28 16:21:27.435015 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:27.434774    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qm9v7"
Jul 28 16:21:27.435015 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:27.434957    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qm9v7" podUID="92ef4728-9e89-4494-8bdf-72dcfe9b3ff4"
Jul 28 16:21:27.471634 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:27.471563    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-154.ec2.internal" event={"ID":"49d5be5d46952d796bae73c6a91a14e0","Type":"ContainerStarted","Data":"14aa582d333dd5b4e4cd033e71bef77e6d4aca4e54e47decbe25125f41f4cce2"}
Jul 28 16:21:27.487068 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:27.486708    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-154.ec2.internal" podStartSLOduration=3.486691173 podStartE2EDuration="3.486691173s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 16:21:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:21:27.485754294 +0000 UTC m=+4.624223709" watchObservedRunningTime="2026-07-28 16:21:27.486691173 +0000 UTC m=+4.625160604"
Jul 28 16:21:28.428316 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:28.428247    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qzkm5"
Jul 28 16:21:28.428756 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:28.428380    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qzkm5" podUID="c687045e-f5d0-4a22-b71e-9c01463677e6"
Jul 28 16:21:28.880980 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:28.880876    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/92ef4728-9e89-4494-8bdf-72dcfe9b3ff4-metrics-certs\") pod \"network-metrics-daemon-qm9v7\" (UID: \"92ef4728-9e89-4494-8bdf-72dcfe9b3ff4\") " pod="openshift-multus/network-metrics-daemon-qm9v7"
Jul 28 16:21:28.880980 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:28.880929    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/31992b1b-4643-467f-a2b1-674af56e8c0e-original-pull-secret\") pod \"global-pull-secret-syncer-9vtwz\" (UID: \"31992b1b-4643-467f-a2b1-674af56e8c0e\") " pod="kube-system/global-pull-secret-syncer-9vtwz"
Jul 28 16:21:28.881187 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:28.881061    2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 16:21:28.881187 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:28.881118    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31992b1b-4643-467f-a2b1-674af56e8c0e-original-pull-secret podName:31992b1b-4643-467f-a2b1-674af56e8c0e nodeName:}" failed. No retries permitted until 2026-07-28 16:21:32.881099463 +0000 UTC m=+10.019568858 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/31992b1b-4643-467f-a2b1-674af56e8c0e-original-pull-secret") pod "global-pull-secret-syncer-9vtwz" (UID: "31992b1b-4643-467f-a2b1-674af56e8c0e") : object "kube-system"/"original-pull-secret" not registered
Jul 28 16:21:28.881509 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:28.881479    2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 16:21:28.881689 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:28.881658    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92ef4728-9e89-4494-8bdf-72dcfe9b3ff4-metrics-certs podName:92ef4728-9e89-4494-8bdf-72dcfe9b3ff4 nodeName:}" failed. No retries permitted until 2026-07-28 16:21:32.881640502 +0000 UTC m=+10.020109907 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/92ef4728-9e89-4494-8bdf-72dcfe9b3ff4-metrics-certs") pod "network-metrics-daemon-qm9v7" (UID: "92ef4728-9e89-4494-8bdf-72dcfe9b3ff4") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 16:21:28.982110 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:28.981317    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xl2sp\" (UniqueName: \"kubernetes.io/projected/c687045e-f5d0-4a22-b71e-9c01463677e6-kube-api-access-xl2sp\") pod \"network-check-target-qzkm5\" (UID: \"c687045e-f5d0-4a22-b71e-9c01463677e6\") " pod="openshift-network-diagnostics/network-check-target-qzkm5"
Jul 28 16:21:28.982110 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:28.981506    2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 16:21:28.982110 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:28.981525    2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 16:21:28.982110 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:28.981537    2557 projected.go:196] Error preparing data for projected volume kube-api-access-xl2sp for pod openshift-network-diagnostics/network-check-target-qzkm5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 16:21:28.982110 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:28.981592    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c687045e-f5d0-4a22-b71e-9c01463677e6-kube-api-access-xl2sp podName:c687045e-f5d0-4a22-b71e-9c01463677e6 nodeName:}" failed. No retries permitted until 2026-07-28 16:21:32.981573309 +0000 UTC m=+10.120042718 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-xl2sp" (UniqueName: "kubernetes.io/projected/c687045e-f5d0-4a22-b71e-9c01463677e6-kube-api-access-xl2sp") pod "network-check-target-qzkm5" (UID: "c687045e-f5d0-4a22-b71e-9c01463677e6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 16:21:29.428628 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:29.428598    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9vtwz"
Jul 28 16:21:29.429070 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:29.428731    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9vtwz" podUID="31992b1b-4643-467f-a2b1-674af56e8c0e"
Jul 28 16:21:29.429070 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:29.428774    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qm9v7"
Jul 28 16:21:29.429070 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:29.428892    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qm9v7" podUID="92ef4728-9e89-4494-8bdf-72dcfe9b3ff4"
Jul 28 16:21:30.428641 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:30.428581    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qzkm5"
Jul 28 16:21:30.429039 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:30.428716    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qzkm5" podUID="c687045e-f5d0-4a22-b71e-9c01463677e6"
Jul 28 16:21:31.427761 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:31.427719    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9vtwz"
Jul 28 16:21:31.427987 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:31.427862    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9vtwz" podUID="31992b1b-4643-467f-a2b1-674af56e8c0e"
Jul 28 16:21:31.428368 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:31.428345    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qm9v7"
Jul 28 16:21:31.428548 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:31.428489    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qm9v7" podUID="92ef4728-9e89-4494-8bdf-72dcfe9b3ff4"
Jul 28 16:21:32.429011 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:32.428462    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qzkm5"
Jul 28 16:21:32.429011 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:32.428612    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qzkm5" podUID="c687045e-f5d0-4a22-b71e-9c01463677e6"
Jul 28 16:21:32.915282 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:32.915189    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/31992b1b-4643-467f-a2b1-674af56e8c0e-original-pull-secret\") pod \"global-pull-secret-syncer-9vtwz\" (UID: \"31992b1b-4643-467f-a2b1-674af56e8c0e\") " pod="kube-system/global-pull-secret-syncer-9vtwz"
Jul 28 16:21:32.915448 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:32.915289    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/92ef4728-9e89-4494-8bdf-72dcfe9b3ff4-metrics-certs\") pod \"network-metrics-daemon-qm9v7\" (UID: \"92ef4728-9e89-4494-8bdf-72dcfe9b3ff4\") " pod="openshift-multus/network-metrics-daemon-qm9v7"
Jul 28 16:21:32.915448 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:32.915353    2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 16:21:32.915448 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:32.915390    2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 16:21:32.915448 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:32.915435    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31992b1b-4643-467f-a2b1-674af56e8c0e-original-pull-secret podName:31992b1b-4643-467f-a2b1-674af56e8c0e nodeName:}" failed. No retries permitted until 2026-07-28 16:21:40.915402205 +0000 UTC m=+18.053871599 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/31992b1b-4643-467f-a2b1-674af56e8c0e-original-pull-secret") pod "global-pull-secret-syncer-9vtwz" (UID: "31992b1b-4643-467f-a2b1-674af56e8c0e") : object "kube-system"/"original-pull-secret" not registered
Jul 28 16:21:32.915746 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:32.915456    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92ef4728-9e89-4494-8bdf-72dcfe9b3ff4-metrics-certs podName:92ef4728-9e89-4494-8bdf-72dcfe9b3ff4 nodeName:}" failed. No retries permitted until 2026-07-28 16:21:40.915446148 +0000 UTC m=+18.053915544 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/92ef4728-9e89-4494-8bdf-72dcfe9b3ff4-metrics-certs") pod "network-metrics-daemon-qm9v7" (UID: "92ef4728-9e89-4494-8bdf-72dcfe9b3ff4") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 16:21:33.016101 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:33.016066    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xl2sp\" (UniqueName: \"kubernetes.io/projected/c687045e-f5d0-4a22-b71e-9c01463677e6-kube-api-access-xl2sp\") pod \"network-check-target-qzkm5\" (UID: \"c687045e-f5d0-4a22-b71e-9c01463677e6\") " pod="openshift-network-diagnostics/network-check-target-qzkm5"
Jul 28 16:21:33.016304 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:33.016243    2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 16:21:33.016304 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:33.016272    2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 16:21:33.016304 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:33.016286    2557 projected.go:196] Error preparing data for projected volume kube-api-access-xl2sp for pod openshift-network-diagnostics/network-check-target-qzkm5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 16:21:33.016483 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:33.016352    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c687045e-f5d0-4a22-b71e-9c01463677e6-kube-api-access-xl2sp podName:c687045e-f5d0-4a22-b71e-9c01463677e6 nodeName:}" failed. No retries permitted until 2026-07-28 16:21:41.01633272 +0000 UTC m=+18.154802129 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-xl2sp" (UniqueName: "kubernetes.io/projected/c687045e-f5d0-4a22-b71e-9c01463677e6-kube-api-access-xl2sp") pod "network-check-target-qzkm5" (UID: "c687045e-f5d0-4a22-b71e-9c01463677e6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 16:21:33.429356 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:33.429322    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9vtwz"
Jul 28 16:21:33.430043 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:33.429456    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9vtwz" podUID="31992b1b-4643-467f-a2b1-674af56e8c0e"
Jul 28 16:21:33.430043 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:33.429927    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qm9v7"
Jul 28 16:21:33.430043 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:33.430031    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qm9v7" podUID="92ef4728-9e89-4494-8bdf-72dcfe9b3ff4"
Jul 28 16:21:34.427703 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:34.427671    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qzkm5"
Jul 28 16:21:34.427887 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:34.427811    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qzkm5" podUID="c687045e-f5d0-4a22-b71e-9c01463677e6"
Jul 28 16:21:35.429961 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:35.429928    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9vtwz"
Jul 28 16:21:35.430374 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:35.430052    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9vtwz" podUID="31992b1b-4643-467f-a2b1-674af56e8c0e"
Jul 28 16:21:35.430479 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:35.430463    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qm9v7"
Jul 28 16:21:35.430604 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:35.430585    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qm9v7" podUID="92ef4728-9e89-4494-8bdf-72dcfe9b3ff4"
Jul 28 16:21:36.428323 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:36.428279    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qzkm5"
Jul 28 16:21:36.428522 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:36.428380    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qzkm5" podUID="c687045e-f5d0-4a22-b71e-9c01463677e6"
Jul 28 16:21:37.428033 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:37.427956    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9vtwz"
Jul 28 16:21:37.428507 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:37.428071    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9vtwz" podUID="31992b1b-4643-467f-a2b1-674af56e8c0e"
Jul 28 16:21:37.428507 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:37.427916    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qm9v7"
Jul 28 16:21:37.428623 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:37.428554    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qm9v7" podUID="92ef4728-9e89-4494-8bdf-72dcfe9b3ff4"
Jul 28 16:21:38.427882 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:38.427844    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qzkm5"
Jul 28 16:21:38.428048 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:38.427979    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qzkm5" podUID="c687045e-f5d0-4a22-b71e-9c01463677e6"
Jul 28 16:21:39.428610 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:39.428575    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qm9v7"
Jul 28 16:21:39.429063 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:39.428584    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9vtwz"
Jul 28 16:21:39.429063 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:39.428711    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qm9v7" podUID="92ef4728-9e89-4494-8bdf-72dcfe9b3ff4"
Jul 28 16:21:39.429063 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:39.428796    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9vtwz" podUID="31992b1b-4643-467f-a2b1-674af56e8c0e"
Jul 28 16:21:40.428106 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:40.428071    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qzkm5"
Jul 28 16:21:40.428329 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:40.428199    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qzkm5" podUID="c687045e-f5d0-4a22-b71e-9c01463677e6"
Jul 28 16:21:40.975155 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:40.975115    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/92ef4728-9e89-4494-8bdf-72dcfe9b3ff4-metrics-certs\") pod \"network-metrics-daemon-qm9v7\" (UID: \"92ef4728-9e89-4494-8bdf-72dcfe9b3ff4\") " pod="openshift-multus/network-metrics-daemon-qm9v7"
Jul 28 16:21:40.975576 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:40.975170    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/31992b1b-4643-467f-a2b1-674af56e8c0e-original-pull-secret\") pod \"global-pull-secret-syncer-9vtwz\" (UID: \"31992b1b-4643-467f-a2b1-674af56e8c0e\") " pod="kube-system/global-pull-secret-syncer-9vtwz"
Jul 28 16:21:40.975576 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:40.975274    2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 16:21:40.975576 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:40.975304    2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 16:21:40.975576 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:40.975337    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92ef4728-9e89-4494-8bdf-72dcfe9b3ff4-metrics-certs podName:92ef4728-9e89-4494-8bdf-72dcfe9b3ff4 nodeName:}" failed. No retries permitted until 2026-07-28 16:21:56.975321638 +0000 UTC m=+34.113791029 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/92ef4728-9e89-4494-8bdf-72dcfe9b3ff4-metrics-certs") pod "network-metrics-daemon-qm9v7" (UID: "92ef4728-9e89-4494-8bdf-72dcfe9b3ff4") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 16:21:40.975576 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:40.975359    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31992b1b-4643-467f-a2b1-674af56e8c0e-original-pull-secret podName:31992b1b-4643-467f-a2b1-674af56e8c0e nodeName:}" failed. No retries permitted until 2026-07-28 16:21:56.975343551 +0000 UTC m=+34.113812944 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/31992b1b-4643-467f-a2b1-674af56e8c0e-original-pull-secret") pod "global-pull-secret-syncer-9vtwz" (UID: "31992b1b-4643-467f-a2b1-674af56e8c0e") : object "kube-system"/"original-pull-secret" not registered
Jul 28 16:21:41.075861 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:41.075830    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xl2sp\" (UniqueName: \"kubernetes.io/projected/c687045e-f5d0-4a22-b71e-9c01463677e6-kube-api-access-xl2sp\") pod \"network-check-target-qzkm5\" (UID: \"c687045e-f5d0-4a22-b71e-9c01463677e6\") " pod="openshift-network-diagnostics/network-check-target-qzkm5"
Jul 28 16:21:41.075994 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:41.075953    2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 16:21:41.075994 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:41.075967    2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 16:21:41.075994 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:41.075975    2557 projected.go:196] Error preparing data for projected volume kube-api-access-xl2sp for pod openshift-network-diagnostics/network-check-target-qzkm5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 16:21:41.076089 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:41.076019    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c687045e-f5d0-4a22-b71e-9c01463677e6-kube-api-access-xl2sp podName:c687045e-f5d0-4a22-b71e-9c01463677e6 nodeName:}" failed. No retries permitted until 2026-07-28 16:21:57.076007763 +0000 UTC m=+34.214477155 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-xl2sp" (UniqueName: "kubernetes.io/projected/c687045e-f5d0-4a22-b71e-9c01463677e6-kube-api-access-xl2sp") pod "network-check-target-qzkm5" (UID: "c687045e-f5d0-4a22-b71e-9c01463677e6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 16:21:41.428637 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:41.428556    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9vtwz"
Jul 28 16:21:41.428637 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:41.428575    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qm9v7"
Jul 28 16:21:41.428826 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:41.428688    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9vtwz" podUID="31992b1b-4643-467f-a2b1-674af56e8c0e"
Jul 28 16:21:41.428866 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:41.428837    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qm9v7" podUID="92ef4728-9e89-4494-8bdf-72dcfe9b3ff4"
Jul 28 16:21:42.428159 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:42.428124    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qzkm5"
Jul 28 16:21:42.428556 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:42.428250    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qzkm5" podUID="c687045e-f5d0-4a22-b71e-9c01463677e6"
Jul 28 16:21:43.429164 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:43.429125    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9vtwz"
Jul 28 16:21:43.429590 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:43.429244    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9vtwz" podUID="31992b1b-4643-467f-a2b1-674af56e8c0e"
Jul 28 16:21:43.429590 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:43.429342    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qm9v7"
Jul 28 16:21:43.429590 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:43.429468    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qm9v7" podUID="92ef4728-9e89-4494-8bdf-72dcfe9b3ff4"
Jul 28 16:21:44.428025 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:44.427989    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qzkm5"
Jul 28 16:21:44.428203 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:44.428124    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qzkm5" podUID="c687045e-f5d0-4a22-b71e-9c01463677e6"
Jul 28 16:21:45.428244 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:45.427958    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9vtwz"
Jul 28 16:21:45.428842 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:45.428017    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qm9v7"
Jul 28 16:21:45.428842 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:45.428313    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9vtwz" podUID="31992b1b-4643-467f-a2b1-674af56e8c0e"
Jul 28 16:21:45.428842 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:45.428387    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qm9v7" podUID="92ef4728-9e89-4494-8bdf-72dcfe9b3ff4"
Jul 28 16:21:45.516092 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:45.516064    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-6mmzq" event={"ID":"228cc225-7549-41ea-928b-23d353595cd3","Type":"ContainerStarted","Data":"daf6107766bce66a64e15cb7a0c4d58011dcde90857ed40f65b86be3e0059df9"}
Jul 28 16:21:45.517730 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:45.517704    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-f8l6z" event={"ID":"ff81c208-be53-4aff-9993-47b1965201b3","Type":"ContainerStarted","Data":"8dabae39b844ba437f45d54dace8fe67029bdb88bc0716f740285882b24b0144"}
Jul 28 16:21:45.519234 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:45.519214    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nlb66" event={"ID":"38125fdc-3616-4f16-8683-aabde8a95527","Type":"ContainerStarted","Data":"14e5c1683656cde9e338389bbc5d1dd1d92885db5fcb55361bed3a120c7577ae"}
Jul 28 16:21:45.522705 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:45.522685    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sbf8h" event={"ID":"396c56d3-19a2-49b4-bc4a-c3dabeb144b4","Type":"ContainerStarted","Data":"3b6cc59d193f49c3da9f29132caaf9b2de68f44f59e715d422d679c930530bf1"}
Jul 28 16:21:45.522796 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:45.522750    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sbf8h" event={"ID":"396c56d3-19a2-49b4-bc4a-c3dabeb144b4","Type":"ContainerStarted","Data":"c51bb362ece26d9deaf7314a7df7238e48da97bcdf68ce7d018e1b1aa63b255d"}
Jul 28 16:21:45.522852 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:45.522803    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sbf8h" event={"ID":"396c56d3-19a2-49b4-bc4a-c3dabeb144b4","Type":"ContainerStarted","Data":"79d69687c79ca70333f51df5de1eb9fc59c4801da02aa43d94c7803af7417b25"}
Jul 28 16:21:45.522903 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:45.522858    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sbf8h" event={"ID":"396c56d3-19a2-49b4-bc4a-c3dabeb144b4","Type":"ContainerStarted","Data":"92420db0223464d92ea30a363a42cf43a44ff24e43092e86a87c39509f34aabf"}
Jul 28 16:21:45.522948 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:45.522908    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sbf8h" event={"ID":"396c56d3-19a2-49b4-bc4a-c3dabeb144b4","Type":"ContainerStarted","Data":"ec6f513518520ef529db962c58a996e36e72a605eef9d376dade2cc428624a2e"}
Jul 28 16:21:45.523013 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:45.522956    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sbf8h" event={"ID":"396c56d3-19a2-49b4-bc4a-c3dabeb144b4","Type":"ContainerStarted","Data":"af07582e580d8f2de689538e75dff3daf0c48aec79c39048c7951b23a279bdd6"}
Jul 28 16:21:45.528888 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:45.528864    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-rn2z4" event={"ID":"b6c82e3b-ae45-41a0-b919-d4fbd8c3fc64","Type":"ContainerStarted","Data":"e4a796416994317882790cadcfbb93d0e8b8d81e57c500d79f02dcff8d4e4ef4"}
Jul 28 16:21:45.531656 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:45.531597    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-6mmzq" podStartSLOduration=3.832249395 podStartE2EDuration="22.531560976s" totalImagesPullingTime="18.699311581s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 16:21:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:21:45.531251024 +0000 UTC m=+22.669720438" watchObservedRunningTime="2026-07-28 16:21:45.531560976 +0000 UTC m=+22.670030370"
Jul 28 16:21:45.536698 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:45.536642    2557 generic.go:356] "Generic (PLEG): container finished" podID="695dbc5b-d1b4-42f3-921a-cc62010ddc14" containerID="5cfe18e2a3a91752c9a9e96b0239eb9094f69bcf33f811bc5ca07aa239e5effa" exitCode=0
Jul 28 16:21:45.536978 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:45.536953    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-x2pv7" event={"ID":"695dbc5b-d1b4-42f3-921a-cc62010ddc14","Type":"ContainerDied","Data":"5cfe18e2a3a91752c9a9e96b0239eb9094f69bcf33f811bc5ca07aa239e5effa"}
Jul 28 16:21:45.540149 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:45.539587    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-6rk8j" event={"ID":"da6efa62-02e3-4cac-8881-e62018734a17","Type":"ContainerStarted","Data":"53712ddb19611dd9ccdfc7d7c33193da3d935d014ff7709af21d9a3bbbc3bcf7"}
Jul 28 16:21:45.545075 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:45.545036    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-f8l6z" podStartSLOduration=3.901716384 podStartE2EDuration="22.545024144s" totalImagesPullingTime="18.64330776s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 16:21:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:21:45.54435942 +0000 UTC m=+22.682828834" watchObservedRunningTime="2026-07-28 16:21:45.545024144 +0000 UTC m=+22.683493578"
Jul 28 16:21:45.551679 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:45.551623    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-nmwkn" event={"ID":"9af11dde-7687-459a-8857-0d41fd189315","Type":"ContainerStarted","Data":"e55dfa0e5760a0905155a354b1fdd9a7a4e50a5cd8c1cfb9c6dfe601321abb7c"}
Jul 28 16:21:45.580745 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:45.580706    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-rn2z4" podStartSLOduration=3.937169882 podStartE2EDuration="22.580693434s" totalImagesPullingTime="18.643523552s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 16:21:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:21:45.56121032 +0000 UTC m=+22.699679735" watchObservedRunningTime="2026-07-28 16:21:45.580693434 +0000 UTC m=+22.719162835"
Jul 28 16:21:45.595026 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:45.594991    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-nmwkn" podStartSLOduration=3.932757551 podStartE2EDuration="22.594980266s" totalImagesPullingTime="18.662222715s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 16:21:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:21:45.594695944 +0000 UTC m=+22.733165376" watchObservedRunningTime="2026-07-28 16:21:45.594980266 +0000 UTC m=+22.733449679"
Jul 28 16:21:45.611088 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:45.611049    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-6rk8j" podStartSLOduration=11.741415675 podStartE2EDuration="22.611018853s" totalImagesPullingTime="10.869603178s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 16:21:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:21:45.61099942 +0000 UTC m=+22.749468835" watchObservedRunningTime="2026-07-28 16:21:45.611018853 +0000 UTC m=+22.749488262"
Jul 28 16:21:45.678684 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:45.678646    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:21:45.731720 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:45.731677    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:21:46.273832 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:46.273807    2557 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 28 16:21:46.304253 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:46.304171    2557 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-28T16:21:46.273828182Z","UUID":"3fe256d7-4790-42ec-b800-b40c93cd3d0e","Handler":null,"Name":"","Endpoint":""}
Jul 28 16:21:46.305690 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:46.305673    2557 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 28 16:21:46.305798 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:46.305713    2557 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 28 16:21:46.428227 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:46.428140    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qzkm5"
Jul 28 16:21:46.428410 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:46.428275    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qzkm5" podUID="c687045e-f5d0-4a22-b71e-9c01463677e6"
Jul 28 16:21:46.556372 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:46.555954    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-2lpp7" event={"ID":"ecff0248-22af-459d-ae9a-083ed7508975","Type":"ContainerStarted","Data":"2262b499be8f29cc58d4b36ed6376d942efb52684475b7ce80bb602af6aa51d3"}
Jul 28 16:21:46.557964 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:46.557932    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nlb66" event={"ID":"38125fdc-3616-4f16-8683-aabde8a95527","Type":"ContainerStarted","Data":"477a1a99f1780921805617b2a3ab9661745f49a5aa59e21eaeb8e40da540b227"}
Jul 28 16:21:46.572636 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:46.572589    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-2lpp7" podStartSLOduration=4.927103726 podStartE2EDuration="23.572573299s" totalImagesPullingTime="18.645469573s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 16:21:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:21:46.571925595 +0000 UTC m=+23.710395022" watchObservedRunningTime="2026-07-28 16:21:46.572573299 +0000 UTC m=+23.711042714"
Jul 28 16:21:46.691259 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:46.691184    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:21:46.742355 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:46.742326    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:21:47.428193 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:47.427807    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qm9v7"
Jul 28 16:21:47.428193 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:47.427806    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9vtwz"
Jul 28 16:21:47.428193 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:47.427930    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qm9v7" podUID="92ef4728-9e89-4494-8bdf-72dcfe9b3ff4"
Jul 28 16:21:47.428193 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:47.427971    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9vtwz" podUID="31992b1b-4643-467f-a2b1-674af56e8c0e"
Jul 28 16:21:47.562130 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:47.562085    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nlb66" event={"ID":"38125fdc-3616-4f16-8683-aabde8a95527","Type":"ContainerStarted","Data":"451e81869ddb7843958b99b5d14266a841cb5ea5ba0a750fb67ce4bee0fbdc66"}
Jul 28 16:21:47.583610 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:47.583560    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nlb66" podStartSLOduration=3.672360841 podStartE2EDuration="24.583545281s" totalImagesPullingTime="20.91118444s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 16:21:23 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:21:47.582981734 +0000 UTC m=+24.721451147" watchObservedRunningTime="2026-07-28 16:21:47.583545281 +0000 UTC m=+24.722014694"
Jul 28 16:21:48.072913 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:48.072821    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:21:48.127517 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:48.127457    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:21:48.428387 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:48.428312    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qzkm5"
Jul 28 16:21:48.428573 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:48.428427    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qzkm5" podUID="c687045e-f5d0-4a22-b71e-9c01463677e6"
Jul 28 16:21:48.568418 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:48.568383    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sbf8h" event={"ID":"396c56d3-19a2-49b4-bc4a-c3dabeb144b4","Type":"ContainerStarted","Data":"9a2e7c99dd7282a5b5858ac6ee705fbd97e5cc8e08385baa294da9130fa69990"}
Jul 28 16:21:49.103416 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:49.103378    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:21:49.157201 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:49.157156    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:21:49.431538 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:49.431311    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qm9v7"
Jul 28 16:21:49.431689 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:49.431344    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9vtwz"
Jul 28 16:21:49.431689 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:49.431588    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qm9v7" podUID="92ef4728-9e89-4494-8bdf-72dcfe9b3ff4"
Jul 28 16:21:49.431804 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:49.431685    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9vtwz" podUID="31992b1b-4643-467f-a2b1-674af56e8c0e"
Jul 28 16:21:49.559408 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:49.559374    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-6rk8j"
Jul 28 16:21:49.559997 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:49.559981    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-6rk8j"
Jul 28 16:21:49.803794 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:49.803700    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:21:49.904644 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:49.904614    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:21:50.427773 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:50.427711    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qzkm5"
Jul 28 16:21:50.427863 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:50.427815    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qzkm5" podUID="c687045e-f5d0-4a22-b71e-9c01463677e6"
Jul 28 16:21:50.575784 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:50.575752    2557 generic.go:356] "Generic (PLEG): container finished" podID="695dbc5b-d1b4-42f3-921a-cc62010ddc14" containerID="88446b24d146d897eecf95e5fb172e08d4dcce763c9f7f08d2120034b6cdaf99" exitCode=0
Jul 28 16:21:50.575906 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:50.575834    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-x2pv7" event={"ID":"695dbc5b-d1b4-42f3-921a-cc62010ddc14","Type":"ContainerDied","Data":"88446b24d146d897eecf95e5fb172e08d4dcce763c9f7f08d2120034b6cdaf99"}
Jul 28 16:21:50.579590 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:50.579571    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sbf8h" event={"ID":"396c56d3-19a2-49b4-bc4a-c3dabeb144b4","Type":"ContainerStarted","Data":"209f57d429fe339f4e8fd6fd8bd5b9f6b0d6ba74e7ef5beb5a879477a546bc08"}
Jul 28 16:21:50.579843 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:50.579829    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-sbf8h"
Jul 28 16:21:50.579898 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:50.579891    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-sbf8h"
Jul 28 16:21:50.595294 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:50.595276    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-sbf8h"
Jul 28 16:21:50.617903 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:50.617862    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-sbf8h" podStartSLOduration=8.90013188 podStartE2EDuration="27.617850005s" totalImagesPullingTime="18.717718125s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 16:21:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:21:50.616323954 +0000 UTC m=+27.754793367" watchObservedRunningTime="2026-07-28 16:21:50.617850005 +0000 UTC m=+27.756319420"
Jul 28 16:21:50.958840 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:50.958815    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-6rk8j"
Jul 28 16:21:50.959559 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:50.958923    2557 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 28 16:21:50.959620 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:50.959604    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-6rk8j"
Jul 28 16:21:51.018564 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:51.018523    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:21:51.431352 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:51.431328    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qm9v7"
Jul 28 16:21:51.431518 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:51.431427    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qm9v7" podUID="92ef4728-9e89-4494-8bdf-72dcfe9b3ff4"
Jul 28 16:21:51.431518 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:51.431481    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9vtwz"
Jul 28 16:21:51.431605 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:51.431584    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9vtwz" podUID="31992b1b-4643-467f-a2b1-674af56e8c0e"
Jul 28 16:21:51.581878 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:51.581718    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-sbf8h"
Jul 28 16:21:51.596396 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:51.596375    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-sbf8h"
Jul 28 16:21:52.164771 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:52.164662    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-qm9v7"]
Jul 28 16:21:52.165198 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:52.164831    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qm9v7"
Jul 28 16:21:52.165198 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:52.164951    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qm9v7" podUID="92ef4728-9e89-4494-8bdf-72dcfe9b3ff4"
Jul 28 16:21:52.165454 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:52.165411    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-9vtwz"]
Jul 28 16:21:52.165600 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:52.165542    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9vtwz"
Jul 28 16:21:52.165668 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:52.165626    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9vtwz" podUID="31992b1b-4643-467f-a2b1-674af56e8c0e"
Jul 28 16:21:52.166132 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:52.166109    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-qzkm5"]
Jul 28 16:21:52.166693 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:52.166211    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qzkm5"
Jul 28 16:21:52.166693 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:52.166297    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qzkm5" podUID="c687045e-f5d0-4a22-b71e-9c01463677e6"
Jul 28 16:21:52.585801 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:52.585765    2557 generic.go:356] "Generic (PLEG): container finished" podID="695dbc5b-d1b4-42f3-921a-cc62010ddc14" containerID="957d96e64ef7f376ce69e4289dac03201c544e34f3b22ab71ac18d7c67fed871" exitCode=0
Jul 28 16:21:52.586020 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:52.585844    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-x2pv7" event={"ID":"695dbc5b-d1b4-42f3-921a-cc62010ddc14","Type":"ContainerDied","Data":"957d96e64ef7f376ce69e4289dac03201c544e34f3b22ab71ac18d7c67fed871"}
Jul 28 16:21:53.089047 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:53.089002    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:21:53.429077 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:53.429004    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9vtwz"
Jul 28 16:21:53.429512 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:53.429080    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9vtwz" podUID="31992b1b-4643-467f-a2b1-674af56e8c0e"
Jul 28 16:21:53.429512 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:53.429164    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qm9v7"
Jul 28 16:21:53.429512 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:53.429283    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qm9v7" podUID="92ef4728-9e89-4494-8bdf-72dcfe9b3ff4"
Jul 28 16:21:53.429512 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:53.429331    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qzkm5"
Jul 28 16:21:53.429512 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:53.429401    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qzkm5" podUID="c687045e-f5d0-4a22-b71e-9c01463677e6"
Jul 28 16:21:54.594938 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:54.594899    2557 generic.go:356] "Generic (PLEG): container finished" podID="695dbc5b-d1b4-42f3-921a-cc62010ddc14" containerID="9ed933026deff76b696d75c59de77265298210f0e7d408222e34b744f1e2e5a4" exitCode=0
Jul 28 16:21:54.595317 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:54.594947    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-x2pv7" event={"ID":"695dbc5b-d1b4-42f3-921a-cc62010ddc14","Type":"ContainerDied","Data":"9ed933026deff76b696d75c59de77265298210f0e7d408222e34b744f1e2e5a4"}
Jul 28 16:21:55.431391 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:55.431357    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qzkm5"
Jul 28 16:21:55.431561 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:55.431454    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9vtwz"
Jul 28 16:21:55.431561 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:55.431462    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qzkm5" podUID="c687045e-f5d0-4a22-b71e-9c01463677e6"
Jul 28 16:21:55.431561 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:55.431487    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qm9v7"
Jul 28 16:21:55.431675 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:55.431601    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9vtwz" podUID="31992b1b-4643-467f-a2b1-674af56e8c0e"
Jul 28 16:21:55.431741 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:55.431711    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qm9v7" podUID="92ef4728-9e89-4494-8bdf-72dcfe9b3ff4"
Jul 28 16:21:57.006906 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:57.006864    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/92ef4728-9e89-4494-8bdf-72dcfe9b3ff4-metrics-certs\") pod \"network-metrics-daemon-qm9v7\" (UID: \"92ef4728-9e89-4494-8bdf-72dcfe9b3ff4\") " pod="openshift-multus/network-metrics-daemon-qm9v7"
Jul 28 16:21:57.007365 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:57.006928    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/31992b1b-4643-467f-a2b1-674af56e8c0e-original-pull-secret\") pod \"global-pull-secret-syncer-9vtwz\" (UID: \"31992b1b-4643-467f-a2b1-674af56e8c0e\") " pod="kube-system/global-pull-secret-syncer-9vtwz"
Jul 28 16:21:57.007365 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:57.007041    2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 16:21:57.007365 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:57.007057    2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 16:21:57.007365 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:57.007116    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92ef4728-9e89-4494-8bdf-72dcfe9b3ff4-metrics-certs podName:92ef4728-9e89-4494-8bdf-72dcfe9b3ff4 nodeName:}" failed. No retries permitted until 2026-07-28 16:22:29.007096596 +0000 UTC m=+66.145566004 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/92ef4728-9e89-4494-8bdf-72dcfe9b3ff4-metrics-certs") pod "network-metrics-daemon-qm9v7" (UID: "92ef4728-9e89-4494-8bdf-72dcfe9b3ff4") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 16:21:57.007365 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:57.007140    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31992b1b-4643-467f-a2b1-674af56e8c0e-original-pull-secret podName:31992b1b-4643-467f-a2b1-674af56e8c0e nodeName:}" failed. No retries permitted until 2026-07-28 16:22:29.007127148 +0000 UTC m=+66.145596543 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/31992b1b-4643-467f-a2b1-674af56e8c0e-original-pull-secret") pod "global-pull-secret-syncer-9vtwz" (UID: "31992b1b-4643-467f-a2b1-674af56e8c0e") : object "kube-system"/"original-pull-secret" not registered
Jul 28 16:21:57.108151 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:57.108119    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xl2sp\" (UniqueName: \"kubernetes.io/projected/c687045e-f5d0-4a22-b71e-9c01463677e6-kube-api-access-xl2sp\") pod \"network-check-target-qzkm5\" (UID: \"c687045e-f5d0-4a22-b71e-9c01463677e6\") " pod="openshift-network-diagnostics/network-check-target-qzkm5"
Jul 28 16:21:57.108311 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:57.108283    2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 16:21:57.108311 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:57.108300    2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 16:21:57.108311 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:57.108312    2557 projected.go:196] Error preparing data for projected volume kube-api-access-xl2sp for pod openshift-network-diagnostics/network-check-target-qzkm5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 16:21:57.108443 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:57.108366    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c687045e-f5d0-4a22-b71e-9c01463677e6-kube-api-access-xl2sp podName:c687045e-f5d0-4a22-b71e-9c01463677e6 nodeName:}" failed. No retries permitted until 2026-07-28 16:22:29.108348089 +0000 UTC m=+66.246817483 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-xl2sp" (UniqueName: "kubernetes.io/projected/c687045e-f5d0-4a22-b71e-9c01463677e6-kube-api-access-xl2sp") pod "network-check-target-qzkm5" (UID: "c687045e-f5d0-4a22-b71e-9c01463677e6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 16:21:57.428671 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:57.428638    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qm9v7"
Jul 28 16:21:57.428671 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:57.428675    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qzkm5"
Jul 28 16:21:57.428929 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:57.428638    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9vtwz"
Jul 28 16:21:57.428929 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:57.428786    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qm9v7" podUID="92ef4728-9e89-4494-8bdf-72dcfe9b3ff4"
Jul 28 16:21:57.428929 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:57.428859    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9vtwz" podUID="31992b1b-4643-467f-a2b1-674af56e8c0e"
Jul 28 16:21:57.429079 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:57.428944    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qzkm5" podUID="c687045e-f5d0-4a22-b71e-9c01463677e6"
Jul 28 16:21:58.417120 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:58.416893    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:21:58.469058 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:58.469015    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:21:58.665543 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:58.665443    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-154.ec2.internal" event="NodeReady"
Jul 28 16:21:58.665692 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:58.665658    2557 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 28 16:21:58.699914 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:58.699887    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6cf9d7745c-2f4rl"]
Jul 28 16:21:58.701557 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:58.701539    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6cf9d7745c-2f4rl"
Jul 28 16:21:58.703775 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:58.703754    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 28 16:21:58.703897 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:58.703845    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 28 16:21:58.703960 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:58.703895    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-zshnb\""
Jul 28 16:21:58.704006 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:58.703993    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 28 16:21:58.709269 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:58.709100    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 28 16:21:58.709812 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:58.709796    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-wbnl5"]
Jul 28 16:21:58.711538 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:58.711519    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-wbnl5"
Jul 28 16:21:58.712567 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:58.712547    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-rt2ch"]
Jul 28 16:21:58.713822 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:58.713802    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 28 16:21:58.713910 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:58.713895    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-hmx4q\""
Jul 28 16:21:58.714061 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:58.714038    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 28 16:21:58.714158 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:58.714134    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-rt2ch"
Jul 28 16:21:58.716308 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:58.716291    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-sllng\""
Jul 28 16:21:58.716404 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:58.716293    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 28 16:21:58.716591 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:58.716574    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 28 16:21:58.717030 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:58.717016    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 28 16:21:58.717708 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:58.717689    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6cf9d7745c-2f4rl"]
Jul 28 16:21:58.725244 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:58.725223    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-wbnl5"]
Jul 28 16:21:58.728127 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:58.728107    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-rt2ch"]
Jul 28 16:21:58.822246 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:58.822226    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qlkh5\" (UniqueName: \"kubernetes.io/projected/9b7c5d20-5762-4a8d-9420-ca25360cd351-kube-api-access-qlkh5\") pod \"dns-default-wbnl5\" (UID: \"9b7c5d20-5762-4a8d-9420-ca25360cd351\") " pod="openshift-dns/dns-default-wbnl5"
Jul 28 16:21:58.822305 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:58.822255    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/915e2010-9827-4845-b395-a7ee1fedcfe7-trusted-ca\") pod \"image-registry-6cf9d7745c-2f4rl\" (UID: \"915e2010-9827-4845-b395-a7ee1fedcfe7\") " pod="openshift-image-registry/image-registry-6cf9d7745c-2f4rl"
Jul 28 16:21:58.822305 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:58.822278    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/915e2010-9827-4845-b395-a7ee1fedcfe7-bound-sa-token\") pod \"image-registry-6cf9d7745c-2f4rl\" (UID: \"915e2010-9827-4845-b395-a7ee1fedcfe7\") " pod="openshift-image-registry/image-registry-6cf9d7745c-2f4rl"
Jul 28 16:21:58.822410 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:58.822384    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9b7c5d20-5762-4a8d-9420-ca25360cd351-tmp-dir\") pod \"dns-default-wbnl5\" (UID: \"9b7c5d20-5762-4a8d-9420-ca25360cd351\") " pod="openshift-dns/dns-default-wbnl5"
Jul 28 16:21:58.822623 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:58.822438    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/915e2010-9827-4845-b395-a7ee1fedcfe7-installation-pull-secrets\") pod \"image-registry-6cf9d7745c-2f4rl\" (UID: \"915e2010-9827-4845-b395-a7ee1fedcfe7\") " pod="openshift-image-registry/image-registry-6cf9d7745c-2f4rl"
Jul 28 16:21:58.825960 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:58.822894    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9b7c5d20-5762-4a8d-9420-ca25360cd351-metrics-tls\") pod \"dns-default-wbnl5\" (UID: \"9b7c5d20-5762-4a8d-9420-ca25360cd351\") " pod="openshift-dns/dns-default-wbnl5"
Jul 28 16:21:58.825960 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:58.822979    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e958d4e8-df2b-4009-b83a-2625c110d4f9-cert\") pod \"ingress-canary-rt2ch\" (UID: \"e958d4e8-df2b-4009-b83a-2625c110d4f9\") " pod="openshift-ingress-canary/ingress-canary-rt2ch"
Jul 28 16:21:58.825960 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:58.823062    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9b7c5d20-5762-4a8d-9420-ca25360cd351-config-volume\") pod \"dns-default-wbnl5\" (UID: \"9b7c5d20-5762-4a8d-9420-ca25360cd351\") " pod="openshift-dns/dns-default-wbnl5"
Jul 28 16:21:58.825960 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:58.823100    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mrdrm\" (UniqueName: \"kubernetes.io/projected/e958d4e8-df2b-4009-b83a-2625c110d4f9-kube-api-access-mrdrm\") pod \"ingress-canary-rt2ch\" (UID: \"e958d4e8-df2b-4009-b83a-2625c110d4f9\") " pod="openshift-ingress-canary/ingress-canary-rt2ch"
Jul 28 16:21:58.825960 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:58.823135    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/915e2010-9827-4845-b395-a7ee1fedcfe7-image-registry-private-configuration\") pod \"image-registry-6cf9d7745c-2f4rl\" (UID: \"915e2010-9827-4845-b395-a7ee1fedcfe7\") " pod="openshift-image-registry/image-registry-6cf9d7745c-2f4rl"
Jul 28 16:21:58.825960 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:58.823171    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-59ccc\" (UniqueName: \"kubernetes.io/projected/915e2010-9827-4845-b395-a7ee1fedcfe7-kube-api-access-59ccc\") pod \"image-registry-6cf9d7745c-2f4rl\" (UID: \"915e2010-9827-4845-b395-a7ee1fedcfe7\") " pod="openshift-image-registry/image-registry-6cf9d7745c-2f4rl"
Jul 28 16:21:58.825960 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:58.823225    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/915e2010-9827-4845-b395-a7ee1fedcfe7-ca-trust-extracted\") pod \"image-registry-6cf9d7745c-2f4rl\" (UID: \"915e2010-9827-4845-b395-a7ee1fedcfe7\") " pod="openshift-image-registry/image-registry-6cf9d7745c-2f4rl"
Jul 28 16:21:58.825960 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:58.823255    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/915e2010-9827-4845-b395-a7ee1fedcfe7-registry-tls\") pod \"image-registry-6cf9d7745c-2f4rl\" (UID: \"915e2010-9827-4845-b395-a7ee1fedcfe7\") " pod="openshift-image-registry/image-registry-6cf9d7745c-2f4rl"
Jul 28 16:21:58.825960 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:58.823279    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/915e2010-9827-4845-b395-a7ee1fedcfe7-registry-certificates\") pod \"image-registry-6cf9d7745c-2f4rl\" (UID: \"915e2010-9827-4845-b395-a7ee1fedcfe7\") " pod="openshift-image-registry/image-registry-6cf9d7745c-2f4rl"
Jul 28 16:21:58.924122 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:58.924061    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qlkh5\" (UniqueName: \"kubernetes.io/projected/9b7c5d20-5762-4a8d-9420-ca25360cd351-kube-api-access-qlkh5\") pod \"dns-default-wbnl5\" (UID: \"9b7c5d20-5762-4a8d-9420-ca25360cd351\") " pod="openshift-dns/dns-default-wbnl5"
Jul 28 16:21:58.924122 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:58.924101    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/915e2010-9827-4845-b395-a7ee1fedcfe7-trusted-ca\") pod \"image-registry-6cf9d7745c-2f4rl\" (UID: \"915e2010-9827-4845-b395-a7ee1fedcfe7\") " pod="openshift-image-registry/image-registry-6cf9d7745c-2f4rl"
Jul 28 16:21:58.924267 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:58.924131    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/915e2010-9827-4845-b395-a7ee1fedcfe7-bound-sa-token\") pod \"image-registry-6cf9d7745c-2f4rl\" (UID: \"915e2010-9827-4845-b395-a7ee1fedcfe7\") " pod="openshift-image-registry/image-registry-6cf9d7745c-2f4rl"
Jul 28 16:21:58.924267 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:58.924179    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9b7c5d20-5762-4a8d-9420-ca25360cd351-tmp-dir\") pod \"dns-default-wbnl5\" (UID: \"9b7c5d20-5762-4a8d-9420-ca25360cd351\") " pod="openshift-dns/dns-default-wbnl5"
Jul 28 16:21:58.924267 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:58.924211    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/915e2010-9827-4845-b395-a7ee1fedcfe7-installation-pull-secrets\") pod \"image-registry-6cf9d7745c-2f4rl\" (UID: \"915e2010-9827-4845-b395-a7ee1fedcfe7\") " pod="openshift-image-registry/image-registry-6cf9d7745c-2f4rl"
Jul 28 16:21:58.924267 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:58.924240    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9b7c5d20-5762-4a8d-9420-ca25360cd351-metrics-tls\") pod \"dns-default-wbnl5\" (UID: \"9b7c5d20-5762-4a8d-9420-ca25360cd351\") " pod="openshift-dns/dns-default-wbnl5"
Jul 28 16:21:58.924267 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:58.924263    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e958d4e8-df2b-4009-b83a-2625c110d4f9-cert\") pod \"ingress-canary-rt2ch\" (UID: \"e958d4e8-df2b-4009-b83a-2625c110d4f9\") " pod="openshift-ingress-canary/ingress-canary-rt2ch"
Jul 28 16:21:58.924549 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:58.924315    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9b7c5d20-5762-4a8d-9420-ca25360cd351-config-volume\") pod \"dns-default-wbnl5\" (UID: \"9b7c5d20-5762-4a8d-9420-ca25360cd351\") " pod="openshift-dns/dns-default-wbnl5"
Jul 28 16:21:58.924549 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:58.924342    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mrdrm\" (UniqueName: \"kubernetes.io/projected/e958d4e8-df2b-4009-b83a-2625c110d4f9-kube-api-access-mrdrm\") pod \"ingress-canary-rt2ch\" (UID: \"e958d4e8-df2b-4009-b83a-2625c110d4f9\") " pod="openshift-ingress-canary/ingress-canary-rt2ch"
Jul 28 16:21:58.924549 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:58.924367    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/915e2010-9827-4845-b395-a7ee1fedcfe7-image-registry-private-configuration\") pod \"image-registry-6cf9d7745c-2f4rl\" (UID: \"915e2010-9827-4845-b395-a7ee1fedcfe7\") " pod="openshift-image-registry/image-registry-6cf9d7745c-2f4rl"
Jul 28 16:21:58.924549 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:58.924379    2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 16:21:58.924549 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:58.924402    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-59ccc\" (UniqueName: \"kubernetes.io/projected/915e2010-9827-4845-b395-a7ee1fedcfe7-kube-api-access-59ccc\") pod \"image-registry-6cf9d7745c-2f4rl\" (UID: \"915e2010-9827-4845-b395-a7ee1fedcfe7\") " pod="openshift-image-registry/image-registry-6cf9d7745c-2f4rl"
Jul 28 16:21:58.924549 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:58.924443    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/915e2010-9827-4845-b395-a7ee1fedcfe7-ca-trust-extracted\") pod \"image-registry-6cf9d7745c-2f4rl\" (UID: \"915e2010-9827-4845-b395-a7ee1fedcfe7\") " pod="openshift-image-registry/image-registry-6cf9d7745c-2f4rl"
Jul 28 16:21:58.924549 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:58.924460    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9b7c5d20-5762-4a8d-9420-ca25360cd351-metrics-tls podName:9b7c5d20-5762-4a8d-9420-ca25360cd351 nodeName:}" failed. No retries permitted until 2026-07-28 16:21:59.424438108 +0000 UTC m=+36.562907503 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9b7c5d20-5762-4a8d-9420-ca25360cd351-metrics-tls") pod "dns-default-wbnl5" (UID: "9b7c5d20-5762-4a8d-9420-ca25360cd351") : secret "dns-default-metrics-tls" not found
Jul 28 16:21:58.924957 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:58.924577    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9b7c5d20-5762-4a8d-9420-ca25360cd351-tmp-dir\") pod \"dns-default-wbnl5\" (UID: \"9b7c5d20-5762-4a8d-9420-ca25360cd351\") " pod="openshift-dns/dns-default-wbnl5"
Jul 28 16:21:58.924957 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:58.924796    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/915e2010-9827-4845-b395-a7ee1fedcfe7-ca-trust-extracted\") pod \"image-registry-6cf9d7745c-2f4rl\" (UID: \"915e2010-9827-4845-b395-a7ee1fedcfe7\") " pod="openshift-image-registry/image-registry-6cf9d7745c-2f4rl"
Jul 28 16:21:58.924957 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:58.924843    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/915e2010-9827-4845-b395-a7ee1fedcfe7-registry-tls\") pod \"image-registry-6cf9d7745c-2f4rl\" (UID: \"915e2010-9827-4845-b395-a7ee1fedcfe7\") " pod="openshift-image-registry/image-registry-6cf9d7745c-2f4rl"
Jul 28 16:21:58.924957 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:58.924869    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/915e2010-9827-4845-b395-a7ee1fedcfe7-registry-certificates\") pod \"image-registry-6cf9d7745c-2f4rl\" (UID: \"915e2010-9827-4845-b395-a7ee1fedcfe7\") " pod="openshift-image-registry/image-registry-6cf9d7745c-2f4rl"
Jul 28 16:21:58.925160 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:58.924972    2557 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 28 16:21:58.925160 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:58.924988    2557 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6cf9d7745c-2f4rl: secret "image-registry-tls" not found
Jul 28 16:21:58.925160 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:58.925053    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/915e2010-9827-4845-b395-a7ee1fedcfe7-registry-tls podName:915e2010-9827-4845-b395-a7ee1fedcfe7 nodeName:}" failed. No retries permitted until 2026-07-28 16:21:59.425037704 +0000 UTC m=+36.563507101 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/915e2010-9827-4845-b395-a7ee1fedcfe7-registry-tls") pod "image-registry-6cf9d7745c-2f4rl" (UID: "915e2010-9827-4845-b395-a7ee1fedcfe7") : secret "image-registry-tls" not found
Jul 28 16:21:58.925160 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:58.925091    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9b7c5d20-5762-4a8d-9420-ca25360cd351-config-volume\") pod \"dns-default-wbnl5\" (UID: \"9b7c5d20-5762-4a8d-9420-ca25360cd351\") " pod="openshift-dns/dns-default-wbnl5"
Jul 28 16:21:58.925160 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:58.925106    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/915e2010-9827-4845-b395-a7ee1fedcfe7-trusted-ca\") pod \"image-registry-6cf9d7745c-2f4rl\" (UID: \"915e2010-9827-4845-b395-a7ee1fedcfe7\") " pod="openshift-image-registry/image-registry-6cf9d7745c-2f4rl"
Jul 28 16:21:58.925398 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:58.925344    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/915e2010-9827-4845-b395-a7ee1fedcfe7-registry-certificates\") pod \"image-registry-6cf9d7745c-2f4rl\" (UID: \"915e2010-9827-4845-b395-a7ee1fedcfe7\") " pod="openshift-image-registry/image-registry-6cf9d7745c-2f4rl"
Jul 28 16:21:58.928217 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:58.928200    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/915e2010-9827-4845-b395-a7ee1fedcfe7-installation-pull-secrets\") pod \"image-registry-6cf9d7745c-2f4rl\" (UID: \"915e2010-9827-4845-b395-a7ee1fedcfe7\") " pod="openshift-image-registry/image-registry-6cf9d7745c-2f4rl"
Jul 28 16:21:58.928217 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:58.928210    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/915e2010-9827-4845-b395-a7ee1fedcfe7-image-registry-private-configuration\") pod \"image-registry-6cf9d7745c-2f4rl\" (UID: \"915e2010-9827-4845-b395-a7ee1fedcfe7\") " pod="openshift-image-registry/image-registry-6cf9d7745c-2f4rl"
Jul 28 16:21:58.928319 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:58.928242    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e958d4e8-df2b-4009-b83a-2625c110d4f9-cert\") pod \"ingress-canary-rt2ch\" (UID: \"e958d4e8-df2b-4009-b83a-2625c110d4f9\") " pod="openshift-ingress-canary/ingress-canary-rt2ch"
Jul 28 16:21:58.932403 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:58.932383    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-59ccc\" (UniqueName: \"kubernetes.io/projected/915e2010-9827-4845-b395-a7ee1fedcfe7-kube-api-access-59ccc\") pod \"image-registry-6cf9d7745c-2f4rl\" (UID: \"915e2010-9827-4845-b395-a7ee1fedcfe7\") " pod="openshift-image-registry/image-registry-6cf9d7745c-2f4rl"
Jul 28 16:21:58.932914 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:58.932889    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qlkh5\" (UniqueName: \"kubernetes.io/projected/9b7c5d20-5762-4a8d-9420-ca25360cd351-kube-api-access-qlkh5\") pod \"dns-default-wbnl5\" (UID: \"9b7c5d20-5762-4a8d-9420-ca25360cd351\") " pod="openshift-dns/dns-default-wbnl5"
Jul 28 16:21:58.933692 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:58.933667    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/915e2010-9827-4845-b395-a7ee1fedcfe7-bound-sa-token\") pod \"image-registry-6cf9d7745c-2f4rl\" (UID: \"915e2010-9827-4845-b395-a7ee1fedcfe7\") " pod="openshift-image-registry/image-registry-6cf9d7745c-2f4rl"
Jul 28 16:21:58.933766 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:58.933713    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mrdrm\" (UniqueName: \"kubernetes.io/projected/e958d4e8-df2b-4009-b83a-2625c110d4f9-kube-api-access-mrdrm\") pod \"ingress-canary-rt2ch\" (UID: \"e958d4e8-df2b-4009-b83a-2625c110d4f9\") " pod="openshift-ingress-canary/ingress-canary-rt2ch"
Jul 28 16:21:59.028155 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:59.028125    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-rt2ch"
Jul 28 16:21:59.153900 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:59.153868    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-rt2ch"]
Jul 28 16:21:59.157851 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:21:59.157821    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode958d4e8_df2b_4009_b83a_2625c110d4f9.slice/crio-c32b3481397431a0430356320befc47a2cce43cfc2e04142a348cbe678b6497d WatchSource:0}: Error finding container c32b3481397431a0430356320befc47a2cce43cfc2e04142a348cbe678b6497d: Status 404 returned error can't find the container with id c32b3481397431a0430356320befc47a2cce43cfc2e04142a348cbe678b6497d
Jul 28 16:21:59.428020 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:59.427989    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qzkm5"
Jul 28 16:21:59.428813 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:59.428037    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qm9v7"
Jul 28 16:21:59.428813 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:59.427997    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9vtwz"
Jul 28 16:21:59.429315 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:59.429296    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9b7c5d20-5762-4a8d-9420-ca25360cd351-metrics-tls\") pod \"dns-default-wbnl5\" (UID: \"9b7c5d20-5762-4a8d-9420-ca25360cd351\") " pod="openshift-dns/dns-default-wbnl5"
Jul 28 16:21:59.429443 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:59.429425    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/915e2010-9827-4845-b395-a7ee1fedcfe7-registry-tls\") pod \"image-registry-6cf9d7745c-2f4rl\" (UID: \"915e2010-9827-4845-b395-a7ee1fedcfe7\") " pod="openshift-image-registry/image-registry-6cf9d7745c-2f4rl"
Jul 28 16:21:59.429477 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:59.429459    2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 16:21:59.429711 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:59.429556    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9b7c5d20-5762-4a8d-9420-ca25360cd351-metrics-tls podName:9b7c5d20-5762-4a8d-9420-ca25360cd351 nodeName:}" failed. No retries permitted until 2026-07-28 16:22:00.429535807 +0000 UTC m=+37.568005201 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9b7c5d20-5762-4a8d-9420-ca25360cd351-metrics-tls") pod "dns-default-wbnl5" (UID: "9b7c5d20-5762-4a8d-9420-ca25360cd351") : secret "dns-default-metrics-tls" not found
Jul 28 16:21:59.429711 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:59.429571    2557 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 28 16:21:59.429711 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:59.429583    2557 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6cf9d7745c-2f4rl: secret "image-registry-tls" not found
Jul 28 16:21:59.429711 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:21:59.429658    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/915e2010-9827-4845-b395-a7ee1fedcfe7-registry-tls podName:915e2010-9827-4845-b395-a7ee1fedcfe7 nodeName:}" failed. No retries permitted until 2026-07-28 16:22:00.429645686 +0000 UTC m=+37.568115081 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/915e2010-9827-4845-b395-a7ee1fedcfe7-registry-tls") pod "image-registry-6cf9d7745c-2f4rl" (UID: "915e2010-9827-4845-b395-a7ee1fedcfe7") : secret "image-registry-tls" not found
Jul 28 16:21:59.430648 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:59.430632    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 28 16:21:59.431565 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:59.431548    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-gnm96\""
Jul 28 16:21:59.431649 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:59.431574    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 28 16:21:59.431649 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:59.431577    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-nr6mp\""
Jul 28 16:21:59.431649 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:59.431621    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 28 16:21:59.431649 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:59.431549    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 28 16:21:59.608928 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:59.608900    2557 generic.go:356] "Generic (PLEG): container finished" podID="695dbc5b-d1b4-42f3-921a-cc62010ddc14" containerID="699bcf97fee65a64a4e2f4b28baa4fbbe71caf24793aef771edcad0e61f0ffc0" exitCode=0
Jul 28 16:21:59.609089 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:59.608989    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-x2pv7" event={"ID":"695dbc5b-d1b4-42f3-921a-cc62010ddc14","Type":"ContainerDied","Data":"699bcf97fee65a64a4e2f4b28baa4fbbe71caf24793aef771edcad0e61f0ffc0"}
Jul 28 16:21:59.610094 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:21:59.610071    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-rt2ch" event={"ID":"e958d4e8-df2b-4009-b83a-2625c110d4f9","Type":"ContainerStarted","Data":"c32b3481397431a0430356320befc47a2cce43cfc2e04142a348cbe678b6497d"}
Jul 28 16:22:00.079389 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:00.079212    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-vctpt"]
Jul 28 16:22:00.081237 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:00.081209    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-vctpt"
Jul 28 16:22:00.083735 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:00.083705    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Jul 28 16:22:00.083846 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:00.083753    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Jul 28 16:22:00.083922 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:00.083914    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Jul 28 16:22:00.084000 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:00.083980    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-4kb9z\""
Jul 28 16:22:00.084154 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:00.084138    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Jul 28 16:22:00.091656 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:00.091616    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-vctpt"]
Jul 28 16:22:00.234509 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:00.234472    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ts4t9\" (UniqueName: \"kubernetes.io/projected/9d3954a2-15d1-4000-9941-6555dc3aabd2-kube-api-access-ts4t9\") pod \"service-ca-operator-5d6c75f84f-vctpt\" (UID: \"9d3954a2-15d1-4000-9941-6555dc3aabd2\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-vctpt"
Jul 28 16:22:00.234675 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:00.234527    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9d3954a2-15d1-4000-9941-6555dc3aabd2-serving-cert\") pod \"service-ca-operator-5d6c75f84f-vctpt\" (UID: \"9d3954a2-15d1-4000-9941-6555dc3aabd2\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-vctpt"
Jul 28 16:22:00.234675 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:00.234593    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9d3954a2-15d1-4000-9941-6555dc3aabd2-config\") pod \"service-ca-operator-5d6c75f84f-vctpt\" (UID: \"9d3954a2-15d1-4000-9941-6555dc3aabd2\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-vctpt"
Jul 28 16:22:00.335188 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:00.335106    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9d3954a2-15d1-4000-9941-6555dc3aabd2-config\") pod \"service-ca-operator-5d6c75f84f-vctpt\" (UID: \"9d3954a2-15d1-4000-9941-6555dc3aabd2\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-vctpt"
Jul 28 16:22:00.335345 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:00.335211    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ts4t9\" (UniqueName: \"kubernetes.io/projected/9d3954a2-15d1-4000-9941-6555dc3aabd2-kube-api-access-ts4t9\") pod \"service-ca-operator-5d6c75f84f-vctpt\" (UID: \"9d3954a2-15d1-4000-9941-6555dc3aabd2\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-vctpt"
Jul 28 16:22:00.335345 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:00.335241    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9d3954a2-15d1-4000-9941-6555dc3aabd2-serving-cert\") pod \"service-ca-operator-5d6c75f84f-vctpt\" (UID: \"9d3954a2-15d1-4000-9941-6555dc3aabd2\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-vctpt"
Jul 28 16:22:00.335856 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:00.335824    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9d3954a2-15d1-4000-9941-6555dc3aabd2-config\") pod \"service-ca-operator-5d6c75f84f-vctpt\" (UID: \"9d3954a2-15d1-4000-9941-6555dc3aabd2\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-vctpt"
Jul 28 16:22:00.338963 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:00.338942    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9d3954a2-15d1-4000-9941-6555dc3aabd2-serving-cert\") pod \"service-ca-operator-5d6c75f84f-vctpt\" (UID: \"9d3954a2-15d1-4000-9941-6555dc3aabd2\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-vctpt"
Jul 28 16:22:00.343526 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:00.343486    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ts4t9\" (UniqueName: \"kubernetes.io/projected/9d3954a2-15d1-4000-9941-6555dc3aabd2-kube-api-access-ts4t9\") pod \"service-ca-operator-5d6c75f84f-vctpt\" (UID: \"9d3954a2-15d1-4000-9941-6555dc3aabd2\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-vctpt"
Jul 28 16:22:00.392666 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:00.392645    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-vctpt"
Jul 28 16:22:00.436554 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:00.436523    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9b7c5d20-5762-4a8d-9420-ca25360cd351-metrics-tls\") pod \"dns-default-wbnl5\" (UID: \"9b7c5d20-5762-4a8d-9420-ca25360cd351\") " pod="openshift-dns/dns-default-wbnl5"
Jul 28 16:22:00.436874 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:00.436631    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/915e2010-9827-4845-b395-a7ee1fedcfe7-registry-tls\") pod \"image-registry-6cf9d7745c-2f4rl\" (UID: \"915e2010-9827-4845-b395-a7ee1fedcfe7\") " pod="openshift-image-registry/image-registry-6cf9d7745c-2f4rl"
Jul 28 16:22:00.436874 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:22:00.436758    2557 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 28 16:22:00.436874 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:22:00.436774    2557 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6cf9d7745c-2f4rl: secret "image-registry-tls" not found
Jul 28 16:22:00.436874 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:22:00.436834    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/915e2010-9827-4845-b395-a7ee1fedcfe7-registry-tls podName:915e2010-9827-4845-b395-a7ee1fedcfe7 nodeName:}" failed. No retries permitted until 2026-07-28 16:22:02.436814961 +0000 UTC m=+39.575284356 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/915e2010-9827-4845-b395-a7ee1fedcfe7-registry-tls") pod "image-registry-6cf9d7745c-2f4rl" (UID: "915e2010-9827-4845-b395-a7ee1fedcfe7") : secret "image-registry-tls" not found
Jul 28 16:22:00.437198 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:22:00.437178    2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 16:22:00.437310 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:22:00.437241    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9b7c5d20-5762-4a8d-9420-ca25360cd351-metrics-tls podName:9b7c5d20-5762-4a8d-9420-ca25360cd351 nodeName:}" failed. No retries permitted until 2026-07-28 16:22:02.437215558 +0000 UTC m=+39.575684957 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9b7c5d20-5762-4a8d-9420-ca25360cd351-metrics-tls") pod "dns-default-wbnl5" (UID: "9b7c5d20-5762-4a8d-9420-ca25360cd351") : secret "dns-default-metrics-tls" not found
Jul 28 16:22:00.522089 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:00.522044    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:22:00.525087 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:00.525056    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-vctpt"]
Jul 28 16:22:00.529814 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:22:00.529776    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d3954a2_15d1_4000_9941_6555dc3aabd2.slice/crio-7ef5e452685ca3d8f2b32f5770a9cef601e3f49bcaaa1ace526b3e0cedbe366f WatchSource:0}: Error finding container 7ef5e452685ca3d8f2b32f5770a9cef601e3f49bcaaa1ace526b3e0cedbe366f: Status 404 returned error can't find the container with id 7ef5e452685ca3d8f2b32f5770a9cef601e3f49bcaaa1ace526b3e0cedbe366f
Jul 28 16:22:00.586644 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:00.586545    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:22:00.613854 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:00.613821    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-vctpt" event={"ID":"9d3954a2-15d1-4000-9941-6555dc3aabd2","Type":"ContainerStarted","Data":"7ef5e452685ca3d8f2b32f5770a9cef601e3f49bcaaa1ace526b3e0cedbe366f"}
Jul 28 16:22:00.617953 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:00.617927    2557 generic.go:356] "Generic (PLEG): container finished" podID="695dbc5b-d1b4-42f3-921a-cc62010ddc14" containerID="f9c7a2f80b0db24a9dc185a808834983352153cc5781c39dc3620c9a34717f9f" exitCode=0
Jul 28 16:22:00.618088 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:00.617971    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-x2pv7" event={"ID":"695dbc5b-d1b4-42f3-921a-cc62010ddc14","Type":"ContainerDied","Data":"f9c7a2f80b0db24a9dc185a808834983352153cc5781c39dc3620c9a34717f9f"}
Jul 28 16:22:00.663150 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:00.663103    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:22:01.625069 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:01.625031    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-x2pv7" event={"ID":"695dbc5b-d1b4-42f3-921a-cc62010ddc14","Type":"ContainerStarted","Data":"e25b56888258ee07b2112eb03e6a53f03e0d9600a1ecb6848d2c155f8b166689"}
Jul 28 16:22:01.648285 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:01.648075    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-x2pv7" podStartSLOduration=9.111082299 podStartE2EDuration="38.648059735s" totalImagesPullingTime="29.536977436s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 16:21:23 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:22:01.647371326 +0000 UTC m=+38.785840742" watchObservedRunningTime="2026-07-28 16:22:01.648059735 +0000 UTC m=+38.786529141"
Jul 28 16:22:01.766811 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:01.766771    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:22:01.820122 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:01.820030    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:22:02.181929 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:02.181843    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:22:02.282722 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:02.282616    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:22:02.334460 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:02.334421    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:22:02.452336 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:02.452251    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9b7c5d20-5762-4a8d-9420-ca25360cd351-metrics-tls\") pod \"dns-default-wbnl5\" (UID: \"9b7c5d20-5762-4a8d-9420-ca25360cd351\") " pod="openshift-dns/dns-default-wbnl5"
Jul 28 16:22:02.452552 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:02.452335    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/915e2010-9827-4845-b395-a7ee1fedcfe7-registry-tls\") pod \"image-registry-6cf9d7745c-2f4rl\" (UID: \"915e2010-9827-4845-b395-a7ee1fedcfe7\") " pod="openshift-image-registry/image-registry-6cf9d7745c-2f4rl"
Jul 28 16:22:02.452552 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:22:02.452437    2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 16:22:02.452552 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:22:02.452456    2557 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 28 16:22:02.452552 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:22:02.452470    2557 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6cf9d7745c-2f4rl: secret "image-registry-tls" not found
Jul 28 16:22:02.452552 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:22:02.452537    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9b7c5d20-5762-4a8d-9420-ca25360cd351-metrics-tls podName:9b7c5d20-5762-4a8d-9420-ca25360cd351 nodeName:}" failed. No retries permitted until 2026-07-28 16:22:06.452513358 +0000 UTC m=+43.590982763 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9b7c5d20-5762-4a8d-9420-ca25360cd351-metrics-tls") pod "dns-default-wbnl5" (UID: "9b7c5d20-5762-4a8d-9420-ca25360cd351") : secret "dns-default-metrics-tls" not found
Jul 28 16:22:02.452811 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:22:02.452562    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/915e2010-9827-4845-b395-a7ee1fedcfe7-registry-tls podName:915e2010-9827-4845-b395-a7ee1fedcfe7 nodeName:}" failed. No retries permitted until 2026-07-28 16:22:06.452550643 +0000 UTC m=+43.591020078 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/915e2010-9827-4845-b395-a7ee1fedcfe7-registry-tls") pod "image-registry-6cf9d7745c-2f4rl" (UID: "915e2010-9827-4845-b395-a7ee1fedcfe7") : secret "image-registry-tls" not found
Jul 28 16:22:02.543870 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:02.543836    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-6fjcf"]
Jul 28 16:22:02.567654 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:02.567608    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-6fjcf"]
Jul 28 16:22:02.567822 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:02.567742    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-6fjcf"
Jul 28 16:22:02.571574 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:02.571552    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-lx2xj\""
Jul 28 16:22:02.571997 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:02.571977    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 28 16:22:02.572265 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:02.572235    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 28 16:22:02.573292 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:02.573268    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 28 16:22:02.573508 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:02.573474    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 28 16:22:02.578920 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:02.578900    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:22:02.647441 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:02.647378    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:22:02.755435 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:02.755371    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kf7nd\" (UniqueName: \"kubernetes.io/projected/f59115d6-8481-469e-a651-21e7faa6c392-kube-api-access-kf7nd\") pod \"insights-runtime-extractor-6fjcf\" (UID: \"f59115d6-8481-469e-a651-21e7faa6c392\") " pod="openshift-insights/insights-runtime-extractor-6fjcf"
Jul 28 16:22:02.755603 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:02.755451    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f59115d6-8481-469e-a651-21e7faa6c392-data-volume\") pod \"insights-runtime-extractor-6fjcf\" (UID: \"f59115d6-8481-469e-a651-21e7faa6c392\") " pod="openshift-insights/insights-runtime-extractor-6fjcf"
Jul 28 16:22:02.755603 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:02.755468    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f59115d6-8481-469e-a651-21e7faa6c392-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6fjcf\" (UID: \"f59115d6-8481-469e-a651-21e7faa6c392\") " pod="openshift-insights/insights-runtime-extractor-6fjcf"
Jul 28 16:22:02.755603 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:02.755570    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f59115d6-8481-469e-a651-21e7faa6c392-crio-socket\") pod \"insights-runtime-extractor-6fjcf\" (UID: \"f59115d6-8481-469e-a651-21e7faa6c392\") " pod="openshift-insights/insights-runtime-extractor-6fjcf"
Jul 28 16:22:02.755733 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:02.755683    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f59115d6-8481-469e-a651-21e7faa6c392-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6fjcf\" (UID: \"f59115d6-8481-469e-a651-21e7faa6c392\") " pod="openshift-insights/insights-runtime-extractor-6fjcf"
Jul 28 16:22:02.856279 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:02.856256    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f59115d6-8481-469e-a651-21e7faa6c392-data-volume\") pod \"insights-runtime-extractor-6fjcf\" (UID: \"f59115d6-8481-469e-a651-21e7faa6c392\") " pod="openshift-insights/insights-runtime-extractor-6fjcf"
Jul 28 16:22:02.856393 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:02.856291    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f59115d6-8481-469e-a651-21e7faa6c392-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6fjcf\" (UID: \"f59115d6-8481-469e-a651-21e7faa6c392\") " pod="openshift-insights/insights-runtime-extractor-6fjcf"
Jul 28 16:22:02.856393 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:22:02.856362    2557 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 28 16:22:02.856484 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:22:02.856415    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f59115d6-8481-469e-a651-21e7faa6c392-kube-rbac-proxy-cm podName:f59115d6-8481-469e-a651-21e7faa6c392 nodeName:}" failed. No retries permitted until 2026-07-28 16:22:03.356402587 +0000 UTC m=+40.494871978 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f59115d6-8481-469e-a651-21e7faa6c392-kube-rbac-proxy-cm") pod "insights-runtime-extractor-6fjcf" (UID: "f59115d6-8481-469e-a651-21e7faa6c392") : configmap "kube-rbac-proxy" not found
Jul 28 16:22:02.856484 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:02.856415    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f59115d6-8481-469e-a651-21e7faa6c392-crio-socket\") pod \"insights-runtime-extractor-6fjcf\" (UID: \"f59115d6-8481-469e-a651-21e7faa6c392\") " pod="openshift-insights/insights-runtime-extractor-6fjcf"
Jul 28 16:22:02.856484 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:02.856474    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f59115d6-8481-469e-a651-21e7faa6c392-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6fjcf\" (UID: \"f59115d6-8481-469e-a651-21e7faa6c392\") " pod="openshift-insights/insights-runtime-extractor-6fjcf"
Jul 28 16:22:02.856628 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:02.856535    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kf7nd\" (UniqueName: \"kubernetes.io/projected/f59115d6-8481-469e-a651-21e7faa6c392-kube-api-access-kf7nd\") pod \"insights-runtime-extractor-6fjcf\" (UID: \"f59115d6-8481-469e-a651-21e7faa6c392\") " pod="openshift-insights/insights-runtime-extractor-6fjcf"
Jul 28 16:22:02.856628 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:02.856578    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f59115d6-8481-469e-a651-21e7faa6c392-data-volume\") pod \"insights-runtime-extractor-6fjcf\" (UID: \"f59115d6-8481-469e-a651-21e7faa6c392\") " pod="openshift-insights/insights-runtime-extractor-6fjcf"
Jul 28 16:22:02.856727 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:22:02.856648    2557 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 16:22:02.856727 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:02.856679    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f59115d6-8481-469e-a651-21e7faa6c392-crio-socket\") pod \"insights-runtime-extractor-6fjcf\" (UID: \"f59115d6-8481-469e-a651-21e7faa6c392\") " pod="openshift-insights/insights-runtime-extractor-6fjcf"
Jul 28 16:22:02.856727 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:22:02.856694    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f59115d6-8481-469e-a651-21e7faa6c392-insights-runtime-extractor-tls podName:f59115d6-8481-469e-a651-21e7faa6c392 nodeName:}" failed. No retries permitted until 2026-07-28 16:22:03.356681651 +0000 UTC m=+40.495151043 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f59115d6-8481-469e-a651-21e7faa6c392-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6fjcf" (UID: "f59115d6-8481-469e-a651-21e7faa6c392") : secret "insights-runtime-extractor-tls" not found
Jul 28 16:22:02.865363 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:02.865337    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kf7nd\" (UniqueName: \"kubernetes.io/projected/f59115d6-8481-469e-a651-21e7faa6c392-kube-api-access-kf7nd\") pod \"insights-runtime-extractor-6fjcf\" (UID: \"f59115d6-8481-469e-a651-21e7faa6c392\") " pod="openshift-insights/insights-runtime-extractor-6fjcf"
Jul 28 16:22:03.359705 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:03.359631    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f59115d6-8481-469e-a651-21e7faa6c392-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6fjcf\" (UID: \"f59115d6-8481-469e-a651-21e7faa6c392\") " pod="openshift-insights/insights-runtime-extractor-6fjcf"
Jul 28 16:22:03.359843 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:03.359725    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f59115d6-8481-469e-a651-21e7faa6c392-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6fjcf\" (UID: \"f59115d6-8481-469e-a651-21e7faa6c392\") " pod="openshift-insights/insights-runtime-extractor-6fjcf"
Jul 28 16:22:03.359843 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:22:03.359763    2557 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 28 16:22:03.359843 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:22:03.359820    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f59115d6-8481-469e-a651-21e7faa6c392-kube-rbac-proxy-cm podName:f59115d6-8481-469e-a651-21e7faa6c392 nodeName:}" failed. No retries permitted until 2026-07-28 16:22:04.359805935 +0000 UTC m=+41.498275331 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f59115d6-8481-469e-a651-21e7faa6c392-kube-rbac-proxy-cm") pod "insights-runtime-extractor-6fjcf" (UID: "f59115d6-8481-469e-a651-21e7faa6c392") : configmap "kube-rbac-proxy" not found
Jul 28 16:22:03.359964 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:22:03.359848    2557 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 16:22:03.359964 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:22:03.359898    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f59115d6-8481-469e-a651-21e7faa6c392-insights-runtime-extractor-tls podName:f59115d6-8481-469e-a651-21e7faa6c392 nodeName:}" failed. No retries permitted until 2026-07-28 16:22:04.359884099 +0000 UTC m=+41.498353497 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f59115d6-8481-469e-a651-21e7faa6c392-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6fjcf" (UID: "f59115d6-8481-469e-a651-21e7faa6c392") : secret "insights-runtime-extractor-tls" not found
Jul 28 16:22:03.603382 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:03.603331    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:22:03.631717 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:03.631649    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-rt2ch" event={"ID":"e958d4e8-df2b-4009-b83a-2625c110d4f9","Type":"ContainerStarted","Data":"69a3724af199d0950e2a7d0a19f9a71f78956ba84f55a6846ead5fc1f506b46f"}
Jul 28 16:22:03.633026 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:03.632996    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-vctpt" event={"ID":"9d3954a2-15d1-4000-9941-6555dc3aabd2","Type":"ContainerStarted","Data":"311e70562fbc859a24e39d30cb426bffaec3e3f41b478d1f4854aecdc07f9af4"}
Jul 28 16:22:03.646828 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:03.646730    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-rt2ch" podStartSLOduration=2.045038414 podStartE2EDuration="5.646718735s" totalImagesPullingTime="3.601680321s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 16:21:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:22:03.645816834 +0000 UTC m=+40.784286250" watchObservedRunningTime="2026-07-28 16:22:03.646718735 +0000 UTC m=+40.785188149"
Jul 28 16:22:03.659577 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:03.659536    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-vctpt" podStartSLOduration=1.111261655 podStartE2EDuration="3.659518678s" totalImagesPullingTime="2.548257023s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 16:22:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:22:03.65890681 +0000 UTC m=+40.797376224" watchObservedRunningTime="2026-07-28 16:22:03.659518678 +0000 UTC m=+40.797988091"
Jul 28 16:22:03.774290 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:03.774247    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:22:03.848093 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:03.848049    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:22:03.908851 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:03.908757    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:22:04.078449 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:04.078414    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:22:04.129305 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:04.129274    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:22:04.369368 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:04.369337    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f59115d6-8481-469e-a651-21e7faa6c392-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6fjcf\" (UID: \"f59115d6-8481-469e-a651-21e7faa6c392\") " pod="openshift-insights/insights-runtime-extractor-6fjcf"
Jul 28 16:22:04.369529 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:04.369417    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f59115d6-8481-469e-a651-21e7faa6c392-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6fjcf\" (UID: \"f59115d6-8481-469e-a651-21e7faa6c392\") " pod="openshift-insights/insights-runtime-extractor-6fjcf"
Jul 28 16:22:04.369529 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:22:04.369508    2557 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 16:22:04.369529 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:22:04.369512    2557 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 28 16:22:04.369670 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:22:04.369571    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f59115d6-8481-469e-a651-21e7faa6c392-insights-runtime-extractor-tls podName:f59115d6-8481-469e-a651-21e7faa6c392 nodeName:}" failed. No retries permitted until 2026-07-28 16:22:06.369556802 +0000 UTC m=+43.508026195 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f59115d6-8481-469e-a651-21e7faa6c392-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6fjcf" (UID: "f59115d6-8481-469e-a651-21e7faa6c392") : secret "insights-runtime-extractor-tls" not found
Jul 28 16:22:04.369670 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:22:04.369590    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f59115d6-8481-469e-a651-21e7faa6c392-kube-rbac-proxy-cm podName:f59115d6-8481-469e-a651-21e7faa6c392 nodeName:}" failed. No retries permitted until 2026-07-28 16:22:06.369582715 +0000 UTC m=+43.508052107 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f59115d6-8481-469e-a651-21e7faa6c392-kube-rbac-proxy-cm") pod "insights-runtime-extractor-6fjcf" (UID: "f59115d6-8481-469e-a651-21e7faa6c392") : configmap "kube-rbac-proxy" not found
Jul 28 16:22:04.516104 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:04.516070    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:22:04.697184 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:04.697094    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:22:04.711700 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:04.711676    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-rn2z4_b6c82e3b-ae45-41a0-b919-d4fbd8c3fc64/dns-node-resolver/0.log"
Jul 28 16:22:04.797148 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:04.797109    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:22:04.897699 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:04.897667    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:22:05.512403 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:05.512374    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-f8l6z_ff81c208-be53-4aff-9993-47b1965201b3/node-ca/0.log"
Jul 28 16:22:06.311740 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:06.311704    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-rt2ch_e958d4e8-df2b-4009-b83a-2625c110d4f9/serve-healthcheck-canary/0.log"
Jul 28 16:22:06.384510 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:06.384465    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f59115d6-8481-469e-a651-21e7faa6c392-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6fjcf\" (UID: \"f59115d6-8481-469e-a651-21e7faa6c392\") " pod="openshift-insights/insights-runtime-extractor-6fjcf"
Jul 28 16:22:06.384646 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:06.384584    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f59115d6-8481-469e-a651-21e7faa6c392-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6fjcf\" (UID: \"f59115d6-8481-469e-a651-21e7faa6c392\") " pod="openshift-insights/insights-runtime-extractor-6fjcf"
Jul 28 16:22:06.384646 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:22:06.384623    2557 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 16:22:06.384779 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:22:06.384683    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f59115d6-8481-469e-a651-21e7faa6c392-insights-runtime-extractor-tls podName:f59115d6-8481-469e-a651-21e7faa6c392 nodeName:}" failed. No retries permitted until 2026-07-28 16:22:10.384665088 +0000 UTC m=+47.523134511 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f59115d6-8481-469e-a651-21e7faa6c392-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6fjcf" (UID: "f59115d6-8481-469e-a651-21e7faa6c392") : secret "insights-runtime-extractor-tls" not found
Jul 28 16:22:06.384779 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:22:06.384728    2557 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 28 16:22:06.384875 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:22:06.384791    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f59115d6-8481-469e-a651-21e7faa6c392-kube-rbac-proxy-cm podName:f59115d6-8481-469e-a651-21e7faa6c392 nodeName:}" failed. No retries permitted until 2026-07-28 16:22:10.384773837 +0000 UTC m=+47.523243231 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f59115d6-8481-469e-a651-21e7faa6c392-kube-rbac-proxy-cm") pod "insights-runtime-extractor-6fjcf" (UID: "f59115d6-8481-469e-a651-21e7faa6c392") : configmap "kube-rbac-proxy" not found
Jul 28 16:22:06.385886 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:06.385866    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-lrpfl"]
Jul 28 16:22:06.418157 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:06.418129    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:22:06.420222 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:06.420204    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-lrpfl"]
Jul 28 16:22:06.420309 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:06.420297    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-658d7b7c78-lrpfl"
Jul 28 16:22:06.422961 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:06.422940    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Jul 28 16:22:06.423064 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:06.422962    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Jul 28 16:22:06.423064 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:06.422962    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-2ncl9\""
Jul 28 16:22:06.423886 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:06.423868    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Jul 28 16:22:06.424128 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:06.424112    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Jul 28 16:22:06.485333 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:06.485304    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7r984\" (UniqueName: \"kubernetes.io/projected/41d7581c-38b0-4508-aa40-062e122e1f6b-kube-api-access-7r984\") pod \"service-ca-658d7b7c78-lrpfl\" (UID: \"41d7581c-38b0-4508-aa40-062e122e1f6b\") " pod="openshift-service-ca/service-ca-658d7b7c78-lrpfl"
Jul 28 16:22:06.485436 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:06.485340    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9b7c5d20-5762-4a8d-9420-ca25360cd351-metrics-tls\") pod \"dns-default-wbnl5\" (UID: \"9b7c5d20-5762-4a8d-9420-ca25360cd351\") " pod="openshift-dns/dns-default-wbnl5"
Jul 28 16:22:06.485436 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:06.485405    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/41d7581c-38b0-4508-aa40-062e122e1f6b-signing-key\") pod \"service-ca-658d7b7c78-lrpfl\" (UID: \"41d7581c-38b0-4508-aa40-062e122e1f6b\") " pod="openshift-service-ca/service-ca-658d7b7c78-lrpfl"
Jul 28 16:22:06.485526 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:06.485433    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/41d7581c-38b0-4508-aa40-062e122e1f6b-signing-cabundle\") pod \"service-ca-658d7b7c78-lrpfl\" (UID: \"41d7581c-38b0-4508-aa40-062e122e1f6b\") " pod="openshift-service-ca/service-ca-658d7b7c78-lrpfl"
Jul 28 16:22:06.485526 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:22:06.485459    2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 16:22:06.485590 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:06.485537    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/915e2010-9827-4845-b395-a7ee1fedcfe7-registry-tls\") pod \"image-registry-6cf9d7745c-2f4rl\" (UID: \"915e2010-9827-4845-b395-a7ee1fedcfe7\") " pod="openshift-image-registry/image-registry-6cf9d7745c-2f4rl"
Jul 28 16:22:06.485590 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:22:06.485563    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9b7c5d20-5762-4a8d-9420-ca25360cd351-metrics-tls podName:9b7c5d20-5762-4a8d-9420-ca25360cd351 nodeName:}" failed. No retries permitted until 2026-07-28 16:22:14.485542926 +0000 UTC m=+51.624012319 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9b7c5d20-5762-4a8d-9420-ca25360cd351-metrics-tls") pod "dns-default-wbnl5" (UID: "9b7c5d20-5762-4a8d-9420-ca25360cd351") : secret "dns-default-metrics-tls" not found
Jul 28 16:22:06.485657 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:22:06.485620    2557 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 28 16:22:06.485657 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:22:06.485633    2557 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6cf9d7745c-2f4rl: secret "image-registry-tls" not found
Jul 28 16:22:06.485722 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:22:06.485678    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/915e2010-9827-4845-b395-a7ee1fedcfe7-registry-tls podName:915e2010-9827-4845-b395-a7ee1fedcfe7 nodeName:}" failed. No retries permitted until 2026-07-28 16:22:14.485664379 +0000 UTC m=+51.624133797 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/915e2010-9827-4845-b395-a7ee1fedcfe7-registry-tls") pod "image-registry-6cf9d7745c-2f4rl" (UID: "915e2010-9827-4845-b395-a7ee1fedcfe7") : secret "image-registry-tls" not found
Jul 28 16:22:06.518300 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:06.518276    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:22:06.586526 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:06.586431    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7r984\" (UniqueName: \"kubernetes.io/projected/41d7581c-38b0-4508-aa40-062e122e1f6b-kube-api-access-7r984\") pod \"service-ca-658d7b7c78-lrpfl\" (UID: \"41d7581c-38b0-4508-aa40-062e122e1f6b\") " pod="openshift-service-ca/service-ca-658d7b7c78-lrpfl"
Jul 28 16:22:06.586657 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:06.586534    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/41d7581c-38b0-4508-aa40-062e122e1f6b-signing-key\") pod \"service-ca-658d7b7c78-lrpfl\" (UID: \"41d7581c-38b0-4508-aa40-062e122e1f6b\") " pod="openshift-service-ca/service-ca-658d7b7c78-lrpfl"
Jul 28 16:22:06.586657 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:06.586562    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/41d7581c-38b0-4508-aa40-062e122e1f6b-signing-cabundle\") pod \"service-ca-658d7b7c78-lrpfl\" (UID: \"41d7581c-38b0-4508-aa40-062e122e1f6b\") " pod="openshift-service-ca/service-ca-658d7b7c78-lrpfl"
Jul 28 16:22:06.587331 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:06.587235    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/41d7581c-38b0-4508-aa40-062e122e1f6b-signing-cabundle\") pod \"service-ca-658d7b7c78-lrpfl\" (UID: \"41d7581c-38b0-4508-aa40-062e122e1f6b\") " pod="openshift-service-ca/service-ca-658d7b7c78-lrpfl"
Jul 28 16:22:06.589038 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:06.589018    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/41d7581c-38b0-4508-aa40-062e122e1f6b-signing-key\") pod \"service-ca-658d7b7c78-lrpfl\" (UID: \"41d7581c-38b0-4508-aa40-062e122e1f6b\") " pod="openshift-service-ca/service-ca-658d7b7c78-lrpfl"
Jul 28 16:22:06.594273 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:06.594247    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7r984\" (UniqueName: \"kubernetes.io/projected/41d7581c-38b0-4508-aa40-062e122e1f6b-kube-api-access-7r984\") pod \"service-ca-658d7b7c78-lrpfl\" (UID: \"41d7581c-38b0-4508-aa40-062e122e1f6b\") " pod="openshift-service-ca/service-ca-658d7b7c78-lrpfl"
Jul 28 16:22:06.729067 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:06.729034    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-658d7b7c78-lrpfl"
Jul 28 16:22:06.882458 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:06.882322    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-lrpfl"]
Jul 28 16:22:06.884917 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:22:06.884890    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41d7581c_38b0_4508_aa40_062e122e1f6b.slice/crio-0b2fae2f7fb868b9ce8f3701cee6fa91c60419027f5eab8036eea6d72539a1ac WatchSource:0}: Error finding container 0b2fae2f7fb868b9ce8f3701cee6fa91c60419027f5eab8036eea6d72539a1ac: Status 404 returned error can't find the container with id 0b2fae2f7fb868b9ce8f3701cee6fa91c60419027f5eab8036eea6d72539a1ac
Jul 28 16:22:07.364379 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:07.364345    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:22:07.644442 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:07.644369    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-658d7b7c78-lrpfl" event={"ID":"41d7581c-38b0-4508-aa40-062e122e1f6b","Type":"ContainerStarted","Data":"1794354523d0c81fa1d4a9ebb7ed7cd493a9210e70e3e650ac68ef43c356c034"}
Jul 28 16:22:07.644442 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:07.644419    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-658d7b7c78-lrpfl" event={"ID":"41d7581c-38b0-4508-aa40-062e122e1f6b","Type":"ContainerStarted","Data":"0b2fae2f7fb868b9ce8f3701cee6fa91c60419027f5eab8036eea6d72539a1ac"}
Jul 28 16:22:07.659676 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:07.659627    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-658d7b7c78-lrpfl" podStartSLOduration=1.659614254 podStartE2EDuration="1.659614254s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 16:22:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:22:07.658781193 +0000 UTC m=+44.797250618" watchObservedRunningTime="2026-07-28 16:22:07.659614254 +0000 UTC m=+44.798083667"
Jul 28 16:22:09.003289 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:09.003254    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:22:09.102719 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:09.102683    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:22:09.203597 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:09.203555    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:22:09.305107 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:09.305078    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:22:09.404022 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:09.403980    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:22:09.513898 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:09.513863    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:22:09.648472 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:09.648378    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:22:10.413678 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:10.413644    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f59115d6-8481-469e-a651-21e7faa6c392-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6fjcf\" (UID: \"f59115d6-8481-469e-a651-21e7faa6c392\") " pod="openshift-insights/insights-runtime-extractor-6fjcf"
Jul 28 16:22:10.414091 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:10.413721    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f59115d6-8481-469e-a651-21e7faa6c392-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6fjcf\" (UID: \"f59115d6-8481-469e-a651-21e7faa6c392\") " pod="openshift-insights/insights-runtime-extractor-6fjcf"
Jul 28 16:22:10.414091 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:22:10.413787    2557 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 28 16:22:10.414091 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:22:10.413860    2557 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 16:22:10.414091 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:22:10.413869    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f59115d6-8481-469e-a651-21e7faa6c392-kube-rbac-proxy-cm podName:f59115d6-8481-469e-a651-21e7faa6c392 nodeName:}" failed. No retries permitted until 2026-07-28 16:22:18.413847743 +0000 UTC m=+55.552317140 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f59115d6-8481-469e-a651-21e7faa6c392-kube-rbac-proxy-cm") pod "insights-runtime-extractor-6fjcf" (UID: "f59115d6-8481-469e-a651-21e7faa6c392") : configmap "kube-rbac-proxy" not found
Jul 28 16:22:10.414091 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:22:10.413913    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f59115d6-8481-469e-a651-21e7faa6c392-insights-runtime-extractor-tls podName:f59115d6-8481-469e-a651-21e7faa6c392 nodeName:}" failed. No retries permitted until 2026-07-28 16:22:18.413896897 +0000 UTC m=+55.552366306 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f59115d6-8481-469e-a651-21e7faa6c392-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6fjcf" (UID: "f59115d6-8481-469e-a651-21e7faa6c392") : secret "insights-runtime-extractor-tls" not found
Jul 28 16:22:10.434780 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:10.434747    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:22:10.633141 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:10.633105    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:22:11.041251 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:11.041219    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:22:11.731019 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:11.730984    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:22:11.834690 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:11.834655    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:22:12.237928 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:12.237873    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:22:12.434757 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:12.434726    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:22:12.636137 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:12.636099    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:22:12.694835 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:12.694795    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:22:12.865002 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:12.864966    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:22:13.357428 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:13.357394    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:22:14.035345 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:14.035305    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:22:14.233334 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:14.233295    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:22:14.288413 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:14.288338    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:22:14.482350 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:14.482317    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:22:14.547464 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:14.547398    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9b7c5d20-5762-4a8d-9420-ca25360cd351-metrics-tls\") pod \"dns-default-wbnl5\" (UID: \"9b7c5d20-5762-4a8d-9420-ca25360cd351\") " pod="openshift-dns/dns-default-wbnl5"
Jul 28 16:22:14.547621 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:14.547463    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/915e2010-9827-4845-b395-a7ee1fedcfe7-registry-tls\") pod \"image-registry-6cf9d7745c-2f4rl\" (UID: \"915e2010-9827-4845-b395-a7ee1fedcfe7\") " pod="openshift-image-registry/image-registry-6cf9d7745c-2f4rl"
Jul 28 16:22:14.549887 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:14.549869    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9b7c5d20-5762-4a8d-9420-ca25360cd351-metrics-tls\") pod \"dns-default-wbnl5\" (UID: \"9b7c5d20-5762-4a8d-9420-ca25360cd351\") " pod="openshift-dns/dns-default-wbnl5"
Jul 28 16:22:14.549976 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:14.549952    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/915e2010-9827-4845-b395-a7ee1fedcfe7-registry-tls\") pod \"image-registry-6cf9d7745c-2f4rl\" (UID: \"915e2010-9827-4845-b395-a7ee1fedcfe7\") " pod="openshift-image-registry/image-registry-6cf9d7745c-2f4rl"
Jul 28 16:22:14.611779 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:14.611751    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6cf9d7745c-2f4rl"
Jul 28 16:22:14.621858 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:14.621833    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-wbnl5"
Jul 28 16:22:14.630268 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:14.630227    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:22:14.778090 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:14.778063    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6cf9d7745c-2f4rl"]
Jul 28 16:22:14.782287 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:22:14.782264    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod915e2010_9827_4845_b395_a7ee1fedcfe7.slice/crio-fe2224a5e380cd33c6d2a26afc6d7bbdfa97ae9de9af334678b46b747abbf9b7 WatchSource:0}: Error finding container fe2224a5e380cd33c6d2a26afc6d7bbdfa97ae9de9af334678b46b747abbf9b7: Status 404 returned error can't find the container with id fe2224a5e380cd33c6d2a26afc6d7bbdfa97ae9de9af334678b46b747abbf9b7
Jul 28 16:22:14.790564 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:14.790537    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-wbnl5"]
Jul 28 16:22:14.792354 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:22:14.792321    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b7c5d20_5762_4a8d_9420_ca25360cd351.slice/crio-8d6f57c94552aa8f3db129f2645b5bb81758c50a9d2fc17807ca0b18fcc1b847 WatchSource:0}: Error finding container 8d6f57c94552aa8f3db129f2645b5bb81758c50a9d2fc17807ca0b18fcc1b847: Status 404 returned error can't find the container with id 8d6f57c94552aa8f3db129f2645b5bb81758c50a9d2fc17807ca0b18fcc1b847
Jul 28 16:22:14.830997 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:14.830955    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:22:15.665783 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:15.665751    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-wbnl5" event={"ID":"9b7c5d20-5762-4a8d-9420-ca25360cd351","Type":"ContainerStarted","Data":"8d6f57c94552aa8f3db129f2645b5bb81758c50a9d2fc17807ca0b18fcc1b847"}
Jul 28 16:22:15.667454 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:15.667419    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6cf9d7745c-2f4rl" event={"ID":"915e2010-9827-4845-b395-a7ee1fedcfe7","Type":"ContainerStarted","Data":"9d12be9b0cbc42c7312b0a10a6fff9043dec7bd209fe0f2d38a8b10efd218a6e"}
Jul 28 16:22:15.667593 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:15.667470    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6cf9d7745c-2f4rl" event={"ID":"915e2010-9827-4845-b395-a7ee1fedcfe7","Type":"ContainerStarted","Data":"fe2224a5e380cd33c6d2a26afc6d7bbdfa97ae9de9af334678b46b747abbf9b7"}
Jul 28 16:22:15.667593 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:15.667580    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6cf9d7745c-2f4rl"
Jul 28 16:22:15.688428 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:15.688384    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6cf9d7745c-2f4rl" podStartSLOduration=17.688369945 podStartE2EDuration="17.688369945s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 16:21:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:22:15.687101808 +0000 UTC m=+52.825571222" watchObservedRunningTime="2026-07-28 16:22:15.688369945 +0000 UTC m=+52.826839359"
Jul 28 16:22:15.691941 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:15.691915    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:22:15.832315 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:15.832277    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:22:16.032931 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:16.032840    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:22:16.522796 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:16.522765    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:22:16.588281 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:16.588250    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:22:16.651731 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:16.651701    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:22:17.035435 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:17.035406    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:22:17.635802 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:17.635772    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:22:17.674162 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:17.674131    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-wbnl5" event={"ID":"9b7c5d20-5762-4a8d-9420-ca25360cd351","Type":"ContainerStarted","Data":"db7e2e428769b12caa4f9f57651e7050717599a91736d58c5ec87e0e4dd4b578"}
Jul 28 16:22:17.674307 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:17.674180    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-wbnl5" event={"ID":"9b7c5d20-5762-4a8d-9420-ca25360cd351","Type":"ContainerStarted","Data":"a8b1fc6d6a890d6b16fb82a22590e1b60e6fea0723aa67a061e9d3b91ee373d0"}
Jul 28 16:22:17.674307 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:17.674263    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-wbnl5"
Jul 28 16:22:17.691131 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:17.691088    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-wbnl5" podStartSLOduration=17.516424381 podStartE2EDuration="19.691074591s" totalImagesPullingTime="2.17465021s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 16:21:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:22:17.690393904 +0000 UTC m=+54.828863320" watchObservedRunningTime="2026-07-28 16:22:17.691074591 +0000 UTC m=+54.829544002"
Jul 28 16:22:18.036051 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:18.035977    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:22:18.235227 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:18.235200    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:22:18.481890 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:18.481863    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f59115d6-8481-469e-a651-21e7faa6c392-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6fjcf\" (UID: \"f59115d6-8481-469e-a651-21e7faa6c392\") " pod="openshift-insights/insights-runtime-extractor-6fjcf"
Jul 28 16:22:18.482041 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:18.481918    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f59115d6-8481-469e-a651-21e7faa6c392-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6fjcf\" (UID: \"f59115d6-8481-469e-a651-21e7faa6c392\") " pod="openshift-insights/insights-runtime-extractor-6fjcf"
Jul 28 16:22:18.482082 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:22:18.482054    2557 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 16:22:18.482126 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:22:18.482120    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f59115d6-8481-469e-a651-21e7faa6c392-insights-runtime-extractor-tls podName:f59115d6-8481-469e-a651-21e7faa6c392 nodeName:}" failed. No retries permitted until 2026-07-28 16:22:34.482100416 +0000 UTC m=+71.620569810 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f59115d6-8481-469e-a651-21e7faa6c392-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6fjcf" (UID: "f59115d6-8481-469e-a651-21e7faa6c392") : secret "insights-runtime-extractor-tls" not found
Jul 28 16:22:18.482512 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:18.482473    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f59115d6-8481-469e-a651-21e7faa6c392-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6fjcf\" (UID: \"f59115d6-8481-469e-a651-21e7faa6c392\") " pod="openshift-insights/insights-runtime-extractor-6fjcf"
Jul 28 16:22:18.674841 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:18.674804    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:22:18.894144 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:18.894110    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:22:19.236180 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:19.236070    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:22:19.473940 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:19.473904    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:22:19.638521 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:19.638466    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:22:19.821005 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:19.820961    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:22:19.878110 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:19.878077    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:22:20.404910 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:20.404872    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:22:23.608683 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:23.608655    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-sbf8h"
Jul 28 16:22:27.680038 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:27.680009    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-wbnl5"
Jul 28 16:22:29.061996 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:29.061943    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/31992b1b-4643-467f-a2b1-674af56e8c0e-original-pull-secret\") pod \"global-pull-secret-syncer-9vtwz\" (UID: \"31992b1b-4643-467f-a2b1-674af56e8c0e\") " pod="kube-system/global-pull-secret-syncer-9vtwz"
Jul 28 16:22:29.062455 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:29.062026    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/92ef4728-9e89-4494-8bdf-72dcfe9b3ff4-metrics-certs\") pod \"network-metrics-daemon-qm9v7\" (UID: \"92ef4728-9e89-4494-8bdf-72dcfe9b3ff4\") " pod="openshift-multus/network-metrics-daemon-qm9v7"
Jul 28 16:22:29.064585 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:29.064568    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 28 16:22:29.065402 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:29.065388    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 28 16:22:29.075344 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:29.075313    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/92ef4728-9e89-4494-8bdf-72dcfe9b3ff4-metrics-certs\") pod \"network-metrics-daemon-qm9v7\" (UID: \"92ef4728-9e89-4494-8bdf-72dcfe9b3ff4\") " pod="openshift-multus/network-metrics-daemon-qm9v7"
Jul 28 16:22:29.075458 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:29.075369    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/31992b1b-4643-467f-a2b1-674af56e8c0e-original-pull-secret\") pod \"global-pull-secret-syncer-9vtwz\" (UID: \"31992b1b-4643-467f-a2b1-674af56e8c0e\") " pod="kube-system/global-pull-secret-syncer-9vtwz"
Jul 28 16:22:29.143673 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:29.143642    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9vtwz"
Jul 28 16:22:29.151265 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:29.151242    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-nr6mp\""
Jul 28 16:22:29.159193 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:29.159172    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qm9v7"
Jul 28 16:22:29.163165 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:29.163147    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xl2sp\" (UniqueName: \"kubernetes.io/projected/c687045e-f5d0-4a22-b71e-9c01463677e6-kube-api-access-xl2sp\") pod \"network-check-target-qzkm5\" (UID: \"c687045e-f5d0-4a22-b71e-9c01463677e6\") " pod="openshift-network-diagnostics/network-check-target-qzkm5"
Jul 28 16:22:29.165459 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:29.165441    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 28 16:22:29.175891 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:29.175858    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 28 16:22:29.186366 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:29.186340    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xl2sp\" (UniqueName: \"kubernetes.io/projected/c687045e-f5d0-4a22-b71e-9c01463677e6-kube-api-access-xl2sp\") pod \"network-check-target-qzkm5\" (UID: \"c687045e-f5d0-4a22-b71e-9c01463677e6\") " pod="openshift-network-diagnostics/network-check-target-qzkm5"
Jul 28 16:22:29.275898 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:29.275860    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-9vtwz"]
Jul 28 16:22:29.278791 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:22:29.278767    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31992b1b_4643_467f_a2b1_674af56e8c0e.slice/crio-f4ee1e103e83542cd418a79cfb847341540bb70d337786b69903eb335bce61ef WatchSource:0}: Error finding container f4ee1e103e83542cd418a79cfb847341540bb70d337786b69903eb335bce61ef: Status 404 returned error can't find the container with id f4ee1e103e83542cd418a79cfb847341540bb70d337786b69903eb335bce61ef
Jul 28 16:22:29.292041 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:29.292013    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-qm9v7"]
Jul 28 16:22:29.294658 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:22:29.294618    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92ef4728_9e89_4494_8bdf_72dcfe9b3ff4.slice/crio-609d4f76f9ad3b6bfdb45b1831c96c79b0e4af43f02c6065a797e1d27f006b74 WatchSource:0}: Error finding container 609d4f76f9ad3b6bfdb45b1831c96c79b0e4af43f02c6065a797e1d27f006b74: Status 404 returned error can't find the container with id 609d4f76f9ad3b6bfdb45b1831c96c79b0e4af43f02c6065a797e1d27f006b74
Jul 28 16:22:29.441186 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:29.441108    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-gnm96\""
Jul 28 16:22:29.449355 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:29.449337    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qzkm5"
Jul 28 16:22:29.568813 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:29.568691    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-qzkm5"]
Jul 28 16:22:29.571105 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:22:29.571082    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc687045e_f5d0_4a22_b71e_9c01463677e6.slice/crio-e0e0dab406645a11bd6291358695682fd5f152d426913b71559abd2d49f67344 WatchSource:0}: Error finding container e0e0dab406645a11bd6291358695682fd5f152d426913b71559abd2d49f67344: Status 404 returned error can't find the container with id e0e0dab406645a11bd6291358695682fd5f152d426913b71559abd2d49f67344
Jul 28 16:22:29.685013 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:29.684957    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5f5d967454-44c94"]
Jul 28 16:22:29.686927 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:29.686906    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5f5d967454-44c94"
Jul 28 16:22:29.694137 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:29.693000    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 28 16:22:29.694137 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:29.693003    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 28 16:22:29.694137 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:29.693275    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 28 16:22:29.694137 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:29.693274    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Jul 28 16:22:29.701676 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:29.701653    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5f5d967454-44c94"]
Jul 28 16:22:29.709468 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:29.709443    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6cf9d7745c-2f4rl"]
Jul 28 16:22:29.715164 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:29.715117    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-qzkm5" event={"ID":"c687045e-f5d0-4a22-b71e-9c01463677e6","Type":"ContainerStarted","Data":"e0e0dab406645a11bd6291358695682fd5f152d426913b71559abd2d49f67344"}
Jul 28 16:22:29.715908 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:29.715880    2557 patch_prober.go:28] interesting pod/image-registry-6cf9d7745c-2f4rl container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 28 16:22:29.716022 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:29.715932    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-6cf9d7745c-2f4rl" podUID="915e2010-9827-4845-b395-a7ee1fedcfe7" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 16:22:29.717019 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:29.716974    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-qm9v7" event={"ID":"92ef4728-9e89-4494-8bdf-72dcfe9b3ff4","Type":"ContainerStarted","Data":"609d4f76f9ad3b6bfdb45b1831c96c79b0e4af43f02c6065a797e1d27f006b74"}
Jul 28 16:22:29.718571 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:29.718544    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-9vtwz" event={"ID":"31992b1b-4643-467f-a2b1-674af56e8c0e","Type":"ContainerStarted","Data":"f4ee1e103e83542cd418a79cfb847341540bb70d337786b69903eb335bce61ef"}
Jul 28 16:22:29.768150 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:29.768109    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9ff2b357-1fdf-4265-96fd-ee69cf6554a4-tmp\") pod \"klusterlet-addon-workmgr-5f5d967454-44c94\" (UID: \"9ff2b357-1fdf-4265-96fd-ee69cf6554a4\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5f5d967454-44c94"
Jul 28 16:22:29.768322 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:29.768238    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tzx8g\" (UniqueName: \"kubernetes.io/projected/9ff2b357-1fdf-4265-96fd-ee69cf6554a4-kube-api-access-tzx8g\") pod \"klusterlet-addon-workmgr-5f5d967454-44c94\" (UID: \"9ff2b357-1fdf-4265-96fd-ee69cf6554a4\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5f5d967454-44c94"
Jul 28 16:22:29.768382 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:29.768337    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/9ff2b357-1fdf-4265-96fd-ee69cf6554a4-klusterlet-config\") pod \"klusterlet-addon-workmgr-5f5d967454-44c94\" (UID: \"9ff2b357-1fdf-4265-96fd-ee69cf6554a4\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5f5d967454-44c94"
Jul 28 16:22:29.783641 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:29.783596    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-77fbd874ff-997dw"]
Jul 28 16:22:29.786223 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:29.785924    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-77fbd874ff-997dw"
Jul 28 16:22:29.787223 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:29.787181    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:22:29.800914 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:29.800887    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-77fbd874ff-997dw"]
Jul 28 16:22:29.870681 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:29.869322    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/63f9d5f1-d1c2-4932-9cbd-46aa1d644e69-registry-certificates\") pod \"image-registry-77fbd874ff-997dw\" (UID: \"63f9d5f1-d1c2-4932-9cbd-46aa1d644e69\") " pod="openshift-image-registry/image-registry-77fbd874ff-997dw"
Jul 28 16:22:29.870681 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:29.869373    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9ff2b357-1fdf-4265-96fd-ee69cf6554a4-tmp\") pod \"klusterlet-addon-workmgr-5f5d967454-44c94\" (UID: \"9ff2b357-1fdf-4265-96fd-ee69cf6554a4\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5f5d967454-44c94"
Jul 28 16:22:29.870681 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:29.869408    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tzx8g\" (UniqueName: \"kubernetes.io/projected/9ff2b357-1fdf-4265-96fd-ee69cf6554a4-kube-api-access-tzx8g\") pod \"klusterlet-addon-workmgr-5f5d967454-44c94\" (UID: \"9ff2b357-1fdf-4265-96fd-ee69cf6554a4\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5f5d967454-44c94"
Jul 28 16:22:29.870681 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:29.869433    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/63f9d5f1-d1c2-4932-9cbd-46aa1d644e69-ca-trust-extracted\") pod \"image-registry-77fbd874ff-997dw\" (UID: \"63f9d5f1-d1c2-4932-9cbd-46aa1d644e69\") " pod="openshift-image-registry/image-registry-77fbd874ff-997dw"
Jul 28 16:22:29.870681 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:29.869489    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/63f9d5f1-d1c2-4932-9cbd-46aa1d644e69-image-registry-private-configuration\") pod \"image-registry-77fbd874ff-997dw\" (UID: \"63f9d5f1-d1c2-4932-9cbd-46aa1d644e69\") " pod="openshift-image-registry/image-registry-77fbd874ff-997dw"
Jul 28 16:22:29.870681 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:29.869539    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/63f9d5f1-d1c2-4932-9cbd-46aa1d644e69-trusted-ca\") pod \"image-registry-77fbd874ff-997dw\" (UID: \"63f9d5f1-d1c2-4932-9cbd-46aa1d644e69\") " pod="openshift-image-registry/image-registry-77fbd874ff-997dw"
Jul 28 16:22:29.870681 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:29.869591    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/9ff2b357-1fdf-4265-96fd-ee69cf6554a4-klusterlet-config\") pod \"klusterlet-addon-workmgr-5f5d967454-44c94\" (UID: \"9ff2b357-1fdf-4265-96fd-ee69cf6554a4\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5f5d967454-44c94"
Jul 28 16:22:29.870681 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:29.869623    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/63f9d5f1-d1c2-4932-9cbd-46aa1d644e69-installation-pull-secrets\") pod \"image-registry-77fbd874ff-997dw\" (UID: \"63f9d5f1-d1c2-4932-9cbd-46aa1d644e69\") " pod="openshift-image-registry/image-registry-77fbd874ff-997dw"
Jul 28 16:22:29.870681 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:29.869654    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/63f9d5f1-d1c2-4932-9cbd-46aa1d644e69-registry-tls\") pod \"image-registry-77fbd874ff-997dw\" (UID: \"63f9d5f1-d1c2-4932-9cbd-46aa1d644e69\") " pod="openshift-image-registry/image-registry-77fbd874ff-997dw"
Jul 28 16:22:29.870681 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:29.869667    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/63f9d5f1-d1c2-4932-9cbd-46aa1d644e69-bound-sa-token\") pod \"image-registry-77fbd874ff-997dw\" (UID: \"63f9d5f1-d1c2-4932-9cbd-46aa1d644e69\") " pod="openshift-image-registry/image-registry-77fbd874ff-997dw"
Jul 28 16:22:29.870681 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:29.869682    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8qw6p\" (UniqueName: \"kubernetes.io/projected/63f9d5f1-d1c2-4932-9cbd-46aa1d644e69-kube-api-access-8qw6p\") pod \"image-registry-77fbd874ff-997dw\" (UID: \"63f9d5f1-d1c2-4932-9cbd-46aa1d644e69\") " pod="openshift-image-registry/image-registry-77fbd874ff-997dw"
Jul 28 16:22:29.870681 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:29.870039    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9ff2b357-1fdf-4265-96fd-ee69cf6554a4-tmp\") pod \"klusterlet-addon-workmgr-5f5d967454-44c94\" (UID: \"9ff2b357-1fdf-4265-96fd-ee69cf6554a4\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5f5d967454-44c94"
Jul 28 16:22:29.873371 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:29.873345    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/9ff2b357-1fdf-4265-96fd-ee69cf6554a4-klusterlet-config\") pod \"klusterlet-addon-workmgr-5f5d967454-44c94\" (UID: \"9ff2b357-1fdf-4265-96fd-ee69cf6554a4\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5f5d967454-44c94"
Jul 28 16:22:29.878837 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:29.878817    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tzx8g\" (UniqueName: \"kubernetes.io/projected/9ff2b357-1fdf-4265-96fd-ee69cf6554a4-kube-api-access-tzx8g\") pod \"klusterlet-addon-workmgr-5f5d967454-44c94\" (UID: \"9ff2b357-1fdf-4265-96fd-ee69cf6554a4\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5f5d967454-44c94"
Jul 28 16:22:29.879099 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:29.879080    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:22:29.970338 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:29.970251    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/63f9d5f1-d1c2-4932-9cbd-46aa1d644e69-ca-trust-extracted\") pod \"image-registry-77fbd874ff-997dw\" (UID: \"63f9d5f1-d1c2-4932-9cbd-46aa1d644e69\") " pod="openshift-image-registry/image-registry-77fbd874ff-997dw"
Jul 28 16:22:29.970338 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:29.970325    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/63f9d5f1-d1c2-4932-9cbd-46aa1d644e69-image-registry-private-configuration\") pod \"image-registry-77fbd874ff-997dw\" (UID: \"63f9d5f1-d1c2-4932-9cbd-46aa1d644e69\") " pod="openshift-image-registry/image-registry-77fbd874ff-997dw"
Jul 28 16:22:29.970589 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:29.970352    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/63f9d5f1-d1c2-4932-9cbd-46aa1d644e69-trusted-ca\") pod \"image-registry-77fbd874ff-997dw\" (UID: \"63f9d5f1-d1c2-4932-9cbd-46aa1d644e69\") " pod="openshift-image-registry/image-registry-77fbd874ff-997dw"
Jul 28 16:22:29.970589 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:29.970390    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/63f9d5f1-d1c2-4932-9cbd-46aa1d644e69-installation-pull-secrets\") pod \"image-registry-77fbd874ff-997dw\" (UID: \"63f9d5f1-d1c2-4932-9cbd-46aa1d644e69\") " pod="openshift-image-registry/image-registry-77fbd874ff-997dw"
Jul 28 16:22:29.970589 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:29.970427    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/63f9d5f1-d1c2-4932-9cbd-46aa1d644e69-registry-tls\") pod \"image-registry-77fbd874ff-997dw\" (UID: \"63f9d5f1-d1c2-4932-9cbd-46aa1d644e69\") " pod="openshift-image-registry/image-registry-77fbd874ff-997dw"
Jul 28 16:22:29.970589 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:29.970448    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/63f9d5f1-d1c2-4932-9cbd-46aa1d644e69-bound-sa-token\") pod \"image-registry-77fbd874ff-997dw\" (UID: \"63f9d5f1-d1c2-4932-9cbd-46aa1d644e69\") " pod="openshift-image-registry/image-registry-77fbd874ff-997dw"
Jul 28 16:22:29.970589 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:29.970469    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8qw6p\" (UniqueName: \"kubernetes.io/projected/63f9d5f1-d1c2-4932-9cbd-46aa1d644e69-kube-api-access-8qw6p\") pod \"image-registry-77fbd874ff-997dw\" (UID: \"63f9d5f1-d1c2-4932-9cbd-46aa1d644e69\") " pod="openshift-image-registry/image-registry-77fbd874ff-997dw"
Jul 28 16:22:29.970589 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:29.970529    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/63f9d5f1-d1c2-4932-9cbd-46aa1d644e69-registry-certificates\") pod \"image-registry-77fbd874ff-997dw\" (UID: \"63f9d5f1-d1c2-4932-9cbd-46aa1d644e69\") " pod="openshift-image-registry/image-registry-77fbd874ff-997dw"
Jul 28 16:22:29.970886 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:29.970679    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/63f9d5f1-d1c2-4932-9cbd-46aa1d644e69-ca-trust-extracted\") pod \"image-registry-77fbd874ff-997dw\" (UID: \"63f9d5f1-d1c2-4932-9cbd-46aa1d644e69\") " pod="openshift-image-registry/image-registry-77fbd874ff-997dw"
Jul 28 16:22:29.971431 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:29.971398    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/63f9d5f1-d1c2-4932-9cbd-46aa1d644e69-registry-certificates\") pod \"image-registry-77fbd874ff-997dw\" (UID: \"63f9d5f1-d1c2-4932-9cbd-46aa1d644e69\") " pod="openshift-image-registry/image-registry-77fbd874ff-997dw"
Jul 28 16:22:29.971633 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:29.971611    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/63f9d5f1-d1c2-4932-9cbd-46aa1d644e69-trusted-ca\") pod \"image-registry-77fbd874ff-997dw\" (UID: \"63f9d5f1-d1c2-4932-9cbd-46aa1d644e69\") " pod="openshift-image-registry/image-registry-77fbd874ff-997dw"
Jul 28 16:22:29.974148 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:29.974104    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/63f9d5f1-d1c2-4932-9cbd-46aa1d644e69-installation-pull-secrets\") pod \"image-registry-77fbd874ff-997dw\" (UID: \"63f9d5f1-d1c2-4932-9cbd-46aa1d644e69\") " pod="openshift-image-registry/image-registry-77fbd874ff-997dw"
Jul 28 16:22:29.974391 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:29.974373    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/63f9d5f1-d1c2-4932-9cbd-46aa1d644e69-image-registry-private-configuration\") pod \"image-registry-77fbd874ff-997dw\" (UID: \"63f9d5f1-d1c2-4932-9cbd-46aa1d644e69\") " pod="openshift-image-registry/image-registry-77fbd874ff-997dw"
Jul 28 16:22:29.975960 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:29.975920    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/63f9d5f1-d1c2-4932-9cbd-46aa1d644e69-registry-tls\") pod \"image-registry-77fbd874ff-997dw\" (UID: \"63f9d5f1-d1c2-4932-9cbd-46aa1d644e69\") " pod="openshift-image-registry/image-registry-77fbd874ff-997dw"
Jul 28 16:22:29.979291 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:29.979263    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/63f9d5f1-d1c2-4932-9cbd-46aa1d644e69-bound-sa-token\") pod \"image-registry-77fbd874ff-997dw\" (UID: \"63f9d5f1-d1c2-4932-9cbd-46aa1d644e69\") " pod="openshift-image-registry/image-registry-77fbd874ff-997dw"
Jul 28 16:22:29.979727 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:29.979713    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8qw6p\" (UniqueName: \"kubernetes.io/projected/63f9d5f1-d1c2-4932-9cbd-46aa1d644e69-kube-api-access-8qw6p\") pod \"image-registry-77fbd874ff-997dw\" (UID: \"63f9d5f1-d1c2-4932-9cbd-46aa1d644e69\") " pod="openshift-image-registry/image-registry-77fbd874ff-997dw"
Jul 28 16:22:30.000606 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:30.000577    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5f5d967454-44c94"
Jul 28 16:22:30.098137 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:30.098102    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-77fbd874ff-997dw"
Jul 28 16:22:30.422638 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:30.422259    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-77fbd874ff-997dw"]
Jul 28 16:22:30.439781 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:30.439742    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5f5d967454-44c94"]
Jul 28 16:22:30.526266 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:22:30.526229    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63f9d5f1_d1c2_4932_9cbd_46aa1d644e69.slice/crio-65a493f643769babe145a2f66afea7c620e7195160a5172bb206631349d2a05d WatchSource:0}: Error finding container 65a493f643769babe145a2f66afea7c620e7195160a5172bb206631349d2a05d: Status 404 returned error can't find the container with id 65a493f643769babe145a2f66afea7c620e7195160a5172bb206631349d2a05d
Jul 28 16:22:30.528733 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:22:30.528704    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ff2b357_1fdf_4265_96fd_ee69cf6554a4.slice/crio-39092eb7a723fe91a98d12b36c8215beefae7a8e9fff293b32f31785273cdc5a WatchSource:0}: Error finding container 39092eb7a723fe91a98d12b36c8215beefae7a8e9fff293b32f31785273cdc5a: Status 404 returned error can't find the container with id 39092eb7a723fe91a98d12b36c8215beefae7a8e9fff293b32f31785273cdc5a
Jul 28 16:22:30.727337 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:30.726332    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-77fbd874ff-997dw" event={"ID":"63f9d5f1-d1c2-4932-9cbd-46aa1d644e69","Type":"ContainerStarted","Data":"ebde911d2fb83e63aa9f7b2613f1c12f86a0e433c3f22ac7fe8d458ea498221d"}
Jul 28 16:22:30.727337 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:30.726389    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-77fbd874ff-997dw" event={"ID":"63f9d5f1-d1c2-4932-9cbd-46aa1d644e69","Type":"ContainerStarted","Data":"65a493f643769babe145a2f66afea7c620e7195160a5172bb206631349d2a05d"}
Jul 28 16:22:30.727337 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:30.727045    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-77fbd874ff-997dw"
Jul 28 16:22:30.730939 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:30.730352    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5f5d967454-44c94" event={"ID":"9ff2b357-1fdf-4265-96fd-ee69cf6554a4","Type":"ContainerStarted","Data":"39092eb7a723fe91a98d12b36c8215beefae7a8e9fff293b32f31785273cdc5a"}
Jul 28 16:22:30.732610 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:30.732566    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-qm9v7" event={"ID":"92ef4728-9e89-4494-8bdf-72dcfe9b3ff4","Type":"ContainerStarted","Data":"4b29e4d8aeeea54d0b1af89fb134c28209750f4deedc4a1c4ce874f05bbcba25"}
Jul 28 16:22:30.747343 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:30.745958    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-77fbd874ff-997dw" podStartSLOduration=1.745942836 podStartE2EDuration="1.745942836s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 16:22:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:22:30.745092664 +0000 UTC m=+67.883562092" watchObservedRunningTime="2026-07-28 16:22:30.745942836 +0000 UTC m=+67.884412227"
Jul 28 16:22:31.741024 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:31.740935    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-qm9v7" event={"ID":"92ef4728-9e89-4494-8bdf-72dcfe9b3ff4","Type":"ContainerStarted","Data":"9c9ff9b7a17b00441ab8cff3e784ca60e4eb947e1d369b5201b7cd163bb31c65"}
Jul 28 16:22:31.762511 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:31.762448    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-qm9v7" podStartSLOduration=67.501481794 podStartE2EDuration="1m8.762429192s" totalImagesPullingTime="1.260947398s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 16:21:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:22:31.760946687 +0000 UTC m=+68.899416102" watchObservedRunningTime="2026-07-28 16:22:31.762429192 +0000 UTC m=+68.900898612"
Jul 28 16:22:31.854686 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:31.854646    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:22:31.913628 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:31.913561    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:22:32.317249 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:32.317209    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:22:32.505645 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:32.505509    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:22:32.605914 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:32.605778    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:22:32.683746 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:32.683711    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:22:32.765034 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:32.765001    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:22:33.751448 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:33.751412    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:22:34.514194 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:34.514151    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f59115d6-8481-469e-a651-21e7faa6c392-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6fjcf\" (UID: \"f59115d6-8481-469e-a651-21e7faa6c392\") " pod="openshift-insights/insights-runtime-extractor-6fjcf"
Jul 28 16:22:34.516815 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:34.516784    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f59115d6-8481-469e-a651-21e7faa6c392-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6fjcf\" (UID: \"f59115d6-8481-469e-a651-21e7faa6c392\") " pod="openshift-insights/insights-runtime-extractor-6fjcf"
Jul 28 16:22:34.682661 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:34.682627    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-lx2xj\""
Jul 28 16:22:34.691284 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:34.691252    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-6fjcf"
Jul 28 16:22:36.395174 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:36.395089    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:22:36.448852 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:36.448774    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:22:36.984719 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:36.984688    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-6fjcf"]
Jul 28 16:22:36.988008 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:22:36.987926    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf59115d6_8481_469e_a651_21e7faa6c392.slice/crio-0a84c5838b109f4abb2862f77fda8124b1ea4e9e2e129038562203ef6f0e2e11 WatchSource:0}: Error finding container 0a84c5838b109f4abb2862f77fda8124b1ea4e9e2e129038562203ef6f0e2e11: Status 404 returned error can't find the container with id 0a84c5838b109f4abb2862f77fda8124b1ea4e9e2e129038562203ef6f0e2e11
Jul 28 16:22:37.706620 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:37.706525    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:22:37.764465 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:37.764423    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-9vtwz" event={"ID":"31992b1b-4643-467f-a2b1-674af56e8c0e","Type":"ContainerStarted","Data":"cb16cb91b3c7971a1f4026a9287ce629434e26d6fb645737833fae4d8b8a428f"}
Jul 28 16:22:37.766874 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:37.766843    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:22:37.767265 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:37.767242    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5f5d967454-44c94" event={"ID":"9ff2b357-1fdf-4265-96fd-ee69cf6554a4","Type":"ContainerStarted","Data":"f1c4cf4dcfb328172a7bd1d978a254ed50bb3885e59c77d33a821bae02b5c24e"}
Jul 28 16:22:37.767905 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:37.767840    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5f5d967454-44c94"
Jul 28 16:22:37.769609 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:37.769585    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5f5d967454-44c94"
Jul 28 16:22:37.770427 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:37.770402    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6fjcf" event={"ID":"f59115d6-8481-469e-a651-21e7faa6c392","Type":"ContainerStarted","Data":"0a84c5838b109f4abb2862f77fda8124b1ea4e9e2e129038562203ef6f0e2e11"}
Jul 28 16:22:37.774731 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:37.774710    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-qzkm5" event={"ID":"c687045e-f5d0-4a22-b71e-9c01463677e6","Type":"ContainerStarted","Data":"574f3014bdcaed2673d6fd8a799cdc598ed961f6ba6876dfad2edf427508d21e"}
Jul 28 16:22:37.774878 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:37.774862    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-qzkm5"
Jul 28 16:22:37.780917 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:37.780874    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-9vtwz" podStartSLOduration=67.218207995 podStartE2EDuration="1m14.780856935s" totalImagesPullingTime="7.56264894s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 16:21:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:22:37.780007956 +0000 UTC m=+74.918477371" watchObservedRunningTime="2026-07-28 16:22:37.780856935 +0000 UTC m=+74.919326352"
Jul 28 16:22:37.795368 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:37.795303    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5f5d967454-44c94" podStartSLOduration=2.455240665 podStartE2EDuration="8.795287855s" totalImagesPullingTime="6.34004719s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 16:22:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:22:37.794412613 +0000 UTC m=+74.932882029" watchObservedRunningTime="2026-07-28 16:22:37.795287855 +0000 UTC m=+74.933757270"
Jul 28 16:22:37.810189 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:37.809129    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-qzkm5" podStartSLOduration=67.511739465 podStartE2EDuration="1m14.80911108s" totalImagesPullingTime="7.297371615s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 16:21:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:22:37.808747897 +0000 UTC m=+74.947217309" watchObservedRunningTime="2026-07-28 16:22:37.80911108 +0000 UTC m=+74.947580499"
Jul 28 16:22:37.838943 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:37.838914    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:22:38.761709 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:38.761654    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:22:38.823135 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:38.823101    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:22:39.678718 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:39.678686    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:22:39.715086 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:39.715054    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6cf9d7745c-2f4rl"
Jul 28 16:22:39.739584 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:39.739552    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:22:42.166338 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:42.166252    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:22:42.794615 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:42.794583    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6fjcf" event={"ID":"f59115d6-8481-469e-a651-21e7faa6c392","Type":"ContainerStarted","Data":"1e6441ebb2ec7a3da883a9b4f10d8f80ed7cdccc61c60cf461525063b1238378"}
Jul 28 16:22:43.676867 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:43.676827    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-7c97dc4555-lk8fs"]
Jul 28 16:22:43.681219 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:43.681193    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-7c97dc4555-lk8fs"
Jul 28 16:22:43.684036 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:43.684008    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-554c2\""
Jul 28 16:22:43.684173 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:43.684016    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 28 16:22:43.684363 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:43.684343    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 28 16:22:43.690136 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:43.690081    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-7c97dc4555-lk8fs"]
Jul 28 16:22:43.693962 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:43.693809    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:22:43.780050 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:43.780022    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:22:43.789219 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:43.789108    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2nvjv\" (UniqueName: \"kubernetes.io/projected/d5e68bc9-29a9-4ed7-a692-c28fe6ddad08-kube-api-access-2nvjv\") pod \"downloads-7c97dc4555-lk8fs\" (UID: \"d5e68bc9-29a9-4ed7-a692-c28fe6ddad08\") " pod="openshift-console/downloads-7c97dc4555-lk8fs"
Jul 28 16:22:43.879170 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:43.879126    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:22:43.890165 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:43.890138    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2nvjv\" (UniqueName: \"kubernetes.io/projected/d5e68bc9-29a9-4ed7-a692-c28fe6ddad08-kube-api-access-2nvjv\") pod \"downloads-7c97dc4555-lk8fs\" (UID: \"d5e68bc9-29a9-4ed7-a692-c28fe6ddad08\") " pod="openshift-console/downloads-7c97dc4555-lk8fs"
Jul 28 16:22:43.905293 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:43.905267    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2nvjv\" (UniqueName: \"kubernetes.io/projected/d5e68bc9-29a9-4ed7-a692-c28fe6ddad08-kube-api-access-2nvjv\") pod \"downloads-7c97dc4555-lk8fs\" (UID: \"d5e68bc9-29a9-4ed7-a692-c28fe6ddad08\") " pod="openshift-console/downloads-7c97dc4555-lk8fs"
Jul 28 16:22:43.980334 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:43.980176    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:22:43.993034 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:43.993010    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-7c97dc4555-lk8fs"
Jul 28 16:22:44.078678 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:44.078649    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:22:44.134530 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:44.134479    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-6pvg7"]
Jul 28 16:22:44.137824 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:44.137801    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-6pvg7"
Jul 28 16:22:44.144925 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:44.144078    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Jul 28 16:22:44.144925 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:44.144379    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Jul 28 16:22:44.144925 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:44.144524    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-hh5cb\""
Jul 28 16:22:44.144925 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:44.144592    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 28 16:22:44.145549 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:44.145313    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 28 16:22:44.146904 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:44.146211    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 28 16:22:44.156768 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:44.156747    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-6pvg7"]
Jul 28 16:22:44.161339 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:44.161318    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-lxckg"]
Jul 28 16:22:44.163705 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:44.163686    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-lxckg"
Jul 28 16:22:44.168324 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:22:44.168258    2557 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"node-exporter-tls\" is forbidden: User \"system:node:ip-10-0-140-154.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-140-154.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" type="*v1.Secret"
Jul 28 16:22:44.173930 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:22:44.172922    2557 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"node-exporter-kube-rbac-proxy-config\" is forbidden: User \"system:node:ip-10-0-140-154.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-140-154.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" type="*v1.Secret"
Jul 28 16:22:44.173930 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:22:44.173272    2557 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"node-exporter-accelerators-collector-config\" is forbidden: User \"system:node:ip-10-0-140-154.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-140-154.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" type="*v1.ConfigMap"
Jul 28 16:22:44.173930 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:22:44.173378    2557 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"node-exporter-dockercfg-bqvfh\" is forbidden: User \"system:node:ip-10-0-140-154.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-140-154.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-bqvfh\"" type="*v1.Secret"
Jul 28 16:22:44.175141 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:44.175063    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-v8smk"]
Jul 28 16:22:44.177755 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:44.177735    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-v8smk"
Jul 28 16:22:44.183099 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:44.183078    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Jul 28 16:22:44.183825 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:44.183786    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Jul 28 16:22:44.184048 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:44.184032    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-qhjtp\""
Jul 28 16:22:44.186347 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:44.186314    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Jul 28 16:22:44.194821 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:44.194799    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-v8smk"]
Jul 28 16:22:44.201561 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:44.201539    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:22:44.293044 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:44.292964    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/04e4322a-db5d-4979-9106-21478fe43277-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-v8smk\" (UID: \"04e4322a-db5d-4979-9106-21478fe43277\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-v8smk"
Jul 28 16:22:44.293044 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:44.293015    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/04e4322a-db5d-4979-9106-21478fe43277-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-v8smk\" (UID: \"04e4322a-db5d-4979-9106-21478fe43277\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-v8smk"
Jul 28 16:22:44.293044 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:44.293043    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r66dr\" (UniqueName: \"kubernetes.io/projected/796d8289-12d1-431b-9b33-d144ab4d9932-kube-api-access-r66dr\") pod \"openshift-state-metrics-b8f49f5c6-6pvg7\" (UID: \"796d8289-12d1-431b-9b33-d144ab4d9932\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-6pvg7"
Jul 28 16:22:44.293325 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:44.293070    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7cb32b37-dc8a-4928-858b-2a40c09523bc-node-exporter-tls\") pod \"node-exporter-lxckg\" (UID: \"7cb32b37-dc8a-4928-858b-2a40c09523bc\") " pod="openshift-monitoring/node-exporter-lxckg"
Jul 28 16:22:44.293325 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:44.293096    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7cb32b37-dc8a-4928-858b-2a40c09523bc-node-exporter-wtmp\") pod \"node-exporter-lxckg\" (UID: \"7cb32b37-dc8a-4928-858b-2a40c09523bc\") " pod="openshift-monitoring/node-exporter-lxckg"
Jul 28 16:22:44.293325 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:44.293144    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7cb32b37-dc8a-4928-858b-2a40c09523bc-sys\") pod \"node-exporter-lxckg\" (UID: \"7cb32b37-dc8a-4928-858b-2a40c09523bc\") " pod="openshift-monitoring/node-exporter-lxckg"
Jul 28 16:22:44.293325 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:44.293170    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7cb32b37-dc8a-4928-858b-2a40c09523bc-root\") pod \"node-exporter-lxckg\" (UID: \"7cb32b37-dc8a-4928-858b-2a40c09523bc\") " pod="openshift-monitoring/node-exporter-lxckg"
Jul 28 16:22:44.293325 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:44.293190    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7cb32b37-dc8a-4928-858b-2a40c09523bc-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-lxckg\" (UID: \"7cb32b37-dc8a-4928-858b-2a40c09523bc\") " pod="openshift-monitoring/node-exporter-lxckg"
Jul 28 16:22:44.293325 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:44.293224    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/04e4322a-db5d-4979-9106-21478fe43277-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-v8smk\" (UID: \"04e4322a-db5d-4979-9106-21478fe43277\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-v8smk"
Jul 28 16:22:44.293325 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:44.293253    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/04e4322a-db5d-4979-9106-21478fe43277-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-v8smk\" (UID: \"04e4322a-db5d-4979-9106-21478fe43277\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-v8smk"
Jul 28 16:22:44.293325 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:44.293288    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tmdqv\" (UniqueName: \"kubernetes.io/projected/04e4322a-db5d-4979-9106-21478fe43277-kube-api-access-tmdqv\") pod \"kube-state-metrics-56d9f944b4-v8smk\" (UID: \"04e4322a-db5d-4979-9106-21478fe43277\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-v8smk"
Jul 28 16:22:44.293748 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:44.293347    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7cb32b37-dc8a-4928-858b-2a40c09523bc-metrics-client-ca\") pod \"node-exporter-lxckg\" (UID: \"7cb32b37-dc8a-4928-858b-2a40c09523bc\") " pod="openshift-monitoring/node-exporter-lxckg"
Jul 28 16:22:44.293748 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:44.293371    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/796d8289-12d1-431b-9b33-d144ab4d9932-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-6pvg7\" (UID: \"796d8289-12d1-431b-9b33-d144ab4d9932\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-6pvg7"
Jul 28 16:22:44.293748 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:44.293430    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/796d8289-12d1-431b-9b33-d144ab4d9932-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-6pvg7\" (UID: \"796d8289-12d1-431b-9b33-d144ab4d9932\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-6pvg7"
Jul 28 16:22:44.293748 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:44.293459    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-42tp4\" (UniqueName: \"kubernetes.io/projected/7cb32b37-dc8a-4928-858b-2a40c09523bc-kube-api-access-42tp4\") pod \"node-exporter-lxckg\" (UID: \"7cb32b37-dc8a-4928-858b-2a40c09523bc\") " pod="openshift-monitoring/node-exporter-lxckg"
Jul 28 16:22:44.293748 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:44.293486    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/796d8289-12d1-431b-9b33-d144ab4d9932-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-6pvg7\" (UID: \"796d8289-12d1-431b-9b33-d144ab4d9932\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-6pvg7"
Jul 28 16:22:44.293748 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:44.293536    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7cb32b37-dc8a-4928-858b-2a40c09523bc-node-exporter-accelerators-collector-config\") pod \"node-exporter-lxckg\" (UID: \"7cb32b37-dc8a-4928-858b-2a40c09523bc\") " pod="openshift-monitoring/node-exporter-lxckg"
Jul 28 16:22:44.293748 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:44.293567    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7cb32b37-dc8a-4928-858b-2a40c09523bc-node-exporter-textfile\") pod \"node-exporter-lxckg\" (UID: \"7cb32b37-dc8a-4928-858b-2a40c09523bc\") " pod="openshift-monitoring/node-exporter-lxckg"
Jul 28 16:22:44.293748 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:44.293599    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/04e4322a-db5d-4979-9106-21478fe43277-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-v8smk\" (UID: \"04e4322a-db5d-4979-9106-21478fe43277\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-v8smk"
Jul 28 16:22:44.314323 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:44.314230    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:22:44.394926 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:44.394894    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tmdqv\" (UniqueName: \"kubernetes.io/projected/04e4322a-db5d-4979-9106-21478fe43277-kube-api-access-tmdqv\") pod \"kube-state-metrics-56d9f944b4-v8smk\" (UID: \"04e4322a-db5d-4979-9106-21478fe43277\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-v8smk"
Jul 28 16:22:44.395089 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:44.394950    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7cb32b37-dc8a-4928-858b-2a40c09523bc-metrics-client-ca\") pod \"node-exporter-lxckg\" (UID: \"7cb32b37-dc8a-4928-858b-2a40c09523bc\") " pod="openshift-monitoring/node-exporter-lxckg"
Jul 28 16:22:44.395089 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:44.394982    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/796d8289-12d1-431b-9b33-d144ab4d9932-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-6pvg7\" (UID: \"796d8289-12d1-431b-9b33-d144ab4d9932\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-6pvg7"
Jul 28 16:22:44.395089 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:44.395019    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/796d8289-12d1-431b-9b33-d144ab4d9932-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-6pvg7\" (UID: \"796d8289-12d1-431b-9b33-d144ab4d9932\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-6pvg7"
Jul 28 16:22:44.395089 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:44.395047    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-42tp4\" (UniqueName: \"kubernetes.io/projected/7cb32b37-dc8a-4928-858b-2a40c09523bc-kube-api-access-42tp4\") pod \"node-exporter-lxckg\" (UID: \"7cb32b37-dc8a-4928-858b-2a40c09523bc\") " pod="openshift-monitoring/node-exporter-lxckg"
Jul 28 16:22:44.395089 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:44.395076    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/796d8289-12d1-431b-9b33-d144ab4d9932-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-6pvg7\" (UID: \"796d8289-12d1-431b-9b33-d144ab4d9932\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-6pvg7"
Jul 28 16:22:44.395335 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:44.395106    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7cb32b37-dc8a-4928-858b-2a40c09523bc-node-exporter-accelerators-collector-config\") pod \"node-exporter-lxckg\" (UID: \"7cb32b37-dc8a-4928-858b-2a40c09523bc\") " pod="openshift-monitoring/node-exporter-lxckg"
Jul 28 16:22:44.395335 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:44.395134    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7cb32b37-dc8a-4928-858b-2a40c09523bc-node-exporter-textfile\") pod \"node-exporter-lxckg\" (UID: \"7cb32b37-dc8a-4928-858b-2a40c09523bc\") " pod="openshift-monitoring/node-exporter-lxckg"
Jul 28 16:22:44.395335 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:44.395166    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/04e4322a-db5d-4979-9106-21478fe43277-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-v8smk\" (UID: \"04e4322a-db5d-4979-9106-21478fe43277\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-v8smk"
Jul 28 16:22:44.395335 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:44.395193    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/04e4322a-db5d-4979-9106-21478fe43277-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-v8smk\" (UID: \"04e4322a-db5d-4979-9106-21478fe43277\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-v8smk"
Jul 28 16:22:44.395335 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:44.395237    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/04e4322a-db5d-4979-9106-21478fe43277-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-v8smk\" (UID: \"04e4322a-db5d-4979-9106-21478fe43277\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-v8smk"
Jul 28 16:22:44.395335 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:44.395264    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r66dr\" (UniqueName: \"kubernetes.io/projected/796d8289-12d1-431b-9b33-d144ab4d9932-kube-api-access-r66dr\") pod \"openshift-state-metrics-b8f49f5c6-6pvg7\" (UID: \"796d8289-12d1-431b-9b33-d144ab4d9932\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-6pvg7"
Jul 28 16:22:44.395335 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:44.395314    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7cb32b37-dc8a-4928-858b-2a40c09523bc-node-exporter-tls\") pod \"node-exporter-lxckg\" (UID: \"7cb32b37-dc8a-4928-858b-2a40c09523bc\") " pod="openshift-monitoring/node-exporter-lxckg"
Jul 28 16:22:44.395711 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:44.395355    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7cb32b37-dc8a-4928-858b-2a40c09523bc-node-exporter-wtmp\") pod \"node-exporter-lxckg\" (UID: \"7cb32b37-dc8a-4928-858b-2a40c09523bc\") " pod="openshift-monitoring/node-exporter-lxckg"
Jul 28 16:22:44.395711 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:44.395401    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7cb32b37-dc8a-4928-858b-2a40c09523bc-sys\") pod \"node-exporter-lxckg\" (UID: \"7cb32b37-dc8a-4928-858b-2a40c09523bc\") " pod="openshift-monitoring/node-exporter-lxckg"
Jul 28 16:22:44.395711 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:44.395422    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7cb32b37-dc8a-4928-858b-2a40c09523bc-root\") pod \"node-exporter-lxckg\" (UID: \"7cb32b37-dc8a-4928-858b-2a40c09523bc\") " pod="openshift-monitoring/node-exporter-lxckg"
Jul 28 16:22:44.395711 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:44.395445    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7cb32b37-dc8a-4928-858b-2a40c09523bc-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-lxckg\" (UID: \"7cb32b37-dc8a-4928-858b-2a40c09523bc\") " pod="openshift-monitoring/node-exporter-lxckg"
Jul 28 16:22:44.395711 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:44.395477    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/04e4322a-db5d-4979-9106-21478fe43277-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-v8smk\" (UID: \"04e4322a-db5d-4979-9106-21478fe43277\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-v8smk"
Jul 28 16:22:44.395711 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:44.395524    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/04e4322a-db5d-4979-9106-21478fe43277-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-v8smk\" (UID: \"04e4322a-db5d-4979-9106-21478fe43277\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-v8smk"
Jul 28 16:22:44.395711 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:44.395568    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7cb32b37-dc8a-4928-858b-2a40c09523bc-metrics-client-ca\") pod \"node-exporter-lxckg\" (UID: \"7cb32b37-dc8a-4928-858b-2a40c09523bc\") " pod="openshift-monitoring/node-exporter-lxckg"
Jul 28 16:22:44.395931 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:44.395737    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7cb32b37-dc8a-4928-858b-2a40c09523bc-node-exporter-textfile\") pod \"node-exporter-lxckg\" (UID: \"7cb32b37-dc8a-4928-858b-2a40c09523bc\") " pod="openshift-monitoring/node-exporter-lxckg"
Jul 28 16:22:44.395982 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:44.395965    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/04e4322a-db5d-4979-9106-21478fe43277-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-v8smk\" (UID: \"04e4322a-db5d-4979-9106-21478fe43277\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-v8smk"
Jul 28 16:22:44.396078 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:44.396056    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7cb32b37-dc8a-4928-858b-2a40c09523bc-root\") pod \"node-exporter-lxckg\" (UID: \"7cb32b37-dc8a-4928-858b-2a40c09523bc\") " pod="openshift-monitoring/node-exporter-lxckg"
Jul 28 16:22:44.396212 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:44.396190    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7cb32b37-dc8a-4928-858b-2a40c09523bc-node-exporter-wtmp\") pod \"node-exporter-lxckg\" (UID: \"7cb32b37-dc8a-4928-858b-2a40c09523bc\") " pod="openshift-monitoring/node-exporter-lxckg"
Jul 28 16:22:44.396284 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:44.396246    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7cb32b37-dc8a-4928-858b-2a40c09523bc-sys\") pod \"node-exporter-lxckg\" (UID: \"7cb32b37-dc8a-4928-858b-2a40c09523bc\") " pod="openshift-monitoring/node-exporter-lxckg"
Jul 28 16:22:44.396642 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:44.396599    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/04e4322a-db5d-4979-9106-21478fe43277-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-v8smk\" (UID: \"04e4322a-db5d-4979-9106-21478fe43277\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-v8smk"
Jul 28 16:22:44.396763 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:44.396732    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/796d8289-12d1-431b-9b33-d144ab4d9932-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-6pvg7\" (UID: \"796d8289-12d1-431b-9b33-d144ab4d9932\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-6pvg7"
Jul 28 16:22:44.396969 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:44.396906    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/04e4322a-db5d-4979-9106-21478fe43277-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-v8smk\" (UID: \"04e4322a-db5d-4979-9106-21478fe43277\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-v8smk"
Jul 28 16:22:44.398728 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:44.398702    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/04e4322a-db5d-4979-9106-21478fe43277-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-v8smk\" (UID: \"04e4322a-db5d-4979-9106-21478fe43277\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-v8smk"
Jul 28 16:22:44.399046 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:44.399004    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/796d8289-12d1-431b-9b33-d144ab4d9932-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-6pvg7\" (UID: \"796d8289-12d1-431b-9b33-d144ab4d9932\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-6pvg7"
Jul 28 16:22:44.399560 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:44.399134    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/796d8289-12d1-431b-9b33-d144ab4d9932-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-6pvg7\" (UID: \"796d8289-12d1-431b-9b33-d144ab4d9932\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-6pvg7"
Jul 28 16:22:44.399560 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:44.399532    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/04e4322a-db5d-4979-9106-21478fe43277-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-v8smk\" (UID: \"04e4322a-db5d-4979-9106-21478fe43277\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-v8smk"
Jul 28 16:22:44.405092 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:44.405069    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tmdqv\" (UniqueName: \"kubernetes.io/projected/04e4322a-db5d-4979-9106-21478fe43277-kube-api-access-tmdqv\") pod \"kube-state-metrics-56d9f944b4-v8smk\" (UID: \"04e4322a-db5d-4979-9106-21478fe43277\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-v8smk"
Jul 28 16:22:44.405202 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:44.405180    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r66dr\" (UniqueName: \"kubernetes.io/projected/796d8289-12d1-431b-9b33-d144ab4d9932-kube-api-access-r66dr\") pod \"openshift-state-metrics-b8f49f5c6-6pvg7\" (UID: \"796d8289-12d1-431b-9b33-d144ab4d9932\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-6pvg7"
Jul 28 16:22:44.405470 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:44.405454    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-42tp4\" (UniqueName: \"kubernetes.io/projected/7cb32b37-dc8a-4928-858b-2a40c09523bc-kube-api-access-42tp4\") pod \"node-exporter-lxckg\" (UID: \"7cb32b37-dc8a-4928-858b-2a40c09523bc\") " pod="openshift-monitoring/node-exporter-lxckg"
Jul 28 16:22:44.414298 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:44.414273    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:22:44.458427 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:44.458399    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-6pvg7"
Jul 28 16:22:44.490395 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:44.490366    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-v8smk"
Jul 28 16:22:44.515306 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:44.515273    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:22:44.613962 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:44.613930    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:22:44.719608 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:44.719566    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:22:44.814751 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:44.814719    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:22:44.966420 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:44.966336    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:22:45.066614 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:45.066560    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:22:45.099991 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:45.099960    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 28 16:22:45.102897 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:45.102869    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 28 16:22:45.109357 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:45.109332    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7cb32b37-dc8a-4928-858b-2a40c09523bc-node-exporter-tls\") pod \"node-exporter-lxckg\" (UID: \"7cb32b37-dc8a-4928-858b-2a40c09523bc\") " pod="openshift-monitoring/node-exporter-lxckg"
Jul 28 16:22:45.111929 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:45.111902    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7cb32b37-dc8a-4928-858b-2a40c09523bc-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-lxckg\" (UID: \"7cb32b37-dc8a-4928-858b-2a40c09523bc\") " pod="openshift-monitoring/node-exporter-lxckg"
Jul 28 16:22:45.169192 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:45.169157    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:22:45.233870 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:45.233791    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:22:45.396237 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:22:45.396210    2557 configmap.go:193] Couldn't get configMap openshift-monitoring/node-exporter-accelerators-collector-config: failed to sync configmap cache: timed out waiting for the condition
Jul 28 16:22:45.396408 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:22:45.396295    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7cb32b37-dc8a-4928-858b-2a40c09523bc-node-exporter-accelerators-collector-config podName:7cb32b37-dc8a-4928-858b-2a40c09523bc nodeName:}" failed. No retries permitted until 2026-07-28 16:22:45.896269064 +0000 UTC m=+83.034738470 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-accelerators-collector-config" (UniqueName: "kubernetes.io/configmap/7cb32b37-dc8a-4928-858b-2a40c09523bc-node-exporter-accelerators-collector-config") pod "node-exporter-lxckg" (UID: "7cb32b37-dc8a-4928-858b-2a40c09523bc") : failed to sync configmap cache: timed out waiting for the condition
Jul 28 16:22:45.398938 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:45.398911    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:22:45.488243 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:45.488174    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:22:45.584779 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:45.584750    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:22:45.600613 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:45.600589    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 28 16:22:45.717617 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:45.715719    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:22:45.743224 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:45.742830    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-bqvfh\""
Jul 28 16:22:45.792680 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:45.792596    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-6pvg7"]
Jul 28 16:22:45.800048 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:22:45.800024    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod796d8289_12d1_431b_9b33_d144ab4d9932.slice/crio-db3f804e55fe46d7ca69f9583cdfcdaeb02eaa84fd8c917ddae0e3f39a7bcd0f WatchSource:0}: Error finding container db3f804e55fe46d7ca69f9583cdfcdaeb02eaa84fd8c917ddae0e3f39a7bcd0f: Status 404 returned error can't find the container with id db3f804e55fe46d7ca69f9583cdfcdaeb02eaa84fd8c917ddae0e3f39a7bcd0f
Jul 28 16:22:45.810374 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:45.810341    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6fjcf" event={"ID":"f59115d6-8481-469e-a651-21e7faa6c392","Type":"ContainerStarted","Data":"b36bc82b0a5df80491b27f818ed5c2375fe7dee8622405368609f69fda0c600e"}
Jul 28 16:22:45.811687 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:45.811659    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-6pvg7" event={"ID":"796d8289-12d1-431b-9b33-d144ab4d9932","Type":"ContainerStarted","Data":"db3f804e55fe46d7ca69f9583cdfcdaeb02eaa84fd8c917ddae0e3f39a7bcd0f"}
Jul 28 16:22:45.815272 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:45.815248    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-7c97dc4555-lk8fs"]
Jul 28 16:22:45.821113 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:22:45.821089    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5e68bc9_29a9_4ed7_a692_c28fe6ddad08.slice/crio-7fe94fdc91719c4201705af0b9501bcd29eebb9092ce2c5685715ef8625fa4ac WatchSource:0}: Error finding container 7fe94fdc91719c4201705af0b9501bcd29eebb9092ce2c5685715ef8625fa4ac: Status 404 returned error can't find the container with id 7fe94fdc91719c4201705af0b9501bcd29eebb9092ce2c5685715ef8625fa4ac
Jul 28 16:22:45.839175 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:45.836684    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-v8smk"]
Jul 28 16:22:45.908144 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:45.908119    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7cb32b37-dc8a-4928-858b-2a40c09523bc-node-exporter-accelerators-collector-config\") pod \"node-exporter-lxckg\" (UID: \"7cb32b37-dc8a-4928-858b-2a40c09523bc\") " pod="openshift-monitoring/node-exporter-lxckg"
Jul 28 16:22:45.908624 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:45.908602    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7cb32b37-dc8a-4928-858b-2a40c09523bc-node-exporter-accelerators-collector-config\") pod \"node-exporter-lxckg\" (UID: \"7cb32b37-dc8a-4928-858b-2a40c09523bc\") " pod="openshift-monitoring/node-exporter-lxckg"
Jul 28 16:22:45.978044 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:45.978020    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-lxckg"
Jul 28 16:22:45.987038 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:22:45.987010    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7cb32b37_dc8a_4928_858b_2a40c09523bc.slice/crio-c963d1f17110e00c71dd8313b8f7d6067b7476abbedd6766f6477f2fa47d22c7 WatchSource:0}: Error finding container c963d1f17110e00c71dd8313b8f7d6067b7476abbedd6766f6477f2fa47d22c7: Status 404 returned error can't find the container with id c963d1f17110e00c71dd8313b8f7d6067b7476abbedd6766f6477f2fa47d22c7
Jul 28 16:22:46.053238 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:46.053213    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:22:46.609294 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:46.609222    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:22:46.714030 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:46.713981    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:22:46.823332 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:46.823197    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-v8smk" event={"ID":"04e4322a-db5d-4979-9106-21478fe43277","Type":"ContainerStarted","Data":"a413b44f6c9fa69bc287cf14031013541fd69d9c3917aade350f15c76c64efec"}
Jul 28 16:22:46.828854 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:46.828720    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-6pvg7" event={"ID":"796d8289-12d1-431b-9b33-d144ab4d9932","Type":"ContainerStarted","Data":"6a9786830bf7305ec3a140ce058e6f5f996a49043b1443d981ac29dc5581d4e0"}
Jul 28 16:22:46.828854 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:46.828779    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-6pvg7" event={"ID":"796d8289-12d1-431b-9b33-d144ab4d9932","Type":"ContainerStarted","Data":"c168b2b605a3c7759017d421133a6a7f61f385b61cf1e411f8e025378147d3ca"}
Jul 28 16:22:46.832519 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:46.832432    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-lxckg" event={"ID":"7cb32b37-dc8a-4928-858b-2a40c09523bc","Type":"ContainerStarted","Data":"c963d1f17110e00c71dd8313b8f7d6067b7476abbedd6766f6477f2fa47d22c7"}
Jul 28 16:22:46.834100 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:46.834044    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-7c97dc4555-lk8fs" event={"ID":"d5e68bc9-29a9-4ed7-a692-c28fe6ddad08","Type":"ContainerStarted","Data":"7fe94fdc91719c4201705af0b9501bcd29eebb9092ce2c5685715ef8625fa4ac"}
Jul 28 16:22:47.073104 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:47.073041    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:22:47.131741 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:47.131690    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:22:47.635341 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:47.635299    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:22:47.732827 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:47.732797    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:22:48.128436 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:48.128409    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-7c5674d886-pqfzc"]
Jul 28 16:22:48.132994 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:48.132976    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7c5674d886-pqfzc"
Jul 28 16:22:48.135756 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:48.135731    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Jul 28 16:22:48.135904 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:48.135848    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-1gdl94c2cq81p\""
Jul 28 16:22:48.136137 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:48.136119    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-qkz26\""
Jul 28 16:22:48.136224 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:48.135738    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Jul 28 16:22:48.137111 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:48.136839    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Jul 28 16:22:48.137111 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:48.136895    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Jul 28 16:22:48.137111 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:48.136846    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Jul 28 16:22:48.142102 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:48.142082    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7c5674d886-pqfzc"]
Jul 28 16:22:48.229728 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:48.229585    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/2d437116-f45c-4765-b8aa-14ad42f54af0-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7c5674d886-pqfzc\" (UID: \"2d437116-f45c-4765-b8aa-14ad42f54af0\") " pod="openshift-monitoring/thanos-querier-7c5674d886-pqfzc"
Jul 28 16:22:48.229728 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:48.229633    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/2d437116-f45c-4765-b8aa-14ad42f54af0-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7c5674d886-pqfzc\" (UID: \"2d437116-f45c-4765-b8aa-14ad42f54af0\") " pod="openshift-monitoring/thanos-querier-7c5674d886-pqfzc"
Jul 28 16:22:48.229728 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:48.229669    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/2d437116-f45c-4765-b8aa-14ad42f54af0-secret-thanos-querier-tls\") pod \"thanos-querier-7c5674d886-pqfzc\" (UID: \"2d437116-f45c-4765-b8aa-14ad42f54af0\") " pod="openshift-monitoring/thanos-querier-7c5674d886-pqfzc"
Jul 28 16:22:48.229728 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:48.229694    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2d437116-f45c-4765-b8aa-14ad42f54af0-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7c5674d886-pqfzc\" (UID: \"2d437116-f45c-4765-b8aa-14ad42f54af0\") " pod="openshift-monitoring/thanos-querier-7c5674d886-pqfzc"
Jul 28 16:22:48.229728 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:48.229716    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bb42d\" (UniqueName: \"kubernetes.io/projected/2d437116-f45c-4765-b8aa-14ad42f54af0-kube-api-access-bb42d\") pod \"thanos-querier-7c5674d886-pqfzc\" (UID: \"2d437116-f45c-4765-b8aa-14ad42f54af0\") " pod="openshift-monitoring/thanos-querier-7c5674d886-pqfzc"
Jul 28 16:22:48.230052 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:48.229763    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2d437116-f45c-4765-b8aa-14ad42f54af0-secret-grpc-tls\") pod \"thanos-querier-7c5674d886-pqfzc\" (UID: \"2d437116-f45c-4765-b8aa-14ad42f54af0\") " pod="openshift-monitoring/thanos-querier-7c5674d886-pqfzc"
Jul 28 16:22:48.230052 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:48.229792    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2d437116-f45c-4765-b8aa-14ad42f54af0-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7c5674d886-pqfzc\" (UID: \"2d437116-f45c-4765-b8aa-14ad42f54af0\") " pod="openshift-monitoring/thanos-querier-7c5674d886-pqfzc"
Jul 28 16:22:48.230052 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:48.229824    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2d437116-f45c-4765-b8aa-14ad42f54af0-metrics-client-ca\") pod \"thanos-querier-7c5674d886-pqfzc\" (UID: \"2d437116-f45c-4765-b8aa-14ad42f54af0\") " pod="openshift-monitoring/thanos-querier-7c5674d886-pqfzc"
Jul 28 16:22:48.330244 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:48.330168    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/2d437116-f45c-4765-b8aa-14ad42f54af0-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7c5674d886-pqfzc\" (UID: \"2d437116-f45c-4765-b8aa-14ad42f54af0\") " pod="openshift-monitoring/thanos-querier-7c5674d886-pqfzc"
Jul 28 16:22:48.330244 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:48.330208    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/2d437116-f45c-4765-b8aa-14ad42f54af0-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7c5674d886-pqfzc\" (UID: \"2d437116-f45c-4765-b8aa-14ad42f54af0\") " pod="openshift-monitoring/thanos-querier-7c5674d886-pqfzc"
Jul 28 16:22:48.330244 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:48.330232    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/2d437116-f45c-4765-b8aa-14ad42f54af0-secret-thanos-querier-tls\") pod \"thanos-querier-7c5674d886-pqfzc\" (UID: \"2d437116-f45c-4765-b8aa-14ad42f54af0\") " pod="openshift-monitoring/thanos-querier-7c5674d886-pqfzc"
Jul 28 16:22:48.330472 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:48.330248    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2d437116-f45c-4765-b8aa-14ad42f54af0-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7c5674d886-pqfzc\" (UID: \"2d437116-f45c-4765-b8aa-14ad42f54af0\") " pod="openshift-monitoring/thanos-querier-7c5674d886-pqfzc"
Jul 28 16:22:48.330472 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:48.330264    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bb42d\" (UniqueName: \"kubernetes.io/projected/2d437116-f45c-4765-b8aa-14ad42f54af0-kube-api-access-bb42d\") pod \"thanos-querier-7c5674d886-pqfzc\" (UID: \"2d437116-f45c-4765-b8aa-14ad42f54af0\") " pod="openshift-monitoring/thanos-querier-7c5674d886-pqfzc"
Jul 28 16:22:48.330472 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:48.330297    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2d437116-f45c-4765-b8aa-14ad42f54af0-secret-grpc-tls\") pod \"thanos-querier-7c5674d886-pqfzc\" (UID: \"2d437116-f45c-4765-b8aa-14ad42f54af0\") " pod="openshift-monitoring/thanos-querier-7c5674d886-pqfzc"
Jul 28 16:22:48.330472 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:48.330334    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2d437116-f45c-4765-b8aa-14ad42f54af0-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7c5674d886-pqfzc\" (UID: \"2d437116-f45c-4765-b8aa-14ad42f54af0\") " pod="openshift-monitoring/thanos-querier-7c5674d886-pqfzc"
Jul 28 16:22:48.330472 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:48.330355    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2d437116-f45c-4765-b8aa-14ad42f54af0-metrics-client-ca\") pod \"thanos-querier-7c5674d886-pqfzc\" (UID: \"2d437116-f45c-4765-b8aa-14ad42f54af0\") " pod="openshift-monitoring/thanos-querier-7c5674d886-pqfzc"
Jul 28 16:22:48.331372 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:48.331115    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2d437116-f45c-4765-b8aa-14ad42f54af0-metrics-client-ca\") pod \"thanos-querier-7c5674d886-pqfzc\" (UID: \"2d437116-f45c-4765-b8aa-14ad42f54af0\") " pod="openshift-monitoring/thanos-querier-7c5674d886-pqfzc"
Jul 28 16:22:48.334580 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:48.334425    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/2d437116-f45c-4765-b8aa-14ad42f54af0-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7c5674d886-pqfzc\" (UID: \"2d437116-f45c-4765-b8aa-14ad42f54af0\") " pod="openshift-monitoring/thanos-querier-7c5674d886-pqfzc"
Jul 28 16:22:48.334580 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:48.334530    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2d437116-f45c-4765-b8aa-14ad42f54af0-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7c5674d886-pqfzc\" (UID: \"2d437116-f45c-4765-b8aa-14ad42f54af0\") " pod="openshift-monitoring/thanos-querier-7c5674d886-pqfzc"
Jul 28 16:22:48.335247 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:48.334917    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/2d437116-f45c-4765-b8aa-14ad42f54af0-secret-thanos-querier-tls\") pod \"thanos-querier-7c5674d886-pqfzc\" (UID: \"2d437116-f45c-4765-b8aa-14ad42f54af0\") " pod="openshift-monitoring/thanos-querier-7c5674d886-pqfzc"
Jul 28 16:22:48.335247 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:48.335209    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2d437116-f45c-4765-b8aa-14ad42f54af0-secret-grpc-tls\") pod \"thanos-querier-7c5674d886-pqfzc\" (UID: \"2d437116-f45c-4765-b8aa-14ad42f54af0\") " pod="openshift-monitoring/thanos-querier-7c5674d886-pqfzc"
Jul 28 16:22:48.336225 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:48.336184    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2d437116-f45c-4765-b8aa-14ad42f54af0-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7c5674d886-pqfzc\" (UID: \"2d437116-f45c-4765-b8aa-14ad42f54af0\") " pod="openshift-monitoring/thanos-querier-7c5674d886-pqfzc"
Jul 28 16:22:48.337238 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:48.337188    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/2d437116-f45c-4765-b8aa-14ad42f54af0-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7c5674d886-pqfzc\" (UID: \"2d437116-f45c-4765-b8aa-14ad42f54af0\") " pod="openshift-monitoring/thanos-querier-7c5674d886-pqfzc"
Jul 28 16:22:48.345748 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:48.345710    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bb42d\" (UniqueName: \"kubernetes.io/projected/2d437116-f45c-4765-b8aa-14ad42f54af0-kube-api-access-bb42d\") pod \"thanos-querier-7c5674d886-pqfzc\" (UID: \"2d437116-f45c-4765-b8aa-14ad42f54af0\") " pod="openshift-monitoring/thanos-querier-7c5674d886-pqfzc"
Jul 28 16:22:48.462266 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:48.462233    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7c5674d886-pqfzc"
Jul 28 16:22:48.767710 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:48.767593    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:22:48.852273 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:48.852234    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-v8smk" event={"ID":"04e4322a-db5d-4979-9106-21478fe43277","Type":"ContainerStarted","Data":"a269db839136e94b48b5b6c049bee75d6ccb0455a83c8c2fa8b7091eb996d511"}
Jul 28 16:22:48.852533 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:48.852330    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-v8smk" event={"ID":"04e4322a-db5d-4979-9106-21478fe43277","Type":"ContainerStarted","Data":"d8223ccc7ba762b3bff0f267d26b5fbb5c363d8a50f55aeecf0ded831d87757b"}
Jul 28 16:22:48.852533 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:48.852362    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-v8smk" event={"ID":"04e4322a-db5d-4979-9106-21478fe43277","Type":"ContainerStarted","Data":"92b63fad35ef1c9c3e2bb11c51b08b5e5fa471f494c38a7a81e2aa1a8ae9ada8"}
Jul 28 16:22:48.856154 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:48.856126    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-6pvg7" event={"ID":"796d8289-12d1-431b-9b33-d144ab4d9932","Type":"ContainerStarted","Data":"1297839794fef6eb5c472d64fa2e17a24a92238e6c85c8fd05b87a2997d04aa3"}
Jul 28 16:22:48.858307 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:48.858279    2557 generic.go:356] "Generic (PLEG): container finished" podID="7cb32b37-dc8a-4928-858b-2a40c09523bc" containerID="69f2656124703848139766ebe337075745ed6c6a324a37765be74dd04b9020cd" exitCode=0
Jul 28 16:22:48.858409 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:48.858334    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-lxckg" event={"ID":"7cb32b37-dc8a-4928-858b-2a40c09523bc","Type":"ContainerDied","Data":"69f2656124703848139766ebe337075745ed6c6a324a37765be74dd04b9020cd"}
Jul 28 16:22:48.874375 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:48.874355    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:22:48.898723 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:48.898667    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-v8smk" podStartSLOduration=2.653435524 podStartE2EDuration="4.898649424s" totalImagesPullingTime="2.2452139s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 16:22:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:22:48.873678276 +0000 UTC m=+86.012147692" watchObservedRunningTime="2026-07-28 16:22:48.898649424 +0000 UTC m=+86.037118838"
Jul 28 16:22:48.899208 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:48.899162    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-6pvg7" podStartSLOduration=2.741122527 podStartE2EDuration="4.899149245s" totalImagesPullingTime="2.158026718s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 16:22:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:22:48.898755326 +0000 UTC m=+86.037224743" watchObservedRunningTime="2026-07-28 16:22:48.899149245 +0000 UTC m=+86.037618659"
Jul 28 16:22:48.968326 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:48.968258    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:22:49.078105 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:49.078070    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:22:49.817101 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:49.817054    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:22:49.865022 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:49.864978    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:22:50.392853 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:50.392816    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:22:50.562040 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:50.561991    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7c5674d886-pqfzc"]
Jul 28 16:22:50.564147 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:22:50.564113    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d437116_f45c_4765_b8aa_14ad42f54af0.slice/crio-ede638244bf2b124918251d095cd4a0b08c31d062318ca76fdaf6c6445a47cbc WatchSource:0}: Error finding container ede638244bf2b124918251d095cd4a0b08c31d062318ca76fdaf6c6445a47cbc: Status 404 returned error can't find the container with id ede638244bf2b124918251d095cd4a0b08c31d062318ca76fdaf6c6445a47cbc
Jul 28 16:22:50.871904 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:50.871857    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-lxckg" event={"ID":"7cb32b37-dc8a-4928-858b-2a40c09523bc","Type":"ContainerStarted","Data":"8070bf8a1101e820f5ce5614d3c32be8cd1f083784f88cf8212ac83c5a90cea7"}
Jul 28 16:22:50.872310 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:50.871933    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-lxckg" event={"ID":"7cb32b37-dc8a-4928-858b-2a40c09523bc","Type":"ContainerStarted","Data":"8f4415a5d3e77d97d5444c34185158ae1999a5d3f12772c5f56f061f9f5d2f87"}
Jul 28 16:22:50.874700 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:50.874199    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6fjcf" event={"ID":"f59115d6-8481-469e-a651-21e7faa6c392","Type":"ContainerStarted","Data":"9e6491c25fabfa245cd88950e77ee5141b625f12fefae4afe6dc6b425a81cb3c"}
Jul 28 16:22:50.875660 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:50.875620    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7c5674d886-pqfzc" event={"ID":"2d437116-f45c-4765-b8aa-14ad42f54af0","Type":"ContainerStarted","Data":"ede638244bf2b124918251d095cd4a0b08c31d062318ca76fdaf6c6445a47cbc"}
Jul 28 16:22:50.892643 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:50.892595    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-lxckg" podStartSLOduration=4.683286653 podStartE2EDuration="6.892576632s" totalImagesPullingTime="2.09441747s" totalInitContainerRuntime="114.872509ms" isStatefulPod=true podCreationTimestamp="2026-07-28 16:22:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:22:50.890390971 +0000 UTC m=+88.028860385" watchObservedRunningTime="2026-07-28 16:22:50.892576632 +0000 UTC m=+88.031046047"
Jul 28 16:22:50.907655 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:50.907620    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-6fjcf" podStartSLOduration=35.606286092 podStartE2EDuration="48.907607252s" totalImagesPullingTime="13.30132116s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 16:22:02 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:22:50.906343242 +0000 UTC m=+88.044812657" watchObservedRunningTime="2026-07-28 16:22:50.907607252 +0000 UTC m=+88.046076665"
Jul 28 16:22:51.788307 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:51.788253    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:22:51.843926 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:51.843888    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:22:52.749685 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:52.749648    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-77fbd874ff-997dw"
Jul 28 16:22:52.894000 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:52.893966    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:22:52.997647 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:52.997620    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:22:53.001289 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:53.001264    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-66f76ff476-wb8td"]
Jul 28 16:22:53.004650 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:53.004632    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66f76ff476-wb8td"
Jul 28 16:22:53.007279 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:53.007261    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 28 16:22:53.007381 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:53.007343    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 28 16:22:53.007521 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:53.007474    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 28 16:22:53.007771 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:53.007741    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-5fjfh\""
Jul 28 16:22:53.007866 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:53.007794    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 28 16:22:53.007866 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:53.007722    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 28 16:22:53.015217 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:53.015200    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-66f76ff476-wb8td"]
Jul 28 16:22:53.073142 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:53.073112    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/03d0cb65-9f7e-49d7-8d1d-715b986a0bff-console-oauth-config\") pod \"console-66f76ff476-wb8td\" (UID: \"03d0cb65-9f7e-49d7-8d1d-715b986a0bff\") " pod="openshift-console/console-66f76ff476-wb8td"
Jul 28 16:22:53.073254 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:53.073161    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/03d0cb65-9f7e-49d7-8d1d-715b986a0bff-service-ca\") pod \"console-66f76ff476-wb8td\" (UID: \"03d0cb65-9f7e-49d7-8d1d-715b986a0bff\") " pod="openshift-console/console-66f76ff476-wb8td"
Jul 28 16:22:53.073254 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:53.073223    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/03d0cb65-9f7e-49d7-8d1d-715b986a0bff-console-config\") pod \"console-66f76ff476-wb8td\" (UID: \"03d0cb65-9f7e-49d7-8d1d-715b986a0bff\") " pod="openshift-console/console-66f76ff476-wb8td"
Jul 28 16:22:53.073254 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:53.073250    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/03d0cb65-9f7e-49d7-8d1d-715b986a0bff-console-serving-cert\") pod \"console-66f76ff476-wb8td\" (UID: \"03d0cb65-9f7e-49d7-8d1d-715b986a0bff\") " pod="openshift-console/console-66f76ff476-wb8td"
Jul 28 16:22:53.073394 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:53.073276    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8dcbg\" (UniqueName: \"kubernetes.io/projected/03d0cb65-9f7e-49d7-8d1d-715b986a0bff-kube-api-access-8dcbg\") pod \"console-66f76ff476-wb8td\" (UID: \"03d0cb65-9f7e-49d7-8d1d-715b986a0bff\") " pod="openshift-console/console-66f76ff476-wb8td"
Jul 28 16:22:53.073486 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:53.073465    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/03d0cb65-9f7e-49d7-8d1d-715b986a0bff-oauth-serving-cert\") pod \"console-66f76ff476-wb8td\" (UID: \"03d0cb65-9f7e-49d7-8d1d-715b986a0bff\") " pod="openshift-console/console-66f76ff476-wb8td"
Jul 28 16:22:53.174505 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:53.174462    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/03d0cb65-9f7e-49d7-8d1d-715b986a0bff-console-oauth-config\") pod \"console-66f76ff476-wb8td\" (UID: \"03d0cb65-9f7e-49d7-8d1d-715b986a0bff\") " pod="openshift-console/console-66f76ff476-wb8td"
Jul 28 16:22:53.174505 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:53.174509    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/03d0cb65-9f7e-49d7-8d1d-715b986a0bff-service-ca\") pod \"console-66f76ff476-wb8td\" (UID: \"03d0cb65-9f7e-49d7-8d1d-715b986a0bff\") " pod="openshift-console/console-66f76ff476-wb8td"
Jul 28 16:22:53.174731 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:53.174710    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/03d0cb65-9f7e-49d7-8d1d-715b986a0bff-console-config\") pod \"console-66f76ff476-wb8td\" (UID: \"03d0cb65-9f7e-49d7-8d1d-715b986a0bff\") " pod="openshift-console/console-66f76ff476-wb8td"
Jul 28 16:22:53.174772 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:53.174746    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/03d0cb65-9f7e-49d7-8d1d-715b986a0bff-console-serving-cert\") pod \"console-66f76ff476-wb8td\" (UID: \"03d0cb65-9f7e-49d7-8d1d-715b986a0bff\") " pod="openshift-console/console-66f76ff476-wb8td"
Jul 28 16:22:53.174822 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:53.174773    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8dcbg\" (UniqueName: \"kubernetes.io/projected/03d0cb65-9f7e-49d7-8d1d-715b986a0bff-kube-api-access-8dcbg\") pod \"console-66f76ff476-wb8td\" (UID: \"03d0cb65-9f7e-49d7-8d1d-715b986a0bff\") " pod="openshift-console/console-66f76ff476-wb8td"
Jul 28 16:22:53.174822 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:53.174802    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/03d0cb65-9f7e-49d7-8d1d-715b986a0bff-oauth-serving-cert\") pod \"console-66f76ff476-wb8td\" (UID: \"03d0cb65-9f7e-49d7-8d1d-715b986a0bff\") " pod="openshift-console/console-66f76ff476-wb8td"
Jul 28 16:22:53.175265 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:53.175239    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/03d0cb65-9f7e-49d7-8d1d-715b986a0bff-service-ca\") pod \"console-66f76ff476-wb8td\" (UID: \"03d0cb65-9f7e-49d7-8d1d-715b986a0bff\") " pod="openshift-console/console-66f76ff476-wb8td"
Jul 28 16:22:53.175393 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:53.175349    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/03d0cb65-9f7e-49d7-8d1d-715b986a0bff-oauth-serving-cert\") pod \"console-66f76ff476-wb8td\" (UID: \"03d0cb65-9f7e-49d7-8d1d-715b986a0bff\") " pod="openshift-console/console-66f76ff476-wb8td"
Jul 28 16:22:53.175535 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:53.175516    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/03d0cb65-9f7e-49d7-8d1d-715b986a0bff-console-config\") pod \"console-66f76ff476-wb8td\" (UID: \"03d0cb65-9f7e-49d7-8d1d-715b986a0bff\") " pod="openshift-console/console-66f76ff476-wb8td"
Jul 28 16:22:53.177298 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:53.177275    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/03d0cb65-9f7e-49d7-8d1d-715b986a0bff-console-oauth-config\") pod \"console-66f76ff476-wb8td\" (UID: \"03d0cb65-9f7e-49d7-8d1d-715b986a0bff\") " pod="openshift-console/console-66f76ff476-wb8td"
Jul 28 16:22:53.177708 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:53.177690    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/03d0cb65-9f7e-49d7-8d1d-715b986a0bff-console-serving-cert\") pod \"console-66f76ff476-wb8td\" (UID: \"03d0cb65-9f7e-49d7-8d1d-715b986a0bff\") " pod="openshift-console/console-66f76ff476-wb8td"
Jul 28 16:22:53.183122 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:53.183102    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8dcbg\" (UniqueName: \"kubernetes.io/projected/03d0cb65-9f7e-49d7-8d1d-715b986a0bff-kube-api-access-8dcbg\") pod \"console-66f76ff476-wb8td\" (UID: \"03d0cb65-9f7e-49d7-8d1d-715b986a0bff\") " pod="openshift-console/console-66f76ff476-wb8td"
Jul 28 16:22:53.326268 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:53.326236    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66f76ff476-wb8td"
Jul 28 16:22:53.470535 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:53.470506    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-66f76ff476-wb8td"]
Jul 28 16:22:53.473333 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:22:53.473307    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03d0cb65_9f7e_49d7_8d1d_715b986a0bff.slice/crio-e745c48ebba7258cd2ddb92c9c4423068a0695f2542917e5e00a2bc82bff8531 WatchSource:0}: Error finding container e745c48ebba7258cd2ddb92c9c4423068a0695f2542917e5e00a2bc82bff8531: Status 404 returned error can't find the container with id e745c48ebba7258cd2ddb92c9c4423068a0695f2542917e5e00a2bc82bff8531
Jul 28 16:22:53.892651 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:53.892620    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7c5674d886-pqfzc" event={"ID":"2d437116-f45c-4765-b8aa-14ad42f54af0","Type":"ContainerStarted","Data":"c5d50f2b48f2c0d1acdeb48832463903e269d34e53edf544bba0da1e08f0c1ba"}
Jul 28 16:22:53.892986 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:53.892702    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7c5674d886-pqfzc" event={"ID":"2d437116-f45c-4765-b8aa-14ad42f54af0","Type":"ContainerStarted","Data":"be98c80cc051d49460d8184e550224fbb85d3b3cbe6b5fcf46f1ba78291b82f5"}
Jul 28 16:22:53.892986 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:53.892743    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7c5674d886-pqfzc" event={"ID":"2d437116-f45c-4765-b8aa-14ad42f54af0","Type":"ContainerStarted","Data":"1106cf24ef577183bb5781f59c8030c15748bb3539b70fd05568c09da33eb7a1"}
Jul 28 16:22:53.894045 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:53.894017    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66f76ff476-wb8td" event={"ID":"03d0cb65-9f7e-49d7-8d1d-715b986a0bff","Type":"ContainerStarted","Data":"e745c48ebba7258cd2ddb92c9c4423068a0695f2542917e5e00a2bc82bff8531"}
Jul 28 16:22:53.996014 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:53.995989    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:22:54.097089 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:54.097008    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:22:54.734967 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:54.734929    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-6cf9d7745c-2f4rl" podUID="915e2010-9827-4845-b395-a7ee1fedcfe7" containerName="registry" containerID="cri-o://9d12be9b0cbc42c7312b0a10a6fff9043dec7bd209fe0f2d38a8b10efd218a6e" gracePeriod=30
Jul 28 16:22:54.900260 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:54.900083    2557 generic.go:356] "Generic (PLEG): container finished" podID="915e2010-9827-4845-b395-a7ee1fedcfe7" containerID="9d12be9b0cbc42c7312b0a10a6fff9043dec7bd209fe0f2d38a8b10efd218a6e" exitCode=0
Jul 28 16:22:54.900260 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:54.900173    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6cf9d7745c-2f4rl" event={"ID":"915e2010-9827-4845-b395-a7ee1fedcfe7","Type":"ContainerDied","Data":"9d12be9b0cbc42c7312b0a10a6fff9043dec7bd209fe0f2d38a8b10efd218a6e"}
Jul 28 16:22:54.905641 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:54.904372    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7c5674d886-pqfzc" event={"ID":"2d437116-f45c-4765-b8aa-14ad42f54af0","Type":"ContainerStarted","Data":"d41a89f354c99d1d7c921869c0b7681f3a006ee8fe9c4b410fcb3fc46e6fef7c"}
Jul 28 16:22:54.905641 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:54.904449    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7c5674d886-pqfzc" event={"ID":"2d437116-f45c-4765-b8aa-14ad42f54af0","Type":"ContainerStarted","Data":"1a71cadc1c2b8d008aafeebe64e0d52675ea59fd572246e4ab0f3bbcb07b5d83"}
Jul 28 16:22:54.905641 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:54.904510    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7c5674d886-pqfzc" event={"ID":"2d437116-f45c-4765-b8aa-14ad42f54af0","Type":"ContainerStarted","Data":"a9d4473adbd0fad4fc1dab4e59549b03b842599a110ebe9ced1a2d9693257dad"}
Jul 28 16:22:54.905641 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:54.905468    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-7c5674d886-pqfzc"
Jul 28 16:22:54.930290 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:54.930241    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-7c5674d886-pqfzc" podStartSLOduration=3.881158346 podStartE2EDuration="6.930220706s" totalImagesPullingTime="3.04906236s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 16:22:48 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:22:54.926258169 +0000 UTC m=+92.064727584" watchObservedRunningTime="2026-07-28 16:22:54.930220706 +0000 UTC m=+92.068690121"
Jul 28 16:22:55.004444 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:55.004418    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6cf9d7745c-2f4rl"
Jul 28 16:22:55.090785 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:55.090752    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/915e2010-9827-4845-b395-a7ee1fedcfe7-registry-certificates\") pod \"915e2010-9827-4845-b395-a7ee1fedcfe7\" (UID: \"915e2010-9827-4845-b395-a7ee1fedcfe7\") "
Jul 28 16:22:55.090926 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:55.090802    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/915e2010-9827-4845-b395-a7ee1fedcfe7-trusted-ca\") pod \"915e2010-9827-4845-b395-a7ee1fedcfe7\" (UID: \"915e2010-9827-4845-b395-a7ee1fedcfe7\") "
Jul 28 16:22:55.090926 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:55.090833    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/915e2010-9827-4845-b395-a7ee1fedcfe7-image-registry-private-configuration\") pod \"915e2010-9827-4845-b395-a7ee1fedcfe7\" (UID: \"915e2010-9827-4845-b395-a7ee1fedcfe7\") "
Jul 28 16:22:55.090926 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:55.090863    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/915e2010-9827-4845-b395-a7ee1fedcfe7-bound-sa-token\") pod \"915e2010-9827-4845-b395-a7ee1fedcfe7\" (UID: \"915e2010-9827-4845-b395-a7ee1fedcfe7\") "
Jul 28 16:22:55.090926 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:55.090888    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/915e2010-9827-4845-b395-a7ee1fedcfe7-ca-trust-extracted\") pod \"915e2010-9827-4845-b395-a7ee1fedcfe7\" (UID: \"915e2010-9827-4845-b395-a7ee1fedcfe7\") "
Jul 28 16:22:55.090926 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:55.090919    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-59ccc\" (UniqueName: \"kubernetes.io/projected/915e2010-9827-4845-b395-a7ee1fedcfe7-kube-api-access-59ccc\") pod \"915e2010-9827-4845-b395-a7ee1fedcfe7\" (UID: \"915e2010-9827-4845-b395-a7ee1fedcfe7\") "
Jul 28 16:22:55.091135 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:55.090996    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/915e2010-9827-4845-b395-a7ee1fedcfe7-installation-pull-secrets\") pod \"915e2010-9827-4845-b395-a7ee1fedcfe7\" (UID: \"915e2010-9827-4845-b395-a7ee1fedcfe7\") "
Jul 28 16:22:55.091135 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:55.091056    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/915e2010-9827-4845-b395-a7ee1fedcfe7-registry-tls\") pod \"915e2010-9827-4845-b395-a7ee1fedcfe7\" (UID: \"915e2010-9827-4845-b395-a7ee1fedcfe7\") "
Jul 28 16:22:55.092460 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:55.092397    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/915e2010-9827-4845-b395-a7ee1fedcfe7-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "915e2010-9827-4845-b395-a7ee1fedcfe7" (UID: "915e2010-9827-4845-b395-a7ee1fedcfe7"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 16:22:55.092739 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:55.092708    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/915e2010-9827-4845-b395-a7ee1fedcfe7-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "915e2010-9827-4845-b395-a7ee1fedcfe7" (UID: "915e2010-9827-4845-b395-a7ee1fedcfe7"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 16:22:55.094888 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:55.094862    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/915e2010-9827-4845-b395-a7ee1fedcfe7-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "915e2010-9827-4845-b395-a7ee1fedcfe7" (UID: "915e2010-9827-4845-b395-a7ee1fedcfe7"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 16:22:55.095129 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:55.095107    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/915e2010-9827-4845-b395-a7ee1fedcfe7-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "915e2010-9827-4845-b395-a7ee1fedcfe7" (UID: "915e2010-9827-4845-b395-a7ee1fedcfe7"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 16:22:55.096888 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:55.096853    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/915e2010-9827-4845-b395-a7ee1fedcfe7-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "915e2010-9827-4845-b395-a7ee1fedcfe7" (UID: "915e2010-9827-4845-b395-a7ee1fedcfe7"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 16:22:55.097131 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:55.097097    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/915e2010-9827-4845-b395-a7ee1fedcfe7-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "915e2010-9827-4845-b395-a7ee1fedcfe7" (UID: "915e2010-9827-4845-b395-a7ee1fedcfe7"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 16:22:55.097589 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:55.097562    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/915e2010-9827-4845-b395-a7ee1fedcfe7-kube-api-access-59ccc" (OuterVolumeSpecName: "kube-api-access-59ccc") pod "915e2010-9827-4845-b395-a7ee1fedcfe7" (UID: "915e2010-9827-4845-b395-a7ee1fedcfe7"). InnerVolumeSpecName "kube-api-access-59ccc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 16:22:55.104068 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:55.104045    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/915e2010-9827-4845-b395-a7ee1fedcfe7-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "915e2010-9827-4845-b395-a7ee1fedcfe7" (UID: "915e2010-9827-4845-b395-a7ee1fedcfe7"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 16:22:55.192372 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:55.192338    2557 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/915e2010-9827-4845-b395-a7ee1fedcfe7-installation-pull-secrets\") on node \"ip-10-0-140-154.ec2.internal\" DevicePath \"\""
Jul 28 16:22:55.192372 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:55.192374    2557 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/915e2010-9827-4845-b395-a7ee1fedcfe7-registry-tls\") on node \"ip-10-0-140-154.ec2.internal\" DevicePath \"\""
Jul 28 16:22:55.192588 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:55.192389    2557 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/915e2010-9827-4845-b395-a7ee1fedcfe7-registry-certificates\") on node \"ip-10-0-140-154.ec2.internal\" DevicePath \"\""
Jul 28 16:22:55.192588 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:55.192400    2557 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/915e2010-9827-4845-b395-a7ee1fedcfe7-trusted-ca\") on node \"ip-10-0-140-154.ec2.internal\" DevicePath \"\""
Jul 28 16:22:55.192588 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:55.192410    2557 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/915e2010-9827-4845-b395-a7ee1fedcfe7-image-registry-private-configuration\") on node \"ip-10-0-140-154.ec2.internal\" DevicePath \"\""
Jul 28 16:22:55.192588 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:55.192419    2557 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/915e2010-9827-4845-b395-a7ee1fedcfe7-bound-sa-token\") on node \"ip-10-0-140-154.ec2.internal\" DevicePath \"\""
Jul 28 16:22:55.192588 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:55.192427    2557 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/915e2010-9827-4845-b395-a7ee1fedcfe7-ca-trust-extracted\") on node \"ip-10-0-140-154.ec2.internal\" DevicePath \"\""
Jul 28 16:22:55.192588 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:55.192435    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-59ccc\" (UniqueName: \"kubernetes.io/projected/915e2010-9827-4845-b395-a7ee1fedcfe7-kube-api-access-59ccc\") on node \"ip-10-0-140-154.ec2.internal\" DevicePath \"\""
Jul 28 16:22:55.911004 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:55.910973    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6cf9d7745c-2f4rl"
Jul 28 16:22:55.911457 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:55.911004    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6cf9d7745c-2f4rl" event={"ID":"915e2010-9827-4845-b395-a7ee1fedcfe7","Type":"ContainerDied","Data":"fe2224a5e380cd33c6d2a26afc6d7bbdfa97ae9de9af334678b46b747abbf9b7"}
Jul 28 16:22:55.911457 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:55.911067    2557 scope.go:117] "RemoveContainer" containerID="9d12be9b0cbc42c7312b0a10a6fff9043dec7bd209fe0f2d38a8b10efd218a6e"
Jul 28 16:22:55.936022 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:55.935990    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6cf9d7745c-2f4rl"]
Jul 28 16:22:55.939322 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:55.939296    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-6cf9d7745c-2f4rl"]
Jul 28 16:22:56.919064 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:56.919025    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66f76ff476-wb8td" event={"ID":"03d0cb65-9f7e-49d7-8d1d-715b986a0bff","Type":"ContainerStarted","Data":"4997b464d39daf73ac81f0a25b58d2c14bde004b2761c287ff4fcd72fc28f1b1"}
Jul 28 16:22:56.936996 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:56.936937    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-66f76ff476-wb8td" podStartSLOduration=1.647401301 podStartE2EDuration="4.936917206s" totalImagesPullingTime="3.289515905s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 16:22:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:22:56.935275866 +0000 UTC m=+94.073745294" watchObservedRunningTime="2026-07-28 16:22:56.936917206 +0000 UTC m=+94.075386622"
Jul 28 16:22:57.434073 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:57.434034    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="915e2010-9827-4845-b395-a7ee1fedcfe7" path="/var/lib/kubelet/pods/915e2010-9827-4845-b395-a7ee1fedcfe7/volumes"
Jul 28 16:22:58.196922 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:58.196888    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:22:58.257160 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:22:58.257130    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:23:01.386152 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:23:01.386118    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-675796f599-dfngp"]
Jul 28 16:23:01.391003 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:23:01.390974    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="915e2010-9827-4845-b395-a7ee1fedcfe7" containerName="registry"
Jul 28 16:23:01.391181 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:23:01.391008    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="915e2010-9827-4845-b395-a7ee1fedcfe7" containerName="registry"
Jul 28 16:23:01.391316 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:23:01.391292    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="915e2010-9827-4845-b395-a7ee1fedcfe7" containerName="registry"
Jul 28 16:23:01.543539 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:23:01.543484    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-675796f599-dfngp"]
Jul 28 16:23:01.543732 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:23:01.543642    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-675796f599-dfngp"
Jul 28 16:23:01.551820 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:23:01.551757    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 28 16:23:01.649604 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:23:01.649522    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4ca992e9-b957-4c69-b3e3-78487b3f9d2e-console-config\") pod \"console-675796f599-dfngp\" (UID: \"4ca992e9-b957-4c69-b3e3-78487b3f9d2e\") " pod="openshift-console/console-675796f599-dfngp"
Jul 28 16:23:01.649604 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:23:01.649594    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4ca992e9-b957-4c69-b3e3-78487b3f9d2e-oauth-serving-cert\") pod \"console-675796f599-dfngp\" (UID: \"4ca992e9-b957-4c69-b3e3-78487b3f9d2e\") " pod="openshift-console/console-675796f599-dfngp"
Jul 28 16:23:01.649800 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:23:01.649618    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-llbwl\" (UniqueName: \"kubernetes.io/projected/4ca992e9-b957-4c69-b3e3-78487b3f9d2e-kube-api-access-llbwl\") pod \"console-675796f599-dfngp\" (UID: \"4ca992e9-b957-4c69-b3e3-78487b3f9d2e\") " pod="openshift-console/console-675796f599-dfngp"
Jul 28 16:23:01.649800 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:23:01.649662    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4ca992e9-b957-4c69-b3e3-78487b3f9d2e-console-oauth-config\") pod \"console-675796f599-dfngp\" (UID: \"4ca992e9-b957-4c69-b3e3-78487b3f9d2e\") " pod="openshift-console/console-675796f599-dfngp"
Jul 28 16:23:01.649800 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:23:01.649736    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4ca992e9-b957-4c69-b3e3-78487b3f9d2e-console-serving-cert\") pod \"console-675796f599-dfngp\" (UID: \"4ca992e9-b957-4c69-b3e3-78487b3f9d2e\") " pod="openshift-console/console-675796f599-dfngp"
Jul 28 16:23:01.649800 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:23:01.649766    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4ca992e9-b957-4c69-b3e3-78487b3f9d2e-trusted-ca-bundle\") pod \"console-675796f599-dfngp\" (UID: \"4ca992e9-b957-4c69-b3e3-78487b3f9d2e\") " pod="openshift-console/console-675796f599-dfngp"
Jul 28 16:23:01.649964 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:23:01.649809    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4ca992e9-b957-4c69-b3e3-78487b3f9d2e-service-ca\") pod \"console-675796f599-dfngp\" (UID: \"4ca992e9-b957-4c69-b3e3-78487b3f9d2e\") " pod="openshift-console/console-675796f599-dfngp"
Jul 28 16:23:01.750226 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:23:01.750190    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4ca992e9-b957-4c69-b3e3-78487b3f9d2e-trusted-ca-bundle\") pod \"console-675796f599-dfngp\" (UID: \"4ca992e9-b957-4c69-b3e3-78487b3f9d2e\") " pod="openshift-console/console-675796f599-dfngp"
Jul 28 16:23:01.750372 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:23:01.750248    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4ca992e9-b957-4c69-b3e3-78487b3f9d2e-service-ca\") pod \"console-675796f599-dfngp\" (UID: \"4ca992e9-b957-4c69-b3e3-78487b3f9d2e\") " pod="openshift-console/console-675796f599-dfngp"
Jul 28 16:23:01.750372 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:23:01.750293    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4ca992e9-b957-4c69-b3e3-78487b3f9d2e-console-config\") pod \"console-675796f599-dfngp\" (UID: \"4ca992e9-b957-4c69-b3e3-78487b3f9d2e\") " pod="openshift-console/console-675796f599-dfngp"
Jul 28 16:23:01.750372 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:23:01.750332    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4ca992e9-b957-4c69-b3e3-78487b3f9d2e-oauth-serving-cert\") pod \"console-675796f599-dfngp\" (UID: \"4ca992e9-b957-4c69-b3e3-78487b3f9d2e\") " pod="openshift-console/console-675796f599-dfngp"
Jul 28 16:23:01.750372 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:23:01.750350    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-llbwl\" (UniqueName: \"kubernetes.io/projected/4ca992e9-b957-4c69-b3e3-78487b3f9d2e-kube-api-access-llbwl\") pod \"console-675796f599-dfngp\" (UID: \"4ca992e9-b957-4c69-b3e3-78487b3f9d2e\") " pod="openshift-console/console-675796f599-dfngp"
Jul 28 16:23:01.750607 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:23:01.750384    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4ca992e9-b957-4c69-b3e3-78487b3f9d2e-console-oauth-config\") pod \"console-675796f599-dfngp\" (UID: \"4ca992e9-b957-4c69-b3e3-78487b3f9d2e\") " pod="openshift-console/console-675796f599-dfngp"
Jul 28 16:23:01.750607 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:23:01.750432    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4ca992e9-b957-4c69-b3e3-78487b3f9d2e-console-serving-cert\") pod \"console-675796f599-dfngp\" (UID: \"4ca992e9-b957-4c69-b3e3-78487b3f9d2e\") " pod="openshift-console/console-675796f599-dfngp"
Jul 28 16:23:01.751482 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:23:01.751053    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4ca992e9-b957-4c69-b3e3-78487b3f9d2e-service-ca\") pod \"console-675796f599-dfngp\" (UID: \"4ca992e9-b957-4c69-b3e3-78487b3f9d2e\") " pod="openshift-console/console-675796f599-dfngp"
Jul 28 16:23:01.751482 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:23:01.751172    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4ca992e9-b957-4c69-b3e3-78487b3f9d2e-trusted-ca-bundle\") pod \"console-675796f599-dfngp\" (UID: \"4ca992e9-b957-4c69-b3e3-78487b3f9d2e\") " pod="openshift-console/console-675796f599-dfngp"
Jul 28 16:23:01.751482 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:23:01.751252    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4ca992e9-b957-4c69-b3e3-78487b3f9d2e-console-config\") pod \"console-675796f599-dfngp\" (UID: \"4ca992e9-b957-4c69-b3e3-78487b3f9d2e\") " pod="openshift-console/console-675796f599-dfngp"
Jul 28 16:23:01.751482 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:23:01.751430    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4ca992e9-b957-4c69-b3e3-78487b3f9d2e-oauth-serving-cert\") pod \"console-675796f599-dfngp\" (UID: \"4ca992e9-b957-4c69-b3e3-78487b3f9d2e\") " pod="openshift-console/console-675796f599-dfngp"
Jul 28 16:23:01.753254 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:23:01.753229    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4ca992e9-b957-4c69-b3e3-78487b3f9d2e-console-serving-cert\") pod \"console-675796f599-dfngp\" (UID: \"4ca992e9-b957-4c69-b3e3-78487b3f9d2e\") " pod="openshift-console/console-675796f599-dfngp"
Jul 28 16:23:01.753457 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:23:01.753430    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4ca992e9-b957-4c69-b3e3-78487b3f9d2e-console-oauth-config\") pod \"console-675796f599-dfngp\" (UID: \"4ca992e9-b957-4c69-b3e3-78487b3f9d2e\") " pod="openshift-console/console-675796f599-dfngp"
Jul 28 16:23:01.759231 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:23:01.759209    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-llbwl\" (UniqueName: \"kubernetes.io/projected/4ca992e9-b957-4c69-b3e3-78487b3f9d2e-kube-api-access-llbwl\") pod \"console-675796f599-dfngp\" (UID: \"4ca992e9-b957-4c69-b3e3-78487b3f9d2e\") " pod="openshift-console/console-675796f599-dfngp"
Jul 28 16:23:01.856074 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:23:01.855909    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-675796f599-dfngp"
Jul 28 16:23:01.926304 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:23:01.926239    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-7c5674d886-pqfzc"
Jul 28 16:23:01.996258 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:23:01.996212    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:23:02.096016 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:23:02.095973    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:23:02.196540 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:23:02.196442    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:23:03.327308 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:23:03.327280    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-66f76ff476-wb8td"
Jul 28 16:23:03.327789 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:23:03.327337    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-66f76ff476-wb8td"
Jul 28 16:23:03.332969 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:23:03.332946    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-66f76ff476-wb8td"
Jul 28 16:23:03.954606 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:23:03.954580    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-66f76ff476-wb8td"
Jul 28 16:23:04.716876 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:23:04.716851    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-675796f599-dfngp"]
Jul 28 16:23:04.718986 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:23:04.718961    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ca992e9_b957_4c69_b3e3_78487b3f9d2e.slice/crio-cd849c26fe7009a7415830453eaa0af137a789e26f229559b7085d3a7c40710a WatchSource:0}: Error finding container cd849c26fe7009a7415830453eaa0af137a789e26f229559b7085d3a7c40710a: Status 404 returned error can't find the container with id cd849c26fe7009a7415830453eaa0af137a789e26f229559b7085d3a7c40710a
Jul 28 16:23:04.804075 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:23:04.804049    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:23:04.905421 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:23:04.905354    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:23:04.956110 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:23:04.956045    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-675796f599-dfngp" event={"ID":"4ca992e9-b957-4c69-b3e3-78487b3f9d2e","Type":"ContainerStarted","Data":"6274a87e86dd59be4bb51dde99485a9da5bfcb7ec10b564301ffcf1af88503e8"}
Jul 28 16:23:04.956245 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:23:04.956128    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-675796f599-dfngp" event={"ID":"4ca992e9-b957-4c69-b3e3-78487b3f9d2e","Type":"ContainerStarted","Data":"cd849c26fe7009a7415830453eaa0af137a789e26f229559b7085d3a7c40710a"}
Jul 28 16:23:04.974504 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:23:04.974456    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-675796f599-dfngp" podStartSLOduration=3.974440605 podStartE2EDuration="3.974440605s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 16:23:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:23:04.973199909 +0000 UTC m=+102.111669324" watchObservedRunningTime="2026-07-28 16:23:04.974440605 +0000 UTC m=+102.112910081"
Jul 28 16:23:05.962594 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:23:05.962540    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-7c97dc4555-lk8fs" event={"ID":"d5e68bc9-29a9-4ed7-a692-c28fe6ddad08","Type":"ContainerStarted","Data":"3c1cca66bfb6c906c1b3c2aa3e69be98df72d180d1fbeef388d9baab82a0590b"}
Jul 28 16:23:05.963073 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:23:05.962784    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-7c97dc4555-lk8fs"
Jul 28 16:23:05.975621 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:23:05.975592    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-7c97dc4555-lk8fs"
Jul 28 16:23:05.978627 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:23:05.978578    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-7c97dc4555-lk8fs" podStartSLOduration=3.8190551729999997 podStartE2EDuration="22.978561272s" totalImagesPullingTime="19.159506099s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 16:22:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:23:05.978142081 +0000 UTC m=+103.116611519" watchObservedRunningTime="2026-07-28 16:23:05.978561272 +0000 UTC m=+103.117030686"
Jul 28 16:23:08.781239 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:23:08.781196    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-qzkm5"
Jul 28 16:23:09.799603 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:23:09.799562    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:23:09.855429 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:23:09.855386    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:23:10.938208 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:23:10.938169    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:23:11.040285 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:23:11.040260    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:23:11.792021 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:23:11.791989    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:23:11.857015 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:23:11.856982    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-675796f599-dfngp"
Jul 28 16:23:11.857204 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:23:11.857088    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-675796f599-dfngp"
Jul 28 16:23:11.862625 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:23:11.862599    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-675796f599-dfngp"
Jul 28 16:23:11.998399 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:23:11.998364    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-675796f599-dfngp"
Jul 28 16:23:12.055312 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:23:12.055284    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-66f76ff476-wb8td"]
Jul 28 16:23:12.149221 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:23:12.149181    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:23:12.250324 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:23:12.250288    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:23:12.349297 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:23:12.349215    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:23:13.022154 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:23:13.022114    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:23:13.089579 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:23:13.089541    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:23:19.812110 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:23:19.812063    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:23:19.906880 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:23:19.906852    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:23:24.037692 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:23:24.037611    2557 generic.go:356] "Generic (PLEG): container finished" podID="9d3954a2-15d1-4000-9941-6555dc3aabd2" containerID="311e70562fbc859a24e39d30cb426bffaec3e3f41b478d1f4854aecdc07f9af4" exitCode=0
Jul 28 16:23:24.037692 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:23:24.037663    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-vctpt" event={"ID":"9d3954a2-15d1-4000-9941-6555dc3aabd2","Type":"ContainerDied","Data":"311e70562fbc859a24e39d30cb426bffaec3e3f41b478d1f4854aecdc07f9af4"}
Jul 28 16:23:24.038121 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:23:24.037911    2557 scope.go:117] "RemoveContainer" containerID="311e70562fbc859a24e39d30cb426bffaec3e3f41b478d1f4854aecdc07f9af4"
Jul 28 16:23:25.043033 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:23:25.043000    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-vctpt" event={"ID":"9d3954a2-15d1-4000-9941-6555dc3aabd2","Type":"ContainerStarted","Data":"382b9b0f2318cac607359f22401f546fb987d76636416bbb81d4c518bf57db22"}
Jul 28 16:23:34.801753 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:23:34.801711    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:23:34.855739 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:23:34.855705    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:23:37.083174 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:23:37.083128    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-66f76ff476-wb8td" podUID="03d0cb65-9f7e-49d7-8d1d-715b986a0bff" containerName="console" containerID="cri-o://4997b464d39daf73ac81f0a25b58d2c14bde004b2761c287ff4fcd72fc28f1b1" gracePeriod=15
Jul 28 16:23:37.322382 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:23:37.322362    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66f76ff476-wb8td"
Jul 28 16:23:37.433176 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:23:37.433106    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/03d0cb65-9f7e-49d7-8d1d-715b986a0bff-console-serving-cert\") pod \"03d0cb65-9f7e-49d7-8d1d-715b986a0bff\" (UID: \"03d0cb65-9f7e-49d7-8d1d-715b986a0bff\") "
Jul 28 16:23:37.433176 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:23:37.433141    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/03d0cb65-9f7e-49d7-8d1d-715b986a0bff-console-oauth-config\") pod \"03d0cb65-9f7e-49d7-8d1d-715b986a0bff\" (UID: \"03d0cb65-9f7e-49d7-8d1d-715b986a0bff\") "
Jul 28 16:23:37.433370 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:23:37.433192    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/03d0cb65-9f7e-49d7-8d1d-715b986a0bff-console-config\") pod \"03d0cb65-9f7e-49d7-8d1d-715b986a0bff\" (UID: \"03d0cb65-9f7e-49d7-8d1d-715b986a0bff\") "
Jul 28 16:23:37.433370 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:23:37.433217    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/03d0cb65-9f7e-49d7-8d1d-715b986a0bff-service-ca\") pod \"03d0cb65-9f7e-49d7-8d1d-715b986a0bff\" (UID: \"03d0cb65-9f7e-49d7-8d1d-715b986a0bff\") "
Jul 28 16:23:37.433370 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:23:37.433258    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8dcbg\" (UniqueName: \"kubernetes.io/projected/03d0cb65-9f7e-49d7-8d1d-715b986a0bff-kube-api-access-8dcbg\") pod \"03d0cb65-9f7e-49d7-8d1d-715b986a0bff\" (UID: \"03d0cb65-9f7e-49d7-8d1d-715b986a0bff\") "
Jul 28 16:23:37.433370 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:23:37.433300    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/03d0cb65-9f7e-49d7-8d1d-715b986a0bff-oauth-serving-cert\") pod \"03d0cb65-9f7e-49d7-8d1d-715b986a0bff\" (UID: \"03d0cb65-9f7e-49d7-8d1d-715b986a0bff\") "
Jul 28 16:23:37.433745 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:23:37.433717    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/03d0cb65-9f7e-49d7-8d1d-715b986a0bff-console-config" (OuterVolumeSpecName: "console-config") pod "03d0cb65-9f7e-49d7-8d1d-715b986a0bff" (UID: "03d0cb65-9f7e-49d7-8d1d-715b986a0bff"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 16:23:37.433745 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:23:37.433721    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/03d0cb65-9f7e-49d7-8d1d-715b986a0bff-service-ca" (OuterVolumeSpecName: "service-ca") pod "03d0cb65-9f7e-49d7-8d1d-715b986a0bff" (UID: "03d0cb65-9f7e-49d7-8d1d-715b986a0bff"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 16:23:37.433917 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:23:37.433835    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/03d0cb65-9f7e-49d7-8d1d-715b986a0bff-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "03d0cb65-9f7e-49d7-8d1d-715b986a0bff" (UID: "03d0cb65-9f7e-49d7-8d1d-715b986a0bff"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 16:23:37.435661 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:23:37.435641    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/03d0cb65-9f7e-49d7-8d1d-715b986a0bff-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "03d0cb65-9f7e-49d7-8d1d-715b986a0bff" (UID: "03d0cb65-9f7e-49d7-8d1d-715b986a0bff"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 16:23:37.435941 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:23:37.435917    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/03d0cb65-9f7e-49d7-8d1d-715b986a0bff-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "03d0cb65-9f7e-49d7-8d1d-715b986a0bff" (UID: "03d0cb65-9f7e-49d7-8d1d-715b986a0bff"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 16:23:37.435941 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:23:37.435923    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/03d0cb65-9f7e-49d7-8d1d-715b986a0bff-kube-api-access-8dcbg" (OuterVolumeSpecName: "kube-api-access-8dcbg") pod "03d0cb65-9f7e-49d7-8d1d-715b986a0bff" (UID: "03d0cb65-9f7e-49d7-8d1d-715b986a0bff"). InnerVolumeSpecName "kube-api-access-8dcbg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 16:23:37.534798 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:23:37.534765    2557 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/03d0cb65-9f7e-49d7-8d1d-715b986a0bff-oauth-serving-cert\") on node \"ip-10-0-140-154.ec2.internal\" DevicePath \"\""
Jul 28 16:23:37.534798 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:23:37.534798    2557 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/03d0cb65-9f7e-49d7-8d1d-715b986a0bff-console-serving-cert\") on node \"ip-10-0-140-154.ec2.internal\" DevicePath \"\""
Jul 28 16:23:37.534798 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:23:37.534808    2557 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/03d0cb65-9f7e-49d7-8d1d-715b986a0bff-console-oauth-config\") on node \"ip-10-0-140-154.ec2.internal\" DevicePath \"\""
Jul 28 16:23:37.534955 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:23:37.534817    2557 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/03d0cb65-9f7e-49d7-8d1d-715b986a0bff-console-config\") on node \"ip-10-0-140-154.ec2.internal\" DevicePath \"\""
Jul 28 16:23:37.534955 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:23:37.534826    2557 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/03d0cb65-9f7e-49d7-8d1d-715b986a0bff-service-ca\") on node \"ip-10-0-140-154.ec2.internal\" DevicePath \"\""
Jul 28 16:23:37.534955 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:23:37.534833    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8dcbg\" (UniqueName: \"kubernetes.io/projected/03d0cb65-9f7e-49d7-8d1d-715b986a0bff-kube-api-access-8dcbg\") on node \"ip-10-0-140-154.ec2.internal\" DevicePath \"\""
Jul 28 16:23:38.095012 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:23:38.094982    2557 generic.go:356] "Generic (PLEG): container finished" podID="03d0cb65-9f7e-49d7-8d1d-715b986a0bff" containerID="4997b464d39daf73ac81f0a25b58d2c14bde004b2761c287ff4fcd72fc28f1b1" exitCode=0
Jul 28 16:23:38.095442 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:23:38.095046    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66f76ff476-wb8td"
Jul 28 16:23:38.095442 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:23:38.095078    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66f76ff476-wb8td" event={"ID":"03d0cb65-9f7e-49d7-8d1d-715b986a0bff","Type":"ContainerDied","Data":"4997b464d39daf73ac81f0a25b58d2c14bde004b2761c287ff4fcd72fc28f1b1"}
Jul 28 16:23:38.095442 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:23:38.095170    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66f76ff476-wb8td" event={"ID":"03d0cb65-9f7e-49d7-8d1d-715b986a0bff","Type":"ContainerDied","Data":"e745c48ebba7258cd2ddb92c9c4423068a0695f2542917e5e00a2bc82bff8531"}
Jul 28 16:23:38.095442 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:23:38.095199    2557 scope.go:117] "RemoveContainer" containerID="4997b464d39daf73ac81f0a25b58d2c14bde004b2761c287ff4fcd72fc28f1b1"
Jul 28 16:23:38.103887 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:23:38.103871    2557 scope.go:117] "RemoveContainer" containerID="4997b464d39daf73ac81f0a25b58d2c14bde004b2761c287ff4fcd72fc28f1b1"
Jul 28 16:23:38.104141 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:23:38.104122    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4997b464d39daf73ac81f0a25b58d2c14bde004b2761c287ff4fcd72fc28f1b1\": container with ID starting with 4997b464d39daf73ac81f0a25b58d2c14bde004b2761c287ff4fcd72fc28f1b1 not found: ID does not exist" containerID="4997b464d39daf73ac81f0a25b58d2c14bde004b2761c287ff4fcd72fc28f1b1"
Jul 28 16:23:38.104194 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:23:38.104150    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4997b464d39daf73ac81f0a25b58d2c14bde004b2761c287ff4fcd72fc28f1b1"} err="failed to get container status \"4997b464d39daf73ac81f0a25b58d2c14bde004b2761c287ff4fcd72fc28f1b1\": rpc error: code = NotFound desc = could not find container \"4997b464d39daf73ac81f0a25b58d2c14bde004b2761c287ff4fcd72fc28f1b1\": container with ID starting with 4997b464d39daf73ac81f0a25b58d2c14bde004b2761c287ff4fcd72fc28f1b1 not found: ID does not exist"
Jul 28 16:23:38.132248 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:23:38.132227    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-66f76ff476-wb8td"]
Jul 28 16:23:38.146860 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:23:38.146839    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-66f76ff476-wb8td"]
Jul 28 16:23:39.432210 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:23:39.432176    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="03d0cb65-9f7e-49d7-8d1d-715b986a0bff" path="/var/lib/kubelet/pods/03d0cb65-9f7e-49d7-8d1d-715b986a0bff/volumes"
Jul 28 16:24:03.367088 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:24:03.367045    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:24:03.682474 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:24:03.682396    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:24:08.498812 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:24:08.498775    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-77f859f47c-td52t"]
Jul 28 16:24:08.499300 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:24:08.499103    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="03d0cb65-9f7e-49d7-8d1d-715b986a0bff" containerName="console"
Jul 28 16:24:08.499300 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:24:08.499115    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="03d0cb65-9f7e-49d7-8d1d-715b986a0bff" containerName="console"
Jul 28 16:24:08.499300 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:24:08.499167    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="03d0cb65-9f7e-49d7-8d1d-715b986a0bff" containerName="console"
Jul 28 16:24:08.501912 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:24:08.501892    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-77f859f47c-td52t"
Jul 28 16:24:08.504514 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:24:08.504460    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Jul 28 16:24:08.504633 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:24:08.504522    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Jul 28 16:24:08.504633 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:24:08.504625    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Jul 28 16:24:08.504749 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:24:08.504625    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Jul 28 16:24:08.504749 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:24:08.504652    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Jul 28 16:24:08.504921 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:24:08.504908    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-t57bj\""
Jul 28 16:24:08.511423 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:24:08.511397    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Jul 28 16:24:08.516797 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:24:08.516775    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-77f859f47c-td52t"]
Jul 28 16:24:08.556229 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:24:08.556195    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6f414120-ce25-4eac-9aa1-527edd9b92a7-telemeter-trusted-ca-bundle\") pod \"telemeter-client-77f859f47c-td52t\" (UID: \"6f414120-ce25-4eac-9aa1-527edd9b92a7\") " pod="openshift-monitoring/telemeter-client-77f859f47c-td52t"
Jul 28 16:24:08.556398 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:24:08.556254    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/6f414120-ce25-4eac-9aa1-527edd9b92a7-telemeter-client-tls\") pod \"telemeter-client-77f859f47c-td52t\" (UID: \"6f414120-ce25-4eac-9aa1-527edd9b92a7\") " pod="openshift-monitoring/telemeter-client-77f859f47c-td52t"
Jul 28 16:24:08.556398 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:24:08.556310    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6f414120-ce25-4eac-9aa1-527edd9b92a7-metrics-client-ca\") pod \"telemeter-client-77f859f47c-td52t\" (UID: \"6f414120-ce25-4eac-9aa1-527edd9b92a7\") " pod="openshift-monitoring/telemeter-client-77f859f47c-td52t"
Jul 28 16:24:08.556398 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:24:08.556336    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6wsdz\" (UniqueName: \"kubernetes.io/projected/6f414120-ce25-4eac-9aa1-527edd9b92a7-kube-api-access-6wsdz\") pod \"telemeter-client-77f859f47c-td52t\" (UID: \"6f414120-ce25-4eac-9aa1-527edd9b92a7\") " pod="openshift-monitoring/telemeter-client-77f859f47c-td52t"
Jul 28 16:24:08.556616 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:24:08.556412    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/6f414120-ce25-4eac-9aa1-527edd9b92a7-federate-client-tls\") pod \"telemeter-client-77f859f47c-td52t\" (UID: \"6f414120-ce25-4eac-9aa1-527edd9b92a7\") " pod="openshift-monitoring/telemeter-client-77f859f47c-td52t"
Jul 28 16:24:08.556616 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:24:08.556447    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/6f414120-ce25-4eac-9aa1-527edd9b92a7-secret-telemeter-client\") pod \"telemeter-client-77f859f47c-td52t\" (UID: \"6f414120-ce25-4eac-9aa1-527edd9b92a7\") " pod="openshift-monitoring/telemeter-client-77f859f47c-td52t"
Jul 28 16:24:08.556616 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:24:08.556477    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6f414120-ce25-4eac-9aa1-527edd9b92a7-serving-certs-ca-bundle\") pod \"telemeter-client-77f859f47c-td52t\" (UID: \"6f414120-ce25-4eac-9aa1-527edd9b92a7\") " pod="openshift-monitoring/telemeter-client-77f859f47c-td52t"
Jul 28 16:24:08.556616 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:24:08.556563    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6f414120-ce25-4eac-9aa1-527edd9b92a7-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-77f859f47c-td52t\" (UID: \"6f414120-ce25-4eac-9aa1-527edd9b92a7\") " pod="openshift-monitoring/telemeter-client-77f859f47c-td52t"
Jul 28 16:24:08.657758 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:24:08.657716    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/6f414120-ce25-4eac-9aa1-527edd9b92a7-telemeter-client-tls\") pod \"telemeter-client-77f859f47c-td52t\" (UID: \"6f414120-ce25-4eac-9aa1-527edd9b92a7\") " pod="openshift-monitoring/telemeter-client-77f859f47c-td52t"
Jul 28 16:24:08.657952 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:24:08.657772    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6f414120-ce25-4eac-9aa1-527edd9b92a7-metrics-client-ca\") pod \"telemeter-client-77f859f47c-td52t\" (UID: \"6f414120-ce25-4eac-9aa1-527edd9b92a7\") " pod="openshift-monitoring/telemeter-client-77f859f47c-td52t"
Jul 28 16:24:08.657952 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:24:08.657807    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6wsdz\" (UniqueName: \"kubernetes.io/projected/6f414120-ce25-4eac-9aa1-527edd9b92a7-kube-api-access-6wsdz\") pod \"telemeter-client-77f859f47c-td52t\" (UID: \"6f414120-ce25-4eac-9aa1-527edd9b92a7\") " pod="openshift-monitoring/telemeter-client-77f859f47c-td52t"
Jul 28 16:24:08.657952 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:24:08.657830    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/6f414120-ce25-4eac-9aa1-527edd9b92a7-federate-client-tls\") pod \"telemeter-client-77f859f47c-td52t\" (UID: \"6f414120-ce25-4eac-9aa1-527edd9b92a7\") " pod="openshift-monitoring/telemeter-client-77f859f47c-td52t"
Jul 28 16:24:08.657952 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:24:08.657852    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/6f414120-ce25-4eac-9aa1-527edd9b92a7-secret-telemeter-client\") pod \"telemeter-client-77f859f47c-td52t\" (UID: \"6f414120-ce25-4eac-9aa1-527edd9b92a7\") " pod="openshift-monitoring/telemeter-client-77f859f47c-td52t"
Jul 28 16:24:08.657952 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:24:08.657880    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6f414120-ce25-4eac-9aa1-527edd9b92a7-serving-certs-ca-bundle\") pod \"telemeter-client-77f859f47c-td52t\" (UID: \"6f414120-ce25-4eac-9aa1-527edd9b92a7\") " pod="openshift-monitoring/telemeter-client-77f859f47c-td52t"
Jul 28 16:24:08.657952 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:24:08.657911    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6f414120-ce25-4eac-9aa1-527edd9b92a7-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-77f859f47c-td52t\" (UID: \"6f414120-ce25-4eac-9aa1-527edd9b92a7\") " pod="openshift-monitoring/telemeter-client-77f859f47c-td52t"
Jul 28 16:24:08.657952 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:24:08.657944    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6f414120-ce25-4eac-9aa1-527edd9b92a7-telemeter-trusted-ca-bundle\") pod \"telemeter-client-77f859f47c-td52t\" (UID: \"6f414120-ce25-4eac-9aa1-527edd9b92a7\") " pod="openshift-monitoring/telemeter-client-77f859f47c-td52t"
Jul 28 16:24:08.658886 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:24:08.658847    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6f414120-ce25-4eac-9aa1-527edd9b92a7-metrics-client-ca\") pod \"telemeter-client-77f859f47c-td52t\" (UID: \"6f414120-ce25-4eac-9aa1-527edd9b92a7\") " pod="openshift-monitoring/telemeter-client-77f859f47c-td52t"
Jul 28 16:24:08.659002 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:24:08.658959    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6f414120-ce25-4eac-9aa1-527edd9b92a7-telemeter-trusted-ca-bundle\") pod \"telemeter-client-77f859f47c-td52t\" (UID: \"6f414120-ce25-4eac-9aa1-527edd9b92a7\") " pod="openshift-monitoring/telemeter-client-77f859f47c-td52t"
Jul 28 16:24:08.659002 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:24:08.658969    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6f414120-ce25-4eac-9aa1-527edd9b92a7-serving-certs-ca-bundle\") pod \"telemeter-client-77f859f47c-td52t\" (UID: \"6f414120-ce25-4eac-9aa1-527edd9b92a7\") " pod="openshift-monitoring/telemeter-client-77f859f47c-td52t"
Jul 28 16:24:08.660579 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:24:08.660547    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/6f414120-ce25-4eac-9aa1-527edd9b92a7-telemeter-client-tls\") pod \"telemeter-client-77f859f47c-td52t\" (UID: \"6f414120-ce25-4eac-9aa1-527edd9b92a7\") " pod="openshift-monitoring/telemeter-client-77f859f47c-td52t"
Jul 28 16:24:08.660579 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:24:08.660566    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6f414120-ce25-4eac-9aa1-527edd9b92a7-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-77f859f47c-td52t\" (UID: \"6f414120-ce25-4eac-9aa1-527edd9b92a7\") " pod="openshift-monitoring/telemeter-client-77f859f47c-td52t"
Jul 28 16:24:08.660811 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:24:08.660796    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/6f414120-ce25-4eac-9aa1-527edd9b92a7-federate-client-tls\") pod \"telemeter-client-77f859f47c-td52t\" (UID: \"6f414120-ce25-4eac-9aa1-527edd9b92a7\") " pod="openshift-monitoring/telemeter-client-77f859f47c-td52t"
Jul 28 16:24:08.660941 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:24:08.660921    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/6f414120-ce25-4eac-9aa1-527edd9b92a7-secret-telemeter-client\") pod \"telemeter-client-77f859f47c-td52t\" (UID: \"6f414120-ce25-4eac-9aa1-527edd9b92a7\") " pod="openshift-monitoring/telemeter-client-77f859f47c-td52t"
Jul 28 16:24:08.666335 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:24:08.666314    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6wsdz\" (UniqueName: \"kubernetes.io/projected/6f414120-ce25-4eac-9aa1-527edd9b92a7-kube-api-access-6wsdz\") pod \"telemeter-client-77f859f47c-td52t\" (UID: \"6f414120-ce25-4eac-9aa1-527edd9b92a7\") " pod="openshift-monitoring/telemeter-client-77f859f47c-td52t"
Jul 28 16:24:08.817344 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:24:08.817307    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-77f859f47c-td52t"
Jul 28 16:24:08.954346 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:24:08.954303    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-77f859f47c-td52t"]
Jul 28 16:24:08.959117 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:24:08.959088    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f414120_ce25_4eac_9aa1_527edd9b92a7.slice/crio-5bb95ce0fc5fd059f7f338809357bbc4411b19d28b80af31b379a45c5816713d WatchSource:0}: Error finding container 5bb95ce0fc5fd059f7f338809357bbc4411b19d28b80af31b379a45c5816713d: Status 404 returned error can't find the container with id 5bb95ce0fc5fd059f7f338809357bbc4411b19d28b80af31b379a45c5816713d
Jul 28 16:24:09.209693 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:24:09.209604    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-77f859f47c-td52t" event={"ID":"6f414120-ce25-4eac-9aa1-527edd9b92a7","Type":"ContainerStarted","Data":"5bb95ce0fc5fd059f7f338809357bbc4411b19d28b80af31b379a45c5816713d"}
Jul 28 16:24:11.218307 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:24:11.218273    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-77f859f47c-td52t" event={"ID":"6f414120-ce25-4eac-9aa1-527edd9b92a7","Type":"ContainerStarted","Data":"06932bb4e9a2b7436dda79072cf107d4f620d90b1a438261c5a4778a73c9b82f"}
Jul 28 16:24:12.224011 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:24:12.223927    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-77f859f47c-td52t" event={"ID":"6f414120-ce25-4eac-9aa1-527edd9b92a7","Type":"ContainerStarted","Data":"fa13090afd39620cd9d914859310970a0ac4172e45cb0a5996595c9956302fb0"}
Jul 28 16:24:12.224011 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:24:12.223981    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-77f859f47c-td52t" event={"ID":"6f414120-ce25-4eac-9aa1-527edd9b92a7","Type":"ContainerStarted","Data":"8772133c0331029302f0a2a2fe28b729dab03106b28575d7c8f5d1d3cf98acb4"}
Jul 28 16:24:12.249537 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:24:12.249476    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-77f859f47c-td52t" podStartSLOduration=1.3759483430000001 podStartE2EDuration="4.249458186s" totalImagesPullingTime="2.873509843s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 16:24:08 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:24:12.248374411 +0000 UTC m=+169.386843825" watchObservedRunningTime="2026-07-28 16:24:12.249458186 +0000 UTC m=+169.387927599"
Jul 28 16:24:13.056914 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:24:13.056880    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5dfb8b746c-5j9rd"]
Jul 28 16:24:13.058913 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:24:13.058895    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5dfb8b746c-5j9rd"
Jul 28 16:24:13.071875 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:24:13.071851    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5dfb8b746c-5j9rd"]
Jul 28 16:24:13.096455 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:24:13.096429    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b93a9411-aacf-4c82-a8d2-4ae5e77b2a2a-console-config\") pod \"console-5dfb8b746c-5j9rd\" (UID: \"b93a9411-aacf-4c82-a8d2-4ae5e77b2a2a\") " pod="openshift-console/console-5dfb8b746c-5j9rd"
Jul 28 16:24:13.096600 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:24:13.096466    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b93a9411-aacf-4c82-a8d2-4ae5e77b2a2a-console-oauth-config\") pod \"console-5dfb8b746c-5j9rd\" (UID: \"b93a9411-aacf-4c82-a8d2-4ae5e77b2a2a\") " pod="openshift-console/console-5dfb8b746c-5j9rd"
Jul 28 16:24:13.096600 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:24:13.096489    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b93a9411-aacf-4c82-a8d2-4ae5e77b2a2a-trusted-ca-bundle\") pod \"console-5dfb8b746c-5j9rd\" (UID: \"b93a9411-aacf-4c82-a8d2-4ae5e77b2a2a\") " pod="openshift-console/console-5dfb8b746c-5j9rd"
Jul 28 16:24:13.096600 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:24:13.096571    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b93a9411-aacf-4c82-a8d2-4ae5e77b2a2a-service-ca\") pod \"console-5dfb8b746c-5j9rd\" (UID: \"b93a9411-aacf-4c82-a8d2-4ae5e77b2a2a\") " pod="openshift-console/console-5dfb8b746c-5j9rd"
Jul 28 16:24:13.096702 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:24:13.096598    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b93a9411-aacf-4c82-a8d2-4ae5e77b2a2a-oauth-serving-cert\") pod \"console-5dfb8b746c-5j9rd\" (UID: \"b93a9411-aacf-4c82-a8d2-4ae5e77b2a2a\") " pod="openshift-console/console-5dfb8b746c-5j9rd"
Jul 28 16:24:13.096702 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:24:13.096673    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b93a9411-aacf-4c82-a8d2-4ae5e77b2a2a-console-serving-cert\") pod \"console-5dfb8b746c-5j9rd\" (UID: \"b93a9411-aacf-4c82-a8d2-4ae5e77b2a2a\") " pod="openshift-console/console-5dfb8b746c-5j9rd"
Jul 28 16:24:13.096761 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:24:13.096723    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tqsrv\" (UniqueName: \"kubernetes.io/projected/b93a9411-aacf-4c82-a8d2-4ae5e77b2a2a-kube-api-access-tqsrv\") pod \"console-5dfb8b746c-5j9rd\" (UID: \"b93a9411-aacf-4c82-a8d2-4ae5e77b2a2a\") " pod="openshift-console/console-5dfb8b746c-5j9rd"
Jul 28 16:24:13.108640 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:24:13.108616    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:24:13.197316 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:24:13.197273    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b93a9411-aacf-4c82-a8d2-4ae5e77b2a2a-trusted-ca-bundle\") pod \"console-5dfb8b746c-5j9rd\" (UID: \"b93a9411-aacf-4c82-a8d2-4ae5e77b2a2a\") " pod="openshift-console/console-5dfb8b746c-5j9rd"
Jul 28 16:24:13.197316 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:24:13.197321    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b93a9411-aacf-4c82-a8d2-4ae5e77b2a2a-service-ca\") pod \"console-5dfb8b746c-5j9rd\" (UID: \"b93a9411-aacf-4c82-a8d2-4ae5e77b2a2a\") " pod="openshift-console/console-5dfb8b746c-5j9rd"
Jul 28 16:24:13.197574 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:24:13.197338    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b93a9411-aacf-4c82-a8d2-4ae5e77b2a2a-oauth-serving-cert\") pod \"console-5dfb8b746c-5j9rd\" (UID: \"b93a9411-aacf-4c82-a8d2-4ae5e77b2a2a\") " pod="openshift-console/console-5dfb8b746c-5j9rd"
Jul 28 16:24:13.197574 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:24:13.197362    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b93a9411-aacf-4c82-a8d2-4ae5e77b2a2a-console-serving-cert\") pod \"console-5dfb8b746c-5j9rd\" (UID: \"b93a9411-aacf-4c82-a8d2-4ae5e77b2a2a\") " pod="openshift-console/console-5dfb8b746c-5j9rd"
Jul 28 16:24:13.197574 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:24:13.197406    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tqsrv\" (UniqueName: \"kubernetes.io/projected/b93a9411-aacf-4c82-a8d2-4ae5e77b2a2a-kube-api-access-tqsrv\") pod \"console-5dfb8b746c-5j9rd\" (UID: \"b93a9411-aacf-4c82-a8d2-4ae5e77b2a2a\") " pod="openshift-console/console-5dfb8b746c-5j9rd"
Jul 28 16:24:13.197574 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:24:13.197424    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b93a9411-aacf-4c82-a8d2-4ae5e77b2a2a-console-config\") pod \"console-5dfb8b746c-5j9rd\" (UID: \"b93a9411-aacf-4c82-a8d2-4ae5e77b2a2a\") " pod="openshift-console/console-5dfb8b746c-5j9rd"
Jul 28 16:24:13.197574 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:24:13.197450    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b93a9411-aacf-4c82-a8d2-4ae5e77b2a2a-console-oauth-config\") pod \"console-5dfb8b746c-5j9rd\" (UID: \"b93a9411-aacf-4c82-a8d2-4ae5e77b2a2a\") " pod="openshift-console/console-5dfb8b746c-5j9rd"
Jul 28 16:24:13.198247 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:24:13.198162    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b93a9411-aacf-4c82-a8d2-4ae5e77b2a2a-service-ca\") pod \"console-5dfb8b746c-5j9rd\" (UID: \"b93a9411-aacf-4c82-a8d2-4ae5e77b2a2a\") " pod="openshift-console/console-5dfb8b746c-5j9rd"
Jul 28 16:24:13.198247 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:24:13.198245    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b93a9411-aacf-4c82-a8d2-4ae5e77b2a2a-oauth-serving-cert\") pod \"console-5dfb8b746c-5j9rd\" (UID: \"b93a9411-aacf-4c82-a8d2-4ae5e77b2a2a\") " pod="openshift-console/console-5dfb8b746c-5j9rd"
Jul 28 16:24:13.198572 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:24:13.198324    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b93a9411-aacf-4c82-a8d2-4ae5e77b2a2a-trusted-ca-bundle\") pod \"console-5dfb8b746c-5j9rd\" (UID: \"b93a9411-aacf-4c82-a8d2-4ae5e77b2a2a\") " pod="openshift-console/console-5dfb8b746c-5j9rd"
Jul 28 16:24:13.198647 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:24:13.198611    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b93a9411-aacf-4c82-a8d2-4ae5e77b2a2a-console-config\") pod \"console-5dfb8b746c-5j9rd\" (UID: \"b93a9411-aacf-4c82-a8d2-4ae5e77b2a2a\") " pod="openshift-console/console-5dfb8b746c-5j9rd"
Jul 28 16:24:13.200114 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:24:13.200090    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b93a9411-aacf-4c82-a8d2-4ae5e77b2a2a-console-serving-cert\") pod \"console-5dfb8b746c-5j9rd\" (UID: \"b93a9411-aacf-4c82-a8d2-4ae5e77b2a2a\") " pod="openshift-console/console-5dfb8b746c-5j9rd"
Jul 28 16:24:13.200190 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:24:13.200115    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b93a9411-aacf-4c82-a8d2-4ae5e77b2a2a-console-oauth-config\") pod \"console-5dfb8b746c-5j9rd\" (UID: \"b93a9411-aacf-4c82-a8d2-4ae5e77b2a2a\") " pod="openshift-console/console-5dfb8b746c-5j9rd"
Jul 28 16:24:13.204649 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:24:13.204632    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tqsrv\" (UniqueName: \"kubernetes.io/projected/b93a9411-aacf-4c82-a8d2-4ae5e77b2a2a-kube-api-access-tqsrv\") pod \"console-5dfb8b746c-5j9rd\" (UID: \"b93a9411-aacf-4c82-a8d2-4ae5e77b2a2a\") " pod="openshift-console/console-5dfb8b746c-5j9rd"
Jul 28 16:24:13.373392 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:24:13.373309    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5dfb8b746c-5j9rd"
Jul 28 16:24:13.499337 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:24:13.499298    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5dfb8b746c-5j9rd"]
Jul 28 16:24:13.501452 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:24:13.501427    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb93a9411_aacf_4c82_a8d2_4ae5e77b2a2a.slice/crio-4be89a37ed25d52e353586fd104e84c2ba6aeeae95cec2fe1d36da4b28fa1c67 WatchSource:0}: Error finding container 4be89a37ed25d52e353586fd104e84c2ba6aeeae95cec2fe1d36da4b28fa1c67: Status 404 returned error can't find the container with id 4be89a37ed25d52e353586fd104e84c2ba6aeeae95cec2fe1d36da4b28fa1c67
Jul 28 16:24:14.233163 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:24:14.233088    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5dfb8b746c-5j9rd" event={"ID":"b93a9411-aacf-4c82-a8d2-4ae5e77b2a2a","Type":"ContainerStarted","Data":"6e408e3d5146ed40d5d50cd723e9d794f9832a849a69f1f2f2f6f54814a12b4d"}
Jul 28 16:24:14.233163 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:24:14.233136    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5dfb8b746c-5j9rd" event={"ID":"b93a9411-aacf-4c82-a8d2-4ae5e77b2a2a","Type":"ContainerStarted","Data":"4be89a37ed25d52e353586fd104e84c2ba6aeeae95cec2fe1d36da4b28fa1c67"}
Jul 28 16:24:14.253157 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:24:14.253104    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5dfb8b746c-5j9rd" podStartSLOduration=1.25308927 podStartE2EDuration="1.25308927s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 16:24:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:24:14.251965688 +0000 UTC m=+171.390435103" watchObservedRunningTime="2026-07-28 16:24:14.25308927 +0000 UTC m=+171.391558684"
Jul 28 16:24:20.884437 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:24:20.884401    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:24:23.373693 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:24:23.373657    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5dfb8b746c-5j9rd"
Jul 28 16:24:23.374052 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:24:23.373716    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5dfb8b746c-5j9rd"
Jul 28 16:24:23.378412 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:24:23.378393    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5dfb8b746c-5j9rd"
Jul 28 16:24:24.281734 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:24:24.281706    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5dfb8b746c-5j9rd"
Jul 28 16:24:24.336885 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:24:24.336846    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-675796f599-dfngp"]
Jul 28 16:24:24.452509 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:24:24.452462    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:24:24.553607 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:24:24.553583    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:24:24.653758 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:24:24.653721    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:24:48.105213 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:24:48.105177    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:24:48.716606 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:24:48.716571    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:24:48.775924 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:24:48.775885    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:24:49.361287 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:24:49.361249    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-675796f599-dfngp" podUID="4ca992e9-b957-4c69-b3e3-78487b3f9d2e" containerName="console" containerID="cri-o://6274a87e86dd59be4bb51dde99485a9da5bfcb7ec10b564301ffcf1af88503e8" gracePeriod=15
Jul 28 16:24:49.610381 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:24:49.610359    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-675796f599-dfngp"
Jul 28 16:24:49.694605 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:24:49.694533    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4ca992e9-b957-4c69-b3e3-78487b3f9d2e-console-config\") pod \"4ca992e9-b957-4c69-b3e3-78487b3f9d2e\" (UID: \"4ca992e9-b957-4c69-b3e3-78487b3f9d2e\") "
Jul 28 16:24:49.694605 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:24:49.694564    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4ca992e9-b957-4c69-b3e3-78487b3f9d2e-trusted-ca-bundle\") pod \"4ca992e9-b957-4c69-b3e3-78487b3f9d2e\" (UID: \"4ca992e9-b957-4c69-b3e3-78487b3f9d2e\") "
Jul 28 16:24:49.694817 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:24:49.694611    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4ca992e9-b957-4c69-b3e3-78487b3f9d2e-console-oauth-config\") pod \"4ca992e9-b957-4c69-b3e3-78487b3f9d2e\" (UID: \"4ca992e9-b957-4c69-b3e3-78487b3f9d2e\") "
Jul 28 16:24:49.694817 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:24:49.694640    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4ca992e9-b957-4c69-b3e3-78487b3f9d2e-console-serving-cert\") pod \"4ca992e9-b957-4c69-b3e3-78487b3f9d2e\" (UID: \"4ca992e9-b957-4c69-b3e3-78487b3f9d2e\") "
Jul 28 16:24:49.694817 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:24:49.694680    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4ca992e9-b957-4c69-b3e3-78487b3f9d2e-service-ca\") pod \"4ca992e9-b957-4c69-b3e3-78487b3f9d2e\" (UID: \"4ca992e9-b957-4c69-b3e3-78487b3f9d2e\") "
Jul 28 16:24:49.694817 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:24:49.694712    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4ca992e9-b957-4c69-b3e3-78487b3f9d2e-oauth-serving-cert\") pod \"4ca992e9-b957-4c69-b3e3-78487b3f9d2e\" (UID: \"4ca992e9-b957-4c69-b3e3-78487b3f9d2e\") "
Jul 28 16:24:49.694817 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:24:49.694753    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-llbwl\" (UniqueName: \"kubernetes.io/projected/4ca992e9-b957-4c69-b3e3-78487b3f9d2e-kube-api-access-llbwl\") pod \"4ca992e9-b957-4c69-b3e3-78487b3f9d2e\" (UID: \"4ca992e9-b957-4c69-b3e3-78487b3f9d2e\") "
Jul 28 16:24:49.695079 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:24:49.695040    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4ca992e9-b957-4c69-b3e3-78487b3f9d2e-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "4ca992e9-b957-4c69-b3e3-78487b3f9d2e" (UID: "4ca992e9-b957-4c69-b3e3-78487b3f9d2e"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 16:24:49.695079 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:24:49.695053    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4ca992e9-b957-4c69-b3e3-78487b3f9d2e-console-config" (OuterVolumeSpecName: "console-config") pod "4ca992e9-b957-4c69-b3e3-78487b3f9d2e" (UID: "4ca992e9-b957-4c69-b3e3-78487b3f9d2e"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 16:24:49.695251 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:24:49.695132    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4ca992e9-b957-4c69-b3e3-78487b3f9d2e-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "4ca992e9-b957-4c69-b3e3-78487b3f9d2e" (UID: "4ca992e9-b957-4c69-b3e3-78487b3f9d2e"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 16:24:49.695251 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:24:49.695136    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4ca992e9-b957-4c69-b3e3-78487b3f9d2e-service-ca" (OuterVolumeSpecName: "service-ca") pod "4ca992e9-b957-4c69-b3e3-78487b3f9d2e" (UID: "4ca992e9-b957-4c69-b3e3-78487b3f9d2e"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 16:24:49.696980 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:24:49.696953    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4ca992e9-b957-4c69-b3e3-78487b3f9d2e-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "4ca992e9-b957-4c69-b3e3-78487b3f9d2e" (UID: "4ca992e9-b957-4c69-b3e3-78487b3f9d2e"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 16:24:49.697065 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:24:49.697009    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4ca992e9-b957-4c69-b3e3-78487b3f9d2e-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "4ca992e9-b957-4c69-b3e3-78487b3f9d2e" (UID: "4ca992e9-b957-4c69-b3e3-78487b3f9d2e"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 16:24:49.697150 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:24:49.697130    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4ca992e9-b957-4c69-b3e3-78487b3f9d2e-kube-api-access-llbwl" (OuterVolumeSpecName: "kube-api-access-llbwl") pod "4ca992e9-b957-4c69-b3e3-78487b3f9d2e" (UID: "4ca992e9-b957-4c69-b3e3-78487b3f9d2e"). InnerVolumeSpecName "kube-api-access-llbwl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 16:24:49.795749 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:24:49.795726    2557 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4ca992e9-b957-4c69-b3e3-78487b3f9d2e-console-config\") on node \"ip-10-0-140-154.ec2.internal\" DevicePath \"\""
Jul 28 16:24:49.795749 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:24:49.795746    2557 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4ca992e9-b957-4c69-b3e3-78487b3f9d2e-trusted-ca-bundle\") on node \"ip-10-0-140-154.ec2.internal\" DevicePath \"\""
Jul 28 16:24:49.795891 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:24:49.795755    2557 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4ca992e9-b957-4c69-b3e3-78487b3f9d2e-console-oauth-config\") on node \"ip-10-0-140-154.ec2.internal\" DevicePath \"\""
Jul 28 16:24:49.795891 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:24:49.795764    2557 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4ca992e9-b957-4c69-b3e3-78487b3f9d2e-console-serving-cert\") on node \"ip-10-0-140-154.ec2.internal\" DevicePath \"\""
Jul 28 16:24:49.795891 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:24:49.795772    2557 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4ca992e9-b957-4c69-b3e3-78487b3f9d2e-service-ca\") on node \"ip-10-0-140-154.ec2.internal\" DevicePath \"\""
Jul 28 16:24:49.795891 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:24:49.795780    2557 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4ca992e9-b957-4c69-b3e3-78487b3f9d2e-oauth-serving-cert\") on node \"ip-10-0-140-154.ec2.internal\" DevicePath \"\""
Jul 28 16:24:49.795891 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:24:49.795787    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-llbwl\" (UniqueName: \"kubernetes.io/projected/4ca992e9-b957-4c69-b3e3-78487b3f9d2e-kube-api-access-llbwl\") on node \"ip-10-0-140-154.ec2.internal\" DevicePath \"\""
Jul 28 16:24:50.373649 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:24:50.373613    2557 generic.go:356] "Generic (PLEG): container finished" podID="4ca992e9-b957-4c69-b3e3-78487b3f9d2e" containerID="6274a87e86dd59be4bb51dde99485a9da5bfcb7ec10b564301ffcf1af88503e8" exitCode=0
Jul 28 16:24:50.374089 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:24:50.373657    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-675796f599-dfngp"
Jul 28 16:24:50.374089 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:24:50.373662    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-675796f599-dfngp" event={"ID":"4ca992e9-b957-4c69-b3e3-78487b3f9d2e","Type":"ContainerDied","Data":"6274a87e86dd59be4bb51dde99485a9da5bfcb7ec10b564301ffcf1af88503e8"}
Jul 28 16:24:50.374089 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:24:50.373714    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-675796f599-dfngp" event={"ID":"4ca992e9-b957-4c69-b3e3-78487b3f9d2e","Type":"ContainerDied","Data":"cd849c26fe7009a7415830453eaa0af137a789e26f229559b7085d3a7c40710a"}
Jul 28 16:24:50.374089 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:24:50.373743    2557 scope.go:117] "RemoveContainer" containerID="6274a87e86dd59be4bb51dde99485a9da5bfcb7ec10b564301ffcf1af88503e8"
Jul 28 16:24:50.382749 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:24:50.382733    2557 scope.go:117] "RemoveContainer" containerID="6274a87e86dd59be4bb51dde99485a9da5bfcb7ec10b564301ffcf1af88503e8"
Jul 28 16:24:50.382991 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:24:50.382973    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6274a87e86dd59be4bb51dde99485a9da5bfcb7ec10b564301ffcf1af88503e8\": container with ID starting with 6274a87e86dd59be4bb51dde99485a9da5bfcb7ec10b564301ffcf1af88503e8 not found: ID does not exist" containerID="6274a87e86dd59be4bb51dde99485a9da5bfcb7ec10b564301ffcf1af88503e8"
Jul 28 16:24:50.383039 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:24:50.382999    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6274a87e86dd59be4bb51dde99485a9da5bfcb7ec10b564301ffcf1af88503e8"} err="failed to get container status \"6274a87e86dd59be4bb51dde99485a9da5bfcb7ec10b564301ffcf1af88503e8\": rpc error: code = NotFound desc = could not find container \"6274a87e86dd59be4bb51dde99485a9da5bfcb7ec10b564301ffcf1af88503e8\": container with ID starting with 6274a87e86dd59be4bb51dde99485a9da5bfcb7ec10b564301ffcf1af88503e8 not found: ID does not exist"
Jul 28 16:24:50.393741 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:24:50.393717    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-675796f599-dfngp"]
Jul 28 16:24:50.397266 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:24:50.397247    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-675796f599-dfngp"]
Jul 28 16:24:51.433165 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:24:51.433131    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4ca992e9-b957-4c69-b3e3-78487b3f9d2e" path="/var/lib/kubelet/pods/4ca992e9-b957-4c69-b3e3-78487b3f9d2e/volumes"
Jul 28 16:25:18.637291 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:25:18.637257    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:25:18.688197 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:25:18.688171    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:25:21.746951 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:25:21.746921    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6c499c9d9d-cbwrq"]
Jul 28 16:25:21.747373 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:25:21.747210    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4ca992e9-b957-4c69-b3e3-78487b3f9d2e" containerName="console"
Jul 28 16:25:21.747373 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:25:21.747220    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="4ca992e9-b957-4c69-b3e3-78487b3f9d2e" containerName="console"
Jul 28 16:25:21.747373 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:25:21.747265    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="4ca992e9-b957-4c69-b3e3-78487b3f9d2e" containerName="console"
Jul 28 16:25:21.750279 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:25:21.750261    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c499c9d9d-cbwrq"
Jul 28 16:25:21.764941 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:25:21.764916    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6c499c9d9d-cbwrq"]
Jul 28 16:25:21.808102 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:25:21.808074    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:25:21.830218 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:25:21.830193    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/882c3721-c8ce-4ded-adfd-1cd2b5606da6-service-ca\") pod \"console-6c499c9d9d-cbwrq\" (UID: \"882c3721-c8ce-4ded-adfd-1cd2b5606da6\") " pod="openshift-console/console-6c499c9d9d-cbwrq"
Jul 28 16:25:21.830313 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:25:21.830222    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/882c3721-c8ce-4ded-adfd-1cd2b5606da6-console-oauth-config\") pod \"console-6c499c9d9d-cbwrq\" (UID: \"882c3721-c8ce-4ded-adfd-1cd2b5606da6\") " pod="openshift-console/console-6c499c9d9d-cbwrq"
Jul 28 16:25:21.830313 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:25:21.830246    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/882c3721-c8ce-4ded-adfd-1cd2b5606da6-oauth-serving-cert\") pod \"console-6c499c9d9d-cbwrq\" (UID: \"882c3721-c8ce-4ded-adfd-1cd2b5606da6\") " pod="openshift-console/console-6c499c9d9d-cbwrq"
Jul 28 16:25:21.830313 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:25:21.830269    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mvsd5\" (UniqueName: \"kubernetes.io/projected/882c3721-c8ce-4ded-adfd-1cd2b5606da6-kube-api-access-mvsd5\") pod \"console-6c499c9d9d-cbwrq\" (UID: \"882c3721-c8ce-4ded-adfd-1cd2b5606da6\") " pod="openshift-console/console-6c499c9d9d-cbwrq"
Jul 28 16:25:21.830313 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:25:21.830293    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/882c3721-c8ce-4ded-adfd-1cd2b5606da6-console-serving-cert\") pod \"console-6c499c9d9d-cbwrq\" (UID: \"882c3721-c8ce-4ded-adfd-1cd2b5606da6\") " pod="openshift-console/console-6c499c9d9d-cbwrq"
Jul 28 16:25:21.830440 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:25:21.830334    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/882c3721-c8ce-4ded-adfd-1cd2b5606da6-trusted-ca-bundle\") pod \"console-6c499c9d9d-cbwrq\" (UID: \"882c3721-c8ce-4ded-adfd-1cd2b5606da6\") " pod="openshift-console/console-6c499c9d9d-cbwrq"
Jul 28 16:25:21.830440 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:25:21.830357    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/882c3721-c8ce-4ded-adfd-1cd2b5606da6-console-config\") pod \"console-6c499c9d9d-cbwrq\" (UID: \"882c3721-c8ce-4ded-adfd-1cd2b5606da6\") " pod="openshift-console/console-6c499c9d9d-cbwrq"
Jul 28 16:25:21.861124 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:25:21.861091    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:25:21.931278 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:25:21.931253    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/882c3721-c8ce-4ded-adfd-1cd2b5606da6-console-config\") pod \"console-6c499c9d9d-cbwrq\" (UID: \"882c3721-c8ce-4ded-adfd-1cd2b5606da6\") " pod="openshift-console/console-6c499c9d9d-cbwrq"
Jul 28 16:25:21.931382 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:25:21.931285    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/882c3721-c8ce-4ded-adfd-1cd2b5606da6-service-ca\") pod \"console-6c499c9d9d-cbwrq\" (UID: \"882c3721-c8ce-4ded-adfd-1cd2b5606da6\") " pod="openshift-console/console-6c499c9d9d-cbwrq"
Jul 28 16:25:21.931382 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:25:21.931307    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/882c3721-c8ce-4ded-adfd-1cd2b5606da6-console-oauth-config\") pod \"console-6c499c9d9d-cbwrq\" (UID: \"882c3721-c8ce-4ded-adfd-1cd2b5606da6\") " pod="openshift-console/console-6c499c9d9d-cbwrq"
Jul 28 16:25:21.931382 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:25:21.931343    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/882c3721-c8ce-4ded-adfd-1cd2b5606da6-oauth-serving-cert\") pod \"console-6c499c9d9d-cbwrq\" (UID: \"882c3721-c8ce-4ded-adfd-1cd2b5606da6\") " pod="openshift-console/console-6c499c9d9d-cbwrq"
Jul 28 16:25:21.931382 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:25:21.931378    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mvsd5\" (UniqueName: \"kubernetes.io/projected/882c3721-c8ce-4ded-adfd-1cd2b5606da6-kube-api-access-mvsd5\") pod \"console-6c499c9d9d-cbwrq\" (UID: \"882c3721-c8ce-4ded-adfd-1cd2b5606da6\") " pod="openshift-console/console-6c499c9d9d-cbwrq"
Jul 28 16:25:21.931581 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:25:21.931409    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/882c3721-c8ce-4ded-adfd-1cd2b5606da6-console-serving-cert\") pod \"console-6c499c9d9d-cbwrq\" (UID: \"882c3721-c8ce-4ded-adfd-1cd2b5606da6\") " pod="openshift-console/console-6c499c9d9d-cbwrq"
Jul 28 16:25:21.931581 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:25:21.931460    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/882c3721-c8ce-4ded-adfd-1cd2b5606da6-trusted-ca-bundle\") pod \"console-6c499c9d9d-cbwrq\" (UID: \"882c3721-c8ce-4ded-adfd-1cd2b5606da6\") " pod="openshift-console/console-6c499c9d9d-cbwrq"
Jul 28 16:25:21.932108 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:25:21.932080    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/882c3721-c8ce-4ded-adfd-1cd2b5606da6-service-ca\") pod \"console-6c499c9d9d-cbwrq\" (UID: \"882c3721-c8ce-4ded-adfd-1cd2b5606da6\") " pod="openshift-console/console-6c499c9d9d-cbwrq"
Jul 28 16:25:21.932377 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:25:21.932217    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/882c3721-c8ce-4ded-adfd-1cd2b5606da6-oauth-serving-cert\") pod \"console-6c499c9d9d-cbwrq\" (UID: \"882c3721-c8ce-4ded-adfd-1cd2b5606da6\") " pod="openshift-console/console-6c499c9d9d-cbwrq"
Jul 28 16:25:21.932377 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:25:21.932237    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/882c3721-c8ce-4ded-adfd-1cd2b5606da6-console-config\") pod \"console-6c499c9d9d-cbwrq\" (UID: \"882c3721-c8ce-4ded-adfd-1cd2b5606da6\") " pod="openshift-console/console-6c499c9d9d-cbwrq"
Jul 28 16:25:21.932659 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:25:21.932635    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/882c3721-c8ce-4ded-adfd-1cd2b5606da6-trusted-ca-bundle\") pod \"console-6c499c9d9d-cbwrq\" (UID: \"882c3721-c8ce-4ded-adfd-1cd2b5606da6\") " pod="openshift-console/console-6c499c9d9d-cbwrq"
Jul 28 16:25:21.934057 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:25:21.934037    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/882c3721-c8ce-4ded-adfd-1cd2b5606da6-console-oauth-config\") pod \"console-6c499c9d9d-cbwrq\" (UID: \"882c3721-c8ce-4ded-adfd-1cd2b5606da6\") " pod="openshift-console/console-6c499c9d9d-cbwrq"
Jul 28 16:25:21.934140 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:25:21.934101    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/882c3721-c8ce-4ded-adfd-1cd2b5606da6-console-serving-cert\") pod \"console-6c499c9d9d-cbwrq\" (UID: \"882c3721-c8ce-4ded-adfd-1cd2b5606da6\") " pod="openshift-console/console-6c499c9d9d-cbwrq"
Jul 28 16:25:21.939098 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:25:21.939082    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mvsd5\" (UniqueName: \"kubernetes.io/projected/882c3721-c8ce-4ded-adfd-1cd2b5606da6-kube-api-access-mvsd5\") pod \"console-6c499c9d9d-cbwrq\" (UID: \"882c3721-c8ce-4ded-adfd-1cd2b5606da6\") " pod="openshift-console/console-6c499c9d9d-cbwrq"
Jul 28 16:25:22.059808 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:25:22.059777    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c499c9d9d-cbwrq"
Jul 28 16:25:22.188048 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:25:22.188022    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6c499c9d9d-cbwrq"]
Jul 28 16:25:22.190536 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:25:22.190506    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod882c3721_c8ce_4ded_adfd_1cd2b5606da6.slice/crio-625928ce9891049196bebe556d165db86e4a5a37c1ab88c0a0a8f15f3c1b869c WatchSource:0}: Error finding container 625928ce9891049196bebe556d165db86e4a5a37c1ab88c0a0a8f15f3c1b869c: Status 404 returned error can't find the container with id 625928ce9891049196bebe556d165db86e4a5a37c1ab88c0a0a8f15f3c1b869c
Jul 28 16:25:22.489593 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:25:22.489509    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c499c9d9d-cbwrq" event={"ID":"882c3721-c8ce-4ded-adfd-1cd2b5606da6","Type":"ContainerStarted","Data":"ff40227249b925ca3a5b5b9213edfcb9ce838fc917f0db8ff37f77a345ea9455"}
Jul 28 16:25:22.489593 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:25:22.489551    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c499c9d9d-cbwrq" event={"ID":"882c3721-c8ce-4ded-adfd-1cd2b5606da6","Type":"ContainerStarted","Data":"625928ce9891049196bebe556d165db86e4a5a37c1ab88c0a0a8f15f3c1b869c"}
Jul 28 16:25:22.508221 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:25:22.508176    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6c499c9d9d-cbwrq" podStartSLOduration=1.5081608100000001 podStartE2EDuration="1.50816081s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 16:25:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:25:22.506568564 +0000 UTC m=+239.645037977" watchObservedRunningTime="2026-07-28 16:25:22.50816081 +0000 UTC m=+239.646630223"
Jul 28 16:25:30.616343 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:25:30.616307    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:25:30.664451 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:25:30.664421    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:25:32.060595 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:25:32.060553    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6c499c9d9d-cbwrq"
Jul 28 16:25:32.061034 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:25:32.060627    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6c499c9d9d-cbwrq"
Jul 28 16:25:32.065084 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:25:32.065064    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6c499c9d9d-cbwrq"
Jul 28 16:25:32.531840 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:25:32.531760    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6c499c9d9d-cbwrq"
Jul 28 16:25:32.582723 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:25:32.582688    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5dfb8b746c-5j9rd"]
Jul 28 16:25:32.689603 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:25:32.689570    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:25:32.739832 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:25:32.739797    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:25:32.796741 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:25:32.796677    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:25:34.803305 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:25:34.803268    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:25:34.900708 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:25:34.900674    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:25:34.952890 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:25:34.952861    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:25:51.782826 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:25:51.782792    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:25:57.601777 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:25:57.601720    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5dfb8b746c-5j9rd" podUID="b93a9411-aacf-4c82-a8d2-4ae5e77b2a2a" containerName="console" containerID="cri-o://6e408e3d5146ed40d5d50cd723e9d794f9832a849a69f1f2f2f6f54814a12b4d" gracePeriod=15
Jul 28 16:25:57.840354 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:25:57.840332    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5dfb8b746c-5j9rd"
Jul 28 16:25:58.006571 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:25:58.006463    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b93a9411-aacf-4c82-a8d2-4ae5e77b2a2a-console-serving-cert\") pod \"b93a9411-aacf-4c82-a8d2-4ae5e77b2a2a\" (UID: \"b93a9411-aacf-4c82-a8d2-4ae5e77b2a2a\") "
Jul 28 16:25:58.006571 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:25:58.006526    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b93a9411-aacf-4c82-a8d2-4ae5e77b2a2a-console-oauth-config\") pod \"b93a9411-aacf-4c82-a8d2-4ae5e77b2a2a\" (UID: \"b93a9411-aacf-4c82-a8d2-4ae5e77b2a2a\") "
Jul 28 16:25:58.006571 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:25:58.006559    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b93a9411-aacf-4c82-a8d2-4ae5e77b2a2a-console-config\") pod \"b93a9411-aacf-4c82-a8d2-4ae5e77b2a2a\" (UID: \"b93a9411-aacf-4c82-a8d2-4ae5e77b2a2a\") "
Jul 28 16:25:58.006841 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:25:58.006725    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b93a9411-aacf-4c82-a8d2-4ae5e77b2a2a-oauth-serving-cert\") pod \"b93a9411-aacf-4c82-a8d2-4ae5e77b2a2a\" (UID: \"b93a9411-aacf-4c82-a8d2-4ae5e77b2a2a\") "
Jul 28 16:25:58.006841 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:25:58.006781    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b93a9411-aacf-4c82-a8d2-4ae5e77b2a2a-trusted-ca-bundle\") pod \"b93a9411-aacf-4c82-a8d2-4ae5e77b2a2a\" (UID: \"b93a9411-aacf-4c82-a8d2-4ae5e77b2a2a\") "
Jul 28 16:25:58.006841 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:25:58.006802    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b93a9411-aacf-4c82-a8d2-4ae5e77b2a2a-service-ca\") pod \"b93a9411-aacf-4c82-a8d2-4ae5e77b2a2a\" (UID: \"b93a9411-aacf-4c82-a8d2-4ae5e77b2a2a\") "
Jul 28 16:25:58.006841 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:25:58.006820    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tqsrv\" (UniqueName: \"kubernetes.io/projected/b93a9411-aacf-4c82-a8d2-4ae5e77b2a2a-kube-api-access-tqsrv\") pod \"b93a9411-aacf-4c82-a8d2-4ae5e77b2a2a\" (UID: \"b93a9411-aacf-4c82-a8d2-4ae5e77b2a2a\") "
Jul 28 16:25:58.007044 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:25:58.007002    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b93a9411-aacf-4c82-a8d2-4ae5e77b2a2a-console-config" (OuterVolumeSpecName: "console-config") pod "b93a9411-aacf-4c82-a8d2-4ae5e77b2a2a" (UID: "b93a9411-aacf-4c82-a8d2-4ae5e77b2a2a"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 16:25:58.007151 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:25:58.007125    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b93a9411-aacf-4c82-a8d2-4ae5e77b2a2a-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "b93a9411-aacf-4c82-a8d2-4ae5e77b2a2a" (UID: "b93a9411-aacf-4c82-a8d2-4ae5e77b2a2a"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 16:25:58.007272 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:25:58.007191    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b93a9411-aacf-4c82-a8d2-4ae5e77b2a2a-service-ca" (OuterVolumeSpecName: "service-ca") pod "b93a9411-aacf-4c82-a8d2-4ae5e77b2a2a" (UID: "b93a9411-aacf-4c82-a8d2-4ae5e77b2a2a"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 16:25:58.007272 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:25:58.007210    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b93a9411-aacf-4c82-a8d2-4ae5e77b2a2a-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "b93a9411-aacf-4c82-a8d2-4ae5e77b2a2a" (UID: "b93a9411-aacf-4c82-a8d2-4ae5e77b2a2a"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 16:25:58.008813 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:25:58.008791    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b93a9411-aacf-4c82-a8d2-4ae5e77b2a2a-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "b93a9411-aacf-4c82-a8d2-4ae5e77b2a2a" (UID: "b93a9411-aacf-4c82-a8d2-4ae5e77b2a2a"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 16:25:58.008940 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:25:58.008919    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b93a9411-aacf-4c82-a8d2-4ae5e77b2a2a-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "b93a9411-aacf-4c82-a8d2-4ae5e77b2a2a" (UID: "b93a9411-aacf-4c82-a8d2-4ae5e77b2a2a"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 16:25:58.009197 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:25:58.009178    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b93a9411-aacf-4c82-a8d2-4ae5e77b2a2a-kube-api-access-tqsrv" (OuterVolumeSpecName: "kube-api-access-tqsrv") pod "b93a9411-aacf-4c82-a8d2-4ae5e77b2a2a" (UID: "b93a9411-aacf-4c82-a8d2-4ae5e77b2a2a"). InnerVolumeSpecName "kube-api-access-tqsrv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 16:25:58.108219 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:25:58.108194    2557 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b93a9411-aacf-4c82-a8d2-4ae5e77b2a2a-console-config\") on node \"ip-10-0-140-154.ec2.internal\" DevicePath \"\""
Jul 28 16:25:58.108219 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:25:58.108215    2557 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b93a9411-aacf-4c82-a8d2-4ae5e77b2a2a-oauth-serving-cert\") on node \"ip-10-0-140-154.ec2.internal\" DevicePath \"\""
Jul 28 16:25:58.108365 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:25:58.108224    2557 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b93a9411-aacf-4c82-a8d2-4ae5e77b2a2a-trusted-ca-bundle\") on node \"ip-10-0-140-154.ec2.internal\" DevicePath \"\""
Jul 28 16:25:58.108365 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:25:58.108233    2557 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b93a9411-aacf-4c82-a8d2-4ae5e77b2a2a-service-ca\") on node \"ip-10-0-140-154.ec2.internal\" DevicePath \"\""
Jul 28 16:25:58.108365 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:25:58.108241    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tqsrv\" (UniqueName: \"kubernetes.io/projected/b93a9411-aacf-4c82-a8d2-4ae5e77b2a2a-kube-api-access-tqsrv\") on node \"ip-10-0-140-154.ec2.internal\" DevicePath \"\""
Jul 28 16:25:58.108365 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:25:58.108250    2557 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b93a9411-aacf-4c82-a8d2-4ae5e77b2a2a-console-serving-cert\") on node \"ip-10-0-140-154.ec2.internal\" DevicePath \"\""
Jul 28 16:25:58.108365 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:25:58.108258    2557 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b93a9411-aacf-4c82-a8d2-4ae5e77b2a2a-console-oauth-config\") on node \"ip-10-0-140-154.ec2.internal\" DevicePath \"\""
Jul 28 16:25:58.621514 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:25:58.621457    2557 generic.go:356] "Generic (PLEG): container finished" podID="b93a9411-aacf-4c82-a8d2-4ae5e77b2a2a" containerID="6e408e3d5146ed40d5d50cd723e9d794f9832a849a69f1f2f2f6f54814a12b4d" exitCode=0
Jul 28 16:25:58.621514 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:25:58.621510    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5dfb8b746c-5j9rd"
Jul 28 16:25:58.621955 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:25:58.621543    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5dfb8b746c-5j9rd" event={"ID":"b93a9411-aacf-4c82-a8d2-4ae5e77b2a2a","Type":"ContainerDied","Data":"6e408e3d5146ed40d5d50cd723e9d794f9832a849a69f1f2f2f6f54814a12b4d"}
Jul 28 16:25:58.621955 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:25:58.621591    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5dfb8b746c-5j9rd" event={"ID":"b93a9411-aacf-4c82-a8d2-4ae5e77b2a2a","Type":"ContainerDied","Data":"4be89a37ed25d52e353586fd104e84c2ba6aeeae95cec2fe1d36da4b28fa1c67"}
Jul 28 16:25:58.621955 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:25:58.621611    2557 scope.go:117] "RemoveContainer" containerID="6e408e3d5146ed40d5d50cd723e9d794f9832a849a69f1f2f2f6f54814a12b4d"
Jul 28 16:25:58.631677 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:25:58.631616    2557 scope.go:117] "RemoveContainer" containerID="6e408e3d5146ed40d5d50cd723e9d794f9832a849a69f1f2f2f6f54814a12b4d"
Jul 28 16:25:58.649688 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:25:58.649644    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5dfb8b746c-5j9rd"]
Jul 28 16:25:58.653986 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:25:58.651164    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6e408e3d5146ed40d5d50cd723e9d794f9832a849a69f1f2f2f6f54814a12b4d\": container with ID starting with 6e408e3d5146ed40d5d50cd723e9d794f9832a849a69f1f2f2f6f54814a12b4d not found: ID does not exist" containerID="6e408e3d5146ed40d5d50cd723e9d794f9832a849a69f1f2f2f6f54814a12b4d"
Jul 28 16:25:58.653986 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:25:58.651196    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6e408e3d5146ed40d5d50cd723e9d794f9832a849a69f1f2f2f6f54814a12b4d"} err="failed to get container status \"6e408e3d5146ed40d5d50cd723e9d794f9832a849a69f1f2f2f6f54814a12b4d\": rpc error: code = NotFound desc = could not find container \"6e408e3d5146ed40d5d50cd723e9d794f9832a849a69f1f2f2f6f54814a12b4d\": container with ID starting with 6e408e3d5146ed40d5d50cd723e9d794f9832a849a69f1f2f2f6f54814a12b4d not found: ID does not exist"
Jul 28 16:25:58.653986 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:25:58.653574    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5dfb8b746c-5j9rd"]
Jul 28 16:25:59.432780 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:25:59.432747    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b93a9411-aacf-4c82-a8d2-4ae5e77b2a2a" path="/var/lib/kubelet/pods/b93a9411-aacf-4c82-a8d2-4ae5e77b2a2a/volumes"
Jul 28 16:26:23.256096 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:26:23.256049    2557 kubelet.go:1648] "Image garbage collection succeeded"
Jul 28 16:27:16.192182 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:27:16.192094    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:27:17.418635 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:27:17.418599    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:27:17.458926 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:27:17.458890    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5ks2t8"]
Jul 28 16:27:17.459248 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:27:17.459231    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b93a9411-aacf-4c82-a8d2-4ae5e77b2a2a" containerName="console"
Jul 28 16:27:17.459332 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:27:17.459250    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="b93a9411-aacf-4c82-a8d2-4ae5e77b2a2a" containerName="console"
Jul 28 16:27:17.459393 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:27:17.459339    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="b93a9411-aacf-4c82-a8d2-4ae5e77b2a2a" containerName="console"
Jul 28 16:27:17.462307 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:27:17.462287    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5ks2t8"
Jul 28 16:27:17.464998 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:27:17.464978    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 28 16:27:17.465792 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:27:17.465768    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-dxcj5\""
Jul 28 16:27:17.465907 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:27:17.465769    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 28 16:27:17.469767 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:27:17.469735    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5ks2t8"]
Jul 28 16:27:17.564894 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:27:17.564867    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/30d4a32a-4fc3-4451-ae39-70d1ca79be01-bundle\") pod \"320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5ks2t8\" (UID: \"30d4a32a-4fc3-4451-ae39-70d1ca79be01\") " pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5ks2t8"
Jul 28 16:27:17.565026 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:27:17.564917    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gkvwd\" (UniqueName: \"kubernetes.io/projected/30d4a32a-4fc3-4451-ae39-70d1ca79be01-kube-api-access-gkvwd\") pod \"320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5ks2t8\" (UID: \"30d4a32a-4fc3-4451-ae39-70d1ca79be01\") " pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5ks2t8"
Jul 28 16:27:17.565026 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:27:17.564995    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/30d4a32a-4fc3-4451-ae39-70d1ca79be01-util\") pod \"320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5ks2t8\" (UID: \"30d4a32a-4fc3-4451-ae39-70d1ca79be01\") " pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5ks2t8"
Jul 28 16:27:17.666062 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:27:17.666030    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/30d4a32a-4fc3-4451-ae39-70d1ca79be01-bundle\") pod \"320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5ks2t8\" (UID: \"30d4a32a-4fc3-4451-ae39-70d1ca79be01\") " pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5ks2t8"
Jul 28 16:27:17.666187 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:27:17.666075    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gkvwd\" (UniqueName: \"kubernetes.io/projected/30d4a32a-4fc3-4451-ae39-70d1ca79be01-kube-api-access-gkvwd\") pod \"320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5ks2t8\" (UID: \"30d4a32a-4fc3-4451-ae39-70d1ca79be01\") " pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5ks2t8"
Jul 28 16:27:17.666224 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:27:17.666198    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/30d4a32a-4fc3-4451-ae39-70d1ca79be01-util\") pod \"320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5ks2t8\" (UID: \"30d4a32a-4fc3-4451-ae39-70d1ca79be01\") " pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5ks2t8"
Jul 28 16:27:17.666396 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:27:17.666375    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/30d4a32a-4fc3-4451-ae39-70d1ca79be01-bundle\") pod \"320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5ks2t8\" (UID: \"30d4a32a-4fc3-4451-ae39-70d1ca79be01\") " pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5ks2t8"
Jul 28 16:27:17.666452 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:27:17.666435    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/30d4a32a-4fc3-4451-ae39-70d1ca79be01-util\") pod \"320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5ks2t8\" (UID: \"30d4a32a-4fc3-4451-ae39-70d1ca79be01\") " pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5ks2t8"
Jul 28 16:27:17.674347 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:27:17.674291    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gkvwd\" (UniqueName: \"kubernetes.io/projected/30d4a32a-4fc3-4451-ae39-70d1ca79be01-kube-api-access-gkvwd\") pod \"320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5ks2t8\" (UID: \"30d4a32a-4fc3-4451-ae39-70d1ca79be01\") " pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5ks2t8"
Jul 28 16:27:17.771818 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:27:17.771794    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5ks2t8"
Jul 28 16:27:17.896118 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:27:17.896094    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5ks2t8"]
Jul 28 16:27:17.898351 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:27:17.898322    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30d4a32a_4fc3_4451_ae39_70d1ca79be01.slice/crio-0c9def8dacd11b2fa284f0996d0466a2b69cd7603f746f7f1b930a78fe4e7add WatchSource:0}: Error finding container 0c9def8dacd11b2fa284f0996d0466a2b69cd7603f746f7f1b930a78fe4e7add: Status 404 returned error can't find the container with id 0c9def8dacd11b2fa284f0996d0466a2b69cd7603f746f7f1b930a78fe4e7add
Jul 28 16:27:17.900298 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:27:17.900279    2557 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 16:27:18.905877 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:27:18.905840    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5ks2t8" event={"ID":"30d4a32a-4fc3-4451-ae39-70d1ca79be01","Type":"ContainerStarted","Data":"0c9def8dacd11b2fa284f0996d0466a2b69cd7603f746f7f1b930a78fe4e7add"}
Jul 28 16:27:24.930252 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:27:24.930216    2557 generic.go:356] "Generic (PLEG): container finished" podID="30d4a32a-4fc3-4451-ae39-70d1ca79be01" containerID="de5d6e75b46a79cc5c2990c84e6411a36e1b698f765e6b9b96c54ebd38a77147" exitCode=0
Jul 28 16:27:24.930627 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:27:24.930299    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5ks2t8" event={"ID":"30d4a32a-4fc3-4451-ae39-70d1ca79be01","Type":"ContainerDied","Data":"de5d6e75b46a79cc5c2990c84e6411a36e1b698f765e6b9b96c54ebd38a77147"}
Jul 28 16:27:27.943729 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:27:27.943688    2557 generic.go:356] "Generic (PLEG): container finished" podID="30d4a32a-4fc3-4451-ae39-70d1ca79be01" containerID="8b0157f5ad5fd0a0b5a316e1d055cef622241f38edb9abbf527dd1ae324eeb81" exitCode=0
Jul 28 16:27:27.944195 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:27:27.943743    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5ks2t8" event={"ID":"30d4a32a-4fc3-4451-ae39-70d1ca79be01","Type":"ContainerDied","Data":"8b0157f5ad5fd0a0b5a316e1d055cef622241f38edb9abbf527dd1ae324eeb81"}
Jul 28 16:27:34.973309 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:27:34.973267    2557 generic.go:356] "Generic (PLEG): container finished" podID="30d4a32a-4fc3-4451-ae39-70d1ca79be01" containerID="6ca2346d5cb632233f7318e8b027825091b55e13babe85ac5730fbdd91090a7a" exitCode=0
Jul 28 16:27:34.973739 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:27:34.973330    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5ks2t8" event={"ID":"30d4a32a-4fc3-4451-ae39-70d1ca79be01","Type":"ContainerDied","Data":"6ca2346d5cb632233f7318e8b027825091b55e13babe85ac5730fbdd91090a7a"}
Jul 28 16:27:36.100622 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:27:36.100602    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5ks2t8"
Jul 28 16:27:36.216724 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:27:36.216681    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/30d4a32a-4fc3-4451-ae39-70d1ca79be01-util\") pod \"30d4a32a-4fc3-4451-ae39-70d1ca79be01\" (UID: \"30d4a32a-4fc3-4451-ae39-70d1ca79be01\") "
Jul 28 16:27:36.216866 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:27:36.216765    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/30d4a32a-4fc3-4451-ae39-70d1ca79be01-bundle\") pod \"30d4a32a-4fc3-4451-ae39-70d1ca79be01\" (UID: \"30d4a32a-4fc3-4451-ae39-70d1ca79be01\") "
Jul 28 16:27:36.216866 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:27:36.216790    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gkvwd\" (UniqueName: \"kubernetes.io/projected/30d4a32a-4fc3-4451-ae39-70d1ca79be01-kube-api-access-gkvwd\") pod \"30d4a32a-4fc3-4451-ae39-70d1ca79be01\" (UID: \"30d4a32a-4fc3-4451-ae39-70d1ca79be01\") "
Jul 28 16:27:36.217479 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:27:36.217453    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30d4a32a-4fc3-4451-ae39-70d1ca79be01-bundle" (OuterVolumeSpecName: "bundle") pod "30d4a32a-4fc3-4451-ae39-70d1ca79be01" (UID: "30d4a32a-4fc3-4451-ae39-70d1ca79be01"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 16:27:36.219010 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:27:36.218981    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/30d4a32a-4fc3-4451-ae39-70d1ca79be01-kube-api-access-gkvwd" (OuterVolumeSpecName: "kube-api-access-gkvwd") pod "30d4a32a-4fc3-4451-ae39-70d1ca79be01" (UID: "30d4a32a-4fc3-4451-ae39-70d1ca79be01"). InnerVolumeSpecName "kube-api-access-gkvwd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 16:27:36.221050 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:27:36.221021    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30d4a32a-4fc3-4451-ae39-70d1ca79be01-util" (OuterVolumeSpecName: "util") pod "30d4a32a-4fc3-4451-ae39-70d1ca79be01" (UID: "30d4a32a-4fc3-4451-ae39-70d1ca79be01"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 16:27:36.317902 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:27:36.317879    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gkvwd\" (UniqueName: \"kubernetes.io/projected/30d4a32a-4fc3-4451-ae39-70d1ca79be01-kube-api-access-gkvwd\") on node \"ip-10-0-140-154.ec2.internal\" DevicePath \"\""
Jul 28 16:27:36.317902 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:27:36.317901    2557 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/30d4a32a-4fc3-4451-ae39-70d1ca79be01-util\") on node \"ip-10-0-140-154.ec2.internal\" DevicePath \"\""
Jul 28 16:27:36.318030 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:27:36.317910    2557 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/30d4a32a-4fc3-4451-ae39-70d1ca79be01-bundle\") on node \"ip-10-0-140-154.ec2.internal\" DevicePath \"\""
Jul 28 16:27:36.981368 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:27:36.981331    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5ks2t8" event={"ID":"30d4a32a-4fc3-4451-ae39-70d1ca79be01","Type":"ContainerDied","Data":"0c9def8dacd11b2fa284f0996d0466a2b69cd7603f746f7f1b930a78fe4e7add"}
Jul 28 16:27:36.981368 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:27:36.981359    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5ks2t8"
Jul 28 16:27:36.981368 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:27:36.981375    2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0c9def8dacd11b2fa284f0996d0466a2b69cd7603f746f7f1b930a78fe4e7add"
Jul 28 16:27:38.035360 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:27:38.035328    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:27:38.487346 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:27:38.487303    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:27:38.588305 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:27:38.588274    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:27:39.639227 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:27:39.639197    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-qpjtv"]
Jul 28 16:27:39.639600 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:27:39.639480    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="30d4a32a-4fc3-4451-ae39-70d1ca79be01" containerName="util"
Jul 28 16:27:39.639600 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:27:39.639507    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="30d4a32a-4fc3-4451-ae39-70d1ca79be01" containerName="util"
Jul 28 16:27:39.639600 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:27:39.639519    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="30d4a32a-4fc3-4451-ae39-70d1ca79be01" containerName="pull"
Jul 28 16:27:39.639600 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:27:39.639525    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="30d4a32a-4fc3-4451-ae39-70d1ca79be01" containerName="pull"
Jul 28 16:27:39.639600 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:27:39.639531    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="30d4a32a-4fc3-4451-ae39-70d1ca79be01" containerName="extract"
Jul 28 16:27:39.639600 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:27:39.639536    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="30d4a32a-4fc3-4451-ae39-70d1ca79be01" containerName="extract"
Jul 28 16:27:39.639600 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:27:39.639581    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="30d4a32a-4fc3-4451-ae39-70d1ca79be01" containerName="extract"
Jul 28 16:27:39.681525 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:27:39.681483    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-qpjtv"]
Jul 28 16:27:39.681694 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:27:39.681593    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-qpjtv"
Jul 28 16:27:39.684160 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:27:39.684124    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"custom-metrics-autoscaler-operator-dockercfg-lrnc5\""
Jul 28 16:27:39.684160 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:27:39.684129    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"kedaorg-certs\""
Jul 28 16:27:39.684371 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:27:39.684187    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"kube-root-ca.crt\""
Jul 28 16:27:39.684371 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:27:39.684305    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"openshift-service-ca.crt\""
Jul 28 16:27:39.848001 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:27:39.847971    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/8096a792-650c-4990-91d0-4f0ce9494115-certificates\") pod \"custom-metrics-autoscaler-operator-6969cb7dfb-qpjtv\" (UID: \"8096a792-650c-4990-91d0-4f0ce9494115\") " pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-qpjtv"
Jul 28 16:27:39.848001 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:27:39.848003    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2rgpc\" (UniqueName: \"kubernetes.io/projected/8096a792-650c-4990-91d0-4f0ce9494115-kube-api-access-2rgpc\") pod \"custom-metrics-autoscaler-operator-6969cb7dfb-qpjtv\" (UID: \"8096a792-650c-4990-91d0-4f0ce9494115\") " pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-qpjtv"
Jul 28 16:27:39.949323 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:27:39.949252    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/8096a792-650c-4990-91d0-4f0ce9494115-certificates\") pod \"custom-metrics-autoscaler-operator-6969cb7dfb-qpjtv\" (UID: \"8096a792-650c-4990-91d0-4f0ce9494115\") " pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-qpjtv"
Jul 28 16:27:39.949323 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:27:39.949280    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2rgpc\" (UniqueName: \"kubernetes.io/projected/8096a792-650c-4990-91d0-4f0ce9494115-kube-api-access-2rgpc\") pod \"custom-metrics-autoscaler-operator-6969cb7dfb-qpjtv\" (UID: \"8096a792-650c-4990-91d0-4f0ce9494115\") " pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-qpjtv"
Jul 28 16:27:39.951670 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:27:39.951650    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/8096a792-650c-4990-91d0-4f0ce9494115-certificates\") pod \"custom-metrics-autoscaler-operator-6969cb7dfb-qpjtv\" (UID: \"8096a792-650c-4990-91d0-4f0ce9494115\") " pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-qpjtv"
Jul 28 16:27:39.958039 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:27:39.958012    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2rgpc\" (UniqueName: \"kubernetes.io/projected/8096a792-650c-4990-91d0-4f0ce9494115-kube-api-access-2rgpc\") pod \"custom-metrics-autoscaler-operator-6969cb7dfb-qpjtv\" (UID: \"8096a792-650c-4990-91d0-4f0ce9494115\") " pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-qpjtv"
Jul 28 16:27:39.991372 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:27:39.991342    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-qpjtv"
Jul 28 16:27:40.161974 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:27:40.161945    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8096a792_650c_4990_91d0_4f0ce9494115.slice/crio-8121618b9c4c5077bd1cbd24866c71e952cc2441243d3d3bfb2efbd671731aff WatchSource:0}: Error finding container 8121618b9c4c5077bd1cbd24866c71e952cc2441243d3d3bfb2efbd671731aff: Status 404 returned error can't find the container with id 8121618b9c4c5077bd1cbd24866c71e952cc2441243d3d3bfb2efbd671731aff
Jul 28 16:27:40.164370 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:27:40.164273    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-qpjtv"]
Jul 28 16:27:41.003667 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:27:41.003632    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-qpjtv" event={"ID":"8096a792-650c-4990-91d0-4f0ce9494115","Type":"ContainerStarted","Data":"8121618b9c4c5077bd1cbd24866c71e952cc2441243d3d3bfb2efbd671731aff"}
Jul 28 16:27:44.933551 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:27:44.933489    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:27:45.023459 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:27:45.023425    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-qpjtv" event={"ID":"8096a792-650c-4990-91d0-4f0ce9494115","Type":"ContainerStarted","Data":"abe0e66a223bcea3bcaf3f99d9afcbd88725c9b1574fb1343c0f182ebef39b70"}
Jul 28 16:27:45.023615 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:27:45.023521    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-qpjtv"
Jul 28 16:27:45.057798 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:27:45.057742    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-qpjtv" podStartSLOduration=1.930335439 podStartE2EDuration="6.057725713s" totalImagesPullingTime="4.127390274s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 16:27:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:27:45.055488337 +0000 UTC m=+382.193957785" watchObservedRunningTime="2026-07-28 16:27:45.057725713 +0000 UTC m=+382.196195126"
Jul 28 16:27:45.218226 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:27:45.218138    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:27:45.481251 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:27:45.481170    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:27:45.673282 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:27:45.673250    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-metrics-apiserver-55557df8c-n8n2f"]
Jul 28 16:27:45.675685 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:27:45.675665    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-metrics-apiserver-55557df8c-n8n2f"
Jul 28 16:27:45.678009 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:27:45.677989    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-metrics-apiserver-certs\""
Jul 28 16:27:45.678178 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:27:45.678043    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-operator-dockercfg-vzrc6\""
Jul 28 16:27:45.678295 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:27:45.678046    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"keda-ocp-cabundle\""
Jul 28 16:27:45.684873 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:27:45.684819    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-metrics-apiserver-55557df8c-n8n2f"]
Jul 28 16:27:45.797248 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:27:45.797172    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/32940dc7-879b-4c3c-b715-3175c020768e-temp-vol\") pod \"keda-metrics-apiserver-55557df8c-n8n2f\" (UID: \"32940dc7-879b-4c3c-b715-3175c020768e\") " pod="openshift-keda/keda-metrics-apiserver-55557df8c-n8n2f"
Jul 28 16:27:45.797248 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:27:45.797242    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tm7ph\" (UniqueName: \"kubernetes.io/projected/32940dc7-879b-4c3c-b715-3175c020768e-kube-api-access-tm7ph\") pod \"keda-metrics-apiserver-55557df8c-n8n2f\" (UID: \"32940dc7-879b-4c3c-b715-3175c020768e\") " pod="openshift-keda/keda-metrics-apiserver-55557df8c-n8n2f"
Jul 28 16:27:45.797469 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:27:45.797295    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/32940dc7-879b-4c3c-b715-3175c020768e-certificates\") pod \"keda-metrics-apiserver-55557df8c-n8n2f\" (UID: \"32940dc7-879b-4c3c-b715-3175c020768e\") " pod="openshift-keda/keda-metrics-apiserver-55557df8c-n8n2f"
Jul 28 16:27:45.821720 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:27:45.821676    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:27:45.898001 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:27:45.897972    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/32940dc7-879b-4c3c-b715-3175c020768e-certificates\") pod \"keda-metrics-apiserver-55557df8c-n8n2f\" (UID: \"32940dc7-879b-4c3c-b715-3175c020768e\") " pod="openshift-keda/keda-metrics-apiserver-55557df8c-n8n2f"
Jul 28 16:27:45.898134 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:27:45.898007    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/32940dc7-879b-4c3c-b715-3175c020768e-temp-vol\") pod \"keda-metrics-apiserver-55557df8c-n8n2f\" (UID: \"32940dc7-879b-4c3c-b715-3175c020768e\") " pod="openshift-keda/keda-metrics-apiserver-55557df8c-n8n2f"
Jul 28 16:27:45.898134 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:27:45.898068    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tm7ph\" (UniqueName: \"kubernetes.io/projected/32940dc7-879b-4c3c-b715-3175c020768e-kube-api-access-tm7ph\") pod \"keda-metrics-apiserver-55557df8c-n8n2f\" (UID: \"32940dc7-879b-4c3c-b715-3175c020768e\") " pod="openshift-keda/keda-metrics-apiserver-55557df8c-n8n2f"
Jul 28 16:27:45.898134 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:27:45.898124    2557 secret.go:281] references non-existent secret key: tls.crt
Jul 28 16:27:45.898248 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:27:45.898142    2557 projected.go:279] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: tls.crt
Jul 28 16:27:45.898248 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:27:45.898164    2557 projected.go:196] Error preparing data for projected volume certificates for pod openshift-keda/keda-metrics-apiserver-55557df8c-n8n2f: references non-existent secret key: tls.crt
Jul 28 16:27:45.898248 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:27:45.898225    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/32940dc7-879b-4c3c-b715-3175c020768e-certificates podName:32940dc7-879b-4c3c-b715-3175c020768e nodeName:}" failed. No retries permitted until 2026-07-28 16:27:46.398206714 +0000 UTC m=+383.536676111 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/32940dc7-879b-4c3c-b715-3175c020768e-certificates") pod "keda-metrics-apiserver-55557df8c-n8n2f" (UID: "32940dc7-879b-4c3c-b715-3175c020768e") : references non-existent secret key: tls.crt
Jul 28 16:27:45.898407 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:27:45.898386    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/32940dc7-879b-4c3c-b715-3175c020768e-temp-vol\") pod \"keda-metrics-apiserver-55557df8c-n8n2f\" (UID: \"32940dc7-879b-4c3c-b715-3175c020768e\") " pod="openshift-keda/keda-metrics-apiserver-55557df8c-n8n2f"
Jul 28 16:27:45.906938 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:27:45.906917    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tm7ph\" (UniqueName: \"kubernetes.io/projected/32940dc7-879b-4c3c-b715-3175c020768e-kube-api-access-tm7ph\") pod \"keda-metrics-apiserver-55557df8c-n8n2f\" (UID: \"32940dc7-879b-4c3c-b715-3175c020768e\") " pod="openshift-keda/keda-metrics-apiserver-55557df8c-n8n2f"
Jul 28 16:27:46.188538 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:27:46.188476    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:27:46.246567 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:27:46.246534    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:27:46.380505 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:27:46.380472    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-admission-56f9798bb4-ssfvt"]
Jul 28 16:27:46.382925 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:27:46.382905    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-admission-56f9798bb4-ssfvt"
Jul 28 16:27:46.385657 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:27:46.385636    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-admission-webhooks-certs\""
Jul 28 16:27:46.393904 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:27:46.393878    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-admission-56f9798bb4-ssfvt"]
Jul 28 16:27:46.402940 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:27:46.402914    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/32940dc7-879b-4c3c-b715-3175c020768e-certificates\") pod \"keda-metrics-apiserver-55557df8c-n8n2f\" (UID: \"32940dc7-879b-4c3c-b715-3175c020768e\") " pod="openshift-keda/keda-metrics-apiserver-55557df8c-n8n2f"
Jul 28 16:27:46.405872 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:27:46.405849    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/32940dc7-879b-4c3c-b715-3175c020768e-certificates\") pod \"keda-metrics-apiserver-55557df8c-n8n2f\" (UID: \"32940dc7-879b-4c3c-b715-3175c020768e\") " pod="openshift-keda/keda-metrics-apiserver-55557df8c-n8n2f"
Jul 28 16:27:46.418526 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:27:46.418378    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:27:46.503746 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:27:46.503671    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7rnhl\" (UniqueName: \"kubernetes.io/projected/e58dedc5-2f85-404c-af9c-c87cd21e737b-kube-api-access-7rnhl\") pod \"keda-admission-56f9798bb4-ssfvt\" (UID: \"e58dedc5-2f85-404c-af9c-c87cd21e737b\") " pod="openshift-keda/keda-admission-56f9798bb4-ssfvt"
Jul 28 16:27:46.503855 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:27:46.503761    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/e58dedc5-2f85-404c-af9c-c87cd21e737b-certificates\") pod \"keda-admission-56f9798bb4-ssfvt\" (UID: \"e58dedc5-2f85-404c-af9c-c87cd21e737b\") " pod="openshift-keda/keda-admission-56f9798bb4-ssfvt"
Jul 28 16:27:46.511308 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:27:46.511271    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:27:46.588002 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:27:46.587980    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-metrics-apiserver-55557df8c-n8n2f"
Jul 28 16:27:46.604188 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:27:46.604160    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/e58dedc5-2f85-404c-af9c-c87cd21e737b-certificates\") pod \"keda-admission-56f9798bb4-ssfvt\" (UID: \"e58dedc5-2f85-404c-af9c-c87cd21e737b\") " pod="openshift-keda/keda-admission-56f9798bb4-ssfvt"
Jul 28 16:27:46.604299 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:27:46.604218    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7rnhl\" (UniqueName: \"kubernetes.io/projected/e58dedc5-2f85-404c-af9c-c87cd21e737b-kube-api-access-7rnhl\") pod \"keda-admission-56f9798bb4-ssfvt\" (UID: \"e58dedc5-2f85-404c-af9c-c87cd21e737b\") " pod="openshift-keda/keda-admission-56f9798bb4-ssfvt"
Jul 28 16:27:46.604360 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:27:46.604325    2557 projected.go:266] Couldn't get secret openshift-keda/keda-admission-webhooks-certs: secret "keda-admission-webhooks-certs" not found
Jul 28 16:27:46.604360 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:27:46.604353    2557 projected.go:196] Error preparing data for projected volume certificates for pod openshift-keda/keda-admission-56f9798bb4-ssfvt: secret "keda-admission-webhooks-certs" not found
Jul 28 16:27:46.604475 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:27:46.604425    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e58dedc5-2f85-404c-af9c-c87cd21e737b-certificates podName:e58dedc5-2f85-404c-af9c-c87cd21e737b nodeName:}" failed. No retries permitted until 2026-07-28 16:27:47.104403267 +0000 UTC m=+384.242872664 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/e58dedc5-2f85-404c-af9c-c87cd21e737b-certificates") pod "keda-admission-56f9798bb4-ssfvt" (UID: "e58dedc5-2f85-404c-af9c-c87cd21e737b") : secret "keda-admission-webhooks-certs" not found
Jul 28 16:27:46.612608 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:27:46.612580    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7rnhl\" (UniqueName: \"kubernetes.io/projected/e58dedc5-2f85-404c-af9c-c87cd21e737b-kube-api-access-7rnhl\") pod \"keda-admission-56f9798bb4-ssfvt\" (UID: \"e58dedc5-2f85-404c-af9c-c87cd21e737b\") " pod="openshift-keda/keda-admission-56f9798bb4-ssfvt"
Jul 28 16:27:46.730087 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:27:46.730058    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:27:46.749211 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:27:46.749182    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-metrics-apiserver-55557df8c-n8n2f"]
Jul 28 16:27:46.755351 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:27:46.755296    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32940dc7_879b_4c3c_b715_3175c020768e.slice/crio-504db975d41160d09fb80f8e5dd073f8c5563716c9673cb6c2af99a173b2105b WatchSource:0}: Error finding container 504db975d41160d09fb80f8e5dd073f8c5563716c9673cb6c2af99a173b2105b: Status 404 returned error can't find the container with id 504db975d41160d09fb80f8e5dd073f8c5563716c9673cb6c2af99a173b2105b
Jul 28 16:27:46.817900 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:27:46.817876    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:27:46.918700 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:27:46.918667    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:27:47.032611 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:27:47.032538    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-metrics-apiserver-55557df8c-n8n2f" event={"ID":"32940dc7-879b-4c3c-b715-3175c020768e","Type":"ContainerStarted","Data":"504db975d41160d09fb80f8e5dd073f8c5563716c9673cb6c2af99a173b2105b"}
Jul 28 16:27:47.107848 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:27:47.107794    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/e58dedc5-2f85-404c-af9c-c87cd21e737b-certificates\") pod \"keda-admission-56f9798bb4-ssfvt\" (UID: \"e58dedc5-2f85-404c-af9c-c87cd21e737b\") " pod="openshift-keda/keda-admission-56f9798bb4-ssfvt"
Jul 28 16:27:47.110276 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:27:47.110258    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/e58dedc5-2f85-404c-af9c-c87cd21e737b-certificates\") pod \"keda-admission-56f9798bb4-ssfvt\" (UID: \"e58dedc5-2f85-404c-af9c-c87cd21e737b\") " pod="openshift-keda/keda-admission-56f9798bb4-ssfvt"
Jul 28 16:27:47.119866 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:27:47.119838    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:27:47.220310 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:27:47.220285    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:27:47.294526 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:27:47.294445    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-admission-56f9798bb4-ssfvt"
Jul 28 16:27:47.435093 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:27:47.435067    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-admission-56f9798bb4-ssfvt"]
Jul 28 16:27:47.438088 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:27:47.438063    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode58dedc5_2f85_404c_af9c_c87cd21e737b.slice/crio-54c2f3b59f8afe0df70ed46b7817494dc5f776ebf80a6f82c78c5045340a2ff3 WatchSource:0}: Error finding container 54c2f3b59f8afe0df70ed46b7817494dc5f776ebf80a6f82c78c5045340a2ff3: Status 404 returned error can't find the container with id 54c2f3b59f8afe0df70ed46b7817494dc5f776ebf80a6f82c78c5045340a2ff3
Jul 28 16:27:48.037183 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:27:48.037150    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-admission-56f9798bb4-ssfvt" event={"ID":"e58dedc5-2f85-404c-af9c-c87cd21e737b","Type":"ContainerStarted","Data":"54c2f3b59f8afe0df70ed46b7817494dc5f776ebf80a6f82c78c5045340a2ff3"}
Jul 28 16:27:51.053023 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:27:51.052985    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-admission-56f9798bb4-ssfvt" event={"ID":"e58dedc5-2f85-404c-af9c-c87cd21e737b","Type":"ContainerStarted","Data":"98a08a6cb6478708c40d808a00918f36339fb0ff0d49a395249bed9f7cb33858"}
Jul 28 16:27:51.053469 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:27:51.053092    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-admission-56f9798bb4-ssfvt"
Jul 28 16:27:51.054549 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:27:51.054520    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-metrics-apiserver-55557df8c-n8n2f" event={"ID":"32940dc7-879b-4c3c-b715-3175c020768e","Type":"ContainerStarted","Data":"668752b2e41d3b1c1ae608d0c3d634d80fee9ceca34f7e85e0511e3707edf827"}
Jul 28 16:27:51.054664 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:27:51.054631    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-metrics-apiserver-55557df8c-n8n2f"
Jul 28 16:27:51.068853 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:27:51.068811    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/keda-admission-56f9798bb4-ssfvt" podStartSLOduration=2.357815776 podStartE2EDuration="5.0688005s" totalImagesPullingTime="2.710984724s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 16:27:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:27:51.067434673 +0000 UTC m=+388.205904088" watchObservedRunningTime="2026-07-28 16:27:51.0688005 +0000 UTC m=+388.207269913"
Jul 28 16:27:51.084687 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:27:51.084643    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/keda-metrics-apiserver-55557df8c-n8n2f" podStartSLOduration=2.685609344 podStartE2EDuration="6.084630527s" totalImagesPullingTime="3.399021183s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 16:27:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:27:51.082608905 +0000 UTC m=+388.221078330" watchObservedRunningTime="2026-07-28 16:27:51.084630527 +0000 UTC m=+388.223099941"
Jul 28 16:28:02.063016 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:28:02.062985    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-metrics-apiserver-55557df8c-n8n2f"
Jul 28 16:28:06.030426 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:28:06.030397    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-qpjtv"
Jul 28 16:28:12.061151 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:28:12.061114    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-admission-56f9798bb4-ssfvt"
Jul 28 16:28:16.076581 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:28:16.076550    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:28:16.132702 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:28:16.132677    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:28:17.927954 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:28:17.927919    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:28:17.981433 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:28:17.981412    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:29:00.728161 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:29:00.728117    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:29:00.796174 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:29:00.796143    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/llmisvc-controller-manager-84fc959947-2dfkr"]
Jul 28 16:29:00.798783 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:29:00.798762    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/llmisvc-controller-manager-84fc959947-2dfkr"
Jul 28 16:29:00.801291 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:29:00.801269    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"kube-root-ca.crt\""
Jul 28 16:29:00.801516 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:29:00.801316    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"llmisvc-webhook-server-cert\""
Jul 28 16:29:00.801516 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:29:00.801318    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"openshift-service-ca.crt\""
Jul 28 16:29:00.802165 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:29:00.802136    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"llmisvc-controller-manager-dockercfg-zks5d\""
Jul 28 16:29:00.809039 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:29:00.809018    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/llmisvc-controller-manager-84fc959947-2dfkr"]
Jul 28 16:29:00.811069 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:29:00.811047    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:29:00.826215 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:29:00.826195    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/seaweedfs-64568bcd49-vqrgh"]
Jul 28 16:29:00.828203 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:29:00.828188    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/seaweedfs-64568bcd49-vqrgh"
Jul 28 16:29:00.830438 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:29:00.830422    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"default-dockercfg-vcflw\""
Jul 28 16:29:00.830559 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:29:00.830421    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"mlpipeline-s3-artifact\""
Jul 28 16:29:00.836542 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:29:00.836524    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/seaweedfs-64568bcd49-vqrgh"]
Jul 28 16:29:00.857403 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:29:00.857382    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kkzlk\" (UniqueName: \"kubernetes.io/projected/f42f830b-f5e5-4749-a39d-77bbe83c7707-kube-api-access-kkzlk\") pod \"llmisvc-controller-manager-84fc959947-2dfkr\" (UID: \"f42f830b-f5e5-4749-a39d-77bbe83c7707\") " pod="kserve/llmisvc-controller-manager-84fc959947-2dfkr"
Jul 28 16:29:00.857488 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:29:00.857440    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/489f1c36-f573-45dd-b6b3-0c13b3f8bcd3-data\") pod \"seaweedfs-64568bcd49-vqrgh\" (UID: \"489f1c36-f573-45dd-b6b3-0c13b3f8bcd3\") " pod="kserve/seaweedfs-64568bcd49-vqrgh"
Jul 28 16:29:00.857488 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:29:00.857473    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f42f830b-f5e5-4749-a39d-77bbe83c7707-cert\") pod \"llmisvc-controller-manager-84fc959947-2dfkr\" (UID: \"f42f830b-f5e5-4749-a39d-77bbe83c7707\") " pod="kserve/llmisvc-controller-manager-84fc959947-2dfkr"
Jul 28 16:29:00.857601 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:29:00.857545    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wl66d\" (UniqueName: \"kubernetes.io/projected/489f1c36-f573-45dd-b6b3-0c13b3f8bcd3-kube-api-access-wl66d\") pod \"seaweedfs-64568bcd49-vqrgh\" (UID: \"489f1c36-f573-45dd-b6b3-0c13b3f8bcd3\") " pod="kserve/seaweedfs-64568bcd49-vqrgh"
Jul 28 16:29:00.882125 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:29:00.882103    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:29:00.957842 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:29:00.957818    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/489f1c36-f573-45dd-b6b3-0c13b3f8bcd3-data\") pod \"seaweedfs-64568bcd49-vqrgh\" (UID: \"489f1c36-f573-45dd-b6b3-0c13b3f8bcd3\") " pod="kserve/seaweedfs-64568bcd49-vqrgh"
Jul 28 16:29:00.957940 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:29:00.957854    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f42f830b-f5e5-4749-a39d-77bbe83c7707-cert\") pod \"llmisvc-controller-manager-84fc959947-2dfkr\" (UID: \"f42f830b-f5e5-4749-a39d-77bbe83c7707\") " pod="kserve/llmisvc-controller-manager-84fc959947-2dfkr"
Jul 28 16:29:00.957940 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:29:00.957892    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wl66d\" (UniqueName: \"kubernetes.io/projected/489f1c36-f573-45dd-b6b3-0c13b3f8bcd3-kube-api-access-wl66d\") pod \"seaweedfs-64568bcd49-vqrgh\" (UID: \"489f1c36-f573-45dd-b6b3-0c13b3f8bcd3\") " pod="kserve/seaweedfs-64568bcd49-vqrgh"
Jul 28 16:29:00.958054 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:29:00.957942    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kkzlk\" (UniqueName: \"kubernetes.io/projected/f42f830b-f5e5-4749-a39d-77bbe83c7707-kube-api-access-kkzlk\") pod \"llmisvc-controller-manager-84fc959947-2dfkr\" (UID: \"f42f830b-f5e5-4749-a39d-77bbe83c7707\") " pod="kserve/llmisvc-controller-manager-84fc959947-2dfkr"
Jul 28 16:29:00.958054 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:29:00.957978    2557 secret.go:189] Couldn't get secret kserve/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found
Jul 28 16:29:00.958054 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:29:00.958040    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f42f830b-f5e5-4749-a39d-77bbe83c7707-cert podName:f42f830b-f5e5-4749-a39d-77bbe83c7707 nodeName:}" failed. No retries permitted until 2026-07-28 16:29:01.458021482 +0000 UTC m=+458.596490874 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/f42f830b-f5e5-4749-a39d-77bbe83c7707-cert") pod "llmisvc-controller-manager-84fc959947-2dfkr" (UID: "f42f830b-f5e5-4749-a39d-77bbe83c7707") : secret "llmisvc-webhook-server-cert" not found
Jul 28 16:29:00.958230 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:29:00.958211    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/489f1c36-f573-45dd-b6b3-0c13b3f8bcd3-data\") pod \"seaweedfs-64568bcd49-vqrgh\" (UID: \"489f1c36-f573-45dd-b6b3-0c13b3f8bcd3\") " pod="kserve/seaweedfs-64568bcd49-vqrgh"
Jul 28 16:29:00.968480 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:29:00.968459    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wl66d\" (UniqueName: \"kubernetes.io/projected/489f1c36-f573-45dd-b6b3-0c13b3f8bcd3-kube-api-access-wl66d\") pod \"seaweedfs-64568bcd49-vqrgh\" (UID: \"489f1c36-f573-45dd-b6b3-0c13b3f8bcd3\") " pod="kserve/seaweedfs-64568bcd49-vqrgh"
Jul 28 16:29:00.971455 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:29:00.971434    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kkzlk\" (UniqueName: \"kubernetes.io/projected/f42f830b-f5e5-4749-a39d-77bbe83c7707-kube-api-access-kkzlk\") pod \"llmisvc-controller-manager-84fc959947-2dfkr\" (UID: \"f42f830b-f5e5-4749-a39d-77bbe83c7707\") " pod="kserve/llmisvc-controller-manager-84fc959947-2dfkr"
Jul 28 16:29:01.099126 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:29:01.099093    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:29:01.139165 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:29:01.139142    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/seaweedfs-64568bcd49-vqrgh"
Jul 28 16:29:01.221730 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:29:01.221677    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:29:01.269064 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:29:01.269038    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/seaweedfs-64568bcd49-vqrgh"]
Jul 28 16:29:01.273303 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:29:01.273260    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod489f1c36_f573_45dd_b6b3_0c13b3f8bcd3.slice/crio-c6c702824c3ae46e3256b8b65172318ae13320917831a301b1e8f1c0691bb474 WatchSource:0}: Error finding container c6c702824c3ae46e3256b8b65172318ae13320917831a301b1e8f1c0691bb474: Status 404 returned error can't find the container with id c6c702824c3ae46e3256b8b65172318ae13320917831a301b1e8f1c0691bb474
Jul 28 16:29:01.274070 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:29:01.274047    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:29:01.352538 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:29:01.352448    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/seaweedfs-64568bcd49-vqrgh" event={"ID":"489f1c36-f573-45dd-b6b3-0c13b3f8bcd3","Type":"ContainerStarted","Data":"c6c702824c3ae46e3256b8b65172318ae13320917831a301b1e8f1c0691bb474"}
Jul 28 16:29:01.449011 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:29:01.448971    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:29:01.471276 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:29:01.471253    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f42f830b-f5e5-4749-a39d-77bbe83c7707-cert\") pod \"llmisvc-controller-manager-84fc959947-2dfkr\" (UID: \"f42f830b-f5e5-4749-a39d-77bbe83c7707\") " pod="kserve/llmisvc-controller-manager-84fc959947-2dfkr"
Jul 28 16:29:01.473703 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:29:01.473682    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f42f830b-f5e5-4749-a39d-77bbe83c7707-cert\") pod \"llmisvc-controller-manager-84fc959947-2dfkr\" (UID: \"f42f830b-f5e5-4749-a39d-77bbe83c7707\") " pod="kserve/llmisvc-controller-manager-84fc959947-2dfkr"
Jul 28 16:29:01.711988 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:29:01.711924    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/llmisvc-controller-manager-84fc959947-2dfkr"
Jul 28 16:29:01.863173 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:29:01.863137    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/llmisvc-controller-manager-84fc959947-2dfkr"]
Jul 28 16:29:01.976170 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:29:01.976097    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podf42f830b_f5e5_4749_a39d_77bbe83c7707.slice/crio-29e4c99ddc47db3302b875bc7496885489be6295197a578e3faef8fd1812e857 WatchSource:0}: Error finding container 29e4c99ddc47db3302b875bc7496885489be6295197a578e3faef8fd1812e857: Status 404 returned error can't find the container with id 29e4c99ddc47db3302b875bc7496885489be6295197a578e3faef8fd1812e857
Jul 28 16:29:02.359836 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:29:02.359792    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-84fc959947-2dfkr" event={"ID":"f42f830b-f5e5-4749-a39d-77bbe83c7707","Type":"ContainerStarted","Data":"29e4c99ddc47db3302b875bc7496885489be6295197a578e3faef8fd1812e857"}
Jul 28 16:29:05.374844 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:29:05.374806    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/seaweedfs-64568bcd49-vqrgh" event={"ID":"489f1c36-f573-45dd-b6b3-0c13b3f8bcd3","Type":"ContainerStarted","Data":"08aa3c08f527f473b71d1c0baa09f629263dcc9e6c51d3cca472295567ebde38"}
Jul 28 16:29:05.375278 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:29:05.374893    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/seaweedfs-64568bcd49-vqrgh"
Jul 28 16:29:05.376414 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:29:05.376390    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-84fc959947-2dfkr" event={"ID":"f42f830b-f5e5-4749-a39d-77bbe83c7707","Type":"ContainerStarted","Data":"c4fedfcae86a2d9754f72b37a1fe364e71540bf75086599dfe0266a341cce412"}
Jul 28 16:29:05.376571 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:29:05.376454    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/llmisvc-controller-manager-84fc959947-2dfkr"
Jul 28 16:29:05.391169 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:29:05.391130    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/seaweedfs-64568bcd49-vqrgh" podStartSLOduration=1.757219086 podStartE2EDuration="5.391117416s" totalImagesPullingTime="3.63389833s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 16:29:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:29:05.390346625 +0000 UTC m=+462.528816040" watchObservedRunningTime="2026-07-28 16:29:05.391117416 +0000 UTC m=+462.529586830"
Jul 28 16:29:05.405831 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:29:05.405722    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/llmisvc-controller-manager-84fc959947-2dfkr" podStartSLOduration=2.475306494 podStartE2EDuration="5.405707357s" totalImagesPullingTime="2.930400863s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 16:29:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:29:05.405582525 +0000 UTC m=+462.544051939" watchObservedRunningTime="2026-07-28 16:29:05.405707357 +0000 UTC m=+462.544176774"
Jul 28 16:29:11.382956 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:29:11.382927    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/seaweedfs-64568bcd49-vqrgh"
Jul 28 16:29:16.269928 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:29:16.269885    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:29:25.396667 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:29:25.396630    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:29:36.382855 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:29:36.382825    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/llmisvc-controller-manager-84fc959947-2dfkr"
Jul 28 16:30:16.448046 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:30:16.447950    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:30:16.475648 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:30:16.475618    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/model-serving-api-7b954ffc95-7r22c"]
Jul 28 16:30:16.478986 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:30:16.478964    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/model-serving-api-7b954ffc95-7r22c"
Jul 28 16:30:16.481420 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:30:16.481402    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"model-serving-api-tls\""
Jul 28 16:30:16.481573 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:30:16.481552    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"model-serving-api-dockercfg-8zh8r\""
Jul 28 16:30:16.490279 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:30:16.490258    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/model-serving-api-7b954ffc95-7r22c"]
Jul 28 16:30:16.524722 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:30:16.524692    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:30:16.548134 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:30:16.548107    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xn2tt\" (UniqueName: \"kubernetes.io/projected/d1113ad5-4346-4d74-a8ef-06b1c8e48587-kube-api-access-xn2tt\") pod \"model-serving-api-7b954ffc95-7r22c\" (UID: \"d1113ad5-4346-4d74-a8ef-06b1c8e48587\") " pod="kserve/model-serving-api-7b954ffc95-7r22c"
Jul 28 16:30:16.548223 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:30:16.548163    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d1113ad5-4346-4d74-a8ef-06b1c8e48587-tls-certs\") pod \"model-serving-api-7b954ffc95-7r22c\" (UID: \"d1113ad5-4346-4d74-a8ef-06b1c8e48587\") " pod="kserve/model-serving-api-7b954ffc95-7r22c"
Jul 28 16:30:16.648516 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:30:16.648481    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xn2tt\" (UniqueName: \"kubernetes.io/projected/d1113ad5-4346-4d74-a8ef-06b1c8e48587-kube-api-access-xn2tt\") pod \"model-serving-api-7b954ffc95-7r22c\" (UID: \"d1113ad5-4346-4d74-a8ef-06b1c8e48587\") " pod="kserve/model-serving-api-7b954ffc95-7r22c"
Jul 28 16:30:16.648608 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:30:16.648530    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d1113ad5-4346-4d74-a8ef-06b1c8e48587-tls-certs\") pod \"model-serving-api-7b954ffc95-7r22c\" (UID: \"d1113ad5-4346-4d74-a8ef-06b1c8e48587\") " pod="kserve/model-serving-api-7b954ffc95-7r22c"
Jul 28 16:30:16.648653 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:30:16.648628    2557 secret.go:189] Couldn't get secret kserve/model-serving-api-tls: secret "model-serving-api-tls" not found
Jul 28 16:30:16.648690 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:30:16.648674    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d1113ad5-4346-4d74-a8ef-06b1c8e48587-tls-certs podName:d1113ad5-4346-4d74-a8ef-06b1c8e48587 nodeName:}" failed. No retries permitted until 2026-07-28 16:30:17.148660708 +0000 UTC m=+534.287130101 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/d1113ad5-4346-4d74-a8ef-06b1c8e48587-tls-certs") pod "model-serving-api-7b954ffc95-7r22c" (UID: "d1113ad5-4346-4d74-a8ef-06b1c8e48587") : secret "model-serving-api-tls" not found
Jul 28 16:30:16.659208 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:30:16.659180    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xn2tt\" (UniqueName: \"kubernetes.io/projected/d1113ad5-4346-4d74-a8ef-06b1c8e48587-kube-api-access-xn2tt\") pod \"model-serving-api-7b954ffc95-7r22c\" (UID: \"d1113ad5-4346-4d74-a8ef-06b1c8e48587\") " pod="kserve/model-serving-api-7b954ffc95-7r22c"
Jul 28 16:30:16.706314 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:30:16.706253    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:30:16.929348 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:30:16.929321    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:30:17.052649 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:30:17.052625    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:30:17.117021 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:30:17.116992    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:30:17.151783 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:30:17.151763    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d1113ad5-4346-4d74-a8ef-06b1c8e48587-tls-certs\") pod \"model-serving-api-7b954ffc95-7r22c\" (UID: \"d1113ad5-4346-4d74-a8ef-06b1c8e48587\") " pod="kserve/model-serving-api-7b954ffc95-7r22c"
Jul 28 16:30:17.154242 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:30:17.154227    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d1113ad5-4346-4d74-a8ef-06b1c8e48587-tls-certs\") pod \"model-serving-api-7b954ffc95-7r22c\" (UID: \"d1113ad5-4346-4d74-a8ef-06b1c8e48587\") " pod="kserve/model-serving-api-7b954ffc95-7r22c"
Jul 28 16:30:17.391863 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:30:17.391787    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/model-serving-api-7b954ffc95-7r22c"
Jul 28 16:30:17.518881 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:30:17.518860    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/model-serving-api-7b954ffc95-7r22c"]
Jul 28 16:30:17.520840 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:30:17.520812    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1113ad5_4346_4d74_a8ef_06b1c8e48587.slice/crio-779bdb17bef5d5965df4fdd22a6b7c95283cf7ecf2670be637fd278b1b6e3703 WatchSource:0}: Error finding container 779bdb17bef5d5965df4fdd22a6b7c95283cf7ecf2670be637fd278b1b6e3703: Status 404 returned error can't find the container with id 779bdb17bef5d5965df4fdd22a6b7c95283cf7ecf2670be637fd278b1b6e3703
Jul 28 16:30:17.677167 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:30:17.677090    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/model-serving-api-7b954ffc95-7r22c" event={"ID":"d1113ad5-4346-4d74-a8ef-06b1c8e48587","Type":"ContainerStarted","Data":"779bdb17bef5d5965df4fdd22a6b7c95283cf7ecf2670be637fd278b1b6e3703"}
Jul 28 16:30:20.693016 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:30:20.692976    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/model-serving-api-7b954ffc95-7r22c" event={"ID":"d1113ad5-4346-4d74-a8ef-06b1c8e48587","Type":"ContainerStarted","Data":"cfd9dc4969afbfb6c164131eeeb5ecbae43f99f4f4c54b23a9a75ef26cd47b3e"}
Jul 28 16:30:20.693463 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:30:20.693105    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/model-serving-api-7b954ffc95-7r22c"
Jul 28 16:30:20.710738 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:30:20.710695    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/model-serving-api-7b954ffc95-7r22c" podStartSLOduration=2.371411063 podStartE2EDuration="4.710679869s" totalImagesPullingTime="2.339268806s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 16:30:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:30:20.709768293 +0000 UTC m=+537.848237708" watchObservedRunningTime="2026-07-28 16:30:20.710679869 +0000 UTC m=+537.849149283"
Jul 28 16:30:31.702470 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:30:31.702440    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/model-serving-api-7b954ffc95-7r22c"
Jul 28 16:31:38.260284 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:31:38.260248    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:31:44.444700 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:31:44.444595    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:31:55.326834 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:31:55.326793    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:31:55.427024 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:31:55.426989    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:32:06.645837 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:32:06.645802    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:32:06.685058 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:32:06.685027    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-b2ad5-predictor-666fbfd487-dt8k4"]
Jul 28 16:32:06.688291 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:32:06.688273    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-b2ad5-predictor-666fbfd487-dt8k4"
Jul 28 16:32:06.690816 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:32:06.690793    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 28 16:32:06.690923 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:32:06.690794    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-b2ad5-predictor-serving-cert\""
Jul 28 16:32:06.690923 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:32:06.690843    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 28 16:32:06.691589 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:32:06.691571    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-b2ad5-kube-rbac-proxy-sar-config\""
Jul 28 16:32:06.691692 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:32:06.691589    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-zwq7z\""
Jul 28 16:32:06.696164 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:32:06.696146    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-b2ad5-predictor-666fbfd487-dt8k4"]
Jul 28 16:32:06.811845 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:32:06.811797    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cnfc8\" (UniqueName: \"kubernetes.io/projected/225e1d35-44de-4ab7-b20e-d543efa4539b-kube-api-access-cnfc8\") pod \"success-200-isvc-b2ad5-predictor-666fbfd487-dt8k4\" (UID: \"225e1d35-44de-4ab7-b20e-d543efa4539b\") " pod="kserve-ci-e2e-test/success-200-isvc-b2ad5-predictor-666fbfd487-dt8k4"
Jul 28 16:32:06.811845 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:32:06.811857    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/225e1d35-44de-4ab7-b20e-d543efa4539b-proxy-tls\") pod \"success-200-isvc-b2ad5-predictor-666fbfd487-dt8k4\" (UID: \"225e1d35-44de-4ab7-b20e-d543efa4539b\") " pod="kserve-ci-e2e-test/success-200-isvc-b2ad5-predictor-666fbfd487-dt8k4"
Jul 28 16:32:06.812080 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:32:06.811951    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/225e1d35-44de-4ab7-b20e-d543efa4539b-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-b2ad5-predictor-666fbfd487-dt8k4\" (UID: \"225e1d35-44de-4ab7-b20e-d543efa4539b\") " pod="kserve-ci-e2e-test/success-200-isvc-b2ad5-predictor-666fbfd487-dt8k4"
Jul 28 16:32:06.913018 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:32:06.912949    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cnfc8\" (UniqueName: \"kubernetes.io/projected/225e1d35-44de-4ab7-b20e-d543efa4539b-kube-api-access-cnfc8\") pod \"success-200-isvc-b2ad5-predictor-666fbfd487-dt8k4\" (UID: \"225e1d35-44de-4ab7-b20e-d543efa4539b\") " pod="kserve-ci-e2e-test/success-200-isvc-b2ad5-predictor-666fbfd487-dt8k4"
Jul 28 16:32:06.913018 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:32:06.912983    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/225e1d35-44de-4ab7-b20e-d543efa4539b-proxy-tls\") pod \"success-200-isvc-b2ad5-predictor-666fbfd487-dt8k4\" (UID: \"225e1d35-44de-4ab7-b20e-d543efa4539b\") " pod="kserve-ci-e2e-test/success-200-isvc-b2ad5-predictor-666fbfd487-dt8k4"
Jul 28 16:32:06.913213 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:32:06.913036    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/225e1d35-44de-4ab7-b20e-d543efa4539b-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-b2ad5-predictor-666fbfd487-dt8k4\" (UID: \"225e1d35-44de-4ab7-b20e-d543efa4539b\") " pod="kserve-ci-e2e-test/success-200-isvc-b2ad5-predictor-666fbfd487-dt8k4"
Jul 28 16:32:06.913213 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:32:06.913169    2557 secret.go:189] Couldn't get secret kserve-ci-e2e-test/success-200-isvc-b2ad5-predictor-serving-cert: secret "success-200-isvc-b2ad5-predictor-serving-cert" not found
Jul 28 16:32:06.913376 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:32:06.913248    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/225e1d35-44de-4ab7-b20e-d543efa4539b-proxy-tls podName:225e1d35-44de-4ab7-b20e-d543efa4539b nodeName:}" failed. No retries permitted until 2026-07-28 16:32:07.413228409 +0000 UTC m=+644.551697802 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/225e1d35-44de-4ab7-b20e-d543efa4539b-proxy-tls") pod "success-200-isvc-b2ad5-predictor-666fbfd487-dt8k4" (UID: "225e1d35-44de-4ab7-b20e-d543efa4539b") : secret "success-200-isvc-b2ad5-predictor-serving-cert" not found
Jul 28 16:32:06.913734 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:32:06.913717    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/225e1d35-44de-4ab7-b20e-d543efa4539b-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-b2ad5-predictor-666fbfd487-dt8k4\" (UID: \"225e1d35-44de-4ab7-b20e-d543efa4539b\") " pod="kserve-ci-e2e-test/success-200-isvc-b2ad5-predictor-666fbfd487-dt8k4"
Jul 28 16:32:06.921547 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:32:06.921523    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cnfc8\" (UniqueName: \"kubernetes.io/projected/225e1d35-44de-4ab7-b20e-d543efa4539b-kube-api-access-cnfc8\") pod \"success-200-isvc-b2ad5-predictor-666fbfd487-dt8k4\" (UID: \"225e1d35-44de-4ab7-b20e-d543efa4539b\") " pod="kserve-ci-e2e-test/success-200-isvc-b2ad5-predictor-666fbfd487-dt8k4"
Jul 28 16:32:06.994283 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:32:06.994258    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:32:07.063729 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:32:07.063693    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:32:07.143008 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:32:07.142967    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:32:07.246190 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:32:07.246111    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:32:07.312085 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:32:07.312052    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:32:07.370814 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:32:07.370769    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:32:07.418340 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:32:07.418315    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/225e1d35-44de-4ab7-b20e-d543efa4539b-proxy-tls\") pod \"success-200-isvc-b2ad5-predictor-666fbfd487-dt8k4\" (UID: \"225e1d35-44de-4ab7-b20e-d543efa4539b\") " pod="kserve-ci-e2e-test/success-200-isvc-b2ad5-predictor-666fbfd487-dt8k4"
Jul 28 16:32:07.420933 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:32:07.420910    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/225e1d35-44de-4ab7-b20e-d543efa4539b-proxy-tls\") pod \"success-200-isvc-b2ad5-predictor-666fbfd487-dt8k4\" (UID: \"225e1d35-44de-4ab7-b20e-d543efa4539b\") " pod="kserve-ci-e2e-test/success-200-isvc-b2ad5-predictor-666fbfd487-dt8k4"
Jul 28 16:32:07.599935 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:32:07.599904    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-b2ad5-predictor-666fbfd487-dt8k4"
Jul 28 16:32:07.671762 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:32:07.671717    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:32:07.732356 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:32:07.732343    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-b2ad5-predictor-666fbfd487-dt8k4"]
Jul 28 16:32:07.734706 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:32:07.734682    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod225e1d35_44de_4ab7_b20e_d543efa4539b.slice/crio-84a7889412c67f69411ce71df838bcd9adc4dc785b0a2fc56ee6ebfc31a38e8b WatchSource:0}: Error finding container 84a7889412c67f69411ce71df838bcd9adc4dc785b0a2fc56ee6ebfc31a38e8b: Status 404 returned error can't find the container with id 84a7889412c67f69411ce71df838bcd9adc4dc785b0a2fc56ee6ebfc31a38e8b
Jul 28 16:32:08.149435 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:32:08.149401    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-b2ad5-predictor-666fbfd487-dt8k4" event={"ID":"225e1d35-44de-4ab7-b20e-d543efa4539b","Type":"ContainerStarted","Data":"84a7889412c67f69411ce71df838bcd9adc4dc785b0a2fc56ee6ebfc31a38e8b"}
Jul 28 16:32:19.842707 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:32:19.842688    2557 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 16:32:20.222220 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:32:20.222185    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-b2ad5-predictor-666fbfd487-dt8k4" event={"ID":"225e1d35-44de-4ab7-b20e-d543efa4539b","Type":"ContainerStarted","Data":"4b4bb8e77578153d6cd9b4fc988b5ad9d30cf65def61c15e99931e63ca36fd5b"}
Jul 28 16:32:22.235761 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:32:22.235719    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-b2ad5-predictor-666fbfd487-dt8k4" event={"ID":"225e1d35-44de-4ab7-b20e-d543efa4539b","Type":"ContainerStarted","Data":"047efc18884e90f27eaf87f2c4d971f27c9d382425696cbd5ad0dbc752927e02"}
Jul 28 16:32:22.236178 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:32:22.235928    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-b2ad5-predictor-666fbfd487-dt8k4"
Jul 28 16:32:22.236178 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:32:22.235982    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-b2ad5-predictor-666fbfd487-dt8k4"
Jul 28 16:32:22.237443 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:32:22.237416    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-b2ad5-predictor-666fbfd487-dt8k4" podUID="225e1d35-44de-4ab7-b20e-d543efa4539b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.30:8080: connect: connection refused"
Jul 28 16:32:22.254747 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:32:22.254705    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-b2ad5-predictor-666fbfd487-dt8k4" podStartSLOduration=1.980150501 podStartE2EDuration="16.254690543s" totalImagesPullingTime="14.274540042s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 16:32:06 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:32:22.25269256 +0000 UTC m=+659.391161975" watchObservedRunningTime="2026-07-28 16:32:22.254690543 +0000 UTC m=+659.393159957"
Jul 28 16:32:23.240206 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:32:23.240157    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-b2ad5-predictor-666fbfd487-dt8k4" podUID="225e1d35-44de-4ab7-b20e-d543efa4539b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.30:8080: connect: connection refused"
Jul 28 16:32:28.245075 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:32:28.245047    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-b2ad5-predictor-666fbfd487-dt8k4"
Jul 28 16:32:28.245602 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:32:28.245574    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-b2ad5-predictor-666fbfd487-dt8k4" podUID="225e1d35-44de-4ab7-b20e-d543efa4539b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.30:8080: connect: connection refused"
Jul 28 16:32:38.246510 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:32:38.246468    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-b2ad5-predictor-666fbfd487-dt8k4" podUID="225e1d35-44de-4ab7-b20e-d543efa4539b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.30:8080: connect: connection refused"
Jul 28 16:32:40.305904 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:32:40.305869    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:32:41.414576 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:32:41.414545    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:32:48.245558 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:32:48.245513    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-b2ad5-predictor-666fbfd487-dt8k4" podUID="225e1d35-44de-4ab7-b20e-d543efa4539b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.30:8080: connect: connection refused"
Jul 28 16:32:58.245442 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:32:58.245400    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-b2ad5-predictor-666fbfd487-dt8k4" podUID="225e1d35-44de-4ab7-b20e-d543efa4539b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.30:8080: connect: connection refused"
Jul 28 16:33:08.246074 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:33:08.246035    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-b2ad5-predictor-666fbfd487-dt8k4"
Jul 28 16:33:15.800205 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:33:15.800128    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:33:26.690277 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:33:26.690241    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:33:26.914484 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:33:26.914453    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:33:36.818440 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:33:36.818403    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:33:36.903806 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:33:36.903774    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-b2ad5-predictor-666fbfd487-dt8k4"]
Jul 28 16:33:36.904108 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:33:36.904053    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-b2ad5-predictor-666fbfd487-dt8k4" podUID="225e1d35-44de-4ab7-b20e-d543efa4539b" containerName="kserve-container" containerID="cri-o://4b4bb8e77578153d6cd9b4fc988b5ad9d30cf65def61c15e99931e63ca36fd5b" gracePeriod=30
Jul 28 16:33:36.904211 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:33:36.904124    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-b2ad5-predictor-666fbfd487-dt8k4" podUID="225e1d35-44de-4ab7-b20e-d543efa4539b" containerName="kube-rbac-proxy" containerID="cri-o://047efc18884e90f27eaf87f2c4d971f27c9d382425696cbd5ad0dbc752927e02" gracePeriod=30
Jul 28 16:33:36.927631 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:33:36.927600    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:33:36.968929 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:33:36.968898    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-7ba9d-predictor-68cbcf59bf-jt9kq"]
Jul 28 16:33:36.971361 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:33:36.971342    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-7ba9d-predictor-68cbcf59bf-jt9kq"
Jul 28 16:33:36.973625 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:33:36.973603    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-7ba9d-kube-rbac-proxy-sar-config\""
Jul 28 16:33:36.973737 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:33:36.973692    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-7ba9d-predictor-serving-cert\""
Jul 28 16:33:36.983013 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:33:36.982995    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-7ba9d-predictor-68cbcf59bf-jt9kq"]
Jul 28 16:33:37.008911 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:33:37.008872    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:33:37.068365 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:33:37.068320    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:33:37.071454 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:33:37.071391    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/34ba2157-d9a4-42c8-834c-f349e8df2b76-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-7ba9d-predictor-68cbcf59bf-jt9kq\" (UID: \"34ba2157-d9a4-42c8-834c-f349e8df2b76\") " pod="kserve-ci-e2e-test/success-200-isvc-7ba9d-predictor-68cbcf59bf-jt9kq"
Jul 28 16:33:37.071454 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:33:37.071441    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/34ba2157-d9a4-42c8-834c-f349e8df2b76-proxy-tls\") pod \"success-200-isvc-7ba9d-predictor-68cbcf59bf-jt9kq\" (UID: \"34ba2157-d9a4-42c8-834c-f349e8df2b76\") " pod="kserve-ci-e2e-test/success-200-isvc-7ba9d-predictor-68cbcf59bf-jt9kq"
Jul 28 16:33:37.071693 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:33:37.071591    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dwt4l\" (UniqueName: \"kubernetes.io/projected/34ba2157-d9a4-42c8-834c-f349e8df2b76-kube-api-access-dwt4l\") pod \"success-200-isvc-7ba9d-predictor-68cbcf59bf-jt9kq\" (UID: \"34ba2157-d9a4-42c8-834c-f349e8df2b76\") " pod="kserve-ci-e2e-test/success-200-isvc-7ba9d-predictor-68cbcf59bf-jt9kq"
Jul 28 16:33:37.173050 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:33:37.173011    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/34ba2157-d9a4-42c8-834c-f349e8df2b76-proxy-tls\") pod \"success-200-isvc-7ba9d-predictor-68cbcf59bf-jt9kq\" (UID: \"34ba2157-d9a4-42c8-834c-f349e8df2b76\") " pod="kserve-ci-e2e-test/success-200-isvc-7ba9d-predictor-68cbcf59bf-jt9kq"
Jul 28 16:33:37.173235 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:33:37.173106    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dwt4l\" (UniqueName: \"kubernetes.io/projected/34ba2157-d9a4-42c8-834c-f349e8df2b76-kube-api-access-dwt4l\") pod \"success-200-isvc-7ba9d-predictor-68cbcf59bf-jt9kq\" (UID: \"34ba2157-d9a4-42c8-834c-f349e8df2b76\") " pod="kserve-ci-e2e-test/success-200-isvc-7ba9d-predictor-68cbcf59bf-jt9kq"
Jul 28 16:33:37.173235 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:33:37.173169    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/34ba2157-d9a4-42c8-834c-f349e8df2b76-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-7ba9d-predictor-68cbcf59bf-jt9kq\" (UID: \"34ba2157-d9a4-42c8-834c-f349e8df2b76\") " pod="kserve-ci-e2e-test/success-200-isvc-7ba9d-predictor-68cbcf59bf-jt9kq"
Jul 28 16:33:37.173235 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:33:37.173190    2557 secret.go:189] Couldn't get secret kserve-ci-e2e-test/success-200-isvc-7ba9d-predictor-serving-cert: secret "success-200-isvc-7ba9d-predictor-serving-cert" not found
Jul 28 16:33:37.173386 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:33:37.173272    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/34ba2157-d9a4-42c8-834c-f349e8df2b76-proxy-tls podName:34ba2157-d9a4-42c8-834c-f349e8df2b76 nodeName:}" failed. No retries permitted until 2026-07-28 16:33:37.67325007 +0000 UTC m=+734.811719466 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/34ba2157-d9a4-42c8-834c-f349e8df2b76-proxy-tls") pod "success-200-isvc-7ba9d-predictor-68cbcf59bf-jt9kq" (UID: "34ba2157-d9a4-42c8-834c-f349e8df2b76") : secret "success-200-isvc-7ba9d-predictor-serving-cert" not found
Jul 28 16:33:37.173914 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:33:37.173895    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/34ba2157-d9a4-42c8-834c-f349e8df2b76-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-7ba9d-predictor-68cbcf59bf-jt9kq\" (UID: \"34ba2157-d9a4-42c8-834c-f349e8df2b76\") " pod="kserve-ci-e2e-test/success-200-isvc-7ba9d-predictor-68cbcf59bf-jt9kq"
Jul 28 16:33:37.181413 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:33:37.181392    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dwt4l\" (UniqueName: \"kubernetes.io/projected/34ba2157-d9a4-42c8-834c-f349e8df2b76-kube-api-access-dwt4l\") pod \"success-200-isvc-7ba9d-predictor-68cbcf59bf-jt9kq\" (UID: \"34ba2157-d9a4-42c8-834c-f349e8df2b76\") " pod="kserve-ci-e2e-test/success-200-isvc-7ba9d-predictor-68cbcf59bf-jt9kq"
Jul 28 16:33:37.270366 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:33:37.270323    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:33:37.371417 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:33:37.371327    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:33:37.563683 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:33:37.563653    2557 generic.go:356] "Generic (PLEG): container finished" podID="225e1d35-44de-4ab7-b20e-d543efa4539b" containerID="047efc18884e90f27eaf87f2c4d971f27c9d382425696cbd5ad0dbc752927e02" exitCode=2
Jul 28 16:33:37.563857 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:33:37.563730    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-b2ad5-predictor-666fbfd487-dt8k4" event={"ID":"225e1d35-44de-4ab7-b20e-d543efa4539b","Type":"ContainerDied","Data":"047efc18884e90f27eaf87f2c4d971f27c9d382425696cbd5ad0dbc752927e02"}
Jul 28 16:33:37.677700 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:33:37.677607    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/34ba2157-d9a4-42c8-834c-f349e8df2b76-proxy-tls\") pod \"success-200-isvc-7ba9d-predictor-68cbcf59bf-jt9kq\" (UID: \"34ba2157-d9a4-42c8-834c-f349e8df2b76\") " pod="kserve-ci-e2e-test/success-200-isvc-7ba9d-predictor-68cbcf59bf-jt9kq"
Jul 28 16:33:37.680218 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:33:37.680199    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/34ba2157-d9a4-42c8-834c-f349e8df2b76-proxy-tls\") pod \"success-200-isvc-7ba9d-predictor-68cbcf59bf-jt9kq\" (UID: \"34ba2157-d9a4-42c8-834c-f349e8df2b76\") " pod="kserve-ci-e2e-test/success-200-isvc-7ba9d-predictor-68cbcf59bf-jt9kq"
Jul 28 16:33:37.882675 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:33:37.882631    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-7ba9d-predictor-68cbcf59bf-jt9kq"
Jul 28 16:33:38.013393 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:33:38.013370    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-7ba9d-predictor-68cbcf59bf-jt9kq"]
Jul 28 16:33:38.015450 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:33:38.015426    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34ba2157_d9a4_42c8_834c_f349e8df2b76.slice/crio-5006493ae5277f3c4fdb6a60e93c587f2f0b34c9053333da6698ca9489055dd7 WatchSource:0}: Error finding container 5006493ae5277f3c4fdb6a60e93c587f2f0b34c9053333da6698ca9489055dd7: Status 404 returned error can't find the container with id 5006493ae5277f3c4fdb6a60e93c587f2f0b34c9053333da6698ca9489055dd7
Jul 28 16:33:38.241095 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:33:38.241002    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-b2ad5-predictor-666fbfd487-dt8k4" podUID="225e1d35-44de-4ab7-b20e-d543efa4539b" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.30:8643/healthz\": dial tcp 10.133.0.30:8643: connect: connection refused"
Jul 28 16:33:38.246389 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:33:38.246356    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-b2ad5-predictor-666fbfd487-dt8k4" podUID="225e1d35-44de-4ab7-b20e-d543efa4539b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.30:8080: connect: connection refused"
Jul 28 16:33:38.570421 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:33:38.570381    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-7ba9d-predictor-68cbcf59bf-jt9kq" event={"ID":"34ba2157-d9a4-42c8-834c-f349e8df2b76","Type":"ContainerStarted","Data":"4eaaa4483c214375574c0176800eecb47eec1fa6ce726778fdaf83e409effcda"}
Jul 28 16:33:38.570608 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:33:38.570437    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-7ba9d-predictor-68cbcf59bf-jt9kq" event={"ID":"34ba2157-d9a4-42c8-834c-f349e8df2b76","Type":"ContainerStarted","Data":"9e9af98159a4bb899cde7d2d18ecfafded240bf4a8e18664955b86ff8b70b4ba"}
Jul 28 16:33:38.570608 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:33:38.570458    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-7ba9d-predictor-68cbcf59bf-jt9kq" event={"ID":"34ba2157-d9a4-42c8-834c-f349e8df2b76","Type":"ContainerStarted","Data":"5006493ae5277f3c4fdb6a60e93c587f2f0b34c9053333da6698ca9489055dd7"}
Jul 28 16:33:38.570608 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:33:38.570481    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-7ba9d-predictor-68cbcf59bf-jt9kq"
Jul 28 16:33:38.570608 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:33:38.570523    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-7ba9d-predictor-68cbcf59bf-jt9kq"
Jul 28 16:33:38.571910 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:33:38.571882    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-7ba9d-predictor-68cbcf59bf-jt9kq" podUID="34ba2157-d9a4-42c8-834c-f349e8df2b76" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.31:8080: connect: connection refused"
Jul 28 16:33:38.591063 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:33:38.591014    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-7ba9d-predictor-68cbcf59bf-jt9kq" podStartSLOduration=2.590998225 podStartE2EDuration="2.590998225s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 16:33:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:33:38.58831824 +0000 UTC m=+735.726787655" watchObservedRunningTime="2026-07-28 16:33:38.590998225 +0000 UTC m=+735.729467638"
Jul 28 16:33:39.575374 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:33:39.575338    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-7ba9d-predictor-68cbcf59bf-jt9kq" podUID="34ba2157-d9a4-42c8-834c-f349e8df2b76" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.31:8080: connect: connection refused"
Jul 28 16:33:40.256135 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:33:40.256113    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-b2ad5-predictor-666fbfd487-dt8k4"
Jul 28 16:33:40.299958 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:33:40.299931    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/225e1d35-44de-4ab7-b20e-d543efa4539b-proxy-tls\") pod \"225e1d35-44de-4ab7-b20e-d543efa4539b\" (UID: \"225e1d35-44de-4ab7-b20e-d543efa4539b\") "
Jul 28 16:33:40.300128 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:33:40.299984    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cnfc8\" (UniqueName: \"kubernetes.io/projected/225e1d35-44de-4ab7-b20e-d543efa4539b-kube-api-access-cnfc8\") pod \"225e1d35-44de-4ab7-b20e-d543efa4539b\" (UID: \"225e1d35-44de-4ab7-b20e-d543efa4539b\") "
Jul 28 16:33:40.300128 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:33:40.300032    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/225e1d35-44de-4ab7-b20e-d543efa4539b-kube-rbac-proxy-sar-config\") pod \"225e1d35-44de-4ab7-b20e-d543efa4539b\" (UID: \"225e1d35-44de-4ab7-b20e-d543efa4539b\") "
Jul 28 16:33:40.300544 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:33:40.300486    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/225e1d35-44de-4ab7-b20e-d543efa4539b-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "225e1d35-44de-4ab7-b20e-d543efa4539b" (UID: "225e1d35-44de-4ab7-b20e-d543efa4539b"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 16:33:40.302366 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:33:40.302332    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/225e1d35-44de-4ab7-b20e-d543efa4539b-kube-api-access-cnfc8" (OuterVolumeSpecName: "kube-api-access-cnfc8") pod "225e1d35-44de-4ab7-b20e-d543efa4539b" (UID: "225e1d35-44de-4ab7-b20e-d543efa4539b"). InnerVolumeSpecName "kube-api-access-cnfc8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 16:33:40.302366 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:33:40.302341    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/225e1d35-44de-4ab7-b20e-d543efa4539b-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "225e1d35-44de-4ab7-b20e-d543efa4539b" (UID: "225e1d35-44de-4ab7-b20e-d543efa4539b"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 16:33:40.400973 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:33:40.400940    2557 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/225e1d35-44de-4ab7-b20e-d543efa4539b-proxy-tls\") on node \"ip-10-0-140-154.ec2.internal\" DevicePath \"\""
Jul 28 16:33:40.400973 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:33:40.400969    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cnfc8\" (UniqueName: \"kubernetes.io/projected/225e1d35-44de-4ab7-b20e-d543efa4539b-kube-api-access-cnfc8\") on node \"ip-10-0-140-154.ec2.internal\" DevicePath \"\""
Jul 28 16:33:40.401153 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:33:40.400984    2557 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/225e1d35-44de-4ab7-b20e-d543efa4539b-kube-rbac-proxy-sar-config\") on node \"ip-10-0-140-154.ec2.internal\" DevicePath \"\""
Jul 28 16:33:40.581846 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:33:40.581762    2557 generic.go:356] "Generic (PLEG): container finished" podID="225e1d35-44de-4ab7-b20e-d543efa4539b" containerID="4b4bb8e77578153d6cd9b4fc988b5ad9d30cf65def61c15e99931e63ca36fd5b" exitCode=0
Jul 28 16:33:40.581846 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:33:40.581814    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-b2ad5-predictor-666fbfd487-dt8k4"
Jul 28 16:33:40.582304 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:33:40.581847    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-b2ad5-predictor-666fbfd487-dt8k4" event={"ID":"225e1d35-44de-4ab7-b20e-d543efa4539b","Type":"ContainerDied","Data":"4b4bb8e77578153d6cd9b4fc988b5ad9d30cf65def61c15e99931e63ca36fd5b"}
Jul 28 16:33:40.582304 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:33:40.581911    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-b2ad5-predictor-666fbfd487-dt8k4" event={"ID":"225e1d35-44de-4ab7-b20e-d543efa4539b","Type":"ContainerDied","Data":"84a7889412c67f69411ce71df838bcd9adc4dc785b0a2fc56ee6ebfc31a38e8b"}
Jul 28 16:33:40.582304 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:33:40.581938    2557 scope.go:117] "RemoveContainer" containerID="047efc18884e90f27eaf87f2c4d971f27c9d382425696cbd5ad0dbc752927e02"
Jul 28 16:33:40.591088 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:33:40.591069    2557 scope.go:117] "RemoveContainer" containerID="4b4bb8e77578153d6cd9b4fc988b5ad9d30cf65def61c15e99931e63ca36fd5b"
Jul 28 16:33:40.598685 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:33:40.598668    2557 scope.go:117] "RemoveContainer" containerID="047efc18884e90f27eaf87f2c4d971f27c9d382425696cbd5ad0dbc752927e02"
Jul 28 16:33:40.598934 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:33:40.598917    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"047efc18884e90f27eaf87f2c4d971f27c9d382425696cbd5ad0dbc752927e02\": container with ID starting with 047efc18884e90f27eaf87f2c4d971f27c9d382425696cbd5ad0dbc752927e02 not found: ID does not exist" containerID="047efc18884e90f27eaf87f2c4d971f27c9d382425696cbd5ad0dbc752927e02"
Jul 28 16:33:40.599015 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:33:40.598946    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"047efc18884e90f27eaf87f2c4d971f27c9d382425696cbd5ad0dbc752927e02"} err="failed to get container status \"047efc18884e90f27eaf87f2c4d971f27c9d382425696cbd5ad0dbc752927e02\": rpc error: code = NotFound desc = could not find container \"047efc18884e90f27eaf87f2c4d971f27c9d382425696cbd5ad0dbc752927e02\": container with ID starting with 047efc18884e90f27eaf87f2c4d971f27c9d382425696cbd5ad0dbc752927e02 not found: ID does not exist"
Jul 28 16:33:40.599015 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:33:40.598969    2557 scope.go:117] "RemoveContainer" containerID="4b4bb8e77578153d6cd9b4fc988b5ad9d30cf65def61c15e99931e63ca36fd5b"
Jul 28 16:33:40.599196 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:33:40.599179    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4b4bb8e77578153d6cd9b4fc988b5ad9d30cf65def61c15e99931e63ca36fd5b\": container with ID starting with 4b4bb8e77578153d6cd9b4fc988b5ad9d30cf65def61c15e99931e63ca36fd5b not found: ID does not exist" containerID="4b4bb8e77578153d6cd9b4fc988b5ad9d30cf65def61c15e99931e63ca36fd5b"
Jul 28 16:33:40.599237 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:33:40.599203    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4b4bb8e77578153d6cd9b4fc988b5ad9d30cf65def61c15e99931e63ca36fd5b"} err="failed to get container status \"4b4bb8e77578153d6cd9b4fc988b5ad9d30cf65def61c15e99931e63ca36fd5b\": rpc error: code = NotFound desc = could not find container \"4b4bb8e77578153d6cd9b4fc988b5ad9d30cf65def61c15e99931e63ca36fd5b\": container with ID starting with 4b4bb8e77578153d6cd9b4fc988b5ad9d30cf65def61c15e99931e63ca36fd5b not found: ID does not exist"
Jul 28 16:33:40.603620 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:33:40.603601    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-b2ad5-predictor-666fbfd487-dt8k4"]
Jul 28 16:33:40.607125 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:33:40.607106    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-b2ad5-predictor-666fbfd487-dt8k4"]
Jul 28 16:33:41.435816 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:33:41.435757    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="225e1d35-44de-4ab7-b20e-d543efa4539b" path="/var/lib/kubelet/pods/225e1d35-44de-4ab7-b20e-d543efa4539b/volumes"
Jul 28 16:33:44.579705 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:33:44.579677    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-7ba9d-predictor-68cbcf59bf-jt9kq"
Jul 28 16:33:44.580162 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:33:44.580123    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-7ba9d-predictor-68cbcf59bf-jt9kq" podUID="34ba2157-d9a4-42c8-834c-f349e8df2b76" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.31:8080: connect: connection refused"
Jul 28 16:33:50.311186 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:33:50.311155    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:33:50.412916 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:33:50.412877    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:33:54.580217 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:33:54.580174    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-7ba9d-predictor-68cbcf59bf-jt9kq" podUID="34ba2157-d9a4-42c8-834c-f349e8df2b76" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.31:8080: connect: connection refused"
Jul 28 16:34:04.581130 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:34:04.581093    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-7ba9d-predictor-68cbcf59bf-jt9kq" podUID="34ba2157-d9a4-42c8-834c-f349e8df2b76" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.31:8080: connect: connection refused"
Jul 28 16:34:14.580758 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:34:14.580721    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-7ba9d-predictor-68cbcf59bf-jt9kq" podUID="34ba2157-d9a4-42c8-834c-f349e8df2b76" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.31:8080: connect: connection refused"
Jul 28 16:34:24.076197 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:34:24.076162    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5d6c64b478-xxx5w"]
Jul 28 16:34:24.076622 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:34:24.076528    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="225e1d35-44de-4ab7-b20e-d543efa4539b" containerName="kserve-container"
Jul 28 16:34:24.076622 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:34:24.076539    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="225e1d35-44de-4ab7-b20e-d543efa4539b" containerName="kserve-container"
Jul 28 16:34:24.076622 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:34:24.076546    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="225e1d35-44de-4ab7-b20e-d543efa4539b" containerName="kube-rbac-proxy"
Jul 28 16:34:24.076622 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:34:24.076552    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="225e1d35-44de-4ab7-b20e-d543efa4539b" containerName="kube-rbac-proxy"
Jul 28 16:34:24.076622 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:34:24.076607    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="225e1d35-44de-4ab7-b20e-d543efa4539b" containerName="kserve-container"
Jul 28 16:34:24.076622 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:34:24.076614    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="225e1d35-44de-4ab7-b20e-d543efa4539b" containerName="kube-rbac-proxy"
Jul 28 16:34:24.079315 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:34:24.079297    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5d6c64b478-xxx5w"
Jul 28 16:34:24.089908 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:34:24.089883    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5d6c64b478-xxx5w"]
Jul 28 16:34:24.132165 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:34:24.132130    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:34:24.175404 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:34:24.175379    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mvl2w\" (UniqueName: \"kubernetes.io/projected/be7118f9-3ca8-4a48-b9ff-c90bc1478b54-kube-api-access-mvl2w\") pod \"console-5d6c64b478-xxx5w\" (UID: \"be7118f9-3ca8-4a48-b9ff-c90bc1478b54\") " pod="openshift-console/console-5d6c64b478-xxx5w"
Jul 28 16:34:24.175548 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:34:24.175411    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/be7118f9-3ca8-4a48-b9ff-c90bc1478b54-console-serving-cert\") pod \"console-5d6c64b478-xxx5w\" (UID: \"be7118f9-3ca8-4a48-b9ff-c90bc1478b54\") " pod="openshift-console/console-5d6c64b478-xxx5w"
Jul 28 16:34:24.175548 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:34:24.175441    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/be7118f9-3ca8-4a48-b9ff-c90bc1478b54-console-config\") pod \"console-5d6c64b478-xxx5w\" (UID: \"be7118f9-3ca8-4a48-b9ff-c90bc1478b54\") " pod="openshift-console/console-5d6c64b478-xxx5w"
Jul 28 16:34:24.175548 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:34:24.175465    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/be7118f9-3ca8-4a48-b9ff-c90bc1478b54-oauth-serving-cert\") pod \"console-5d6c64b478-xxx5w\" (UID: \"be7118f9-3ca8-4a48-b9ff-c90bc1478b54\") " pod="openshift-console/console-5d6c64b478-xxx5w"
Jul 28 16:34:24.175660 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:34:24.175551    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/be7118f9-3ca8-4a48-b9ff-c90bc1478b54-trusted-ca-bundle\") pod \"console-5d6c64b478-xxx5w\" (UID: \"be7118f9-3ca8-4a48-b9ff-c90bc1478b54\") " pod="openshift-console/console-5d6c64b478-xxx5w"
Jul 28 16:34:24.175660 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:34:24.175596    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/be7118f9-3ca8-4a48-b9ff-c90bc1478b54-console-oauth-config\") pod \"console-5d6c64b478-xxx5w\" (UID: \"be7118f9-3ca8-4a48-b9ff-c90bc1478b54\") " pod="openshift-console/console-5d6c64b478-xxx5w"
Jul 28 16:34:24.175660 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:34:24.175653    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/be7118f9-3ca8-4a48-b9ff-c90bc1478b54-service-ca\") pod \"console-5d6c64b478-xxx5w\" (UID: \"be7118f9-3ca8-4a48-b9ff-c90bc1478b54\") " pod="openshift-console/console-5d6c64b478-xxx5w"
Jul 28 16:34:24.232337 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:34:24.232315    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:34:24.276923 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:34:24.276898    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/be7118f9-3ca8-4a48-b9ff-c90bc1478b54-trusted-ca-bundle\") pod \"console-5d6c64b478-xxx5w\" (UID: \"be7118f9-3ca8-4a48-b9ff-c90bc1478b54\") " pod="openshift-console/console-5d6c64b478-xxx5w"
Jul 28 16:34:24.277048 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:34:24.276937    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/be7118f9-3ca8-4a48-b9ff-c90bc1478b54-console-oauth-config\") pod \"console-5d6c64b478-xxx5w\" (UID: \"be7118f9-3ca8-4a48-b9ff-c90bc1478b54\") " pod="openshift-console/console-5d6c64b478-xxx5w"
Jul 28 16:34:24.277048 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:34:24.276961    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/be7118f9-3ca8-4a48-b9ff-c90bc1478b54-service-ca\") pod \"console-5d6c64b478-xxx5w\" (UID: \"be7118f9-3ca8-4a48-b9ff-c90bc1478b54\") " pod="openshift-console/console-5d6c64b478-xxx5w"
Jul 28 16:34:24.277048 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:34:24.276991    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mvl2w\" (UniqueName: \"kubernetes.io/projected/be7118f9-3ca8-4a48-b9ff-c90bc1478b54-kube-api-access-mvl2w\") pod \"console-5d6c64b478-xxx5w\" (UID: \"be7118f9-3ca8-4a48-b9ff-c90bc1478b54\") " pod="openshift-console/console-5d6c64b478-xxx5w"
Jul 28 16:34:24.277048 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:34:24.277010    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/be7118f9-3ca8-4a48-b9ff-c90bc1478b54-console-serving-cert\") pod \"console-5d6c64b478-xxx5w\" (UID: \"be7118f9-3ca8-4a48-b9ff-c90bc1478b54\") " pod="openshift-console/console-5d6c64b478-xxx5w"
Jul 28 16:34:24.277251 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:34:24.277070    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/be7118f9-3ca8-4a48-b9ff-c90bc1478b54-console-config\") pod \"console-5d6c64b478-xxx5w\" (UID: \"be7118f9-3ca8-4a48-b9ff-c90bc1478b54\") " pod="openshift-console/console-5d6c64b478-xxx5w"
Jul 28 16:34:24.277251 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:34:24.277100    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/be7118f9-3ca8-4a48-b9ff-c90bc1478b54-oauth-serving-cert\") pod \"console-5d6c64b478-xxx5w\" (UID: \"be7118f9-3ca8-4a48-b9ff-c90bc1478b54\") " pod="openshift-console/console-5d6c64b478-xxx5w"
Jul 28 16:34:24.277774 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:34:24.277749    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/be7118f9-3ca8-4a48-b9ff-c90bc1478b54-console-config\") pod \"console-5d6c64b478-xxx5w\" (UID: \"be7118f9-3ca8-4a48-b9ff-c90bc1478b54\") " pod="openshift-console/console-5d6c64b478-xxx5w"
Jul 28 16:34:24.277868 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:34:24.277850    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/be7118f9-3ca8-4a48-b9ff-c90bc1478b54-oauth-serving-cert\") pod \"console-5d6c64b478-xxx5w\" (UID: \"be7118f9-3ca8-4a48-b9ff-c90bc1478b54\") " pod="openshift-console/console-5d6c64b478-xxx5w"
Jul 28 16:34:24.277951 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:34:24.277865    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/be7118f9-3ca8-4a48-b9ff-c90bc1478b54-service-ca\") pod \"console-5d6c64b478-xxx5w\" (UID: \"be7118f9-3ca8-4a48-b9ff-c90bc1478b54\") " pod="openshift-console/console-5d6c64b478-xxx5w"
Jul 28 16:34:24.278035 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:34:24.278015    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/be7118f9-3ca8-4a48-b9ff-c90bc1478b54-trusted-ca-bundle\") pod \"console-5d6c64b478-xxx5w\" (UID: \"be7118f9-3ca8-4a48-b9ff-c90bc1478b54\") " pod="openshift-console/console-5d6c64b478-xxx5w"
Jul 28 16:34:24.279647 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:34:24.279622    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/be7118f9-3ca8-4a48-b9ff-c90bc1478b54-console-oauth-config\") pod \"console-5d6c64b478-xxx5w\" (UID: \"be7118f9-3ca8-4a48-b9ff-c90bc1478b54\") " pod="openshift-console/console-5d6c64b478-xxx5w"
Jul 28 16:34:24.279741 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:34:24.279723    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/be7118f9-3ca8-4a48-b9ff-c90bc1478b54-console-serving-cert\") pod \"console-5d6c64b478-xxx5w\" (UID: \"be7118f9-3ca8-4a48-b9ff-c90bc1478b54\") " pod="openshift-console/console-5d6c64b478-xxx5w"
Jul 28 16:34:24.285896 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:34:24.285875    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mvl2w\" (UniqueName: \"kubernetes.io/projected/be7118f9-3ca8-4a48-b9ff-c90bc1478b54-kube-api-access-mvl2w\") pod \"console-5d6c64b478-xxx5w\" (UID: \"be7118f9-3ca8-4a48-b9ff-c90bc1478b54\") " pod="openshift-console/console-5d6c64b478-xxx5w"
Jul 28 16:34:24.389671 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:34:24.389598    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5d6c64b478-xxx5w"
Jul 28 16:34:24.518194 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:34:24.518170    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5d6c64b478-xxx5w"]
Jul 28 16:34:24.520245 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:34:24.520204    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe7118f9_3ca8_4a48_b9ff_c90bc1478b54.slice/crio-f004c5296227344eb37176ff1f0b97064806849b97833dde38fb02cdf6d4f775 WatchSource:0}: Error finding container f004c5296227344eb37176ff1f0b97064806849b97833dde38fb02cdf6d4f775: Status 404 returned error can't find the container with id f004c5296227344eb37176ff1f0b97064806849b97833dde38fb02cdf6d4f775
Jul 28 16:34:24.580676 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:34:24.580657    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-7ba9d-predictor-68cbcf59bf-jt9kq"
Jul 28 16:34:24.779894 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:34:24.779781    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5d6c64b478-xxx5w" event={"ID":"be7118f9-3ca8-4a48-b9ff-c90bc1478b54","Type":"ContainerStarted","Data":"43ee31443685d9f0e270eedc8a6a2ebbecd0fac564ac421d6a6f91692f402e84"}
Jul 28 16:34:24.779894 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:34:24.779853    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5d6c64b478-xxx5w" event={"ID":"be7118f9-3ca8-4a48-b9ff-c90bc1478b54","Type":"ContainerStarted","Data":"f004c5296227344eb37176ff1f0b97064806849b97833dde38fb02cdf6d4f775"}
Jul 28 16:34:24.795802 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:34:24.795754    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5d6c64b478-xxx5w" podStartSLOduration=0.795740023 podStartE2EDuration="795.740023ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 16:34:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:34:24.795463094 +0000 UTC m=+781.933932510" watchObservedRunningTime="2026-07-28 16:34:24.795740023 +0000 UTC m=+781.934209437"
Jul 28 16:34:26.730208 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:34:26.730170    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:34:26.833137 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:34:26.833097    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:34:34.390333 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:34:34.390291    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5d6c64b478-xxx5w"
Jul 28 16:34:34.390808 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:34:34.390358    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5d6c64b478-xxx5w"
Jul 28 16:34:34.395123 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:34:34.395102    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5d6c64b478-xxx5w"
Jul 28 16:34:34.834020 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:34:34.833993    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5d6c64b478-xxx5w"
Jul 28 16:34:34.881743 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:34:34.881711    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6c499c9d9d-cbwrq"]
Jul 28 16:34:34.981458 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:34:34.981420    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:34:35.045197 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:34:35.045159    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:34:35.129224 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:34:35.129121    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:34:36.865879 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:34:36.865839    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:34:36.987610 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:34:36.987576    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:34:37.029560 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:34:37.029525    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-2b606-predictor-64d46d8575-sbrbh"]
Jul 28 16:34:37.032810 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:34:37.032785    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-2b606-predictor-64d46d8575-sbrbh"
Jul 28 16:34:37.035400 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:34:37.035377    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-2b606-kube-rbac-proxy-sar-config\""
Jul 28 16:34:37.035554 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:34:37.035377    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-2b606-predictor-serving-cert\""
Jul 28 16:34:37.051430 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:34:37.051394    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-2b606-predictor-64d46d8575-sbrbh"]
Jul 28 16:34:37.075390 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:34:37.075353    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:34:37.083350 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:34:37.083324    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/dd8f78ab-362e-4ef5-8aa1-4362e5de1374-proxy-tls\") pod \"success-200-isvc-2b606-predictor-64d46d8575-sbrbh\" (UID: \"dd8f78ab-362e-4ef5-8aa1-4362e5de1374\") " pod="kserve-ci-e2e-test/success-200-isvc-2b606-predictor-64d46d8575-sbrbh"
Jul 28 16:34:37.083515 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:34:37.083377    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ppwq2\" (UniqueName: \"kubernetes.io/projected/dd8f78ab-362e-4ef5-8aa1-4362e5de1374-kube-api-access-ppwq2\") pod \"success-200-isvc-2b606-predictor-64d46d8575-sbrbh\" (UID: \"dd8f78ab-362e-4ef5-8aa1-4362e5de1374\") " pod="kserve-ci-e2e-test/success-200-isvc-2b606-predictor-64d46d8575-sbrbh"
Jul 28 16:34:37.083515 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:34:37.083444    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/dd8f78ab-362e-4ef5-8aa1-4362e5de1374-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-2b606-predictor-64d46d8575-sbrbh\" (UID: \"dd8f78ab-362e-4ef5-8aa1-4362e5de1374\") " pod="kserve-ci-e2e-test/success-200-isvc-2b606-predictor-64d46d8575-sbrbh"
Jul 28 16:34:37.184818 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:34:37.184730    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/dd8f78ab-362e-4ef5-8aa1-4362e5de1374-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-2b606-predictor-64d46d8575-sbrbh\" (UID: \"dd8f78ab-362e-4ef5-8aa1-4362e5de1374\") " pod="kserve-ci-e2e-test/success-200-isvc-2b606-predictor-64d46d8575-sbrbh"
Jul 28 16:34:37.184818 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:34:37.184802    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/dd8f78ab-362e-4ef5-8aa1-4362e5de1374-proxy-tls\") pod \"success-200-isvc-2b606-predictor-64d46d8575-sbrbh\" (UID: \"dd8f78ab-362e-4ef5-8aa1-4362e5de1374\") " pod="kserve-ci-e2e-test/success-200-isvc-2b606-predictor-64d46d8575-sbrbh"
Jul 28 16:34:37.185087 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:34:37.184848    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ppwq2\" (UniqueName: \"kubernetes.io/projected/dd8f78ab-362e-4ef5-8aa1-4362e5de1374-kube-api-access-ppwq2\") pod \"success-200-isvc-2b606-predictor-64d46d8575-sbrbh\" (UID: \"dd8f78ab-362e-4ef5-8aa1-4362e5de1374\") " pod="kserve-ci-e2e-test/success-200-isvc-2b606-predictor-64d46d8575-sbrbh"
Jul 28 16:34:37.185087 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:34:37.184980    2557 secret.go:189] Couldn't get secret kserve-ci-e2e-test/success-200-isvc-2b606-predictor-serving-cert: secret "success-200-isvc-2b606-predictor-serving-cert" not found
Jul 28 16:34:37.185087 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:34:37.185062    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dd8f78ab-362e-4ef5-8aa1-4362e5de1374-proxy-tls podName:dd8f78ab-362e-4ef5-8aa1-4362e5de1374 nodeName:}" failed. No retries permitted until 2026-07-28 16:34:37.685040229 +0000 UTC m=+794.823509626 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/dd8f78ab-362e-4ef5-8aa1-4362e5de1374-proxy-tls") pod "success-200-isvc-2b606-predictor-64d46d8575-sbrbh" (UID: "dd8f78ab-362e-4ef5-8aa1-4362e5de1374") : secret "success-200-isvc-2b606-predictor-serving-cert" not found
Jul 28 16:34:37.185595 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:34:37.185573    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/dd8f78ab-362e-4ef5-8aa1-4362e5de1374-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-2b606-predictor-64d46d8575-sbrbh\" (UID: \"dd8f78ab-362e-4ef5-8aa1-4362e5de1374\") " pod="kserve-ci-e2e-test/success-200-isvc-2b606-predictor-64d46d8575-sbrbh"
Jul 28 16:34:37.193511 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:34:37.193457    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:34:37.201679 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:34:37.201650    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ppwq2\" (UniqueName: \"kubernetes.io/projected/dd8f78ab-362e-4ef5-8aa1-4362e5de1374-kube-api-access-ppwq2\") pod \"success-200-isvc-2b606-predictor-64d46d8575-sbrbh\" (UID: \"dd8f78ab-362e-4ef5-8aa1-4362e5de1374\") " pod="kserve-ci-e2e-test/success-200-isvc-2b606-predictor-64d46d8575-sbrbh"
Jul 28 16:34:37.289665 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:34:37.289635    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:34:37.482462 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:34:37.482383    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:34:37.690904 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:34:37.690877    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/dd8f78ab-362e-4ef5-8aa1-4362e5de1374-proxy-tls\") pod \"success-200-isvc-2b606-predictor-64d46d8575-sbrbh\" (UID: \"dd8f78ab-362e-4ef5-8aa1-4362e5de1374\") " pod="kserve-ci-e2e-test/success-200-isvc-2b606-predictor-64d46d8575-sbrbh"
Jul 28 16:34:37.691088 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:34:37.691017    2557 secret.go:189] Couldn't get secret kserve-ci-e2e-test/success-200-isvc-2b606-predictor-serving-cert: secret "success-200-isvc-2b606-predictor-serving-cert" not found
Jul 28 16:34:37.691088 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:34:37.691075    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dd8f78ab-362e-4ef5-8aa1-4362e5de1374-proxy-tls podName:dd8f78ab-362e-4ef5-8aa1-4362e5de1374 nodeName:}" failed. No retries permitted until 2026-07-28 16:34:38.691061603 +0000 UTC m=+795.829530995 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/dd8f78ab-362e-4ef5-8aa1-4362e5de1374-proxy-tls") pod "success-200-isvc-2b606-predictor-64d46d8575-sbrbh" (UID: "dd8f78ab-362e-4ef5-8aa1-4362e5de1374") : secret "success-200-isvc-2b606-predictor-serving-cert" not found
Jul 28 16:34:37.850095 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:34:37.850060    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:34:38.699771 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:34:38.699732    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/dd8f78ab-362e-4ef5-8aa1-4362e5de1374-proxy-tls\") pod \"success-200-isvc-2b606-predictor-64d46d8575-sbrbh\" (UID: \"dd8f78ab-362e-4ef5-8aa1-4362e5de1374\") " pod="kserve-ci-e2e-test/success-200-isvc-2b606-predictor-64d46d8575-sbrbh"
Jul 28 16:34:38.702367 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:34:38.702338    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/dd8f78ab-362e-4ef5-8aa1-4362e5de1374-proxy-tls\") pod \"success-200-isvc-2b606-predictor-64d46d8575-sbrbh\" (UID: \"dd8f78ab-362e-4ef5-8aa1-4362e5de1374\") " pod="kserve-ci-e2e-test/success-200-isvc-2b606-predictor-64d46d8575-sbrbh"
Jul 28 16:34:38.852354 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:34:38.852322    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-2b606-predictor-64d46d8575-sbrbh"
Jul 28 16:34:38.987741 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:34:38.987715    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-2b606-predictor-64d46d8575-sbrbh"]
Jul 28 16:34:38.989752 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:34:38.989724    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd8f78ab_362e_4ef5_8aa1_4362e5de1374.slice/crio-fc3c9e2652a1c7c1f9f8fb0ee506bcb1728fd275b37020cfdcbb94408d89d5c9 WatchSource:0}: Error finding container fc3c9e2652a1c7c1f9f8fb0ee506bcb1728fd275b37020cfdcbb94408d89d5c9: Status 404 returned error can't find the container with id fc3c9e2652a1c7c1f9f8fb0ee506bcb1728fd275b37020cfdcbb94408d89d5c9
Jul 28 16:34:39.855139 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:34:39.855098    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-2b606-predictor-64d46d8575-sbrbh" event={"ID":"dd8f78ab-362e-4ef5-8aa1-4362e5de1374","Type":"ContainerStarted","Data":"e2e77d4f3b18229b662d12d40a966f5fde6ee6882ab9fc626a89aa6e0a126f3f"}
Jul 28 16:34:39.855668 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:34:39.855198    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-2b606-predictor-64d46d8575-sbrbh" event={"ID":"dd8f78ab-362e-4ef5-8aa1-4362e5de1374","Type":"ContainerStarted","Data":"b147f15f385e3d87a1df8bb0baa2d8747763f5626b212a70958901df79886e54"}
Jul 28 16:34:39.855668 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:34:39.855224    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-2b606-predictor-64d46d8575-sbrbh" event={"ID":"dd8f78ab-362e-4ef5-8aa1-4362e5de1374","Type":"ContainerStarted","Data":"fc3c9e2652a1c7c1f9f8fb0ee506bcb1728fd275b37020cfdcbb94408d89d5c9"}
Jul 28 16:34:39.855668 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:34:39.855246    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-2b606-predictor-64d46d8575-sbrbh"
Jul 28 16:34:39.855668 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:34:39.855265    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-2b606-predictor-64d46d8575-sbrbh"
Jul 28 16:34:39.856828 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:34:39.856798    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-2b606-predictor-64d46d8575-sbrbh" podUID="dd8f78ab-362e-4ef5-8aa1-4362e5de1374" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.33:8080: connect: connection refused"
Jul 28 16:34:39.873041 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:34:39.872998    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-2b606-predictor-64d46d8575-sbrbh" podStartSLOduration=2.872980581 podStartE2EDuration="2.872980581s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 16:34:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:34:39.871635772 +0000 UTC m=+797.010105186" watchObservedRunningTime="2026-07-28 16:34:39.872980581 +0000 UTC m=+797.011449995"
Jul 28 16:34:40.860215 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:34:40.860177    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-2b606-predictor-64d46d8575-sbrbh" podUID="dd8f78ab-362e-4ef5-8aa1-4362e5de1374" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.33:8080: connect: connection refused"
Jul 28 16:34:45.865757 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:34:45.865676    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-2b606-predictor-64d46d8575-sbrbh"
Jul 28 16:34:45.866188 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:34:45.866161    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-2b606-predictor-64d46d8575-sbrbh" podUID="dd8f78ab-362e-4ef5-8aa1-4362e5de1374" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.33:8080: connect: connection refused"
Jul 28 16:34:55.867067 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:34:55.867024    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-2b606-predictor-64d46d8575-sbrbh" podUID="dd8f78ab-362e-4ef5-8aa1-4362e5de1374" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.33:8080: connect: connection refused"
Jul 28 16:34:59.907119 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:34:59.907055    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6c499c9d9d-cbwrq" podUID="882c3721-c8ce-4ded-adfd-1cd2b5606da6" containerName="console" containerID="cri-o://ff40227249b925ca3a5b5b9213edfcb9ce838fc917f0db8ff37f77a345ea9455" gracePeriod=15
Jul 28 16:35:00.155402 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:35:00.155378    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c499c9d9d-cbwrq"
Jul 28 16:35:00.290762 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:35:00.290675    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mvsd5\" (UniqueName: \"kubernetes.io/projected/882c3721-c8ce-4ded-adfd-1cd2b5606da6-kube-api-access-mvsd5\") pod \"882c3721-c8ce-4ded-adfd-1cd2b5606da6\" (UID: \"882c3721-c8ce-4ded-adfd-1cd2b5606da6\") "
Jul 28 16:35:00.290762 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:35:00.290717    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/882c3721-c8ce-4ded-adfd-1cd2b5606da6-console-serving-cert\") pod \"882c3721-c8ce-4ded-adfd-1cd2b5606da6\" (UID: \"882c3721-c8ce-4ded-adfd-1cd2b5606da6\") "
Jul 28 16:35:00.290762 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:35:00.290737    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/882c3721-c8ce-4ded-adfd-1cd2b5606da6-trusted-ca-bundle\") pod \"882c3721-c8ce-4ded-adfd-1cd2b5606da6\" (UID: \"882c3721-c8ce-4ded-adfd-1cd2b5606da6\") "
Jul 28 16:35:00.290762 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:35:00.290755    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/882c3721-c8ce-4ded-adfd-1cd2b5606da6-console-oauth-config\") pod \"882c3721-c8ce-4ded-adfd-1cd2b5606da6\" (UID: \"882c3721-c8ce-4ded-adfd-1cd2b5606da6\") "
Jul 28 16:35:00.291081 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:35:00.290797    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/882c3721-c8ce-4ded-adfd-1cd2b5606da6-console-config\") pod \"882c3721-c8ce-4ded-adfd-1cd2b5606da6\" (UID: \"882c3721-c8ce-4ded-adfd-1cd2b5606da6\") "
Jul 28 16:35:00.291081 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:35:00.290841    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/882c3721-c8ce-4ded-adfd-1cd2b5606da6-oauth-serving-cert\") pod \"882c3721-c8ce-4ded-adfd-1cd2b5606da6\" (UID: \"882c3721-c8ce-4ded-adfd-1cd2b5606da6\") "
Jul 28 16:35:00.291081 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:35:00.290908    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/882c3721-c8ce-4ded-adfd-1cd2b5606da6-service-ca\") pod \"882c3721-c8ce-4ded-adfd-1cd2b5606da6\" (UID: \"882c3721-c8ce-4ded-adfd-1cd2b5606da6\") "
Jul 28 16:35:00.291381 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:35:00.291308    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/882c3721-c8ce-4ded-adfd-1cd2b5606da6-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "882c3721-c8ce-4ded-adfd-1cd2b5606da6" (UID: "882c3721-c8ce-4ded-adfd-1cd2b5606da6"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 16:35:00.291381 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:35:00.291334    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/882c3721-c8ce-4ded-adfd-1cd2b5606da6-console-config" (OuterVolumeSpecName: "console-config") pod "882c3721-c8ce-4ded-adfd-1cd2b5606da6" (UID: "882c3721-c8ce-4ded-adfd-1cd2b5606da6"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 16:35:00.291556 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:35:00.291421    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/882c3721-c8ce-4ded-adfd-1cd2b5606da6-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "882c3721-c8ce-4ded-adfd-1cd2b5606da6" (UID: "882c3721-c8ce-4ded-adfd-1cd2b5606da6"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 16:35:00.291556 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:35:00.291459    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/882c3721-c8ce-4ded-adfd-1cd2b5606da6-service-ca" (OuterVolumeSpecName: "service-ca") pod "882c3721-c8ce-4ded-adfd-1cd2b5606da6" (UID: "882c3721-c8ce-4ded-adfd-1cd2b5606da6"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 16:35:00.293079 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:35:00.293056    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/882c3721-c8ce-4ded-adfd-1cd2b5606da6-kube-api-access-mvsd5" (OuterVolumeSpecName: "kube-api-access-mvsd5") pod "882c3721-c8ce-4ded-adfd-1cd2b5606da6" (UID: "882c3721-c8ce-4ded-adfd-1cd2b5606da6"). InnerVolumeSpecName "kube-api-access-mvsd5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 16:35:00.293567 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:35:00.293542    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/882c3721-c8ce-4ded-adfd-1cd2b5606da6-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "882c3721-c8ce-4ded-adfd-1cd2b5606da6" (UID: "882c3721-c8ce-4ded-adfd-1cd2b5606da6"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 16:35:00.293567 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:35:00.293554    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/882c3721-c8ce-4ded-adfd-1cd2b5606da6-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "882c3721-c8ce-4ded-adfd-1cd2b5606da6" (UID: "882c3721-c8ce-4ded-adfd-1cd2b5606da6"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 16:35:00.391853 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:35:00.391806    2557 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/882c3721-c8ce-4ded-adfd-1cd2b5606da6-console-config\") on node \"ip-10-0-140-154.ec2.internal\" DevicePath \"\""
Jul 28 16:35:00.391853 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:35:00.391849    2557 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/882c3721-c8ce-4ded-adfd-1cd2b5606da6-oauth-serving-cert\") on node \"ip-10-0-140-154.ec2.internal\" DevicePath \"\""
Jul 28 16:35:00.391853 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:35:00.391859    2557 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/882c3721-c8ce-4ded-adfd-1cd2b5606da6-service-ca\") on node \"ip-10-0-140-154.ec2.internal\" DevicePath \"\""
Jul 28 16:35:00.391853 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:35:00.391867    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mvsd5\" (UniqueName: \"kubernetes.io/projected/882c3721-c8ce-4ded-adfd-1cd2b5606da6-kube-api-access-mvsd5\") on node \"ip-10-0-140-154.ec2.internal\" DevicePath \"\""
Jul 28 16:35:00.392082 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:35:00.391877    2557 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/882c3721-c8ce-4ded-adfd-1cd2b5606da6-console-serving-cert\") on node \"ip-10-0-140-154.ec2.internal\" DevicePath \"\""
Jul 28 16:35:00.392082 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:35:00.391885    2557 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/882c3721-c8ce-4ded-adfd-1cd2b5606da6-trusted-ca-bundle\") on node \"ip-10-0-140-154.ec2.internal\" DevicePath \"\""
Jul 28 16:35:00.392082 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:35:00.391893    2557 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/882c3721-c8ce-4ded-adfd-1cd2b5606da6-console-oauth-config\") on node \"ip-10-0-140-154.ec2.internal\" DevicePath \"\""
Jul 28 16:35:00.954803 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:35:00.954766    2557 generic.go:356] "Generic (PLEG): container finished" podID="882c3721-c8ce-4ded-adfd-1cd2b5606da6" containerID="ff40227249b925ca3a5b5b9213edfcb9ce838fc917f0db8ff37f77a345ea9455" exitCode=0
Jul 28 16:35:00.955190 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:35:00.954813    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c499c9d9d-cbwrq"
Jul 28 16:35:00.955190 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:35:00.954847    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c499c9d9d-cbwrq" event={"ID":"882c3721-c8ce-4ded-adfd-1cd2b5606da6","Type":"ContainerDied","Data":"ff40227249b925ca3a5b5b9213edfcb9ce838fc917f0db8ff37f77a345ea9455"}
Jul 28 16:35:00.955190 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:35:00.954898    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c499c9d9d-cbwrq" event={"ID":"882c3721-c8ce-4ded-adfd-1cd2b5606da6","Type":"ContainerDied","Data":"625928ce9891049196bebe556d165db86e4a5a37c1ab88c0a0a8f15f3c1b869c"}
Jul 28 16:35:00.955190 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:35:00.954919    2557 scope.go:117] "RemoveContainer" containerID="ff40227249b925ca3a5b5b9213edfcb9ce838fc917f0db8ff37f77a345ea9455"
Jul 28 16:35:00.966017 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:35:00.965999    2557 scope.go:117] "RemoveContainer" containerID="ff40227249b925ca3a5b5b9213edfcb9ce838fc917f0db8ff37f77a345ea9455"
Jul 28 16:35:00.966277 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:35:00.966259    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ff40227249b925ca3a5b5b9213edfcb9ce838fc917f0db8ff37f77a345ea9455\": container with ID starting with ff40227249b925ca3a5b5b9213edfcb9ce838fc917f0db8ff37f77a345ea9455 not found: ID does not exist" containerID="ff40227249b925ca3a5b5b9213edfcb9ce838fc917f0db8ff37f77a345ea9455"
Jul 28 16:35:00.966317 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:35:00.966285    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ff40227249b925ca3a5b5b9213edfcb9ce838fc917f0db8ff37f77a345ea9455"} err="failed to get container status \"ff40227249b925ca3a5b5b9213edfcb9ce838fc917f0db8ff37f77a345ea9455\": rpc error: code = NotFound desc = could not find container \"ff40227249b925ca3a5b5b9213edfcb9ce838fc917f0db8ff37f77a345ea9455\": container with ID starting with ff40227249b925ca3a5b5b9213edfcb9ce838fc917f0db8ff37f77a345ea9455 not found: ID does not exist"
Jul 28 16:35:00.977232 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:35:00.977206    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6c499c9d9d-cbwrq"]
Jul 28 16:35:00.979224 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:35:00.979206    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6c499c9d9d-cbwrq"]
Jul 28 16:35:01.435467 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:35:01.435431    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="882c3721-c8ce-4ded-adfd-1cd2b5606da6" path="/var/lib/kubelet/pods/882c3721-c8ce-4ded-adfd-1cd2b5606da6/volumes"
Jul 28 16:35:05.866896 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:35:05.866851    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-2b606-predictor-64d46d8575-sbrbh" podUID="dd8f78ab-362e-4ef5-8aa1-4362e5de1374" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.33:8080: connect: connection refused"
Jul 28 16:35:15.866104 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:35:15.866059    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-2b606-predictor-64d46d8575-sbrbh" podUID="dd8f78ab-362e-4ef5-8aa1-4362e5de1374" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.33:8080: connect: connection refused"
Jul 28 16:35:25.867346 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:35:25.867312    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-2b606-predictor-64d46d8575-sbrbh"
Jul 28 16:35:37.043077 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:35:37.043027    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:35:37.336962 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:35:37.336929    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:37:21.229211 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:37:21.229170    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:39:20.275145 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:39:20.275062    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:40:10.884058 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:40:10.884018    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:42:51.725388 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:42:51.725346    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:42:51.812602 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:42:51.812569    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-7ba9d-predictor-68cbcf59bf-jt9kq"]
Jul 28 16:42:51.831609 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:42:51.831580    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:42:51.864129 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:42:51.864105    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-68c6a-predictor-78785fb6b7-4wt4m"]
Jul 28 16:42:51.864473 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:42:51.864460    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="882c3721-c8ce-4ded-adfd-1cd2b5606da6" containerName="console"
Jul 28 16:42:51.864549 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:42:51.864475    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="882c3721-c8ce-4ded-adfd-1cd2b5606da6" containerName="console"
Jul 28 16:42:51.864588 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:42:51.864582    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="882c3721-c8ce-4ded-adfd-1cd2b5606da6" containerName="console"
Jul 28 16:42:51.867727 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:42:51.867708    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-68c6a-predictor-78785fb6b7-4wt4m"
Jul 28 16:42:51.869858 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:42:51.869838    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-68c6a-predictor-serving-cert\""
Jul 28 16:42:51.869973 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:42:51.869841    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-68c6a-kube-rbac-proxy-sar-config\""
Jul 28 16:42:51.878598 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:42:51.878577    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-68c6a-predictor-78785fb6b7-4wt4m"]
Jul 28 16:42:51.914717 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:42:51.914685    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:42:51.929807 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:42:51.929774    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/4e8697a3-088c-43ab-944d-223d98a45b03-proxy-tls\") pod \"success-200-isvc-68c6a-predictor-78785fb6b7-4wt4m\" (UID: \"4e8697a3-088c-43ab-944d-223d98a45b03\") " pod="kserve-ci-e2e-test/success-200-isvc-68c6a-predictor-78785fb6b7-4wt4m"
Jul 28 16:42:51.929936 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:42:51.929813    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/4e8697a3-088c-43ab-944d-223d98a45b03-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-68c6a-predictor-78785fb6b7-4wt4m\" (UID: \"4e8697a3-088c-43ab-944d-223d98a45b03\") " pod="kserve-ci-e2e-test/success-200-isvc-68c6a-predictor-78785fb6b7-4wt4m"
Jul 28 16:42:51.929936 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:42:51.929857    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ntqgm\" (UniqueName: \"kubernetes.io/projected/4e8697a3-088c-43ab-944d-223d98a45b03-kube-api-access-ntqgm\") pod \"success-200-isvc-68c6a-predictor-78785fb6b7-4wt4m\" (UID: \"4e8697a3-088c-43ab-944d-223d98a45b03\") " pod="kserve-ci-e2e-test/success-200-isvc-68c6a-predictor-78785fb6b7-4wt4m"
Jul 28 16:42:51.974742 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:42:51.974706    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:42:52.030858 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:42:52.030789    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/4e8697a3-088c-43ab-944d-223d98a45b03-proxy-tls\") pod \"success-200-isvc-68c6a-predictor-78785fb6b7-4wt4m\" (UID: \"4e8697a3-088c-43ab-944d-223d98a45b03\") " pod="kserve-ci-e2e-test/success-200-isvc-68c6a-predictor-78785fb6b7-4wt4m"
Jul 28 16:42:52.030858 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:42:52.030822    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/4e8697a3-088c-43ab-944d-223d98a45b03-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-68c6a-predictor-78785fb6b7-4wt4m\" (UID: \"4e8697a3-088c-43ab-944d-223d98a45b03\") " pod="kserve-ci-e2e-test/success-200-isvc-68c6a-predictor-78785fb6b7-4wt4m"
Jul 28 16:42:52.030858 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:42:52.030851    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ntqgm\" (UniqueName: \"kubernetes.io/projected/4e8697a3-088c-43ab-944d-223d98a45b03-kube-api-access-ntqgm\") pod \"success-200-isvc-68c6a-predictor-78785fb6b7-4wt4m\" (UID: \"4e8697a3-088c-43ab-944d-223d98a45b03\") " pod="kserve-ci-e2e-test/success-200-isvc-68c6a-predictor-78785fb6b7-4wt4m"
Jul 28 16:42:52.031521 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:42:52.031470    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/4e8697a3-088c-43ab-944d-223d98a45b03-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-68c6a-predictor-78785fb6b7-4wt4m\" (UID: \"4e8697a3-088c-43ab-944d-223d98a45b03\") " pod="kserve-ci-e2e-test/success-200-isvc-68c6a-predictor-78785fb6b7-4wt4m"
Jul 28 16:42:52.033462 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:42:52.033440    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/4e8697a3-088c-43ab-944d-223d98a45b03-proxy-tls\") pod \"success-200-isvc-68c6a-predictor-78785fb6b7-4wt4m\" (UID: \"4e8697a3-088c-43ab-944d-223d98a45b03\") " pod="kserve-ci-e2e-test/success-200-isvc-68c6a-predictor-78785fb6b7-4wt4m"
Jul 28 16:42:52.041674 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:42:52.041652    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ntqgm\" (UniqueName: \"kubernetes.io/projected/4e8697a3-088c-43ab-944d-223d98a45b03-kube-api-access-ntqgm\") pod \"success-200-isvc-68c6a-predictor-78785fb6b7-4wt4m\" (UID: \"4e8697a3-088c-43ab-944d-223d98a45b03\") " pod="kserve-ci-e2e-test/success-200-isvc-68c6a-predictor-78785fb6b7-4wt4m"
Jul 28 16:42:52.075679 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:42:52.075654    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:42:52.097260 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:42:52.097229    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-7ba9d-predictor-68cbcf59bf-jt9kq" podUID="34ba2157-d9a4-42c8-834c-f349e8df2b76" containerName="kserve-container" containerID="cri-o://9e9af98159a4bb899cde7d2d18ecfafded240bf4a8e18664955b86ff8b70b4ba" gracePeriod=30
Jul 28 16:42:52.097373 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:42:52.097238    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-7ba9d-predictor-68cbcf59bf-jt9kq" podUID="34ba2157-d9a4-42c8-834c-f349e8df2b76" containerName="kube-rbac-proxy" containerID="cri-o://4eaaa4483c214375574c0176800eecb47eec1fa6ce726778fdaf83e409effcda" gracePeriod=30
Jul 28 16:42:52.182336 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:42:52.182309    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-68c6a-predictor-78785fb6b7-4wt4m"
Jul 28 16:42:52.295382 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:42:52.295294    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:42:52.318529 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:42:52.318485    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-68c6a-predictor-78785fb6b7-4wt4m"]
Jul 28 16:42:52.320820 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:42:52.320791    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e8697a3_088c_43ab_944d_223d98a45b03.slice/crio-7ba424ce8ea0fdad94ddccece5f64daf882c5647192c6d12b16632b50f40563a WatchSource:0}: Error finding container 7ba424ce8ea0fdad94ddccece5f64daf882c5647192c6d12b16632b50f40563a: Status 404 returned error can't find the container with id 7ba424ce8ea0fdad94ddccece5f64daf882c5647192c6d12b16632b50f40563a
Jul 28 16:42:53.103949 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:42:53.103914    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-68c6a-predictor-78785fb6b7-4wt4m" event={"ID":"4e8697a3-088c-43ab-944d-223d98a45b03","Type":"ContainerStarted","Data":"f3834afe95a769db929bc8b6c700ef201ff138953a3e7eb1812322d1627a90fe"}
Jul 28 16:42:53.104397 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:42:53.103981    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-68c6a-predictor-78785fb6b7-4wt4m" event={"ID":"4e8697a3-088c-43ab-944d-223d98a45b03","Type":"ContainerStarted","Data":"3f2ba570063eaf4ea4cdf95ab368f6cf39db4e773322eecc980ffa9db12adc02"}
Jul 28 16:42:53.104397 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:42:53.104011    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-68c6a-predictor-78785fb6b7-4wt4m" event={"ID":"4e8697a3-088c-43ab-944d-223d98a45b03","Type":"ContainerStarted","Data":"7ba424ce8ea0fdad94ddccece5f64daf882c5647192c6d12b16632b50f40563a"}
Jul 28 16:42:53.104397 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:42:53.104046    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-68c6a-predictor-78785fb6b7-4wt4m"
Jul 28 16:42:53.105817 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:42:53.105792    2557 generic.go:356] "Generic (PLEG): container finished" podID="34ba2157-d9a4-42c8-834c-f349e8df2b76" containerID="4eaaa4483c214375574c0176800eecb47eec1fa6ce726778fdaf83e409effcda" exitCode=2
Jul 28 16:42:53.105934 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:42:53.105837    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-7ba9d-predictor-68cbcf59bf-jt9kq" event={"ID":"34ba2157-d9a4-42c8-834c-f349e8df2b76","Type":"ContainerDied","Data":"4eaaa4483c214375574c0176800eecb47eec1fa6ce726778fdaf83e409effcda"}
Jul 28 16:42:53.120140 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:42:53.120096    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-68c6a-predictor-78785fb6b7-4wt4m" podStartSLOduration=2.120077448 podStartE2EDuration="2.120077448s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 16:42:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:42:53.118869533 +0000 UTC m=+1290.257338947" watchObservedRunningTime="2026-07-28 16:42:53.120077448 +0000 UTC m=+1290.258546863"
Jul 28 16:42:54.111565 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:42:54.111536    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-68c6a-predictor-78785fb6b7-4wt4m"
Jul 28 16:42:54.112984 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:42:54.112960    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-68c6a-predictor-78785fb6b7-4wt4m" podUID="4e8697a3-088c-43ab-944d-223d98a45b03" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.34:8080: connect: connection refused"
Jul 28 16:42:54.576244 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:42:54.576195    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-7ba9d-predictor-68cbcf59bf-jt9kq" podUID="34ba2157-d9a4-42c8-834c-f349e8df2b76" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.31:8643/healthz\": dial tcp 10.133.0.31:8643: connect: connection refused"
Jul 28 16:42:54.580579 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:42:54.580547    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-7ba9d-predictor-68cbcf59bf-jt9kq" podUID="34ba2157-d9a4-42c8-834c-f349e8df2b76" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.31:8080: connect: connection refused"
Jul 28 16:42:55.115570 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:42:55.115528    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-68c6a-predictor-78785fb6b7-4wt4m" podUID="4e8697a3-088c-43ab-944d-223d98a45b03" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.34:8080: connect: connection refused"
Jul 28 16:42:55.338719 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:42:55.338696    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-7ba9d-predictor-68cbcf59bf-jt9kq"
Jul 28 16:42:55.459910 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:42:55.459840    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/34ba2157-d9a4-42c8-834c-f349e8df2b76-proxy-tls\") pod \"34ba2157-d9a4-42c8-834c-f349e8df2b76\" (UID: \"34ba2157-d9a4-42c8-834c-f349e8df2b76\") "
Jul 28 16:42:55.459910 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:42:55.459874    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/34ba2157-d9a4-42c8-834c-f349e8df2b76-kube-rbac-proxy-sar-config\") pod \"34ba2157-d9a4-42c8-834c-f349e8df2b76\" (UID: \"34ba2157-d9a4-42c8-834c-f349e8df2b76\") "
Jul 28 16:42:55.460084 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:42:55.459975    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dwt4l\" (UniqueName: \"kubernetes.io/projected/34ba2157-d9a4-42c8-834c-f349e8df2b76-kube-api-access-dwt4l\") pod \"34ba2157-d9a4-42c8-834c-f349e8df2b76\" (UID: \"34ba2157-d9a4-42c8-834c-f349e8df2b76\") "
Jul 28 16:42:55.460271 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:42:55.460250    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/34ba2157-d9a4-42c8-834c-f349e8df2b76-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "34ba2157-d9a4-42c8-834c-f349e8df2b76" (UID: "34ba2157-d9a4-42c8-834c-f349e8df2b76"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 16:42:55.462118 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:42:55.462096    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/34ba2157-d9a4-42c8-834c-f349e8df2b76-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "34ba2157-d9a4-42c8-834c-f349e8df2b76" (UID: "34ba2157-d9a4-42c8-834c-f349e8df2b76"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 16:42:55.462180 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:42:55.462113    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/34ba2157-d9a4-42c8-834c-f349e8df2b76-kube-api-access-dwt4l" (OuterVolumeSpecName: "kube-api-access-dwt4l") pod "34ba2157-d9a4-42c8-834c-f349e8df2b76" (UID: "34ba2157-d9a4-42c8-834c-f349e8df2b76"). InnerVolumeSpecName "kube-api-access-dwt4l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 16:42:55.561277 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:42:55.561242    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dwt4l\" (UniqueName: \"kubernetes.io/projected/34ba2157-d9a4-42c8-834c-f349e8df2b76-kube-api-access-dwt4l\") on node \"ip-10-0-140-154.ec2.internal\" DevicePath \"\""
Jul 28 16:42:55.561277 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:42:55.561271    2557 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/34ba2157-d9a4-42c8-834c-f349e8df2b76-proxy-tls\") on node \"ip-10-0-140-154.ec2.internal\" DevicePath \"\""
Jul 28 16:42:55.561450 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:42:55.561284    2557 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/34ba2157-d9a4-42c8-834c-f349e8df2b76-kube-rbac-proxy-sar-config\") on node \"ip-10-0-140-154.ec2.internal\" DevicePath \"\""
Jul 28 16:42:56.121022 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:42:56.120985    2557 generic.go:356] "Generic (PLEG): container finished" podID="34ba2157-d9a4-42c8-834c-f349e8df2b76" containerID="9e9af98159a4bb899cde7d2d18ecfafded240bf4a8e18664955b86ff8b70b4ba" exitCode=0
Jul 28 16:42:56.121022 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:42:56.121025    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-7ba9d-predictor-68cbcf59bf-jt9kq" event={"ID":"34ba2157-d9a4-42c8-834c-f349e8df2b76","Type":"ContainerDied","Data":"9e9af98159a4bb899cde7d2d18ecfafded240bf4a8e18664955b86ff8b70b4ba"}
Jul 28 16:42:56.121462 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:42:56.121057    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-7ba9d-predictor-68cbcf59bf-jt9kq"
Jul 28 16:42:56.121462 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:42:56.121096    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-7ba9d-predictor-68cbcf59bf-jt9kq" event={"ID":"34ba2157-d9a4-42c8-834c-f349e8df2b76","Type":"ContainerDied","Data":"5006493ae5277f3c4fdb6a60e93c587f2f0b34c9053333da6698ca9489055dd7"}
Jul 28 16:42:56.121462 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:42:56.121123    2557 scope.go:117] "RemoveContainer" containerID="4eaaa4483c214375574c0176800eecb47eec1fa6ce726778fdaf83e409effcda"
Jul 28 16:42:56.130509 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:42:56.130471    2557 scope.go:117] "RemoveContainer" containerID="9e9af98159a4bb899cde7d2d18ecfafded240bf4a8e18664955b86ff8b70b4ba"
Jul 28 16:42:56.138009 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:42:56.137989    2557 scope.go:117] "RemoveContainer" containerID="4eaaa4483c214375574c0176800eecb47eec1fa6ce726778fdaf83e409effcda"
Jul 28 16:42:56.138267 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:42:56.138243    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4eaaa4483c214375574c0176800eecb47eec1fa6ce726778fdaf83e409effcda\": container with ID starting with 4eaaa4483c214375574c0176800eecb47eec1fa6ce726778fdaf83e409effcda not found: ID does not exist" containerID="4eaaa4483c214375574c0176800eecb47eec1fa6ce726778fdaf83e409effcda"
Jul 28 16:42:56.138356 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:42:56.138272    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4eaaa4483c214375574c0176800eecb47eec1fa6ce726778fdaf83e409effcda"} err="failed to get container status \"4eaaa4483c214375574c0176800eecb47eec1fa6ce726778fdaf83e409effcda\": rpc error: code = NotFound desc = could not find container \"4eaaa4483c214375574c0176800eecb47eec1fa6ce726778fdaf83e409effcda\": container with ID starting with 4eaaa4483c214375574c0176800eecb47eec1fa6ce726778fdaf83e409effcda not found: ID does not exist"
Jul 28 16:42:56.138356 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:42:56.138288    2557 scope.go:117] "RemoveContainer" containerID="9e9af98159a4bb899cde7d2d18ecfafded240bf4a8e18664955b86ff8b70b4ba"
Jul 28 16:42:56.138559 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:42:56.138533    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9e9af98159a4bb899cde7d2d18ecfafded240bf4a8e18664955b86ff8b70b4ba\": container with ID starting with 9e9af98159a4bb899cde7d2d18ecfafded240bf4a8e18664955b86ff8b70b4ba not found: ID does not exist" containerID="9e9af98159a4bb899cde7d2d18ecfafded240bf4a8e18664955b86ff8b70b4ba"
Jul 28 16:42:56.138646 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:42:56.138569    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9e9af98159a4bb899cde7d2d18ecfafded240bf4a8e18664955b86ff8b70b4ba"} err="failed to get container status \"9e9af98159a4bb899cde7d2d18ecfafded240bf4a8e18664955b86ff8b70b4ba\": rpc error: code = NotFound desc = could not find container \"9e9af98159a4bb899cde7d2d18ecfafded240bf4a8e18664955b86ff8b70b4ba\": container with ID starting with 9e9af98159a4bb899cde7d2d18ecfafded240bf4a8e18664955b86ff8b70b4ba not found: ID does not exist"
Jul 28 16:42:56.141952 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:42:56.141930    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-7ba9d-predictor-68cbcf59bf-jt9kq"]
Jul 28 16:42:56.145756 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:42:56.145738    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-7ba9d-predictor-68cbcf59bf-jt9kq"]
Jul 28 16:42:57.434284 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:42:57.434251    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="34ba2157-d9a4-42c8-834c-f349e8df2b76" path="/var/lib/kubelet/pods/34ba2157-d9a4-42c8-834c-f349e8df2b76/volumes"
Jul 28 16:43:00.119962 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:43:00.119930    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-68c6a-predictor-78785fb6b7-4wt4m"
Jul 28 16:43:00.120593 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:43:00.120564    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-68c6a-predictor-78785fb6b7-4wt4m" podUID="4e8697a3-088c-43ab-944d-223d98a45b03" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.34:8080: connect: connection refused"
Jul 28 16:43:10.120906 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:43:10.120866    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-68c6a-predictor-78785fb6b7-4wt4m" podUID="4e8697a3-088c-43ab-944d-223d98a45b03" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.34:8080: connect: connection refused"
Jul 28 16:43:20.120846 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:43:20.120805    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-68c6a-predictor-78785fb6b7-4wt4m" podUID="4e8697a3-088c-43ab-944d-223d98a45b03" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.34:8080: connect: connection refused"
Jul 28 16:43:30.121265 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:43:30.121223    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-68c6a-predictor-78785fb6b7-4wt4m" podUID="4e8697a3-088c-43ab-944d-223d98a45b03" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.34:8080: connect: connection refused"
Jul 28 16:43:40.121587 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:43:40.121555    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-68c6a-predictor-78785fb6b7-4wt4m"
Jul 28 16:43:51.690019 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:43:51.689981    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:43:51.780316 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:43:51.780280    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-2b606-predictor-64d46d8575-sbrbh"]
Jul 28 16:43:51.780682 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:43:51.780625    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-2b606-predictor-64d46d8575-sbrbh" podUID="dd8f78ab-362e-4ef5-8aa1-4362e5de1374" containerName="kserve-container" containerID="cri-o://b147f15f385e3d87a1df8bb0baa2d8747763f5626b212a70958901df79886e54" gracePeriod=30
Jul 28 16:43:51.780682 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:43:51.780660    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-2b606-predictor-64d46d8575-sbrbh" podUID="dd8f78ab-362e-4ef5-8aa1-4362e5de1374" containerName="kube-rbac-proxy" containerID="cri-o://e2e77d4f3b18229b662d12d40a966f5fde6ee6882ab9fc626a89aa6e0a126f3f" gracePeriod=30
Jul 28 16:43:51.840092 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:43:51.840052    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-92dc3-predictor-7d6df59697-gbssk"]
Jul 28 16:43:51.840455 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:43:51.840442    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="34ba2157-d9a4-42c8-834c-f349e8df2b76" containerName="kube-rbac-proxy"
Jul 28 16:43:51.840544 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:43:51.840457    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="34ba2157-d9a4-42c8-834c-f349e8df2b76" containerName="kube-rbac-proxy"
Jul 28 16:43:51.840544 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:43:51.840486    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="34ba2157-d9a4-42c8-834c-f349e8df2b76" containerName="kserve-container"
Jul 28 16:43:51.840544 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:43:51.840521    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="34ba2157-d9a4-42c8-834c-f349e8df2b76" containerName="kserve-container"
Jul 28 16:43:51.840703 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:43:51.840593    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="34ba2157-d9a4-42c8-834c-f349e8df2b76" containerName="kserve-container"
Jul 28 16:43:51.840703 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:43:51.840602    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="34ba2157-d9a4-42c8-834c-f349e8df2b76" containerName="kube-rbac-proxy"
Jul 28 16:43:51.843711 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:43:51.843694    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-92dc3-predictor-7d6df59697-gbssk"
Jul 28 16:43:51.846668 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:43:51.846644    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-92dc3-predictor-serving-cert\""
Jul 28 16:43:51.846763 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:43:51.846658    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-92dc3-kube-rbac-proxy-sar-config\""
Jul 28 16:43:51.855419 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:43:51.855394    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-92dc3-predictor-7d6df59697-gbssk"]
Jul 28 16:43:51.909055 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:43:51.909008    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:43:51.941940 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:43:51.941868    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c2052a1a-c107-4c2c-965a-232dc354050d-proxy-tls\") pod \"success-200-isvc-92dc3-predictor-7d6df59697-gbssk\" (UID: \"c2052a1a-c107-4c2c-965a-232dc354050d\") " pod="kserve-ci-e2e-test/success-200-isvc-92dc3-predictor-7d6df59697-gbssk"
Jul 28 16:43:51.941940 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:43:51.941915    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c2052a1a-c107-4c2c-965a-232dc354050d-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-92dc3-predictor-7d6df59697-gbssk\" (UID: \"c2052a1a-c107-4c2c-965a-232dc354050d\") " pod="kserve-ci-e2e-test/success-200-isvc-92dc3-predictor-7d6df59697-gbssk"
Jul 28 16:43:51.942190 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:43:51.942025    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hx8gg\" (UniqueName: \"kubernetes.io/projected/c2052a1a-c107-4c2c-965a-232dc354050d-kube-api-access-hx8gg\") pod \"success-200-isvc-92dc3-predictor-7d6df59697-gbssk\" (UID: \"c2052a1a-c107-4c2c-965a-232dc354050d\") " pod="kserve-ci-e2e-test/success-200-isvc-92dc3-predictor-7d6df59697-gbssk"
Jul 28 16:43:52.043013 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:43:52.042983    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c2052a1a-c107-4c2c-965a-232dc354050d-proxy-tls\") pod \"success-200-isvc-92dc3-predictor-7d6df59697-gbssk\" (UID: \"c2052a1a-c107-4c2c-965a-232dc354050d\") " pod="kserve-ci-e2e-test/success-200-isvc-92dc3-predictor-7d6df59697-gbssk"
Jul 28 16:43:52.043204 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:43:52.043020    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c2052a1a-c107-4c2c-965a-232dc354050d-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-92dc3-predictor-7d6df59697-gbssk\" (UID: \"c2052a1a-c107-4c2c-965a-232dc354050d\") " pod="kserve-ci-e2e-test/success-200-isvc-92dc3-predictor-7d6df59697-gbssk"
Jul 28 16:43:52.043204 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:43:52.043067    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hx8gg\" (UniqueName: \"kubernetes.io/projected/c2052a1a-c107-4c2c-965a-232dc354050d-kube-api-access-hx8gg\") pod \"success-200-isvc-92dc3-predictor-7d6df59697-gbssk\" (UID: \"c2052a1a-c107-4c2c-965a-232dc354050d\") " pod="kserve-ci-e2e-test/success-200-isvc-92dc3-predictor-7d6df59697-gbssk"
Jul 28 16:43:52.044019 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:43:52.043990    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c2052a1a-c107-4c2c-965a-232dc354050d-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-92dc3-predictor-7d6df59697-gbssk\" (UID: \"c2052a1a-c107-4c2c-965a-232dc354050d\") " pod="kserve-ci-e2e-test/success-200-isvc-92dc3-predictor-7d6df59697-gbssk"
Jul 28 16:43:52.045719 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:43:52.045696    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c2052a1a-c107-4c2c-965a-232dc354050d-proxy-tls\") pod \"success-200-isvc-92dc3-predictor-7d6df59697-gbssk\" (UID: \"c2052a1a-c107-4c2c-965a-232dc354050d\") " pod="kserve-ci-e2e-test/success-200-isvc-92dc3-predictor-7d6df59697-gbssk"
Jul 28 16:43:52.059626 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:43:52.059602    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hx8gg\" (UniqueName: \"kubernetes.io/projected/c2052a1a-c107-4c2c-965a-232dc354050d-kube-api-access-hx8gg\") pod \"success-200-isvc-92dc3-predictor-7d6df59697-gbssk\" (UID: \"c2052a1a-c107-4c2c-965a-232dc354050d\") " pod="kserve-ci-e2e-test/success-200-isvc-92dc3-predictor-7d6df59697-gbssk"
Jul 28 16:43:52.070607 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:43:52.070587    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:43:52.153783 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:43:52.153755    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-92dc3-predictor-7d6df59697-gbssk"
Jul 28 16:43:52.164728 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:43:52.164695    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:43:52.290365 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:43:52.290342    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-92dc3-predictor-7d6df59697-gbssk"]
Jul 28 16:43:52.292588 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:43:52.292561    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2052a1a_c107_4c2c_965a_232dc354050d.slice/crio-37ceee4a7d579099bd05ceca2b54796a7569111883fe22e327d4032b72d971e0 WatchSource:0}: Error finding container 37ceee4a7d579099bd05ceca2b54796a7569111883fe22e327d4032b72d971e0: Status 404 returned error can't find the container with id 37ceee4a7d579099bd05ceca2b54796a7569111883fe22e327d4032b72d971e0
Jul 28 16:43:52.378565 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:43:52.378534    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-92dc3-predictor-7d6df59697-gbssk" event={"ID":"c2052a1a-c107-4c2c-965a-232dc354050d","Type":"ContainerStarted","Data":"f01e15d38f060966af5dbe3e99d1f836112f4d7c7427d80ed0403e567e5d2e66"}
Jul 28 16:43:52.378687 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:43:52.378604    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-92dc3-predictor-7d6df59697-gbssk" event={"ID":"c2052a1a-c107-4c2c-965a-232dc354050d","Type":"ContainerStarted","Data":"37ceee4a7d579099bd05ceca2b54796a7569111883fe22e327d4032b72d971e0"}
Jul 28 16:43:52.380332 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:43:52.380310    2557 generic.go:356] "Generic (PLEG): container finished" podID="dd8f78ab-362e-4ef5-8aa1-4362e5de1374" containerID="e2e77d4f3b18229b662d12d40a966f5fde6ee6882ab9fc626a89aa6e0a126f3f" exitCode=2
Jul 28 16:43:52.380424 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:43:52.380371    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-2b606-predictor-64d46d8575-sbrbh" event={"ID":"dd8f78ab-362e-4ef5-8aa1-4362e5de1374","Type":"ContainerDied","Data":"e2e77d4f3b18229b662d12d40a966f5fde6ee6882ab9fc626a89aa6e0a126f3f"}
Jul 28 16:43:53.387001 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:43:53.386968    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-92dc3-predictor-7d6df59697-gbssk" event={"ID":"c2052a1a-c107-4c2c-965a-232dc354050d","Type":"ContainerStarted","Data":"2ba9c70ca2aebfeea3ce5e0d34d7b987603962a940e0ee3be64658b9dfc6ad43"}
Jul 28 16:43:53.387389 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:43:53.387107    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-92dc3-predictor-7d6df59697-gbssk"
Jul 28 16:43:53.406414 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:43:53.406352    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-92dc3-predictor-7d6df59697-gbssk" podStartSLOduration=2.406333623 podStartE2EDuration="2.406333623s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 16:43:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:43:53.406141272 +0000 UTC m=+1350.544610707" watchObservedRunningTime="2026-07-28 16:43:53.406333623 +0000 UTC m=+1350.544803038"
Jul 28 16:43:54.391739 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:43:54.391705    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-92dc3-predictor-7d6df59697-gbssk"
Jul 28 16:43:54.393014 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:43:54.392984    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-92dc3-predictor-7d6df59697-gbssk" podUID="c2052a1a-c107-4c2c-965a-232dc354050d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.35:8080: connect: connection refused"
Jul 28 16:43:55.128166 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:43:55.128144    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-2b606-predictor-64d46d8575-sbrbh"
Jul 28 16:43:55.170336 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:43:55.170301    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ppwq2\" (UniqueName: \"kubernetes.io/projected/dd8f78ab-362e-4ef5-8aa1-4362e5de1374-kube-api-access-ppwq2\") pod \"dd8f78ab-362e-4ef5-8aa1-4362e5de1374\" (UID: \"dd8f78ab-362e-4ef5-8aa1-4362e5de1374\") "
Jul 28 16:43:55.170528 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:43:55.170368    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/dd8f78ab-362e-4ef5-8aa1-4362e5de1374-proxy-tls\") pod \"dd8f78ab-362e-4ef5-8aa1-4362e5de1374\" (UID: \"dd8f78ab-362e-4ef5-8aa1-4362e5de1374\") "
Jul 28 16:43:55.170528 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:43:55.170413    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/dd8f78ab-362e-4ef5-8aa1-4362e5de1374-kube-rbac-proxy-sar-config\") pod \"dd8f78ab-362e-4ef5-8aa1-4362e5de1374\" (UID: \"dd8f78ab-362e-4ef5-8aa1-4362e5de1374\") "
Jul 28 16:43:55.170877 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:43:55.170852    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dd8f78ab-362e-4ef5-8aa1-4362e5de1374-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "dd8f78ab-362e-4ef5-8aa1-4362e5de1374" (UID: "dd8f78ab-362e-4ef5-8aa1-4362e5de1374"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 16:43:55.172631 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:43:55.172575    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dd8f78ab-362e-4ef5-8aa1-4362e5de1374-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "dd8f78ab-362e-4ef5-8aa1-4362e5de1374" (UID: "dd8f78ab-362e-4ef5-8aa1-4362e5de1374"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 16:43:55.172736 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:43:55.172695    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dd8f78ab-362e-4ef5-8aa1-4362e5de1374-kube-api-access-ppwq2" (OuterVolumeSpecName: "kube-api-access-ppwq2") pod "dd8f78ab-362e-4ef5-8aa1-4362e5de1374" (UID: "dd8f78ab-362e-4ef5-8aa1-4362e5de1374"). InnerVolumeSpecName "kube-api-access-ppwq2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 16:43:55.271416 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:43:55.271370    2557 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/dd8f78ab-362e-4ef5-8aa1-4362e5de1374-proxy-tls\") on node \"ip-10-0-140-154.ec2.internal\" DevicePath \"\""
Jul 28 16:43:55.271416 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:43:55.271412    2557 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/dd8f78ab-362e-4ef5-8aa1-4362e5de1374-kube-rbac-proxy-sar-config\") on node \"ip-10-0-140-154.ec2.internal\" DevicePath \"\""
Jul 28 16:43:55.271416 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:43:55.271426    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ppwq2\" (UniqueName: \"kubernetes.io/projected/dd8f78ab-362e-4ef5-8aa1-4362e5de1374-kube-api-access-ppwq2\") on node \"ip-10-0-140-154.ec2.internal\" DevicePath \"\""
Jul 28 16:43:55.397720 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:43:55.397686    2557 generic.go:356] "Generic (PLEG): container finished" podID="dd8f78ab-362e-4ef5-8aa1-4362e5de1374" containerID="b147f15f385e3d87a1df8bb0baa2d8747763f5626b212a70958901df79886e54" exitCode=0
Jul 28 16:43:55.398173 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:43:55.397753    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-2b606-predictor-64d46d8575-sbrbh"
Jul 28 16:43:55.398173 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:43:55.397773    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-2b606-predictor-64d46d8575-sbrbh" event={"ID":"dd8f78ab-362e-4ef5-8aa1-4362e5de1374","Type":"ContainerDied","Data":"b147f15f385e3d87a1df8bb0baa2d8747763f5626b212a70958901df79886e54"}
Jul 28 16:43:55.398173 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:43:55.397885    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-2b606-predictor-64d46d8575-sbrbh" event={"ID":"dd8f78ab-362e-4ef5-8aa1-4362e5de1374","Type":"ContainerDied","Data":"fc3c9e2652a1c7c1f9f8fb0ee506bcb1728fd275b37020cfdcbb94408d89d5c9"}
Jul 28 16:43:55.398173 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:43:55.397928    2557 scope.go:117] "RemoveContainer" containerID="e2e77d4f3b18229b662d12d40a966f5fde6ee6882ab9fc626a89aa6e0a126f3f"
Jul 28 16:43:55.398554 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:43:55.398520    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-92dc3-predictor-7d6df59697-gbssk" podUID="c2052a1a-c107-4c2c-965a-232dc354050d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.35:8080: connect: connection refused"
Jul 28 16:43:55.407305 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:43:55.407288    2557 scope.go:117] "RemoveContainer" containerID="b147f15f385e3d87a1df8bb0baa2d8747763f5626b212a70958901df79886e54"
Jul 28 16:43:55.417887 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:43:55.417871    2557 scope.go:117] "RemoveContainer" containerID="e2e77d4f3b18229b662d12d40a966f5fde6ee6882ab9fc626a89aa6e0a126f3f"
Jul 28 16:43:55.418158 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:43:55.418139    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e2e77d4f3b18229b662d12d40a966f5fde6ee6882ab9fc626a89aa6e0a126f3f\": container with ID starting with e2e77d4f3b18229b662d12d40a966f5fde6ee6882ab9fc626a89aa6e0a126f3f not found: ID does not exist" containerID="e2e77d4f3b18229b662d12d40a966f5fde6ee6882ab9fc626a89aa6e0a126f3f"
Jul 28 16:43:55.418237 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:43:55.418169    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e2e77d4f3b18229b662d12d40a966f5fde6ee6882ab9fc626a89aa6e0a126f3f"} err="failed to get container status \"e2e77d4f3b18229b662d12d40a966f5fde6ee6882ab9fc626a89aa6e0a126f3f\": rpc error: code = NotFound desc = could not find container \"e2e77d4f3b18229b662d12d40a966f5fde6ee6882ab9fc626a89aa6e0a126f3f\": container with ID starting with e2e77d4f3b18229b662d12d40a966f5fde6ee6882ab9fc626a89aa6e0a126f3f not found: ID does not exist"
Jul 28 16:43:55.418237 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:43:55.418191    2557 scope.go:117] "RemoveContainer" containerID="b147f15f385e3d87a1df8bb0baa2d8747763f5626b212a70958901df79886e54"
Jul 28 16:43:55.418425 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:43:55.418410    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b147f15f385e3d87a1df8bb0baa2d8747763f5626b212a70958901df79886e54\": container with ID starting with b147f15f385e3d87a1df8bb0baa2d8747763f5626b212a70958901df79886e54 not found: ID does not exist" containerID="b147f15f385e3d87a1df8bb0baa2d8747763f5626b212a70958901df79886e54"
Jul 28 16:43:55.418474 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:43:55.418432    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b147f15f385e3d87a1df8bb0baa2d8747763f5626b212a70958901df79886e54"} err="failed to get container status \"b147f15f385e3d87a1df8bb0baa2d8747763f5626b212a70958901df79886e54\": rpc error: code = NotFound desc = could not find container \"b147f15f385e3d87a1df8bb0baa2d8747763f5626b212a70958901df79886e54\": container with ID starting with b147f15f385e3d87a1df8bb0baa2d8747763f5626b212a70958901df79886e54 not found: ID does not exist"
Jul 28 16:43:55.419155 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:43:55.419138    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-2b606-predictor-64d46d8575-sbrbh"]
Jul 28 16:43:55.425412 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:43:55.425361    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-2b606-predictor-64d46d8575-sbrbh"]
Jul 28 16:43:55.433993 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:43:55.433973    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dd8f78ab-362e-4ef5-8aa1-4362e5de1374" path="/var/lib/kubelet/pods/dd8f78ab-362e-4ef5-8aa1-4362e5de1374/volumes"
Jul 28 16:44:00.402741 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:44:00.402715    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-92dc3-predictor-7d6df59697-gbssk"
Jul 28 16:44:00.403166 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:44:00.403143    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-92dc3-predictor-7d6df59697-gbssk" podUID="c2052a1a-c107-4c2c-965a-232dc354050d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.35:8080: connect: connection refused"
Jul 28 16:44:01.992408 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:44:01.992326    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:44:02.082515 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:44:02.082461    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-68c6a-predictor-78785fb6b7-4wt4m"]
Jul 28 16:44:02.082772 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:44:02.082742    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-68c6a-predictor-78785fb6b7-4wt4m" podUID="4e8697a3-088c-43ab-944d-223d98a45b03" containerName="kserve-container" containerID="cri-o://3f2ba570063eaf4ea4cdf95ab368f6cf39db4e773322eecc980ffa9db12adc02" gracePeriod=30
Jul 28 16:44:02.082897 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:44:02.082787    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-68c6a-predictor-78785fb6b7-4wt4m" podUID="4e8697a3-088c-43ab-944d-223d98a45b03" containerName="kube-rbac-proxy" containerID="cri-o://f3834afe95a769db929bc8b6c700ef201ff138953a3e7eb1812322d1627a90fe" gracePeriod=30
Jul 28 16:44:02.114831 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:44:02.114806    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-c3541-predictor-c87f9b99-wtgfm"]
Jul 28 16:44:02.115208 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:44:02.115193    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dd8f78ab-362e-4ef5-8aa1-4362e5de1374" containerName="kube-rbac-proxy"
Jul 28 16:44:02.115287 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:44:02.115211    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="dd8f78ab-362e-4ef5-8aa1-4362e5de1374" containerName="kube-rbac-proxy"
Jul 28 16:44:02.115287 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:44:02.115226    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dd8f78ab-362e-4ef5-8aa1-4362e5de1374" containerName="kserve-container"
Jul 28 16:44:02.115287 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:44:02.115234    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="dd8f78ab-362e-4ef5-8aa1-4362e5de1374" containerName="kserve-container"
Jul 28 16:44:02.115444 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:44:02.115338    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="dd8f78ab-362e-4ef5-8aa1-4362e5de1374" containerName="kserve-container"
Jul 28 16:44:02.115444 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:44:02.115351    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="dd8f78ab-362e-4ef5-8aa1-4362e5de1374" containerName="kube-rbac-proxy"
Jul 28 16:44:02.119960 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:44:02.119943    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-c3541-predictor-c87f9b99-wtgfm"
Jul 28 16:44:02.122257 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:44:02.122234    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-c3541-predictor-serving-cert\""
Jul 28 16:44:02.122352 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:44:02.122278    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-c3541-kube-rbac-proxy-sar-config\""
Jul 28 16:44:02.129783 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:44:02.129764    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-c3541-predictor-c87f9b99-wtgfm"]
Jul 28 16:44:02.143071 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:44:02.143048    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:44:02.230695 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:44:02.230659    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/714337c0-47d4-49a8-83fe-f9861c5e6a8f-proxy-tls\") pod \"success-200-isvc-c3541-predictor-c87f9b99-wtgfm\" (UID: \"714337c0-47d4-49a8-83fe-f9861c5e6a8f\") " pod="kserve-ci-e2e-test/success-200-isvc-c3541-predictor-c87f9b99-wtgfm"
Jul 28 16:44:02.230892 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:44:02.230720    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-688d9\" (UniqueName: \"kubernetes.io/projected/714337c0-47d4-49a8-83fe-f9861c5e6a8f-kube-api-access-688d9\") pod \"success-200-isvc-c3541-predictor-c87f9b99-wtgfm\" (UID: \"714337c0-47d4-49a8-83fe-f9861c5e6a8f\") " pod="kserve-ci-e2e-test/success-200-isvc-c3541-predictor-c87f9b99-wtgfm"
Jul 28 16:44:02.230892 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:44:02.230788    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/714337c0-47d4-49a8-83fe-f9861c5e6a8f-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-c3541-predictor-c87f9b99-wtgfm\" (UID: \"714337c0-47d4-49a8-83fe-f9861c5e6a8f\") " pod="kserve-ci-e2e-test/success-200-isvc-c3541-predictor-c87f9b99-wtgfm"
Jul 28 16:44:02.331867 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:44:02.331816    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/714337c0-47d4-49a8-83fe-f9861c5e6a8f-proxy-tls\") pod \"success-200-isvc-c3541-predictor-c87f9b99-wtgfm\" (UID: \"714337c0-47d4-49a8-83fe-f9861c5e6a8f\") " pod="kserve-ci-e2e-test/success-200-isvc-c3541-predictor-c87f9b99-wtgfm"
Jul 28 16:44:02.332061 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:44:02.331906    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-688d9\" (UniqueName: \"kubernetes.io/projected/714337c0-47d4-49a8-83fe-f9861c5e6a8f-kube-api-access-688d9\") pod \"success-200-isvc-c3541-predictor-c87f9b99-wtgfm\" (UID: \"714337c0-47d4-49a8-83fe-f9861c5e6a8f\") " pod="kserve-ci-e2e-test/success-200-isvc-c3541-predictor-c87f9b99-wtgfm"
Jul 28 16:44:02.332061 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:44:02.331973    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/714337c0-47d4-49a8-83fe-f9861c5e6a8f-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-c3541-predictor-c87f9b99-wtgfm\" (UID: \"714337c0-47d4-49a8-83fe-f9861c5e6a8f\") " pod="kserve-ci-e2e-test/success-200-isvc-c3541-predictor-c87f9b99-wtgfm"
Jul 28 16:44:02.332706 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:44:02.332685    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/714337c0-47d4-49a8-83fe-f9861c5e6a8f-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-c3541-predictor-c87f9b99-wtgfm\" (UID: \"714337c0-47d4-49a8-83fe-f9861c5e6a8f\") " pod="kserve-ci-e2e-test/success-200-isvc-c3541-predictor-c87f9b99-wtgfm"
Jul 28 16:44:02.334765 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:44:02.334742    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/714337c0-47d4-49a8-83fe-f9861c5e6a8f-proxy-tls\") pod \"success-200-isvc-c3541-predictor-c87f9b99-wtgfm\" (UID: \"714337c0-47d4-49a8-83fe-f9861c5e6a8f\") " pod="kserve-ci-e2e-test/success-200-isvc-c3541-predictor-c87f9b99-wtgfm"
Jul 28 16:44:02.334867 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:44:02.334804    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:44:02.339781 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:44:02.339758    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-688d9\" (UniqueName: \"kubernetes.io/projected/714337c0-47d4-49a8-83fe-f9861c5e6a8f-kube-api-access-688d9\") pod \"success-200-isvc-c3541-predictor-c87f9b99-wtgfm\" (UID: \"714337c0-47d4-49a8-83fe-f9861c5e6a8f\") " pod="kserve-ci-e2e-test/success-200-isvc-c3541-predictor-c87f9b99-wtgfm"
Jul 28 16:44:02.431796 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:44:02.431770    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-c3541-predictor-c87f9b99-wtgfm"
Jul 28 16:44:02.432102 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:44:02.432077    2557 generic.go:356] "Generic (PLEG): container finished" podID="4e8697a3-088c-43ab-944d-223d98a45b03" containerID="f3834afe95a769db929bc8b6c700ef201ff138953a3e7eb1812322d1627a90fe" exitCode=2
Jul 28 16:44:02.432181 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:44:02.432157    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-68c6a-predictor-78785fb6b7-4wt4m" event={"ID":"4e8697a3-088c-43ab-944d-223d98a45b03","Type":"ContainerDied","Data":"f3834afe95a769db929bc8b6c700ef201ff138953a3e7eb1812322d1627a90fe"}
Jul 28 16:44:02.436183 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:44:02.436158    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:44:02.512068 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:44:02.512020    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:44:02.566917 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:44:02.566893    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-c3541-predictor-c87f9b99-wtgfm"]
Jul 28 16:44:02.569076 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:44:02.569048    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod714337c0_47d4_49a8_83fe_f9861c5e6a8f.slice/crio-21874b0f0ef8841825f400d2fb3d180a21fe4eafcaf4ca21c133028f4ec8ec1b WatchSource:0}: Error finding container 21874b0f0ef8841825f400d2fb3d180a21fe4eafcaf4ca21c133028f4ec8ec1b: Status 404 returned error can't find the container with id 21874b0f0ef8841825f400d2fb3d180a21fe4eafcaf4ca21c133028f4ec8ec1b
Jul 28 16:44:03.437659 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:44:03.437622    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-c3541-predictor-c87f9b99-wtgfm" event={"ID":"714337c0-47d4-49a8-83fe-f9861c5e6a8f","Type":"ContainerStarted","Data":"90ce6d0f8a07d6dfc16d61be92b5d1cc1efb9c8838be89b233f612a993cc31f5"}
Jul 28 16:44:03.438030 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:44:03.437687    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-c3541-predictor-c87f9b99-wtgfm" event={"ID":"714337c0-47d4-49a8-83fe-f9861c5e6a8f","Type":"ContainerStarted","Data":"d5e22596515f5e7e0104357a102e9cc21669618a3e54d8a43665afcd40407688"}
Jul 28 16:44:03.438030 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:44:03.437727    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-c3541-predictor-c87f9b99-wtgfm"
Jul 28 16:44:03.438030 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:44:03.437759    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-c3541-predictor-c87f9b99-wtgfm"
Jul 28 16:44:03.438030 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:44:03.437785    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-c3541-predictor-c87f9b99-wtgfm" event={"ID":"714337c0-47d4-49a8-83fe-f9861c5e6a8f","Type":"ContainerStarted","Data":"21874b0f0ef8841825f400d2fb3d180a21fe4eafcaf4ca21c133028f4ec8ec1b"}
Jul 28 16:44:03.439200 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:44:03.439172    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-c3541-predictor-c87f9b99-wtgfm" podUID="714337c0-47d4-49a8-83fe-f9861c5e6a8f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.36:8080: connect: connection refused"
Jul 28 16:44:03.456341 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:44:03.456294    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-c3541-predictor-c87f9b99-wtgfm" podStartSLOduration=1.456280564 podStartE2EDuration="1.456280564s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 16:44:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:44:03.455057145 +0000 UTC m=+1360.593526560" watchObservedRunningTime="2026-07-28 16:44:03.456280564 +0000 UTC m=+1360.594749978"
Jul 28 16:44:04.442292 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:44:04.442246    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-c3541-predictor-c87f9b99-wtgfm" podUID="714337c0-47d4-49a8-83fe-f9861c5e6a8f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.36:8080: connect: connection refused"
Jul 28 16:44:05.116611 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:44:05.116578    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-68c6a-predictor-78785fb6b7-4wt4m" podUID="4e8697a3-088c-43ab-944d-223d98a45b03" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.34:8643/healthz\": dial tcp 10.133.0.34:8643: connect: connection refused"
Jul 28 16:44:05.230999 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:44:05.230971    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-68c6a-predictor-78785fb6b7-4wt4m"
Jul 28 16:44:05.360542 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:44:05.360488    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/4e8697a3-088c-43ab-944d-223d98a45b03-kube-rbac-proxy-sar-config\") pod \"4e8697a3-088c-43ab-944d-223d98a45b03\" (UID: \"4e8697a3-088c-43ab-944d-223d98a45b03\") "
Jul 28 16:44:05.360736 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:44:05.360602    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/4e8697a3-088c-43ab-944d-223d98a45b03-proxy-tls\") pod \"4e8697a3-088c-43ab-944d-223d98a45b03\" (UID: \"4e8697a3-088c-43ab-944d-223d98a45b03\") "
Jul 28 16:44:05.360736 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:44:05.360619    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ntqgm\" (UniqueName: \"kubernetes.io/projected/4e8697a3-088c-43ab-944d-223d98a45b03-kube-api-access-ntqgm\") pod \"4e8697a3-088c-43ab-944d-223d98a45b03\" (UID: \"4e8697a3-088c-43ab-944d-223d98a45b03\") "
Jul 28 16:44:05.360915 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:44:05.360890    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4e8697a3-088c-43ab-944d-223d98a45b03-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "4e8697a3-088c-43ab-944d-223d98a45b03" (UID: "4e8697a3-088c-43ab-944d-223d98a45b03"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 16:44:05.362765 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:44:05.362737    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e8697a3-088c-43ab-944d-223d98a45b03-kube-api-access-ntqgm" (OuterVolumeSpecName: "kube-api-access-ntqgm") pod "4e8697a3-088c-43ab-944d-223d98a45b03" (UID: "4e8697a3-088c-43ab-944d-223d98a45b03"). InnerVolumeSpecName "kube-api-access-ntqgm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 16:44:05.362875 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:44:05.362818    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4e8697a3-088c-43ab-944d-223d98a45b03-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "4e8697a3-088c-43ab-944d-223d98a45b03" (UID: "4e8697a3-088c-43ab-944d-223d98a45b03"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 16:44:05.447081 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:44:05.447054    2557 generic.go:356] "Generic (PLEG): container finished" podID="4e8697a3-088c-43ab-944d-223d98a45b03" containerID="3f2ba570063eaf4ea4cdf95ab368f6cf39db4e773322eecc980ffa9db12adc02" exitCode=0
Jul 28 16:44:05.447455 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:44:05.447103    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-68c6a-predictor-78785fb6b7-4wt4m"
Jul 28 16:44:05.447455 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:44:05.447130    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-68c6a-predictor-78785fb6b7-4wt4m" event={"ID":"4e8697a3-088c-43ab-944d-223d98a45b03","Type":"ContainerDied","Data":"3f2ba570063eaf4ea4cdf95ab368f6cf39db4e773322eecc980ffa9db12adc02"}
Jul 28 16:44:05.447455 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:44:05.447189    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-68c6a-predictor-78785fb6b7-4wt4m" event={"ID":"4e8697a3-088c-43ab-944d-223d98a45b03","Type":"ContainerDied","Data":"7ba424ce8ea0fdad94ddccece5f64daf882c5647192c6d12b16632b50f40563a"}
Jul 28 16:44:05.447455 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:44:05.447216    2557 scope.go:117] "RemoveContainer" containerID="f3834afe95a769db929bc8b6c700ef201ff138953a3e7eb1812322d1627a90fe"
Jul 28 16:44:05.455672 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:44:05.455655    2557 scope.go:117] "RemoveContainer" containerID="3f2ba570063eaf4ea4cdf95ab368f6cf39db4e773322eecc980ffa9db12adc02"
Jul 28 16:44:05.461968 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:44:05.461947    2557 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/4e8697a3-088c-43ab-944d-223d98a45b03-proxy-tls\") on node \"ip-10-0-140-154.ec2.internal\" DevicePath \"\""
Jul 28 16:44:05.462053 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:44:05.461972    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ntqgm\" (UniqueName: \"kubernetes.io/projected/4e8697a3-088c-43ab-944d-223d98a45b03-kube-api-access-ntqgm\") on node \"ip-10-0-140-154.ec2.internal\" DevicePath \"\""
Jul 28 16:44:05.462053 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:44:05.461988    2557 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/4e8697a3-088c-43ab-944d-223d98a45b03-kube-rbac-proxy-sar-config\") on node \"ip-10-0-140-154.ec2.internal\" DevicePath \"\""
Jul 28 16:44:05.463428 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:44:05.463404    2557 scope.go:117] "RemoveContainer" containerID="f3834afe95a769db929bc8b6c700ef201ff138953a3e7eb1812322d1627a90fe"
Jul 28 16:44:05.463690 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:44:05.463671    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f3834afe95a769db929bc8b6c700ef201ff138953a3e7eb1812322d1627a90fe\": container with ID starting with f3834afe95a769db929bc8b6c700ef201ff138953a3e7eb1812322d1627a90fe not found: ID does not exist" containerID="f3834afe95a769db929bc8b6c700ef201ff138953a3e7eb1812322d1627a90fe"
Jul 28 16:44:05.463761 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:44:05.463698    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f3834afe95a769db929bc8b6c700ef201ff138953a3e7eb1812322d1627a90fe"} err="failed to get container status \"f3834afe95a769db929bc8b6c700ef201ff138953a3e7eb1812322d1627a90fe\": rpc error: code = NotFound desc = could not find container \"f3834afe95a769db929bc8b6c700ef201ff138953a3e7eb1812322d1627a90fe\": container with ID starting with f3834afe95a769db929bc8b6c700ef201ff138953a3e7eb1812322d1627a90fe not found: ID does not exist"
Jul 28 16:44:05.463761 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:44:05.463717    2557 scope.go:117] "RemoveContainer" containerID="3f2ba570063eaf4ea4cdf95ab368f6cf39db4e773322eecc980ffa9db12adc02"
Jul 28 16:44:05.463952 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:44:05.463937    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3f2ba570063eaf4ea4cdf95ab368f6cf39db4e773322eecc980ffa9db12adc02\": container with ID starting with 3f2ba570063eaf4ea4cdf95ab368f6cf39db4e773322eecc980ffa9db12adc02 not found: ID does not exist" containerID="3f2ba570063eaf4ea4cdf95ab368f6cf39db4e773322eecc980ffa9db12adc02"
Jul 28 16:44:05.464000 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:44:05.463956    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3f2ba570063eaf4ea4cdf95ab368f6cf39db4e773322eecc980ffa9db12adc02"} err="failed to get container status \"3f2ba570063eaf4ea4cdf95ab368f6cf39db4e773322eecc980ffa9db12adc02\": rpc error: code = NotFound desc = could not find container \"3f2ba570063eaf4ea4cdf95ab368f6cf39db4e773322eecc980ffa9db12adc02\": container with ID starting with 3f2ba570063eaf4ea4cdf95ab368f6cf39db4e773322eecc980ffa9db12adc02 not found: ID does not exist"
Jul 28 16:44:05.464664 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:44:05.464646    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-68c6a-predictor-78785fb6b7-4wt4m"]
Jul 28 16:44:05.467692 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:44:05.467670    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-68c6a-predictor-78785fb6b7-4wt4m"]
Jul 28 16:44:07.434380 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:44:07.434348    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4e8697a3-088c-43ab-944d-223d98a45b03" path="/var/lib/kubelet/pods/4e8697a3-088c-43ab-944d-223d98a45b03/volumes"
Jul 28 16:44:09.447834 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:44:09.447796    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-c3541-predictor-c87f9b99-wtgfm"
Jul 28 16:44:09.448510 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:44:09.448463    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-c3541-predictor-c87f9b99-wtgfm" podUID="714337c0-47d4-49a8-83fe-f9861c5e6a8f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.36:8080: connect: connection refused"
Jul 28 16:44:10.403445 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:44:10.403405    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-92dc3-predictor-7d6df59697-gbssk" podUID="c2052a1a-c107-4c2c-965a-232dc354050d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.35:8080: connect: connection refused"
Jul 28 16:44:19.448686 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:44:19.448645    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-c3541-predictor-c87f9b99-wtgfm" podUID="714337c0-47d4-49a8-83fe-f9861c5e6a8f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.36:8080: connect: connection refused"
Jul 28 16:44:20.403782 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:44:20.403735    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-92dc3-predictor-7d6df59697-gbssk" podUID="c2052a1a-c107-4c2c-965a-232dc354050d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.35:8080: connect: connection refused"
Jul 28 16:44:29.448625 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:44:29.448587    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-c3541-predictor-c87f9b99-wtgfm" podUID="714337c0-47d4-49a8-83fe-f9861c5e6a8f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.36:8080: connect: connection refused"
Jul 28 16:44:30.403375 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:44:30.403329    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-92dc3-predictor-7d6df59697-gbssk" podUID="c2052a1a-c107-4c2c-965a-232dc354050d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.35:8080: connect: connection refused"
Jul 28 16:44:39.449026 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:44:39.448985    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-c3541-predictor-c87f9b99-wtgfm" podUID="714337c0-47d4-49a8-83fe-f9861c5e6a8f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.36:8080: connect: connection refused"
Jul 28 16:44:40.404108 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:44:40.404080    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-92dc3-predictor-7d6df59697-gbssk"
Jul 28 16:44:49.452120 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:44:49.452090    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-c3541-predictor-c87f9b99-wtgfm"
Jul 28 16:44:51.838466 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:44:51.838436    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:44:53.398603 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:44:53.398572    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:45:01.955106 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:45:01.955028    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:45:02.041852 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:45:02.041821    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-92dc3-predictor-7d6df59697-gbssk"]
Jul 28 16:45:02.042115 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:45:02.042088    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-92dc3-predictor-7d6df59697-gbssk" podUID="c2052a1a-c107-4c2c-965a-232dc354050d" containerName="kserve-container" containerID="cri-o://f01e15d38f060966af5dbe3e99d1f836112f4d7c7427d80ed0403e567e5d2e66" gracePeriod=30
Jul 28 16:45:02.042208 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:45:02.042159    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-92dc3-predictor-7d6df59697-gbssk" podUID="c2052a1a-c107-4c2c-965a-232dc354050d" containerName="kube-rbac-proxy" containerID="cri-o://2ba9c70ca2aebfeea3ce5e0d34d7b987603962a940e0ee3be64658b9dfc6ad43" gracePeriod=30
Jul 28 16:45:02.075058 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:45:02.075034    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-12f5e-predictor-597f7c7d45-5tthj"]
Jul 28 16:45:02.075408 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:45:02.075395    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e8697a3-088c-43ab-944d-223d98a45b03" containerName="kube-rbac-proxy"
Jul 28 16:45:02.075454 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:45:02.075413    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e8697a3-088c-43ab-944d-223d98a45b03" containerName="kube-rbac-proxy"
Jul 28 16:45:02.075454 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:45:02.075437    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e8697a3-088c-43ab-944d-223d98a45b03" containerName="kserve-container"
Jul 28 16:45:02.075454 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:45:02.075443    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e8697a3-088c-43ab-944d-223d98a45b03" containerName="kserve-container"
Jul 28 16:45:02.075570 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:45:02.075514    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e8697a3-088c-43ab-944d-223d98a45b03" containerName="kserve-container"
Jul 28 16:45:02.075570 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:45:02.075525    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e8697a3-088c-43ab-944d-223d98a45b03" containerName="kube-rbac-proxy"
Jul 28 16:45:02.078606 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:45:02.078587    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-12f5e-predictor-597f7c7d45-5tthj"
Jul 28 16:45:02.080889 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:45:02.080870    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-12f5e-predictor-serving-cert\""
Jul 28 16:45:02.080961 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:45:02.080910    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-12f5e-kube-rbac-proxy-sar-config\""
Jul 28 16:45:02.089033 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:45:02.089013    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-12f5e-predictor-597f7c7d45-5tthj"]
Jul 28 16:45:02.215933 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:45:02.215834    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:45:02.251667 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:45:02.251625    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/dc1a122b-f788-4e7c-b66f-db2ee469681b-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-12f5e-predictor-597f7c7d45-5tthj\" (UID: \"dc1a122b-f788-4e7c-b66f-db2ee469681b\") " pod="kserve-ci-e2e-test/success-200-isvc-12f5e-predictor-597f7c7d45-5tthj"
Jul 28 16:45:02.251856 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:45:02.251704    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/dc1a122b-f788-4e7c-b66f-db2ee469681b-proxy-tls\") pod \"success-200-isvc-12f5e-predictor-597f7c7d45-5tthj\" (UID: \"dc1a122b-f788-4e7c-b66f-db2ee469681b\") " pod="kserve-ci-e2e-test/success-200-isvc-12f5e-predictor-597f7c7d45-5tthj"
Jul 28 16:45:02.251856 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:45:02.251736    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rx7dl\" (UniqueName: \"kubernetes.io/projected/dc1a122b-f788-4e7c-b66f-db2ee469681b-kube-api-access-rx7dl\") pod \"success-200-isvc-12f5e-predictor-597f7c7d45-5tthj\" (UID: \"dc1a122b-f788-4e7c-b66f-db2ee469681b\") " pod="kserve-ci-e2e-test/success-200-isvc-12f5e-predictor-597f7c7d45-5tthj"
Jul 28 16:45:02.353190 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:45:02.353151    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/dc1a122b-f788-4e7c-b66f-db2ee469681b-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-12f5e-predictor-597f7c7d45-5tthj\" (UID: \"dc1a122b-f788-4e7c-b66f-db2ee469681b\") " pod="kserve-ci-e2e-test/success-200-isvc-12f5e-predictor-597f7c7d45-5tthj"
Jul 28 16:45:02.353390 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:45:02.353238    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/dc1a122b-f788-4e7c-b66f-db2ee469681b-proxy-tls\") pod \"success-200-isvc-12f5e-predictor-597f7c7d45-5tthj\" (UID: \"dc1a122b-f788-4e7c-b66f-db2ee469681b\") " pod="kserve-ci-e2e-test/success-200-isvc-12f5e-predictor-597f7c7d45-5tthj"
Jul 28 16:45:02.353390 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:45:02.353272    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rx7dl\" (UniqueName: \"kubernetes.io/projected/dc1a122b-f788-4e7c-b66f-db2ee469681b-kube-api-access-rx7dl\") pod \"success-200-isvc-12f5e-predictor-597f7c7d45-5tthj\" (UID: \"dc1a122b-f788-4e7c-b66f-db2ee469681b\") " pod="kserve-ci-e2e-test/success-200-isvc-12f5e-predictor-597f7c7d45-5tthj"
Jul 28 16:45:02.353539 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:45:02.353408    2557 secret.go:189] Couldn't get secret kserve-ci-e2e-test/success-200-isvc-12f5e-predictor-serving-cert: secret "success-200-isvc-12f5e-predictor-serving-cert" not found
Jul 28 16:45:02.353539 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:45:02.353518    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dc1a122b-f788-4e7c-b66f-db2ee469681b-proxy-tls podName:dc1a122b-f788-4e7c-b66f-db2ee469681b nodeName:}" failed. No retries permitted until 2026-07-28 16:45:02.853457243 +0000 UTC m=+1419.991926638 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/dc1a122b-f788-4e7c-b66f-db2ee469681b-proxy-tls") pod "success-200-isvc-12f5e-predictor-597f7c7d45-5tthj" (UID: "dc1a122b-f788-4e7c-b66f-db2ee469681b") : secret "success-200-isvc-12f5e-predictor-serving-cert" not found
Jul 28 16:45:02.353896 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:45:02.353878    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/dc1a122b-f788-4e7c-b66f-db2ee469681b-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-12f5e-predictor-597f7c7d45-5tthj\" (UID: \"dc1a122b-f788-4e7c-b66f-db2ee469681b\") " pod="kserve-ci-e2e-test/success-200-isvc-12f5e-predictor-597f7c7d45-5tthj"
Jul 28 16:45:02.362828 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:45:02.362807    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rx7dl\" (UniqueName: \"kubernetes.io/projected/dc1a122b-f788-4e7c-b66f-db2ee469681b-kube-api-access-rx7dl\") pod \"success-200-isvc-12f5e-predictor-597f7c7d45-5tthj\" (UID: \"dc1a122b-f788-4e7c-b66f-db2ee469681b\") " pod="kserve-ci-e2e-test/success-200-isvc-12f5e-predictor-597f7c7d45-5tthj"
Jul 28 16:45:02.463434 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:45:02.463404    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:45:02.619649 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:45:02.619617    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:45:02.711455 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:45:02.711419    2557 generic.go:356] "Generic (PLEG): container finished" podID="c2052a1a-c107-4c2c-965a-232dc354050d" containerID="2ba9c70ca2aebfeea3ce5e0d34d7b987603962a940e0ee3be64658b9dfc6ad43" exitCode=2
Jul 28 16:45:02.711641 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:45:02.711475    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-92dc3-predictor-7d6df59697-gbssk" event={"ID":"c2052a1a-c107-4c2c-965a-232dc354050d","Type":"ContainerDied","Data":"2ba9c70ca2aebfeea3ce5e0d34d7b987603962a940e0ee3be64658b9dfc6ad43"}
Jul 28 16:45:02.721794 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:45:02.721762    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:45:02.858679 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:45:02.858647    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/dc1a122b-f788-4e7c-b66f-db2ee469681b-proxy-tls\") pod \"success-200-isvc-12f5e-predictor-597f7c7d45-5tthj\" (UID: \"dc1a122b-f788-4e7c-b66f-db2ee469681b\") " pod="kserve-ci-e2e-test/success-200-isvc-12f5e-predictor-597f7c7d45-5tthj"
Jul 28 16:45:02.861156 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:45:02.861125    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/dc1a122b-f788-4e7c-b66f-db2ee469681b-proxy-tls\") pod \"success-200-isvc-12f5e-predictor-597f7c7d45-5tthj\" (UID: \"dc1a122b-f788-4e7c-b66f-db2ee469681b\") " pod="kserve-ci-e2e-test/success-200-isvc-12f5e-predictor-597f7c7d45-5tthj"
Jul 28 16:45:02.989611 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:45:02.989531    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-12f5e-predictor-597f7c7d45-5tthj"
Jul 28 16:45:02.991382 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:45:02.991354    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:45:03.124262 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:45:03.123953    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-12f5e-predictor-597f7c7d45-5tthj"]
Jul 28 16:45:03.126818 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:45:03.126784    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc1a122b_f788_4e7c_b66f_db2ee469681b.slice/crio-5ae2f0bd97c7fa36dd1394f2c3e693a556207eb54c559a8532aaa9cef41fa010 WatchSource:0}: Error finding container 5ae2f0bd97c7fa36dd1394f2c3e693a556207eb54c559a8532aaa9cef41fa010: Status 404 returned error can't find the container with id 5ae2f0bd97c7fa36dd1394f2c3e693a556207eb54c559a8532aaa9cef41fa010
Jul 28 16:45:03.717188 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:45:03.717147    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-12f5e-predictor-597f7c7d45-5tthj" event={"ID":"dc1a122b-f788-4e7c-b66f-db2ee469681b","Type":"ContainerStarted","Data":"e1c696c1c0d5745db4553bdbcbab454ddde9c56b8fbac6b17104db5f70c6c705"}
Jul 28 16:45:03.717398 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:45:03.717225    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-12f5e-predictor-597f7c7d45-5tthj" event={"ID":"dc1a122b-f788-4e7c-b66f-db2ee469681b","Type":"ContainerStarted","Data":"ca6e4286ea4f6a04073994afc26a89ebd88dcc3661a6fe6b89eb6553c2e464c7"}
Jul 28 16:45:03.717398 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:45:03.717254    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-12f5e-predictor-597f7c7d45-5tthj" event={"ID":"dc1a122b-f788-4e7c-b66f-db2ee469681b","Type":"ContainerStarted","Data":"5ae2f0bd97c7fa36dd1394f2c3e693a556207eb54c559a8532aaa9cef41fa010"}
Jul 28 16:45:03.717398 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:45:03.717285    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-12f5e-predictor-597f7c7d45-5tthj"
Jul 28 16:45:03.736161 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:45:03.736097    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-12f5e-predictor-597f7c7d45-5tthj" podStartSLOduration=1.736081099 podStartE2EDuration="1.736081099s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 16:45:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:45:03.734193414 +0000 UTC m=+1420.872662829" watchObservedRunningTime="2026-07-28 16:45:03.736081099 +0000 UTC m=+1420.874550513"
Jul 28 16:45:04.722287 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:45:04.722249    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-12f5e-predictor-597f7c7d45-5tthj"
Jul 28 16:45:04.723770 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:45:04.723741    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-12f5e-predictor-597f7c7d45-5tthj" podUID="dc1a122b-f788-4e7c-b66f-db2ee469681b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.37:8080: connect: connection refused"
Jul 28 16:45:05.192642 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:45:05.192620    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-92dc3-predictor-7d6df59697-gbssk"
Jul 28 16:45:05.277629 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:45:05.277484    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c2052a1a-c107-4c2c-965a-232dc354050d-kube-rbac-proxy-sar-config\") pod \"c2052a1a-c107-4c2c-965a-232dc354050d\" (UID: \"c2052a1a-c107-4c2c-965a-232dc354050d\") "
Jul 28 16:45:05.277629 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:45:05.277618    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hx8gg\" (UniqueName: \"kubernetes.io/projected/c2052a1a-c107-4c2c-965a-232dc354050d-kube-api-access-hx8gg\") pod \"c2052a1a-c107-4c2c-965a-232dc354050d\" (UID: \"c2052a1a-c107-4c2c-965a-232dc354050d\") "
Jul 28 16:45:05.277859 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:45:05.277747    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c2052a1a-c107-4c2c-965a-232dc354050d-proxy-tls\") pod \"c2052a1a-c107-4c2c-965a-232dc354050d\" (UID: \"c2052a1a-c107-4c2c-965a-232dc354050d\") "
Jul 28 16:45:05.277859 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:45:05.277810    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c2052a1a-c107-4c2c-965a-232dc354050d-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "c2052a1a-c107-4c2c-965a-232dc354050d" (UID: "c2052a1a-c107-4c2c-965a-232dc354050d"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 16:45:05.278002 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:45:05.277986    2557 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c2052a1a-c107-4c2c-965a-232dc354050d-kube-rbac-proxy-sar-config\") on node \"ip-10-0-140-154.ec2.internal\" DevicePath \"\""
Jul 28 16:45:05.279752 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:45:05.279719    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c2052a1a-c107-4c2c-965a-232dc354050d-kube-api-access-hx8gg" (OuterVolumeSpecName: "kube-api-access-hx8gg") pod "c2052a1a-c107-4c2c-965a-232dc354050d" (UID: "c2052a1a-c107-4c2c-965a-232dc354050d"). InnerVolumeSpecName "kube-api-access-hx8gg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 16:45:05.279871 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:45:05.279780    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c2052a1a-c107-4c2c-965a-232dc354050d-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "c2052a1a-c107-4c2c-965a-232dc354050d" (UID: "c2052a1a-c107-4c2c-965a-232dc354050d"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 16:45:05.379349 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:45:05.379312    2557 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c2052a1a-c107-4c2c-965a-232dc354050d-proxy-tls\") on node \"ip-10-0-140-154.ec2.internal\" DevicePath \"\""
Jul 28 16:45:05.379349 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:45:05.379346    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hx8gg\" (UniqueName: \"kubernetes.io/projected/c2052a1a-c107-4c2c-965a-232dc354050d-kube-api-access-hx8gg\") on node \"ip-10-0-140-154.ec2.internal\" DevicePath \"\""
Jul 28 16:45:05.728359 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:45:05.728327    2557 generic.go:356] "Generic (PLEG): container finished" podID="c2052a1a-c107-4c2c-965a-232dc354050d" containerID="f01e15d38f060966af5dbe3e99d1f836112f4d7c7427d80ed0403e567e5d2e66" exitCode=0
Jul 28 16:45:05.728826 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:45:05.728377    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-92dc3-predictor-7d6df59697-gbssk"
Jul 28 16:45:05.728826 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:45:05.728411    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-92dc3-predictor-7d6df59697-gbssk" event={"ID":"c2052a1a-c107-4c2c-965a-232dc354050d","Type":"ContainerDied","Data":"f01e15d38f060966af5dbe3e99d1f836112f4d7c7427d80ed0403e567e5d2e66"}
Jul 28 16:45:05.728826 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:45:05.728486    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-92dc3-predictor-7d6df59697-gbssk" event={"ID":"c2052a1a-c107-4c2c-965a-232dc354050d","Type":"ContainerDied","Data":"37ceee4a7d579099bd05ceca2b54796a7569111883fe22e327d4032b72d971e0"}
Jul 28 16:45:05.728826 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:45:05.728531    2557 scope.go:117] "RemoveContainer" containerID="2ba9c70ca2aebfeea3ce5e0d34d7b987603962a940e0ee3be64658b9dfc6ad43"
Jul 28 16:45:05.729183 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:45:05.729162    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-12f5e-predictor-597f7c7d45-5tthj" podUID="dc1a122b-f788-4e7c-b66f-db2ee469681b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.37:8080: connect: connection refused"
Jul 28 16:45:05.737116 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:45:05.737100    2557 scope.go:117] "RemoveContainer" containerID="f01e15d38f060966af5dbe3e99d1f836112f4d7c7427d80ed0403e567e5d2e66"
Jul 28 16:45:05.744349 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:45:05.744331    2557 scope.go:117] "RemoveContainer" containerID="2ba9c70ca2aebfeea3ce5e0d34d7b987603962a940e0ee3be64658b9dfc6ad43"
Jul 28 16:45:05.744663 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:45:05.744626    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2ba9c70ca2aebfeea3ce5e0d34d7b987603962a940e0ee3be64658b9dfc6ad43\": container with ID starting with 2ba9c70ca2aebfeea3ce5e0d34d7b987603962a940e0ee3be64658b9dfc6ad43 not found: ID does not exist" containerID="2ba9c70ca2aebfeea3ce5e0d34d7b987603962a940e0ee3be64658b9dfc6ad43"
Jul 28 16:45:05.744769 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:45:05.744672    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2ba9c70ca2aebfeea3ce5e0d34d7b987603962a940e0ee3be64658b9dfc6ad43"} err="failed to get container status \"2ba9c70ca2aebfeea3ce5e0d34d7b987603962a940e0ee3be64658b9dfc6ad43\": rpc error: code = NotFound desc = could not find container \"2ba9c70ca2aebfeea3ce5e0d34d7b987603962a940e0ee3be64658b9dfc6ad43\": container with ID starting with 2ba9c70ca2aebfeea3ce5e0d34d7b987603962a940e0ee3be64658b9dfc6ad43 not found: ID does not exist"
Jul 28 16:45:05.744769 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:45:05.744696    2557 scope.go:117] "RemoveContainer" containerID="f01e15d38f060966af5dbe3e99d1f836112f4d7c7427d80ed0403e567e5d2e66"
Jul 28 16:45:05.745117 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:45:05.745074    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f01e15d38f060966af5dbe3e99d1f836112f4d7c7427d80ed0403e567e5d2e66\": container with ID starting with f01e15d38f060966af5dbe3e99d1f836112f4d7c7427d80ed0403e567e5d2e66 not found: ID does not exist" containerID="f01e15d38f060966af5dbe3e99d1f836112f4d7c7427d80ed0403e567e5d2e66"
Jul 28 16:45:05.745213 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:45:05.745114    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f01e15d38f060966af5dbe3e99d1f836112f4d7c7427d80ed0403e567e5d2e66"} err="failed to get container status \"f01e15d38f060966af5dbe3e99d1f836112f4d7c7427d80ed0403e567e5d2e66\": rpc error: code = NotFound desc = could not find container \"f01e15d38f060966af5dbe3e99d1f836112f4d7c7427d80ed0403e567e5d2e66\": container with ID starting with f01e15d38f060966af5dbe3e99d1f836112f4d7c7427d80ed0403e567e5d2e66 not found: ID does not exist"
Jul 28 16:45:05.747283 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:45:05.747263    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-92dc3-predictor-7d6df59697-gbssk"]
Jul 28 16:45:05.752298 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:45:05.752280    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-92dc3-predictor-7d6df59697-gbssk"]
Jul 28 16:45:07.434768 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:45:07.434738    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c2052a1a-c107-4c2c-965a-232dc354050d" path="/var/lib/kubelet/pods/c2052a1a-c107-4c2c-965a-232dc354050d/volumes"
Jul 28 16:45:10.733313 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:45:10.733283    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-12f5e-predictor-597f7c7d45-5tthj"
Jul 28 16:45:10.733924 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:45:10.733895    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-12f5e-predictor-597f7c7d45-5tthj" podUID="dc1a122b-f788-4e7c-b66f-db2ee469681b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.37:8080: connect: connection refused"
Jul 28 16:45:20.734409 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:45:20.734322    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-12f5e-predictor-597f7c7d45-5tthj" podUID="dc1a122b-f788-4e7c-b66f-db2ee469681b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.37:8080: connect: connection refused"
Jul 28 16:45:30.733901 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:45:30.733861    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-12f5e-predictor-597f7c7d45-5tthj" podUID="dc1a122b-f788-4e7c-b66f-db2ee469681b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.37:8080: connect: connection refused"
Jul 28 16:45:40.734331 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:45:40.734291    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-12f5e-predictor-597f7c7d45-5tthj" podUID="dc1a122b-f788-4e7c-b66f-db2ee469681b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.37:8080: connect: connection refused"
Jul 28 16:45:50.734906 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:45:50.734869    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-12f5e-predictor-597f7c7d45-5tthj"
Jul 28 16:45:51.746960 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:45:51.746929    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:46:02.177277 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:46:02.177243    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:46:02.603782 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:46:02.603753    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:48:47.022175 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:48:47.022072    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:48:47.078597 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:48:47.078560    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:53:16.888002 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:53:16.887964    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:53:16.963468 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:53:16.963435    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-c3541-predictor-c87f9b99-wtgfm"]
Jul 28 16:53:16.963732 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:53:16.963703    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-c3541-predictor-c87f9b99-wtgfm" podUID="714337c0-47d4-49a8-83fe-f9861c5e6a8f" containerName="kserve-container" containerID="cri-o://d5e22596515f5e7e0104357a102e9cc21669618a3e54d8a43665afcd40407688" gracePeriod=30
Jul 28 16:53:16.963870 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:53:16.963753    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-c3541-predictor-c87f9b99-wtgfm" podUID="714337c0-47d4-49a8-83fe-f9861c5e6a8f" containerName="kube-rbac-proxy" containerID="cri-o://90ce6d0f8a07d6dfc16d61be92b5d1cc1efb9c8838be89b233f612a993cc31f5" gracePeriod=30
Jul 28 16:53:17.016988 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:53:17.016951    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-2717b-predictor-5c68f8885c-sjxmd"]
Jul 28 16:53:17.017384 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:53:17.017370    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c2052a1a-c107-4c2c-965a-232dc354050d" containerName="kube-rbac-proxy"
Jul 28 16:53:17.017447 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:53:17.017386    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="c2052a1a-c107-4c2c-965a-232dc354050d" containerName="kube-rbac-proxy"
Jul 28 16:53:17.017447 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:53:17.017403    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c2052a1a-c107-4c2c-965a-232dc354050d" containerName="kserve-container"
Jul 28 16:53:17.017447 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:53:17.017408    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="c2052a1a-c107-4c2c-965a-232dc354050d" containerName="kserve-container"
Jul 28 16:53:17.017633 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:53:17.017472    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="c2052a1a-c107-4c2c-965a-232dc354050d" containerName="kserve-container"
Jul 28 16:53:17.017633 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:53:17.017482    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="c2052a1a-c107-4c2c-965a-232dc354050d" containerName="kube-rbac-proxy"
Jul 28 16:53:17.020521 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:53:17.020504    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-2717b-predictor-5c68f8885c-sjxmd"
Jul 28 16:53:17.022910 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:53:17.022887    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-2717b-kube-rbac-proxy-sar-config\""
Jul 28 16:53:17.022985 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:53:17.022955    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-2717b-predictor-serving-cert\""
Jul 28 16:53:17.031206 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:53:17.031184    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-2717b-predictor-5c68f8885c-sjxmd"]
Jul 28 16:53:17.157386 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:53:17.157300    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vp979\" (UniqueName: \"kubernetes.io/projected/7ec846e8-0d27-4b69-bfe0-538d5f6325de-kube-api-access-vp979\") pod \"success-200-isvc-2717b-predictor-5c68f8885c-sjxmd\" (UID: \"7ec846e8-0d27-4b69-bfe0-538d5f6325de\") " pod="kserve-ci-e2e-test/success-200-isvc-2717b-predictor-5c68f8885c-sjxmd"
Jul 28 16:53:17.157386 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:53:17.157361    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7ec846e8-0d27-4b69-bfe0-538d5f6325de-proxy-tls\") pod \"success-200-isvc-2717b-predictor-5c68f8885c-sjxmd\" (UID: \"7ec846e8-0d27-4b69-bfe0-538d5f6325de\") " pod="kserve-ci-e2e-test/success-200-isvc-2717b-predictor-5c68f8885c-sjxmd"
Jul 28 16:53:17.157611 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:53:17.157390    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/7ec846e8-0d27-4b69-bfe0-538d5f6325de-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-2717b-predictor-5c68f8885c-sjxmd\" (UID: \"7ec846e8-0d27-4b69-bfe0-538d5f6325de\") " pod="kserve-ci-e2e-test/success-200-isvc-2717b-predictor-5c68f8885c-sjxmd"
Jul 28 16:53:17.258675 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:53:17.258646    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vp979\" (UniqueName: \"kubernetes.io/projected/7ec846e8-0d27-4b69-bfe0-538d5f6325de-kube-api-access-vp979\") pod \"success-200-isvc-2717b-predictor-5c68f8885c-sjxmd\" (UID: \"7ec846e8-0d27-4b69-bfe0-538d5f6325de\") " pod="kserve-ci-e2e-test/success-200-isvc-2717b-predictor-5c68f8885c-sjxmd"
Jul 28 16:53:17.258855 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:53:17.258702    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7ec846e8-0d27-4b69-bfe0-538d5f6325de-proxy-tls\") pod \"success-200-isvc-2717b-predictor-5c68f8885c-sjxmd\" (UID: \"7ec846e8-0d27-4b69-bfe0-538d5f6325de\") " pod="kserve-ci-e2e-test/success-200-isvc-2717b-predictor-5c68f8885c-sjxmd"
Jul 28 16:53:17.258855 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:53:17.258739    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/7ec846e8-0d27-4b69-bfe0-538d5f6325de-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-2717b-predictor-5c68f8885c-sjxmd\" (UID: \"7ec846e8-0d27-4b69-bfe0-538d5f6325de\") " pod="kserve-ci-e2e-test/success-200-isvc-2717b-predictor-5c68f8885c-sjxmd"
Jul 28 16:53:17.259382 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:53:17.259361    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/7ec846e8-0d27-4b69-bfe0-538d5f6325de-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-2717b-predictor-5c68f8885c-sjxmd\" (UID: \"7ec846e8-0d27-4b69-bfe0-538d5f6325de\") " pod="kserve-ci-e2e-test/success-200-isvc-2717b-predictor-5c68f8885c-sjxmd"
Jul 28 16:53:17.261225 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:53:17.261201    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7ec846e8-0d27-4b69-bfe0-538d5f6325de-proxy-tls\") pod \"success-200-isvc-2717b-predictor-5c68f8885c-sjxmd\" (UID: \"7ec846e8-0d27-4b69-bfe0-538d5f6325de\") " pod="kserve-ci-e2e-test/success-200-isvc-2717b-predictor-5c68f8885c-sjxmd"
Jul 28 16:53:17.268041 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:53:17.268021    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vp979\" (UniqueName: \"kubernetes.io/projected/7ec846e8-0d27-4b69-bfe0-538d5f6325de-kube-api-access-vp979\") pod \"success-200-isvc-2717b-predictor-5c68f8885c-sjxmd\" (UID: \"7ec846e8-0d27-4b69-bfe0-538d5f6325de\") " pod="kserve-ci-e2e-test/success-200-isvc-2717b-predictor-5c68f8885c-sjxmd"
Jul 28 16:53:17.331857 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:53:17.331824    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-2717b-predictor-5c68f8885c-sjxmd"
Jul 28 16:53:17.462885 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:53:17.462861    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-2717b-predictor-5c68f8885c-sjxmd"]
Jul 28 16:53:17.464987 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:53:17.464956    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ec846e8_0d27_4b69_bfe0_538d5f6325de.slice/crio-cb94278e589dbb628a664401761254c4ba48f987bb126e48f0086c150b28d1da WatchSource:0}: Error finding container cb94278e589dbb628a664401761254c4ba48f987bb126e48f0086c150b28d1da: Status 404 returned error can't find the container with id cb94278e589dbb628a664401761254c4ba48f987bb126e48f0086c150b28d1da
Jul 28 16:53:17.926566 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:53:17.926523    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-2717b-predictor-5c68f8885c-sjxmd" event={"ID":"7ec846e8-0d27-4b69-bfe0-538d5f6325de","Type":"ContainerStarted","Data":"506e70955dabce39dce6e3040d82cbea577c661fb4aecbd14ba2b8b2d2540a19"}
Jul 28 16:53:17.927023 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:53:17.926596    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-2717b-predictor-5c68f8885c-sjxmd" event={"ID":"7ec846e8-0d27-4b69-bfe0-538d5f6325de","Type":"ContainerStarted","Data":"8fdb665beaa57bf415f188b36b70519e772c36bb785186030664718be1d44373"}
Jul 28 16:53:17.927023 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:53:17.926634    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-2717b-predictor-5c68f8885c-sjxmd"
Jul 28 16:53:17.927023 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:53:17.926664    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-2717b-predictor-5c68f8885c-sjxmd" event={"ID":"7ec846e8-0d27-4b69-bfe0-538d5f6325de","Type":"ContainerStarted","Data":"cb94278e589dbb628a664401761254c4ba48f987bb126e48f0086c150b28d1da"}
Jul 28 16:53:17.928926 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:53:17.928895    2557 generic.go:356] "Generic (PLEG): container finished" podID="714337c0-47d4-49a8-83fe-f9861c5e6a8f" containerID="90ce6d0f8a07d6dfc16d61be92b5d1cc1efb9c8838be89b233f612a993cc31f5" exitCode=2
Jul 28 16:53:17.929053 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:53:17.928946    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-c3541-predictor-c87f9b99-wtgfm" event={"ID":"714337c0-47d4-49a8-83fe-f9861c5e6a8f","Type":"ContainerDied","Data":"90ce6d0f8a07d6dfc16d61be92b5d1cc1efb9c8838be89b233f612a993cc31f5"}
Jul 28 16:53:17.943244 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:53:17.943198    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-2717b-predictor-5c68f8885c-sjxmd" podStartSLOduration=0.943184973 podStartE2EDuration="943.184973ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 16:53:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:53:17.942051753 +0000 UTC m=+1915.080521178" watchObservedRunningTime="2026-07-28 16:53:17.943184973 +0000 UTC m=+1915.081654386"
Jul 28 16:53:18.056585 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:53:18.056548    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:53:18.934122 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:53:18.934094    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-2717b-predictor-5c68f8885c-sjxmd"
Jul 28 16:53:18.935636 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:53:18.935603    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-2717b-predictor-5c68f8885c-sjxmd" podUID="7ec846e8-0d27-4b69-bfe0-538d5f6325de" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.38:8080: connect: connection refused"
Jul 28 16:53:19.442435 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:53:19.442390    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-c3541-predictor-c87f9b99-wtgfm" podUID="714337c0-47d4-49a8-83fe-f9861c5e6a8f" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.36:8643/healthz\": dial tcp 10.133.0.36:8643: connect: connection refused"
Jul 28 16:53:19.448776 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:53:19.448753    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-c3541-predictor-c87f9b99-wtgfm" podUID="714337c0-47d4-49a8-83fe-f9861c5e6a8f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.36:8080: connect: connection refused"
Jul 28 16:53:19.939153 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:53:19.939115    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-2717b-predictor-5c68f8885c-sjxmd" podUID="7ec846e8-0d27-4b69-bfe0-538d5f6325de" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.38:8080: connect: connection refused"
Jul 28 16:53:20.212173 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:53:20.212145    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-c3541-predictor-c87f9b99-wtgfm"
Jul 28 16:53:20.290457 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:53:20.290424    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/714337c0-47d4-49a8-83fe-f9861c5e6a8f-kube-rbac-proxy-sar-config\") pod \"714337c0-47d4-49a8-83fe-f9861c5e6a8f\" (UID: \"714337c0-47d4-49a8-83fe-f9861c5e6a8f\") "
Jul 28 16:53:20.290643 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:53:20.290525    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-688d9\" (UniqueName: \"kubernetes.io/projected/714337c0-47d4-49a8-83fe-f9861c5e6a8f-kube-api-access-688d9\") pod \"714337c0-47d4-49a8-83fe-f9861c5e6a8f\" (UID: \"714337c0-47d4-49a8-83fe-f9861c5e6a8f\") "
Jul 28 16:53:20.290643 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:53:20.290558    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/714337c0-47d4-49a8-83fe-f9861c5e6a8f-proxy-tls\") pod \"714337c0-47d4-49a8-83fe-f9861c5e6a8f\" (UID: \"714337c0-47d4-49a8-83fe-f9861c5e6a8f\") "
Jul 28 16:53:20.290828 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:53:20.290804    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/714337c0-47d4-49a8-83fe-f9861c5e6a8f-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "714337c0-47d4-49a8-83fe-f9861c5e6a8f" (UID: "714337c0-47d4-49a8-83fe-f9861c5e6a8f"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 16:53:20.292847 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:53:20.292827    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/714337c0-47d4-49a8-83fe-f9861c5e6a8f-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "714337c0-47d4-49a8-83fe-f9861c5e6a8f" (UID: "714337c0-47d4-49a8-83fe-f9861c5e6a8f"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 16:53:20.292847 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:53:20.292839    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/714337c0-47d4-49a8-83fe-f9861c5e6a8f-kube-api-access-688d9" (OuterVolumeSpecName: "kube-api-access-688d9") pod "714337c0-47d4-49a8-83fe-f9861c5e6a8f" (UID: "714337c0-47d4-49a8-83fe-f9861c5e6a8f"). InnerVolumeSpecName "kube-api-access-688d9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 16:53:20.391366 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:53:20.391332    2557 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/714337c0-47d4-49a8-83fe-f9861c5e6a8f-kube-rbac-proxy-sar-config\") on node \"ip-10-0-140-154.ec2.internal\" DevicePath \"\""
Jul 28 16:53:20.391366 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:53:20.391360    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-688d9\" (UniqueName: \"kubernetes.io/projected/714337c0-47d4-49a8-83fe-f9861c5e6a8f-kube-api-access-688d9\") on node \"ip-10-0-140-154.ec2.internal\" DevicePath \"\""
Jul 28 16:53:20.391366 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:53:20.391371    2557 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/714337c0-47d4-49a8-83fe-f9861c5e6a8f-proxy-tls\") on node \"ip-10-0-140-154.ec2.internal\" DevicePath \"\""
Jul 28 16:53:20.947087 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:53:20.947051    2557 generic.go:356] "Generic (PLEG): container finished" podID="714337c0-47d4-49a8-83fe-f9861c5e6a8f" containerID="d5e22596515f5e7e0104357a102e9cc21669618a3e54d8a43665afcd40407688" exitCode=0
Jul 28 16:53:20.947555 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:53:20.947091    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-c3541-predictor-c87f9b99-wtgfm" event={"ID":"714337c0-47d4-49a8-83fe-f9861c5e6a8f","Type":"ContainerDied","Data":"d5e22596515f5e7e0104357a102e9cc21669618a3e54d8a43665afcd40407688"}
Jul 28 16:53:20.947555 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:53:20.947116    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-c3541-predictor-c87f9b99-wtgfm"
Jul 28 16:53:20.947555 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:53:20.947189    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-c3541-predictor-c87f9b99-wtgfm" event={"ID":"714337c0-47d4-49a8-83fe-f9861c5e6a8f","Type":"ContainerDied","Data":"21874b0f0ef8841825f400d2fb3d180a21fe4eafcaf4ca21c133028f4ec8ec1b"}
Jul 28 16:53:20.947555 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:53:20.947223    2557 scope.go:117] "RemoveContainer" containerID="90ce6d0f8a07d6dfc16d61be92b5d1cc1efb9c8838be89b233f612a993cc31f5"
Jul 28 16:53:20.956126 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:53:20.956112    2557 scope.go:117] "RemoveContainer" containerID="d5e22596515f5e7e0104357a102e9cc21669618a3e54d8a43665afcd40407688"
Jul 28 16:53:20.963694 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:53:20.963631    2557 scope.go:117] "RemoveContainer" containerID="90ce6d0f8a07d6dfc16d61be92b5d1cc1efb9c8838be89b233f612a993cc31f5"
Jul 28 16:53:20.963918 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:53:20.963897    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"90ce6d0f8a07d6dfc16d61be92b5d1cc1efb9c8838be89b233f612a993cc31f5\": container with ID starting with 90ce6d0f8a07d6dfc16d61be92b5d1cc1efb9c8838be89b233f612a993cc31f5 not found: ID does not exist" containerID="90ce6d0f8a07d6dfc16d61be92b5d1cc1efb9c8838be89b233f612a993cc31f5"
Jul 28 16:53:20.963973 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:53:20.963927    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"90ce6d0f8a07d6dfc16d61be92b5d1cc1efb9c8838be89b233f612a993cc31f5"} err="failed to get container status \"90ce6d0f8a07d6dfc16d61be92b5d1cc1efb9c8838be89b233f612a993cc31f5\": rpc error: code = NotFound desc = could not find container \"90ce6d0f8a07d6dfc16d61be92b5d1cc1efb9c8838be89b233f612a993cc31f5\": container with ID starting with 90ce6d0f8a07d6dfc16d61be92b5d1cc1efb9c8838be89b233f612a993cc31f5 not found: ID does not exist"
Jul 28 16:53:20.963973 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:53:20.963944    2557 scope.go:117] "RemoveContainer" containerID="d5e22596515f5e7e0104357a102e9cc21669618a3e54d8a43665afcd40407688"
Jul 28 16:53:20.964181 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:53:20.964167    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d5e22596515f5e7e0104357a102e9cc21669618a3e54d8a43665afcd40407688\": container with ID starting with d5e22596515f5e7e0104357a102e9cc21669618a3e54d8a43665afcd40407688 not found: ID does not exist" containerID="d5e22596515f5e7e0104357a102e9cc21669618a3e54d8a43665afcd40407688"
Jul 28 16:53:20.964219 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:53:20.964185    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d5e22596515f5e7e0104357a102e9cc21669618a3e54d8a43665afcd40407688"} err="failed to get container status \"d5e22596515f5e7e0104357a102e9cc21669618a3e54d8a43665afcd40407688\": rpc error: code = NotFound desc = could not find container \"d5e22596515f5e7e0104357a102e9cc21669618a3e54d8a43665afcd40407688\": container with ID starting with d5e22596515f5e7e0104357a102e9cc21669618a3e54d8a43665afcd40407688 not found: ID does not exist"
Jul 28 16:53:20.969754 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:53:20.969731    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-c3541-predictor-c87f9b99-wtgfm"]
Jul 28 16:53:20.977531 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:53:20.977512    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-c3541-predictor-c87f9b99-wtgfm"]
Jul 28 16:53:21.435146 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:53:21.435102    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="714337c0-47d4-49a8-83fe-f9861c5e6a8f" path="/var/lib/kubelet/pods/714337c0-47d4-49a8-83fe-f9861c5e6a8f/volumes"
Jul 28 16:53:24.944023 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:53:24.943995    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-2717b-predictor-5c68f8885c-sjxmd"
Jul 28 16:53:24.944461 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:53:24.944436    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-2717b-predictor-5c68f8885c-sjxmd" podUID="7ec846e8-0d27-4b69-bfe0-538d5f6325de" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.38:8080: connect: connection refused"
Jul 28 16:53:34.945091 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:53:34.945047    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-2717b-predictor-5c68f8885c-sjxmd" podUID="7ec846e8-0d27-4b69-bfe0-538d5f6325de" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.38:8080: connect: connection refused"
Jul 28 16:53:44.944676 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:53:44.944638    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-2717b-predictor-5c68f8885c-sjxmd" podUID="7ec846e8-0d27-4b69-bfe0-538d5f6325de" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.38:8080: connect: connection refused"
Jul 28 16:53:52.684817 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:53:52.684781    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:53:54.944733 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:53:54.944696    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-2717b-predictor-5c68f8885c-sjxmd" podUID="7ec846e8-0d27-4b69-bfe0-538d5f6325de" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.38:8080: connect: connection refused"
Jul 28 16:54:04.945122 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:54:04.945092    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-2717b-predictor-5c68f8885c-sjxmd"
Jul 28 16:54:16.720760 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:54:16.720674    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:54:16.804904 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:54:16.804870    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-12f5e-predictor-597f7c7d45-5tthj"]
Jul 28 16:54:16.805224 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:54:16.805193    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-12f5e-predictor-597f7c7d45-5tthj" podUID="dc1a122b-f788-4e7c-b66f-db2ee469681b" containerName="kserve-container" containerID="cri-o://ca6e4286ea4f6a04073994afc26a89ebd88dcc3661a6fe6b89eb6553c2e464c7" gracePeriod=30
Jul 28 16:54:16.805381 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:54:16.805321    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-12f5e-predictor-597f7c7d45-5tthj" podUID="dc1a122b-f788-4e7c-b66f-db2ee469681b" containerName="kube-rbac-proxy" containerID="cri-o://e1c696c1c0d5745db4553bdbcbab454ddde9c56b8fbac6b17104db5f70c6c705" gracePeriod=30
Jul 28 16:54:16.827914 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:54:16.827835    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:54:16.875637 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:54:16.875585    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-b9a8c-predictor-7dd8d57945-lrxn7"]
Jul 28 16:54:16.876056 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:54:16.876037    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="714337c0-47d4-49a8-83fe-f9861c5e6a8f" containerName="kserve-container"
Jul 28 16:54:16.876154 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:54:16.876058    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="714337c0-47d4-49a8-83fe-f9861c5e6a8f" containerName="kserve-container"
Jul 28 16:54:16.876154 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:54:16.876087    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="714337c0-47d4-49a8-83fe-f9861c5e6a8f" containerName="kube-rbac-proxy"
Jul 28 16:54:16.876154 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:54:16.876097    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="714337c0-47d4-49a8-83fe-f9861c5e6a8f" containerName="kube-rbac-proxy"
Jul 28 16:54:16.876313 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:54:16.876183    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="714337c0-47d4-49a8-83fe-f9861c5e6a8f" containerName="kube-rbac-proxy"
Jul 28 16:54:16.876313 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:54:16.876198    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="714337c0-47d4-49a8-83fe-f9861c5e6a8f" containerName="kserve-container"
Jul 28 16:54:16.879532 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:54:16.879512    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-b9a8c-predictor-7dd8d57945-lrxn7"
Jul 28 16:54:16.882236 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:54:16.882214    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-b9a8c-kube-rbac-proxy-sar-config\""
Jul 28 16:54:16.882236 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:54:16.882223    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-b9a8c-predictor-serving-cert\""
Jul 28 16:54:16.889851 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:54:16.889816    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-b9a8c-predictor-7dd8d57945-lrxn7"]
Jul 28 16:54:16.898655 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:54:16.898599    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:54:16.964769 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:54:16.964741    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vlvxz\" (UniqueName: \"kubernetes.io/projected/7e6417d3-5731-474e-8805-7c66df1f2ed0-kube-api-access-vlvxz\") pod \"success-200-isvc-b9a8c-predictor-7dd8d57945-lrxn7\" (UID: \"7e6417d3-5731-474e-8805-7c66df1f2ed0\") " pod="kserve-ci-e2e-test/success-200-isvc-b9a8c-predictor-7dd8d57945-lrxn7"
Jul 28 16:54:16.964966 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:54:16.964796    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/7e6417d3-5731-474e-8805-7c66df1f2ed0-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-b9a8c-predictor-7dd8d57945-lrxn7\" (UID: \"7e6417d3-5731-474e-8805-7c66df1f2ed0\") " pod="kserve-ci-e2e-test/success-200-isvc-b9a8c-predictor-7dd8d57945-lrxn7"
Jul 28 16:54:16.964966 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:54:16.964828    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7e6417d3-5731-474e-8805-7c66df1f2ed0-proxy-tls\") pod \"success-200-isvc-b9a8c-predictor-7dd8d57945-lrxn7\" (UID: \"7e6417d3-5731-474e-8805-7c66df1f2ed0\") " pod="kserve-ci-e2e-test/success-200-isvc-b9a8c-predictor-7dd8d57945-lrxn7"
Jul 28 16:54:16.965273 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:54:16.965237    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:54:17.066146 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:54:17.066113    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vlvxz\" (UniqueName: \"kubernetes.io/projected/7e6417d3-5731-474e-8805-7c66df1f2ed0-kube-api-access-vlvxz\") pod \"success-200-isvc-b9a8c-predictor-7dd8d57945-lrxn7\" (UID: \"7e6417d3-5731-474e-8805-7c66df1f2ed0\") " pod="kserve-ci-e2e-test/success-200-isvc-b9a8c-predictor-7dd8d57945-lrxn7"
Jul 28 16:54:17.066313 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:54:17.066164    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/7e6417d3-5731-474e-8805-7c66df1f2ed0-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-b9a8c-predictor-7dd8d57945-lrxn7\" (UID: \"7e6417d3-5731-474e-8805-7c66df1f2ed0\") " pod="kserve-ci-e2e-test/success-200-isvc-b9a8c-predictor-7dd8d57945-lrxn7"
Jul 28 16:54:17.066313 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:54:17.066193    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7e6417d3-5731-474e-8805-7c66df1f2ed0-proxy-tls\") pod \"success-200-isvc-b9a8c-predictor-7dd8d57945-lrxn7\" (UID: \"7e6417d3-5731-474e-8805-7c66df1f2ed0\") " pod="kserve-ci-e2e-test/success-200-isvc-b9a8c-predictor-7dd8d57945-lrxn7"
Jul 28 16:54:17.066980 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:54:17.066953    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/7e6417d3-5731-474e-8805-7c66df1f2ed0-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-b9a8c-predictor-7dd8d57945-lrxn7\" (UID: \"7e6417d3-5731-474e-8805-7c66df1f2ed0\") " pod="kserve-ci-e2e-test/success-200-isvc-b9a8c-predictor-7dd8d57945-lrxn7"
Jul 28 16:54:17.068956 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:54:17.068928    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7e6417d3-5731-474e-8805-7c66df1f2ed0-proxy-tls\") pod \"success-200-isvc-b9a8c-predictor-7dd8d57945-lrxn7\" (UID: \"7e6417d3-5731-474e-8805-7c66df1f2ed0\") " pod="kserve-ci-e2e-test/success-200-isvc-b9a8c-predictor-7dd8d57945-lrxn7"
Jul 28 16:54:17.071479 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:54:17.071436    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:54:17.074393 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:54:17.074373    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vlvxz\" (UniqueName: \"kubernetes.io/projected/7e6417d3-5731-474e-8805-7c66df1f2ed0-kube-api-access-vlvxz\") pod \"success-200-isvc-b9a8c-predictor-7dd8d57945-lrxn7\" (UID: \"7e6417d3-5731-474e-8805-7c66df1f2ed0\") " pod="kserve-ci-e2e-test/success-200-isvc-b9a8c-predictor-7dd8d57945-lrxn7"
Jul 28 16:54:17.192745 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:54:17.192713    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-b9a8c-predictor-7dd8d57945-lrxn7"
Jul 28 16:54:17.197049 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:54:17.197021    2557 generic.go:356] "Generic (PLEG): container finished" podID="dc1a122b-f788-4e7c-b66f-db2ee469681b" containerID="e1c696c1c0d5745db4553bdbcbab454ddde9c56b8fbac6b17104db5f70c6c705" exitCode=2
Jul 28 16:54:17.197176 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:54:17.197098    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-12f5e-predictor-597f7c7d45-5tthj" event={"ID":"dc1a122b-f788-4e7c-b66f-db2ee469681b","Type":"ContainerDied","Data":"e1c696c1c0d5745db4553bdbcbab454ddde9c56b8fbac6b17104db5f70c6c705"}
Jul 28 16:54:17.242749 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:54:17.242709    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:54:17.328659 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:54:17.328631    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-b9a8c-predictor-7dd8d57945-lrxn7"]
Jul 28 16:54:17.331440 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:54:17.331412    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e6417d3_5731_474e_8805_7c66df1f2ed0.slice/crio-f6886ec403883328be5070cfdad812407db8316cd0d3fbbf7420956d5e094b07 WatchSource:0}: Error finding container f6886ec403883328be5070cfdad812407db8316cd0d3fbbf7420956d5e094b07: Status 404 returned error can't find the container with id f6886ec403883328be5070cfdad812407db8316cd0d3fbbf7420956d5e094b07
Jul 28 16:54:17.331663 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:54:17.331639    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:54:18.204086 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:54:18.204048    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-b9a8c-predictor-7dd8d57945-lrxn7" event={"ID":"7e6417d3-5731-474e-8805-7c66df1f2ed0","Type":"ContainerStarted","Data":"84841d2d03667c4ea6e39e8f5520ca76a3390db8f0c6e296a5d38bab53582639"}
Jul 28 16:54:18.204544 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:54:18.204109    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-b9a8c-predictor-7dd8d57945-lrxn7" event={"ID":"7e6417d3-5731-474e-8805-7c66df1f2ed0","Type":"ContainerStarted","Data":"57683799e6a8fc66bb3ff5b4b83d9103515cce50d359e8faf9127c19ac455caa"}
Jul 28 16:54:18.204544 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:54:18.204140    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-b9a8c-predictor-7dd8d57945-lrxn7" event={"ID":"7e6417d3-5731-474e-8805-7c66df1f2ed0","Type":"ContainerStarted","Data":"f6886ec403883328be5070cfdad812407db8316cd0d3fbbf7420956d5e094b07"}
Jul 28 16:54:18.204544 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:54:18.204175    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-b9a8c-predictor-7dd8d57945-lrxn7"
Jul 28 16:54:18.222909 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:54:18.222863    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-b9a8c-predictor-7dd8d57945-lrxn7" podStartSLOduration=2.222850141 podStartE2EDuration="2.222850141s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 16:54:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:54:18.22167609 +0000 UTC m=+1975.360145504" watchObservedRunningTime="2026-07-28 16:54:18.222850141 +0000 UTC m=+1975.361319554"
Jul 28 16:54:19.209352 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:54:19.209312    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-b9a8c-predictor-7dd8d57945-lrxn7"
Jul 28 16:54:19.210637 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:54:19.210607    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-b9a8c-predictor-7dd8d57945-lrxn7" podUID="7e6417d3-5731-474e-8805-7c66df1f2ed0" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.39:8080: connect: connection refused"
Jul 28 16:54:20.059372 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:54:20.059344    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-12f5e-predictor-597f7c7d45-5tthj"
Jul 28 16:54:20.193653 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:54:20.193566    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/dc1a122b-f788-4e7c-b66f-db2ee469681b-proxy-tls\") pod \"dc1a122b-f788-4e7c-b66f-db2ee469681b\" (UID: \"dc1a122b-f788-4e7c-b66f-db2ee469681b\") "
Jul 28 16:54:20.193794 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:54:20.193663    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rx7dl\" (UniqueName: \"kubernetes.io/projected/dc1a122b-f788-4e7c-b66f-db2ee469681b-kube-api-access-rx7dl\") pod \"dc1a122b-f788-4e7c-b66f-db2ee469681b\" (UID: \"dc1a122b-f788-4e7c-b66f-db2ee469681b\") "
Jul 28 16:54:20.193794 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:54:20.193697    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/dc1a122b-f788-4e7c-b66f-db2ee469681b-kube-rbac-proxy-sar-config\") pod \"dc1a122b-f788-4e7c-b66f-db2ee469681b\" (UID: \"dc1a122b-f788-4e7c-b66f-db2ee469681b\") "
Jul 28 16:54:20.194076 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:54:20.194051    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dc1a122b-f788-4e7c-b66f-db2ee469681b-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "dc1a122b-f788-4e7c-b66f-db2ee469681b" (UID: "dc1a122b-f788-4e7c-b66f-db2ee469681b"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 16:54:20.195792 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:54:20.195767    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dc1a122b-f788-4e7c-b66f-db2ee469681b-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "dc1a122b-f788-4e7c-b66f-db2ee469681b" (UID: "dc1a122b-f788-4e7c-b66f-db2ee469681b"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 16:54:20.195902 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:54:20.195888    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dc1a122b-f788-4e7c-b66f-db2ee469681b-kube-api-access-rx7dl" (OuterVolumeSpecName: "kube-api-access-rx7dl") pod "dc1a122b-f788-4e7c-b66f-db2ee469681b" (UID: "dc1a122b-f788-4e7c-b66f-db2ee469681b"). InnerVolumeSpecName "kube-api-access-rx7dl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 16:54:20.219868 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:54:20.219838    2557 generic.go:356] "Generic (PLEG): container finished" podID="dc1a122b-f788-4e7c-b66f-db2ee469681b" containerID="ca6e4286ea4f6a04073994afc26a89ebd88dcc3661a6fe6b89eb6553c2e464c7" exitCode=0
Jul 28 16:54:20.220269 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:54:20.219915    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-12f5e-predictor-597f7c7d45-5tthj"
Jul 28 16:54:20.220269 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:54:20.219919    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-12f5e-predictor-597f7c7d45-5tthj" event={"ID":"dc1a122b-f788-4e7c-b66f-db2ee469681b","Type":"ContainerDied","Data":"ca6e4286ea4f6a04073994afc26a89ebd88dcc3661a6fe6b89eb6553c2e464c7"}
Jul 28 16:54:20.220269 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:54:20.220002    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-12f5e-predictor-597f7c7d45-5tthj" event={"ID":"dc1a122b-f788-4e7c-b66f-db2ee469681b","Type":"ContainerDied","Data":"5ae2f0bd97c7fa36dd1394f2c3e693a556207eb54c559a8532aaa9cef41fa010"}
Jul 28 16:54:20.220269 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:54:20.220028    2557 scope.go:117] "RemoveContainer" containerID="e1c696c1c0d5745db4553bdbcbab454ddde9c56b8fbac6b17104db5f70c6c705"
Jul 28 16:54:20.220483 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:54:20.220377    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-b9a8c-predictor-7dd8d57945-lrxn7" podUID="7e6417d3-5731-474e-8805-7c66df1f2ed0" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.39:8080: connect: connection refused"
Jul 28 16:54:20.228828 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:54:20.228809    2557 scope.go:117] "RemoveContainer" containerID="ca6e4286ea4f6a04073994afc26a89ebd88dcc3661a6fe6b89eb6553c2e464c7"
Jul 28 16:54:20.236852 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:54:20.236837    2557 scope.go:117] "RemoveContainer" containerID="e1c696c1c0d5745db4553bdbcbab454ddde9c56b8fbac6b17104db5f70c6c705"
Jul 28 16:54:20.237102 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:54:20.237085    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e1c696c1c0d5745db4553bdbcbab454ddde9c56b8fbac6b17104db5f70c6c705\": container with ID starting with e1c696c1c0d5745db4553bdbcbab454ddde9c56b8fbac6b17104db5f70c6c705 not found: ID does not exist" containerID="e1c696c1c0d5745db4553bdbcbab454ddde9c56b8fbac6b17104db5f70c6c705"
Jul 28 16:54:20.237152 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:54:20.237111    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e1c696c1c0d5745db4553bdbcbab454ddde9c56b8fbac6b17104db5f70c6c705"} err="failed to get container status \"e1c696c1c0d5745db4553bdbcbab454ddde9c56b8fbac6b17104db5f70c6c705\": rpc error: code = NotFound desc = could not find container \"e1c696c1c0d5745db4553bdbcbab454ddde9c56b8fbac6b17104db5f70c6c705\": container with ID starting with e1c696c1c0d5745db4553bdbcbab454ddde9c56b8fbac6b17104db5f70c6c705 not found: ID does not exist"
Jul 28 16:54:20.237152 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:54:20.237127    2557 scope.go:117] "RemoveContainer" containerID="ca6e4286ea4f6a04073994afc26a89ebd88dcc3661a6fe6b89eb6553c2e464c7"
Jul 28 16:54:20.237362 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:54:20.237345    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ca6e4286ea4f6a04073994afc26a89ebd88dcc3661a6fe6b89eb6553c2e464c7\": container with ID starting with ca6e4286ea4f6a04073994afc26a89ebd88dcc3661a6fe6b89eb6553c2e464c7 not found: ID does not exist" containerID="ca6e4286ea4f6a04073994afc26a89ebd88dcc3661a6fe6b89eb6553c2e464c7"
Jul 28 16:54:20.237414 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:54:20.237371    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ca6e4286ea4f6a04073994afc26a89ebd88dcc3661a6fe6b89eb6553c2e464c7"} err="failed to get container status \"ca6e4286ea4f6a04073994afc26a89ebd88dcc3661a6fe6b89eb6553c2e464c7\": rpc error: code = NotFound desc = could not find container \"ca6e4286ea4f6a04073994afc26a89ebd88dcc3661a6fe6b89eb6553c2e464c7\": container with ID starting with ca6e4286ea4f6a04073994afc26a89ebd88dcc3661a6fe6b89eb6553c2e464c7 not found: ID does not exist"
Jul 28 16:54:20.241345 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:54:20.241323    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-12f5e-predictor-597f7c7d45-5tthj"]
Jul 28 16:54:20.245149 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:54:20.245130    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-12f5e-predictor-597f7c7d45-5tthj"]
Jul 28 16:54:20.294750 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:54:20.294726    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rx7dl\" (UniqueName: \"kubernetes.io/projected/dc1a122b-f788-4e7c-b66f-db2ee469681b-kube-api-access-rx7dl\") on node \"ip-10-0-140-154.ec2.internal\" DevicePath \"\""
Jul 28 16:54:20.294750 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:54:20.294750    2557 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/dc1a122b-f788-4e7c-b66f-db2ee469681b-kube-rbac-proxy-sar-config\") on node \"ip-10-0-140-154.ec2.internal\" DevicePath \"\""
Jul 28 16:54:20.294871 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:54:20.294761    2557 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/dc1a122b-f788-4e7c-b66f-db2ee469681b-proxy-tls\") on node \"ip-10-0-140-154.ec2.internal\" DevicePath \"\""
Jul 28 16:54:21.435462 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:54:21.435430    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dc1a122b-f788-4e7c-b66f-db2ee469681b" path="/var/lib/kubelet/pods/dc1a122b-f788-4e7c-b66f-db2ee469681b/volumes"
Jul 28 16:54:25.224754 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:54:25.224721    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-b9a8c-predictor-7dd8d57945-lrxn7"
Jul 28 16:54:25.225297 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:54:25.225271    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-b9a8c-predictor-7dd8d57945-lrxn7" podUID="7e6417d3-5731-474e-8805-7c66df1f2ed0" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.39:8080: connect: connection refused"
Jul 28 16:54:27.151070 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:54:27.150950    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:54:27.239634 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:54:27.239594    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-2717b-predictor-5c68f8885c-sjxmd"]
Jul 28 16:54:27.239891 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:54:27.239864    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-2717b-predictor-5c68f8885c-sjxmd" podUID="7ec846e8-0d27-4b69-bfe0-538d5f6325de" containerName="kserve-container" containerID="cri-o://8fdb665beaa57bf415f188b36b70519e772c36bb785186030664718be1d44373" gracePeriod=30
Jul 28 16:54:27.239989 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:54:27.239901    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-2717b-predictor-5c68f8885c-sjxmd" podUID="7ec846e8-0d27-4b69-bfe0-538d5f6325de" containerName="kube-rbac-proxy" containerID="cri-o://506e70955dabce39dce6e3040d82cbea577c661fb4aecbd14ba2b8b2d2540a19" gracePeriod=30
Jul 28 16:54:27.274891 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:54:27.274857    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:54:27.299866 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:54:27.299839    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-b6e34-predictor-64bd5d8459-mtdzz"]
Jul 28 16:54:27.300238 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:54:27.300224    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dc1a122b-f788-4e7c-b66f-db2ee469681b" containerName="kserve-container"
Jul 28 16:54:27.300315 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:54:27.300239    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="dc1a122b-f788-4e7c-b66f-db2ee469681b" containerName="kserve-container"
Jul 28 16:54:27.300315 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:54:27.300246    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dc1a122b-f788-4e7c-b66f-db2ee469681b" containerName="kube-rbac-proxy"
Jul 28 16:54:27.300315 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:54:27.300252    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="dc1a122b-f788-4e7c-b66f-db2ee469681b" containerName="kube-rbac-proxy"
Jul 28 16:54:27.300473 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:54:27.300333    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="dc1a122b-f788-4e7c-b66f-db2ee469681b" containerName="kserve-container"
Jul 28 16:54:27.300473 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:54:27.300341    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="dc1a122b-f788-4e7c-b66f-db2ee469681b" containerName="kube-rbac-proxy"
Jul 28 16:54:27.303184 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:54:27.303168    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-b6e34-predictor-64bd5d8459-mtdzz"
Jul 28 16:54:27.305409 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:54:27.305391    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-b6e34-kube-rbac-proxy-sar-config\""
Jul 28 16:54:27.305514 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:54:27.305391    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-b6e34-predictor-serving-cert\""
Jul 28 16:54:27.313060 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:54:27.313036    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-b6e34-predictor-64bd5d8459-mtdzz"]
Jul 28 16:54:27.355332 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:54:27.355296    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/309f7841-d173-466f-a580-50dda84e1758-proxy-tls\") pod \"success-200-isvc-b6e34-predictor-64bd5d8459-mtdzz\" (UID: \"309f7841-d173-466f-a580-50dda84e1758\") " pod="kserve-ci-e2e-test/success-200-isvc-b6e34-predictor-64bd5d8459-mtdzz"
Jul 28 16:54:27.355473 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:54:27.355338    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/309f7841-d173-466f-a580-50dda84e1758-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-b6e34-predictor-64bd5d8459-mtdzz\" (UID: \"309f7841-d173-466f-a580-50dda84e1758\") " pod="kserve-ci-e2e-test/success-200-isvc-b6e34-predictor-64bd5d8459-mtdzz"
Jul 28 16:54:27.355473 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:54:27.355391    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kf6mx\" (UniqueName: \"kubernetes.io/projected/309f7841-d173-466f-a580-50dda84e1758-kube-api-access-kf6mx\") pod \"success-200-isvc-b6e34-predictor-64bd5d8459-mtdzz\" (UID: \"309f7841-d173-466f-a580-50dda84e1758\") " pod="kserve-ci-e2e-test/success-200-isvc-b6e34-predictor-64bd5d8459-mtdzz"
Jul 28 16:54:27.369272 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:54:27.369242    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:54:27.456887 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:54:27.456799    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/309f7841-d173-466f-a580-50dda84e1758-proxy-tls\") pod \"success-200-isvc-b6e34-predictor-64bd5d8459-mtdzz\" (UID: \"309f7841-d173-466f-a580-50dda84e1758\") " pod="kserve-ci-e2e-test/success-200-isvc-b6e34-predictor-64bd5d8459-mtdzz"
Jul 28 16:54:27.456887 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:54:27.456841    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/309f7841-d173-466f-a580-50dda84e1758-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-b6e34-predictor-64bd5d8459-mtdzz\" (UID: \"309f7841-d173-466f-a580-50dda84e1758\") " pod="kserve-ci-e2e-test/success-200-isvc-b6e34-predictor-64bd5d8459-mtdzz"
Jul 28 16:54:27.456887 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:54:27.456879    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kf6mx\" (UniqueName: \"kubernetes.io/projected/309f7841-d173-466f-a580-50dda84e1758-kube-api-access-kf6mx\") pod \"success-200-isvc-b6e34-predictor-64bd5d8459-mtdzz\" (UID: \"309f7841-d173-466f-a580-50dda84e1758\") " pod="kserve-ci-e2e-test/success-200-isvc-b6e34-predictor-64bd5d8459-mtdzz"
Jul 28 16:54:27.457145 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:54:27.456966    2557 secret.go:189] Couldn't get secret kserve-ci-e2e-test/success-200-isvc-b6e34-predictor-serving-cert: secret "success-200-isvc-b6e34-predictor-serving-cert" not found
Jul 28 16:54:27.457145 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:54:27.457050    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/309f7841-d173-466f-a580-50dda84e1758-proxy-tls podName:309f7841-d173-466f-a580-50dda84e1758 nodeName:}" failed. No retries permitted until 2026-07-28 16:54:27.957029128 +0000 UTC m=+1985.095498525 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/309f7841-d173-466f-a580-50dda84e1758-proxy-tls") pod "success-200-isvc-b6e34-predictor-64bd5d8459-mtdzz" (UID: "309f7841-d173-466f-a580-50dda84e1758") : secret "success-200-isvc-b6e34-predictor-serving-cert" not found
Jul 28 16:54:27.457567 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:54:27.457550    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/309f7841-d173-466f-a580-50dda84e1758-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-b6e34-predictor-64bd5d8459-mtdzz\" (UID: \"309f7841-d173-466f-a580-50dda84e1758\") " pod="kserve-ci-e2e-test/success-200-isvc-b6e34-predictor-64bd5d8459-mtdzz"
Jul 28 16:54:27.472918 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:54:27.472890    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kf6mx\" (UniqueName: \"kubernetes.io/projected/309f7841-d173-466f-a580-50dda84e1758-kube-api-access-kf6mx\") pod \"success-200-isvc-b6e34-predictor-64bd5d8459-mtdzz\" (UID: \"309f7841-d173-466f-a580-50dda84e1758\") " pod="kserve-ci-e2e-test/success-200-isvc-b6e34-predictor-64bd5d8459-mtdzz"
Jul 28 16:54:27.488063 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:54:27.488037    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:54:27.574207 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:54:27.574174    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:54:27.673167 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:54:27.673138    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:54:27.961098 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:54:27.961064    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/309f7841-d173-466f-a580-50dda84e1758-proxy-tls\") pod \"success-200-isvc-b6e34-predictor-64bd5d8459-mtdzz\" (UID: \"309f7841-d173-466f-a580-50dda84e1758\") " pod="kserve-ci-e2e-test/success-200-isvc-b6e34-predictor-64bd5d8459-mtdzz"
Jul 28 16:54:27.963713 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:54:27.963691    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/309f7841-d173-466f-a580-50dda84e1758-proxy-tls\") pod \"success-200-isvc-b6e34-predictor-64bd5d8459-mtdzz\" (UID: \"309f7841-d173-466f-a580-50dda84e1758\") " pod="kserve-ci-e2e-test/success-200-isvc-b6e34-predictor-64bd5d8459-mtdzz"
Jul 28 16:54:28.214639 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:54:28.214560    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-b6e34-predictor-64bd5d8459-mtdzz"
Jul 28 16:54:28.260553 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:54:28.260520    2557 generic.go:356] "Generic (PLEG): container finished" podID="7ec846e8-0d27-4b69-bfe0-538d5f6325de" containerID="506e70955dabce39dce6e3040d82cbea577c661fb4aecbd14ba2b8b2d2540a19" exitCode=2
Jul 28 16:54:28.260721 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:54:28.260528    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-2717b-predictor-5c68f8885c-sjxmd" event={"ID":"7ec846e8-0d27-4b69-bfe0-538d5f6325de","Type":"ContainerDied","Data":"506e70955dabce39dce6e3040d82cbea577c661fb4aecbd14ba2b8b2d2540a19"}
Jul 28 16:54:28.352422 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:54:28.352398    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-b6e34-predictor-64bd5d8459-mtdzz"]
Jul 28 16:54:28.354530 ip-10-0-140-154 kubenswrapper[2557]: W0728 16:54:28.354482    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod309f7841_d173_466f_a580_50dda84e1758.slice/crio-f5b568eadcd119ef725db5b376d475110e08568dec177e3065f4076cfbdc5cf8 WatchSource:0}: Error finding container f5b568eadcd119ef725db5b376d475110e08568dec177e3065f4076cfbdc5cf8: Status 404 returned error can't find the container with id f5b568eadcd119ef725db5b376d475110e08568dec177e3065f4076cfbdc5cf8
Jul 28 16:54:29.267109 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:54:29.267071    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-b6e34-predictor-64bd5d8459-mtdzz" event={"ID":"309f7841-d173-466f-a580-50dda84e1758","Type":"ContainerStarted","Data":"49677425c4d607c31c96b6bde65471d0bb599ff08167d39f6ae42a5388cef2c4"}
Jul 28 16:54:29.267630 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:54:29.267137    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-b6e34-predictor-64bd5d8459-mtdzz" event={"ID":"309f7841-d173-466f-a580-50dda84e1758","Type":"ContainerStarted","Data":"de2ee333caa7e0726c6d82a4c6a856bfd05cc765c3285df5f13c0acb8ed7e06e"}
Jul 28 16:54:29.267630 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:54:29.267174    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-b6e34-predictor-64bd5d8459-mtdzz"
Jul 28 16:54:29.267630 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:54:29.267203    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-b6e34-predictor-64bd5d8459-mtdzz" event={"ID":"309f7841-d173-466f-a580-50dda84e1758","Type":"ContainerStarted","Data":"f5b568eadcd119ef725db5b376d475110e08568dec177e3065f4076cfbdc5cf8"}
Jul 28 16:54:29.284821 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:54:29.284776    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-b6e34-predictor-64bd5d8459-mtdzz" podStartSLOduration=2.284757738 podStartE2EDuration="2.284757738s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 16:54:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:54:29.283489098 +0000 UTC m=+1986.421958522" watchObservedRunningTime="2026-07-28 16:54:29.284757738 +0000 UTC m=+1986.423227154"
Jul 28 16:54:29.940322 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:54:29.940281    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-2717b-predictor-5c68f8885c-sjxmd" podUID="7ec846e8-0d27-4b69-bfe0-538d5f6325de" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.38:8643/healthz\": dial tcp 10.133.0.38:8643: connect: connection refused"
Jul 28 16:54:30.271395 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:54:30.271313    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-b6e34-predictor-64bd5d8459-mtdzz"
Jul 28 16:54:30.272886 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:54:30.272857    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-b6e34-predictor-64bd5d8459-mtdzz" podUID="309f7841-d173-466f-a580-50dda84e1758" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.40:8080: connect: connection refused"
Jul 28 16:54:30.489845 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:54:30.489823    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-2717b-predictor-5c68f8885c-sjxmd"
Jul 28 16:54:30.583656 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:54:30.583625    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vp979\" (UniqueName: \"kubernetes.io/projected/7ec846e8-0d27-4b69-bfe0-538d5f6325de-kube-api-access-vp979\") pod \"7ec846e8-0d27-4b69-bfe0-538d5f6325de\" (UID: \"7ec846e8-0d27-4b69-bfe0-538d5f6325de\") "
Jul 28 16:54:30.583827 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:54:30.583702    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/7ec846e8-0d27-4b69-bfe0-538d5f6325de-kube-rbac-proxy-sar-config\") pod \"7ec846e8-0d27-4b69-bfe0-538d5f6325de\" (UID: \"7ec846e8-0d27-4b69-bfe0-538d5f6325de\") "
Jul 28 16:54:30.583913 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:54:30.583868    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7ec846e8-0d27-4b69-bfe0-538d5f6325de-proxy-tls\") pod \"7ec846e8-0d27-4b69-bfe0-538d5f6325de\" (UID: \"7ec846e8-0d27-4b69-bfe0-538d5f6325de\") "
Jul 28 16:54:30.584043 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:54:30.584021    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7ec846e8-0d27-4b69-bfe0-538d5f6325de-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "7ec846e8-0d27-4b69-bfe0-538d5f6325de" (UID: "7ec846e8-0d27-4b69-bfe0-538d5f6325de"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 16:54:30.584163 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:54:30.584141    2557 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/7ec846e8-0d27-4b69-bfe0-538d5f6325de-kube-rbac-proxy-sar-config\") on node \"ip-10-0-140-154.ec2.internal\" DevicePath \"\""
Jul 28 16:54:30.585877 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:54:30.585851    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7ec846e8-0d27-4b69-bfe0-538d5f6325de-kube-api-access-vp979" (OuterVolumeSpecName: "kube-api-access-vp979") pod "7ec846e8-0d27-4b69-bfe0-538d5f6325de" (UID: "7ec846e8-0d27-4b69-bfe0-538d5f6325de"). InnerVolumeSpecName "kube-api-access-vp979". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 16:54:30.585988 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:54:30.585936    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7ec846e8-0d27-4b69-bfe0-538d5f6325de-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "7ec846e8-0d27-4b69-bfe0-538d5f6325de" (UID: "7ec846e8-0d27-4b69-bfe0-538d5f6325de"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 16:54:30.685173 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:54:30.685135    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vp979\" (UniqueName: \"kubernetes.io/projected/7ec846e8-0d27-4b69-bfe0-538d5f6325de-kube-api-access-vp979\") on node \"ip-10-0-140-154.ec2.internal\" DevicePath \"\""
Jul 28 16:54:30.685173 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:54:30.685165    2557 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7ec846e8-0d27-4b69-bfe0-538d5f6325de-proxy-tls\") on node \"ip-10-0-140-154.ec2.internal\" DevicePath \"\""
Jul 28 16:54:31.277262 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:54:31.277231    2557 generic.go:356] "Generic (PLEG): container finished" podID="7ec846e8-0d27-4b69-bfe0-538d5f6325de" containerID="8fdb665beaa57bf415f188b36b70519e772c36bb785186030664718be1d44373" exitCode=0
Jul 28 16:54:31.277731 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:54:31.277290    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-2717b-predictor-5c68f8885c-sjxmd"
Jul 28 16:54:31.277731 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:54:31.277319    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-2717b-predictor-5c68f8885c-sjxmd" event={"ID":"7ec846e8-0d27-4b69-bfe0-538d5f6325de","Type":"ContainerDied","Data":"8fdb665beaa57bf415f188b36b70519e772c36bb785186030664718be1d44373"}
Jul 28 16:54:31.277731 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:54:31.277387    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-2717b-predictor-5c68f8885c-sjxmd" event={"ID":"7ec846e8-0d27-4b69-bfe0-538d5f6325de","Type":"ContainerDied","Data":"cb94278e589dbb628a664401761254c4ba48f987bb126e48f0086c150b28d1da"}
Jul 28 16:54:31.277731 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:54:31.277424    2557 scope.go:117] "RemoveContainer" containerID="506e70955dabce39dce6e3040d82cbea577c661fb4aecbd14ba2b8b2d2540a19"
Jul 28 16:54:31.278051 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:54:31.278019    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-b6e34-predictor-64bd5d8459-mtdzz" podUID="309f7841-d173-466f-a580-50dda84e1758" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.40:8080: connect: connection refused"
Jul 28 16:54:31.287303 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:54:31.287286    2557 scope.go:117] "RemoveContainer" containerID="8fdb665beaa57bf415f188b36b70519e772c36bb785186030664718be1d44373"
Jul 28 16:54:31.294851 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:54:31.294833    2557 scope.go:117] "RemoveContainer" containerID="506e70955dabce39dce6e3040d82cbea577c661fb4aecbd14ba2b8b2d2540a19"
Jul 28 16:54:31.295089 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:54:31.295069    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"506e70955dabce39dce6e3040d82cbea577c661fb4aecbd14ba2b8b2d2540a19\": container with ID starting with 506e70955dabce39dce6e3040d82cbea577c661fb4aecbd14ba2b8b2d2540a19 not found: ID does not exist" containerID="506e70955dabce39dce6e3040d82cbea577c661fb4aecbd14ba2b8b2d2540a19"
Jul 28 16:54:31.295149 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:54:31.295098    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"506e70955dabce39dce6e3040d82cbea577c661fb4aecbd14ba2b8b2d2540a19"} err="failed to get container status \"506e70955dabce39dce6e3040d82cbea577c661fb4aecbd14ba2b8b2d2540a19\": rpc error: code = NotFound desc = could not find container \"506e70955dabce39dce6e3040d82cbea577c661fb4aecbd14ba2b8b2d2540a19\": container with ID starting with 506e70955dabce39dce6e3040d82cbea577c661fb4aecbd14ba2b8b2d2540a19 not found: ID does not exist"
Jul 28 16:54:31.295149 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:54:31.295118    2557 scope.go:117] "RemoveContainer" containerID="8fdb665beaa57bf415f188b36b70519e772c36bb785186030664718be1d44373"
Jul 28 16:54:31.295400 ip-10-0-140-154 kubenswrapper[2557]: E0728 16:54:31.295382    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8fdb665beaa57bf415f188b36b70519e772c36bb785186030664718be1d44373\": container with ID starting with 8fdb665beaa57bf415f188b36b70519e772c36bb785186030664718be1d44373 not found: ID does not exist" containerID="8fdb665beaa57bf415f188b36b70519e772c36bb785186030664718be1d44373"
Jul 28 16:54:31.295441 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:54:31.295408    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8fdb665beaa57bf415f188b36b70519e772c36bb785186030664718be1d44373"} err="failed to get container status \"8fdb665beaa57bf415f188b36b70519e772c36bb785186030664718be1d44373\": rpc error: code = NotFound desc = could not find container \"8fdb665beaa57bf415f188b36b70519e772c36bb785186030664718be1d44373\": container with ID starting with 8fdb665beaa57bf415f188b36b70519e772c36bb785186030664718be1d44373 not found: ID does not exist"
Jul 28 16:54:31.298903 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:54:31.298876    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-2717b-predictor-5c68f8885c-sjxmd"]
Jul 28 16:54:31.302653 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:54:31.302634    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-2717b-predictor-5c68f8885c-sjxmd"]
Jul 28 16:54:31.434188 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:54:31.434154    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7ec846e8-0d27-4b69-bfe0-538d5f6325de" path="/var/lib/kubelet/pods/7ec846e8-0d27-4b69-bfe0-538d5f6325de/volumes"
Jul 28 16:54:35.225281 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:54:35.225240    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-b9a8c-predictor-7dd8d57945-lrxn7" podUID="7e6417d3-5731-474e-8805-7c66df1f2ed0" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.39:8080: connect: connection refused"
Jul 28 16:54:36.281790 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:54:36.281760    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-b6e34-predictor-64bd5d8459-mtdzz"
Jul 28 16:54:36.282251 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:54:36.282222    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-b6e34-predictor-64bd5d8459-mtdzz" podUID="309f7841-d173-466f-a580-50dda84e1758" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.40:8080: connect: connection refused"
Jul 28 16:54:45.225474 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:54:45.225431    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-b9a8c-predictor-7dd8d57945-lrxn7" podUID="7e6417d3-5731-474e-8805-7c66df1f2ed0" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.39:8080: connect: connection refused"
Jul 28 16:54:46.282865 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:54:46.282826    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-b6e34-predictor-64bd5d8459-mtdzz" podUID="309f7841-d173-466f-a580-50dda84e1758" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.40:8080: connect: connection refused"
Jul 28 16:54:55.225267 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:54:55.225215    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-b9a8c-predictor-7dd8d57945-lrxn7" podUID="7e6417d3-5731-474e-8805-7c66df1f2ed0" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.39:8080: connect: connection refused"
Jul 28 16:54:56.282390 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:54:56.282353    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-b6e34-predictor-64bd5d8459-mtdzz" podUID="309f7841-d173-466f-a580-50dda84e1758" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.40:8080: connect: connection refused"
Jul 28 16:55:05.226176 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:55:05.226142    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-b9a8c-predictor-7dd8d57945-lrxn7"
Jul 28 16:55:06.282469 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:55:06.282423    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-b6e34-predictor-64bd5d8459-mtdzz" podUID="309f7841-d173-466f-a580-50dda84e1758" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.40:8080: connect: connection refused"
Jul 28 16:55:16.283483 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:55:16.283455    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-b6e34-predictor-64bd5d8459-mtdzz"
Jul 28 16:55:16.925946 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:55:16.925905    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:55:17.060928 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:55:17.060891    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:55:27.373703 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:55:27.373666    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:55:28.131702 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:55:28.131666    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:57:22.480954 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:57:22.480875    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 16:59:52.253000 ip-10-0-140-154 kubenswrapper[2557]: I0728 16:59:52.252964    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 17:01:42.856134 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:01:42.856102    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 17:03:18.662619 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:03:18.662525    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 17:03:29.669943 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:03:29.669910    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 17:03:41.991917 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:03:41.991882    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 17:03:42.084218 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:03:42.084064    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-b6e34-predictor-64bd5d8459-mtdzz"]
Jul 28 17:03:42.084399 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:03:42.084355    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-b6e34-predictor-64bd5d8459-mtdzz" podUID="309f7841-d173-466f-a580-50dda84e1758" containerName="kserve-container" containerID="cri-o://de2ee333caa7e0726c6d82a4c6a856bfd05cc765c3285df5f13c0acb8ed7e06e" gracePeriod=30
Jul 28 17:03:42.084704 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:03:42.084673    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-b6e34-predictor-64bd5d8459-mtdzz" podUID="309f7841-d173-466f-a580-50dda84e1758" containerName="kube-rbac-proxy" containerID="cri-o://49677425c4d607c31c96b6bde65471d0bb599ff08167d39f6ae42a5388cef2c4" gracePeriod=30
Jul 28 17:03:42.105317 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:03:42.105288    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 17:03:42.156687 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:03:42.156664    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 17:03:42.734904 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:03:42.734867    2557 generic.go:356] "Generic (PLEG): container finished" podID="309f7841-d173-466f-a580-50dda84e1758" containerID="49677425c4d607c31c96b6bde65471d0bb599ff08167d39f6ae42a5388cef2c4" exitCode=2
Jul 28 17:03:42.735085 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:03:42.734940    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-b6e34-predictor-64bd5d8459-mtdzz" event={"ID":"309f7841-d173-466f-a580-50dda84e1758","Type":"ContainerDied","Data":"49677425c4d607c31c96b6bde65471d0bb599ff08167d39f6ae42a5388cef2c4"}
Jul 28 17:03:45.329624 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:03:45.329599    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-b6e34-predictor-64bd5d8459-mtdzz"
Jul 28 17:03:45.372828 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:03:45.372760    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/309f7841-d173-466f-a580-50dda84e1758-proxy-tls\") pod \"309f7841-d173-466f-a580-50dda84e1758\" (UID: \"309f7841-d173-466f-a580-50dda84e1758\") "
Jul 28 17:03:45.372828 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:03:45.372794    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/309f7841-d173-466f-a580-50dda84e1758-kube-rbac-proxy-sar-config\") pod \"309f7841-d173-466f-a580-50dda84e1758\" (UID: \"309f7841-d173-466f-a580-50dda84e1758\") "
Jul 28 17:03:45.372828 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:03:45.372813    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kf6mx\" (UniqueName: \"kubernetes.io/projected/309f7841-d173-466f-a580-50dda84e1758-kube-api-access-kf6mx\") pod \"309f7841-d173-466f-a580-50dda84e1758\" (UID: \"309f7841-d173-466f-a580-50dda84e1758\") "
Jul 28 17:03:45.373190 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:03:45.373160    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/309f7841-d173-466f-a580-50dda84e1758-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "309f7841-d173-466f-a580-50dda84e1758" (UID: "309f7841-d173-466f-a580-50dda84e1758"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 17:03:45.375060 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:03:45.375032    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/309f7841-d173-466f-a580-50dda84e1758-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "309f7841-d173-466f-a580-50dda84e1758" (UID: "309f7841-d173-466f-a580-50dda84e1758"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 17:03:45.375146 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:03:45.375071    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/309f7841-d173-466f-a580-50dda84e1758-kube-api-access-kf6mx" (OuterVolumeSpecName: "kube-api-access-kf6mx") pod "309f7841-d173-466f-a580-50dda84e1758" (UID: "309f7841-d173-466f-a580-50dda84e1758"). InnerVolumeSpecName "kube-api-access-kf6mx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 17:03:45.474217 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:03:45.474189    2557 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/309f7841-d173-466f-a580-50dda84e1758-proxy-tls\") on node \"ip-10-0-140-154.ec2.internal\" DevicePath \"\""
Jul 28 17:03:45.474217 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:03:45.474214    2557 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/309f7841-d173-466f-a580-50dda84e1758-kube-rbac-proxy-sar-config\") on node \"ip-10-0-140-154.ec2.internal\" DevicePath \"\""
Jul 28 17:03:45.474393 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:03:45.474223    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kf6mx\" (UniqueName: \"kubernetes.io/projected/309f7841-d173-466f-a580-50dda84e1758-kube-api-access-kf6mx\") on node \"ip-10-0-140-154.ec2.internal\" DevicePath \"\""
Jul 28 17:03:45.750126 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:03:45.750039    2557 generic.go:356] "Generic (PLEG): container finished" podID="309f7841-d173-466f-a580-50dda84e1758" containerID="de2ee333caa7e0726c6d82a4c6a856bfd05cc765c3285df5f13c0acb8ed7e06e" exitCode=0
Jul 28 17:03:45.750126 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:03:45.750091    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-b6e34-predictor-64bd5d8459-mtdzz"
Jul 28 17:03:45.750300 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:03:45.750128    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-b6e34-predictor-64bd5d8459-mtdzz" event={"ID":"309f7841-d173-466f-a580-50dda84e1758","Type":"ContainerDied","Data":"de2ee333caa7e0726c6d82a4c6a856bfd05cc765c3285df5f13c0acb8ed7e06e"}
Jul 28 17:03:45.750300 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:03:45.750191    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-b6e34-predictor-64bd5d8459-mtdzz" event={"ID":"309f7841-d173-466f-a580-50dda84e1758","Type":"ContainerDied","Data":"f5b568eadcd119ef725db5b376d475110e08568dec177e3065f4076cfbdc5cf8"}
Jul 28 17:03:45.750300 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:03:45.750217    2557 scope.go:117] "RemoveContainer" containerID="49677425c4d607c31c96b6bde65471d0bb599ff08167d39f6ae42a5388cef2c4"
Jul 28 17:03:45.758739 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:03:45.758724    2557 scope.go:117] "RemoveContainer" containerID="de2ee333caa7e0726c6d82a4c6a856bfd05cc765c3285df5f13c0acb8ed7e06e"
Jul 28 17:03:45.766125 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:03:45.766110    2557 scope.go:117] "RemoveContainer" containerID="49677425c4d607c31c96b6bde65471d0bb599ff08167d39f6ae42a5388cef2c4"
Jul 28 17:03:45.766388 ip-10-0-140-154 kubenswrapper[2557]: E0728 17:03:45.766369    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"49677425c4d607c31c96b6bde65471d0bb599ff08167d39f6ae42a5388cef2c4\": container with ID starting with 49677425c4d607c31c96b6bde65471d0bb599ff08167d39f6ae42a5388cef2c4 not found: ID does not exist" containerID="49677425c4d607c31c96b6bde65471d0bb599ff08167d39f6ae42a5388cef2c4"
Jul 28 17:03:45.766458 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:03:45.766396    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"49677425c4d607c31c96b6bde65471d0bb599ff08167d39f6ae42a5388cef2c4"} err="failed to get container status \"49677425c4d607c31c96b6bde65471d0bb599ff08167d39f6ae42a5388cef2c4\": rpc error: code = NotFound desc = could not find container \"49677425c4d607c31c96b6bde65471d0bb599ff08167d39f6ae42a5388cef2c4\": container with ID starting with 49677425c4d607c31c96b6bde65471d0bb599ff08167d39f6ae42a5388cef2c4 not found: ID does not exist"
Jul 28 17:03:45.766458 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:03:45.766410    2557 scope.go:117] "RemoveContainer" containerID="de2ee333caa7e0726c6d82a4c6a856bfd05cc765c3285df5f13c0acb8ed7e06e"
Jul 28 17:03:45.766673 ip-10-0-140-154 kubenswrapper[2557]: E0728 17:03:45.766655    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"de2ee333caa7e0726c6d82a4c6a856bfd05cc765c3285df5f13c0acb8ed7e06e\": container with ID starting with de2ee333caa7e0726c6d82a4c6a856bfd05cc765c3285df5f13c0acb8ed7e06e not found: ID does not exist" containerID="de2ee333caa7e0726c6d82a4c6a856bfd05cc765c3285df5f13c0acb8ed7e06e"
Jul 28 17:03:45.766716 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:03:45.766689    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"de2ee333caa7e0726c6d82a4c6a856bfd05cc765c3285df5f13c0acb8ed7e06e"} err="failed to get container status \"de2ee333caa7e0726c6d82a4c6a856bfd05cc765c3285df5f13c0acb8ed7e06e\": rpc error: code = NotFound desc = could not find container \"de2ee333caa7e0726c6d82a4c6a856bfd05cc765c3285df5f13c0acb8ed7e06e\": container with ID starting with de2ee333caa7e0726c6d82a4c6a856bfd05cc765c3285df5f13c0acb8ed7e06e not found: ID does not exist"
Jul 28 17:03:45.769711 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:03:45.769692    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-b6e34-predictor-64bd5d8459-mtdzz"]
Jul 28 17:03:45.779731 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:03:45.779707    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-b6e34-predictor-64bd5d8459-mtdzz"]
Jul 28 17:03:47.433284 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:03:47.433249    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="309f7841-d173-466f-a580-50dda84e1758" path="/var/lib/kubelet/pods/309f7841-d173-466f-a580-50dda84e1758/volumes"
Jul 28 17:04:48.677603 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:04:48.677514    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 17:09:10.016873 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:09:10.016839    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 17:11:36.235101 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:11:36.235067    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 17:11:36.334716 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:11:36.334685    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-b9a8c-predictor-7dd8d57945-lrxn7"]
Jul 28 17:11:36.335035 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:11:36.335002    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-b9a8c-predictor-7dd8d57945-lrxn7" podUID="7e6417d3-5731-474e-8805-7c66df1f2ed0" containerName="kserve-container" containerID="cri-o://57683799e6a8fc66bb3ff5b4b83d9103515cce50d359e8faf9127c19ac455caa" gracePeriod=30
Jul 28 17:11:36.335156 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:11:36.335029    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-b9a8c-predictor-7dd8d57945-lrxn7" podUID="7e6417d3-5731-474e-8805-7c66df1f2ed0" containerName="kube-rbac-proxy" containerID="cri-o://84841d2d03667c4ea6e39e8f5520ca76a3390db8f0c6e296a5d38bab53582639" gracePeriod=30
Jul 28 17:11:36.354774 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:11:36.354752    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 17:11:36.418302 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:11:36.418268    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 17:11:36.844296 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:11:36.844264    2557 generic.go:356] "Generic (PLEG): container finished" podID="7e6417d3-5731-474e-8805-7c66df1f2ed0" containerID="84841d2d03667c4ea6e39e8f5520ca76a3390db8f0c6e296a5d38bab53582639" exitCode=2
Jul 28 17:11:36.844465 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:11:36.844312    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-b9a8c-predictor-7dd8d57945-lrxn7" event={"ID":"7e6417d3-5731-474e-8805-7c66df1f2ed0","Type":"ContainerDied","Data":"84841d2d03667c4ea6e39e8f5520ca76a3390db8f0c6e296a5d38bab53582639"}
Jul 28 17:11:37.535085 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:11:37.535051    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 17:11:37.627350 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:11:37.627315    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 17:11:39.377056 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:11:39.377027    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-b9a8c-predictor-7dd8d57945-lrxn7"
Jul 28 17:11:39.473744 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:11:39.473712    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7e6417d3-5731-474e-8805-7c66df1f2ed0-proxy-tls\") pod \"7e6417d3-5731-474e-8805-7c66df1f2ed0\" (UID: \"7e6417d3-5731-474e-8805-7c66df1f2ed0\") "
Jul 28 17:11:39.473744 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:11:39.473748    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vlvxz\" (UniqueName: \"kubernetes.io/projected/7e6417d3-5731-474e-8805-7c66df1f2ed0-kube-api-access-vlvxz\") pod \"7e6417d3-5731-474e-8805-7c66df1f2ed0\" (UID: \"7e6417d3-5731-474e-8805-7c66df1f2ed0\") "
Jul 28 17:11:39.473938 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:11:39.473788    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/7e6417d3-5731-474e-8805-7c66df1f2ed0-kube-rbac-proxy-sar-config\") pod \"7e6417d3-5731-474e-8805-7c66df1f2ed0\" (UID: \"7e6417d3-5731-474e-8805-7c66df1f2ed0\") "
Jul 28 17:11:39.474154 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:11:39.474126    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7e6417d3-5731-474e-8805-7c66df1f2ed0-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "7e6417d3-5731-474e-8805-7c66df1f2ed0" (UID: "7e6417d3-5731-474e-8805-7c66df1f2ed0"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 17:11:39.476115 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:11:39.476051    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7e6417d3-5731-474e-8805-7c66df1f2ed0-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "7e6417d3-5731-474e-8805-7c66df1f2ed0" (UID: "7e6417d3-5731-474e-8805-7c66df1f2ed0"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 17:11:39.476115 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:11:39.476067    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e6417d3-5731-474e-8805-7c66df1f2ed0-kube-api-access-vlvxz" (OuterVolumeSpecName: "kube-api-access-vlvxz") pod "7e6417d3-5731-474e-8805-7c66df1f2ed0" (UID: "7e6417d3-5731-474e-8805-7c66df1f2ed0"). InnerVolumeSpecName "kube-api-access-vlvxz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 17:11:39.574784 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:11:39.574753    2557 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/7e6417d3-5731-474e-8805-7c66df1f2ed0-kube-rbac-proxy-sar-config\") on node \"ip-10-0-140-154.ec2.internal\" DevicePath \"\""
Jul 28 17:11:39.574944 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:11:39.574803    2557 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7e6417d3-5731-474e-8805-7c66df1f2ed0-proxy-tls\") on node \"ip-10-0-140-154.ec2.internal\" DevicePath \"\""
Jul 28 17:11:39.574944 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:11:39.574813    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vlvxz\" (UniqueName: \"kubernetes.io/projected/7e6417d3-5731-474e-8805-7c66df1f2ed0-kube-api-access-vlvxz\") on node \"ip-10-0-140-154.ec2.internal\" DevicePath \"\""
Jul 28 17:11:39.858139 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:11:39.858107    2557 generic.go:356] "Generic (PLEG): container finished" podID="7e6417d3-5731-474e-8805-7c66df1f2ed0" containerID="57683799e6a8fc66bb3ff5b4b83d9103515cce50d359e8faf9127c19ac455caa" exitCode=0
Jul 28 17:11:39.858310 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:11:39.858157    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-b9a8c-predictor-7dd8d57945-lrxn7"
Jul 28 17:11:39.858310 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:11:39.858207    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-b9a8c-predictor-7dd8d57945-lrxn7" event={"ID":"7e6417d3-5731-474e-8805-7c66df1f2ed0","Type":"ContainerDied","Data":"57683799e6a8fc66bb3ff5b4b83d9103515cce50d359e8faf9127c19ac455caa"}
Jul 28 17:11:39.858310 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:11:39.858272    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-b9a8c-predictor-7dd8d57945-lrxn7" event={"ID":"7e6417d3-5731-474e-8805-7c66df1f2ed0","Type":"ContainerDied","Data":"f6886ec403883328be5070cfdad812407db8316cd0d3fbbf7420956d5e094b07"}
Jul 28 17:11:39.858310 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:11:39.858303    2557 scope.go:117] "RemoveContainer" containerID="84841d2d03667c4ea6e39e8f5520ca76a3390db8f0c6e296a5d38bab53582639"
Jul 28 17:11:39.867773 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:11:39.867756    2557 scope.go:117] "RemoveContainer" containerID="57683799e6a8fc66bb3ff5b4b83d9103515cce50d359e8faf9127c19ac455caa"
Jul 28 17:11:39.875258 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:11:39.875243    2557 scope.go:117] "RemoveContainer" containerID="84841d2d03667c4ea6e39e8f5520ca76a3390db8f0c6e296a5d38bab53582639"
Jul 28 17:11:39.875537 ip-10-0-140-154 kubenswrapper[2557]: E0728 17:11:39.875514    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"84841d2d03667c4ea6e39e8f5520ca76a3390db8f0c6e296a5d38bab53582639\": container with ID starting with 84841d2d03667c4ea6e39e8f5520ca76a3390db8f0c6e296a5d38bab53582639 not found: ID does not exist" containerID="84841d2d03667c4ea6e39e8f5520ca76a3390db8f0c6e296a5d38bab53582639"
Jul 28 17:11:39.875645 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:11:39.875541    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"84841d2d03667c4ea6e39e8f5520ca76a3390db8f0c6e296a5d38bab53582639"} err="failed to get container status \"84841d2d03667c4ea6e39e8f5520ca76a3390db8f0c6e296a5d38bab53582639\": rpc error: code = NotFound desc = could not find container \"84841d2d03667c4ea6e39e8f5520ca76a3390db8f0c6e296a5d38bab53582639\": container with ID starting with 84841d2d03667c4ea6e39e8f5520ca76a3390db8f0c6e296a5d38bab53582639 not found: ID does not exist"
Jul 28 17:11:39.875645 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:11:39.875557    2557 scope.go:117] "RemoveContainer" containerID="57683799e6a8fc66bb3ff5b4b83d9103515cce50d359e8faf9127c19ac455caa"
Jul 28 17:11:39.875819 ip-10-0-140-154 kubenswrapper[2557]: E0728 17:11:39.875802    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"57683799e6a8fc66bb3ff5b4b83d9103515cce50d359e8faf9127c19ac455caa\": container with ID starting with 57683799e6a8fc66bb3ff5b4b83d9103515cce50d359e8faf9127c19ac455caa not found: ID does not exist" containerID="57683799e6a8fc66bb3ff5b4b83d9103515cce50d359e8faf9127c19ac455caa"
Jul 28 17:11:39.875866 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:11:39.875826    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"57683799e6a8fc66bb3ff5b4b83d9103515cce50d359e8faf9127c19ac455caa"} err="failed to get container status \"57683799e6a8fc66bb3ff5b4b83d9103515cce50d359e8faf9127c19ac455caa\": rpc error: code = NotFound desc = could not find container \"57683799e6a8fc66bb3ff5b4b83d9103515cce50d359e8faf9127c19ac455caa\": container with ID starting with 57683799e6a8fc66bb3ff5b4b83d9103515cce50d359e8faf9127c19ac455caa not found: ID does not exist"
Jul 28 17:11:39.879239 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:11:39.879218    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-b9a8c-predictor-7dd8d57945-lrxn7"]
Jul 28 17:11:39.883197 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:11:39.883175    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-b9a8c-predictor-7dd8d57945-lrxn7"]
Jul 28 17:11:41.433414 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:11:41.433381    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7e6417d3-5731-474e-8805-7c66df1f2ed0" path="/var/lib/kubelet/pods/7e6417d3-5731-474e-8805-7c66df1f2ed0/volumes"
Jul 28 17:12:07.476970 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:12:07.476934    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 17:12:07.613152 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:12:07.613120    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 17:12:07.728808 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:12:07.728728    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 17:12:10.289730 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:12:10.289702    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-9vtwz_31992b1b-4643-467f-a2b1-674af56e8c0e/global-pull-secret-syncer/0.log"
Jul 28 17:12:10.397610 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:12:10.397576    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log"
Jul 28 17:12:10.518110 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:12:10.518075    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-140-154.ec2.internal_851ad20362c20c22501be689ae7a68aa/haproxy/0.log"
Jul 28 17:12:13.480832 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:12:13.480796    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 17:12:13.538158 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:12:13.538130    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 17:12:14.492252 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:12:14.492183    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-v8smk_04e4322a-db5d-4979-9106-21478fe43277/kube-state-metrics/0.log"
Jul 28 17:12:14.516849 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:12:14.516823    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-v8smk_04e4322a-db5d-4979-9106-21478fe43277/kube-rbac-proxy-main/0.log"
Jul 28 17:12:14.545219 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:12:14.545192    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-v8smk_04e4322a-db5d-4979-9106-21478fe43277/kube-rbac-proxy-self/0.log"
Jul 28 17:12:14.736588 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:12:14.736559    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-lxckg_7cb32b37-dc8a-4928-858b-2a40c09523bc/node-exporter/0.log"
Jul 28 17:12:14.763196 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:12:14.763131    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-lxckg_7cb32b37-dc8a-4928-858b-2a40c09523bc/kube-rbac-proxy/0.log"
Jul 28 17:12:14.783896 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:12:14.783876    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-lxckg_7cb32b37-dc8a-4928-858b-2a40c09523bc/init-textfile/0.log"
Jul 28 17:12:14.919643 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:12:14.919611    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-6pvg7_796d8289-12d1-431b-9b33-d144ab4d9932/kube-rbac-proxy-main/0.log"
Jul 28 17:12:14.941926 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:12:14.941898    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-6pvg7_796d8289-12d1-431b-9b33-d144ab4d9932/kube-rbac-proxy-self/0.log"
Jul 28 17:12:14.966851 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:12:14.966824    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-6pvg7_796d8289-12d1-431b-9b33-d144ab4d9932/openshift-state-metrics/0.log"
Jul 28 17:12:15.304891 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:12:15.304865    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-77f859f47c-td52t_6f414120-ce25-4eac-9aa1-527edd9b92a7/telemeter-client/0.log"
Jul 28 17:12:15.336865 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:12:15.336841    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-77f859f47c-td52t_6f414120-ce25-4eac-9aa1-527edd9b92a7/reload/0.log"
Jul 28 17:12:15.366538 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:12:15.366511    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-77f859f47c-td52t_6f414120-ce25-4eac-9aa1-527edd9b92a7/kube-rbac-proxy/0.log"
Jul 28 17:12:15.412044 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:12:15.412024    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7c5674d886-pqfzc_2d437116-f45c-4765-b8aa-14ad42f54af0/thanos-query/0.log"
Jul 28 17:12:15.443909 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:12:15.443883    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7c5674d886-pqfzc_2d437116-f45c-4765-b8aa-14ad42f54af0/kube-rbac-proxy-web/0.log"
Jul 28 17:12:15.472428 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:12:15.472411    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7c5674d886-pqfzc_2d437116-f45c-4765-b8aa-14ad42f54af0/kube-rbac-proxy/0.log"
Jul 28 17:12:15.510235 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:12:15.510216    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7c5674d886-pqfzc_2d437116-f45c-4765-b8aa-14ad42f54af0/prom-label-proxy/0.log"
Jul 28 17:12:15.533931 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:12:15.533915    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7c5674d886-pqfzc_2d437116-f45c-4765-b8aa-14ad42f54af0/kube-rbac-proxy-rules/0.log"
Jul 28 17:12:15.557758 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:12:15.557698    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7c5674d886-pqfzc_2d437116-f45c-4765-b8aa-14ad42f54af0/kube-rbac-proxy-metrics/0.log"
Jul 28 17:12:17.857714 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:12:17.857687    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-5d6c64b478-xxx5w_be7118f9-3ca8-4a48-b9ff-c90bc1478b54/console/0.log"
Jul 28 17:12:17.892419 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:12:17.892394    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-7c97dc4555-lk8fs_d5e68bc9-29a9-4ed7-a692-c28fe6ddad08/download-server/0.log"
Jul 28 17:12:18.419923 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:12:18.419892    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-464v8/perf-node-gather-daemonset-6t7cb"]
Jul 28 17:12:18.420316 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:12:18.420297    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="309f7841-d173-466f-a580-50dda84e1758" containerName="kserve-container"
Jul 28 17:12:18.420408 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:12:18.420318    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="309f7841-d173-466f-a580-50dda84e1758" containerName="kserve-container"
Jul 28 17:12:18.420408 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:12:18.420336    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="309f7841-d173-466f-a580-50dda84e1758" containerName="kube-rbac-proxy"
Jul 28 17:12:18.420408 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:12:18.420344    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="309f7841-d173-466f-a580-50dda84e1758" containerName="kube-rbac-proxy"
Jul 28 17:12:18.420408 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:12:18.420356    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ec846e8-0d27-4b69-bfe0-538d5f6325de" containerName="kserve-container"
Jul 28 17:12:18.420408 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:12:18.420365    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ec846e8-0d27-4b69-bfe0-538d5f6325de" containerName="kserve-container"
Jul 28 17:12:18.420408 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:12:18.420375    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ec846e8-0d27-4b69-bfe0-538d5f6325de" containerName="kube-rbac-proxy"
Jul 28 17:12:18.420408 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:12:18.420382    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ec846e8-0d27-4b69-bfe0-538d5f6325de" containerName="kube-rbac-proxy"
Jul 28 17:12:18.420408 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:12:18.420406    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7e6417d3-5731-474e-8805-7c66df1f2ed0" containerName="kube-rbac-proxy"
Jul 28 17:12:18.420824 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:12:18.420415    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e6417d3-5731-474e-8805-7c66df1f2ed0" containerName="kube-rbac-proxy"
Jul 28 17:12:18.420824 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:12:18.420435    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7e6417d3-5731-474e-8805-7c66df1f2ed0" containerName="kserve-container"
Jul 28 17:12:18.420824 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:12:18.420444    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e6417d3-5731-474e-8805-7c66df1f2ed0" containerName="kserve-container"
Jul 28 17:12:18.420824 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:12:18.420544    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="7ec846e8-0d27-4b69-bfe0-538d5f6325de" containerName="kserve-container"
Jul 28 17:12:18.420824 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:12:18.420557    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="7ec846e8-0d27-4b69-bfe0-538d5f6325de" containerName="kube-rbac-proxy"
Jul 28 17:12:18.420824 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:12:18.420570    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="7e6417d3-5731-474e-8805-7c66df1f2ed0" containerName="kube-rbac-proxy"
Jul 28 17:12:18.420824 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:12:18.420580    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="309f7841-d173-466f-a580-50dda84e1758" containerName="kube-rbac-proxy"
Jul 28 17:12:18.420824 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:12:18.420589    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="309f7841-d173-466f-a580-50dda84e1758" containerName="kserve-container"
Jul 28 17:12:18.420824 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:12:18.420598    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="7e6417d3-5731-474e-8805-7c66df1f2ed0" containerName="kserve-container"
Jul 28 17:12:18.423865 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:12:18.423845    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-464v8/perf-node-gather-daemonset-6t7cb"
Jul 28 17:12:18.426080 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:12:18.426058    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-464v8\"/\"kube-root-ca.crt\""
Jul 28 17:12:18.426185 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:12:18.426142    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-464v8\"/\"openshift-service-ca.crt\""
Jul 28 17:12:18.427026 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:12:18.427007    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-464v8\"/\"default-dockercfg-mdmkx\""
Jul 28 17:12:18.432290 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:12:18.432271    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-464v8/perf-node-gather-daemonset-6t7cb"]
Jul 28 17:12:18.601276 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:12:18.601241    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0c6d20bb-9586-4b47-8ca6-b0eaf42a928b-sys\") pod \"perf-node-gather-daemonset-6t7cb\" (UID: \"0c6d20bb-9586-4b47-8ca6-b0eaf42a928b\") " pod="openshift-must-gather-464v8/perf-node-gather-daemonset-6t7cb"
Jul 28 17:12:18.601443 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:12:18.601285    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0c6d20bb-9586-4b47-8ca6-b0eaf42a928b-lib-modules\") pod \"perf-node-gather-daemonset-6t7cb\" (UID: \"0c6d20bb-9586-4b47-8ca6-b0eaf42a928b\") " pod="openshift-must-gather-464v8/perf-node-gather-daemonset-6t7cb"
Jul 28 17:12:18.601443 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:12:18.601324    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wwtx9\" (UniqueName: \"kubernetes.io/projected/0c6d20bb-9586-4b47-8ca6-b0eaf42a928b-kube-api-access-wwtx9\") pod \"perf-node-gather-daemonset-6t7cb\" (UID: \"0c6d20bb-9586-4b47-8ca6-b0eaf42a928b\") " pod="openshift-must-gather-464v8/perf-node-gather-daemonset-6t7cb"
Jul 28 17:12:18.601443 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:12:18.601353    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/0c6d20bb-9586-4b47-8ca6-b0eaf42a928b-podres\") pod \"perf-node-gather-daemonset-6t7cb\" (UID: \"0c6d20bb-9586-4b47-8ca6-b0eaf42a928b\") " pod="openshift-must-gather-464v8/perf-node-gather-daemonset-6t7cb"
Jul 28 17:12:18.601443 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:12:18.601377    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/0c6d20bb-9586-4b47-8ca6-b0eaf42a928b-proc\") pod \"perf-node-gather-daemonset-6t7cb\" (UID: \"0c6d20bb-9586-4b47-8ca6-b0eaf42a928b\") " pod="openshift-must-gather-464v8/perf-node-gather-daemonset-6t7cb"
Jul 28 17:12:18.701879 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:12:18.701794    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0c6d20bb-9586-4b47-8ca6-b0eaf42a928b-lib-modules\") pod \"perf-node-gather-daemonset-6t7cb\" (UID: \"0c6d20bb-9586-4b47-8ca6-b0eaf42a928b\") " pod="openshift-must-gather-464v8/perf-node-gather-daemonset-6t7cb"
Jul 28 17:12:18.701879 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:12:18.701834    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wwtx9\" (UniqueName: \"kubernetes.io/projected/0c6d20bb-9586-4b47-8ca6-b0eaf42a928b-kube-api-access-wwtx9\") pod \"perf-node-gather-daemonset-6t7cb\" (UID: \"0c6d20bb-9586-4b47-8ca6-b0eaf42a928b\") " pod="openshift-must-gather-464v8/perf-node-gather-daemonset-6t7cb"
Jul 28 17:12:18.701879 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:12:18.701860    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/0c6d20bb-9586-4b47-8ca6-b0eaf42a928b-podres\") pod \"perf-node-gather-daemonset-6t7cb\" (UID: \"0c6d20bb-9586-4b47-8ca6-b0eaf42a928b\") " pod="openshift-must-gather-464v8/perf-node-gather-daemonset-6t7cb"
Jul 28 17:12:18.702110 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:12:18.701892    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/0c6d20bb-9586-4b47-8ca6-b0eaf42a928b-proc\") pod \"perf-node-gather-daemonset-6t7cb\" (UID: \"0c6d20bb-9586-4b47-8ca6-b0eaf42a928b\") " pod="openshift-must-gather-464v8/perf-node-gather-daemonset-6t7cb"
Jul 28 17:12:18.702110 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:12:18.701956    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0c6d20bb-9586-4b47-8ca6-b0eaf42a928b-sys\") pod \"perf-node-gather-daemonset-6t7cb\" (UID: \"0c6d20bb-9586-4b47-8ca6-b0eaf42a928b\") " pod="openshift-must-gather-464v8/perf-node-gather-daemonset-6t7cb"
Jul 28 17:12:18.702110 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:12:18.701989    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0c6d20bb-9586-4b47-8ca6-b0eaf42a928b-lib-modules\") pod \"perf-node-gather-daemonset-6t7cb\" (UID: \"0c6d20bb-9586-4b47-8ca6-b0eaf42a928b\") " pod="openshift-must-gather-464v8/perf-node-gather-daemonset-6t7cb"
Jul 28 17:12:18.702110 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:12:18.702020    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/0c6d20bb-9586-4b47-8ca6-b0eaf42a928b-proc\") pod \"perf-node-gather-daemonset-6t7cb\" (UID: \"0c6d20bb-9586-4b47-8ca6-b0eaf42a928b\") " pod="openshift-must-gather-464v8/perf-node-gather-daemonset-6t7cb"
Jul 28 17:12:18.702110 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:12:18.702026    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0c6d20bb-9586-4b47-8ca6-b0eaf42a928b-sys\") pod \"perf-node-gather-daemonset-6t7cb\" (UID: \"0c6d20bb-9586-4b47-8ca6-b0eaf42a928b\") " pod="openshift-must-gather-464v8/perf-node-gather-daemonset-6t7cb"
Jul 28 17:12:18.702110 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:12:18.702054    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/0c6d20bb-9586-4b47-8ca6-b0eaf42a928b-podres\") pod \"perf-node-gather-daemonset-6t7cb\" (UID: \"0c6d20bb-9586-4b47-8ca6-b0eaf42a928b\") " pod="openshift-must-gather-464v8/perf-node-gather-daemonset-6t7cb"
Jul 28 17:12:18.710979 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:12:18.710946    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wwtx9\" (UniqueName: \"kubernetes.io/projected/0c6d20bb-9586-4b47-8ca6-b0eaf42a928b-kube-api-access-wwtx9\") pod \"perf-node-gather-daemonset-6t7cb\" (UID: \"0c6d20bb-9586-4b47-8ca6-b0eaf42a928b\") " pod="openshift-must-gather-464v8/perf-node-gather-daemonset-6t7cb"
Jul 28 17:12:18.734799 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:12:18.734777    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-464v8/perf-node-gather-daemonset-6t7cb"
Jul 28 17:12:18.866049 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:12:18.866021    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-464v8/perf-node-gather-daemonset-6t7cb"]
Jul 28 17:12:18.868321 ip-10-0-140-154 kubenswrapper[2557]: W0728 17:12:18.868289    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod0c6d20bb_9586_4b47_8ca6_b0eaf42a928b.slice/crio-a28ce53580acadfb263e2b6b955952eedd88a2dcf1abc30dd06201580f512709 WatchSource:0}: Error finding container a28ce53580acadfb263e2b6b955952eedd88a2dcf1abc30dd06201580f512709: Status 404 returned error can't find the container with id a28ce53580acadfb263e2b6b955952eedd88a2dcf1abc30dd06201580f512709
Jul 28 17:12:19.030083 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:12:19.029992    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-464v8/perf-node-gather-daemonset-6t7cb" event={"ID":"0c6d20bb-9586-4b47-8ca6-b0eaf42a928b","Type":"ContainerStarted","Data":"230d43b9e8a4c132d0e7fa78aa2ce3221541395485905bc018dfe0f064fa6b1d"}
Jul 28 17:12:19.030083 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:12:19.030049    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-464v8/perf-node-gather-daemonset-6t7cb" event={"ID":"0c6d20bb-9586-4b47-8ca6-b0eaf42a928b","Type":"ContainerStarted","Data":"a28ce53580acadfb263e2b6b955952eedd88a2dcf1abc30dd06201580f512709"}
Jul 28 17:12:19.030272 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:12:19.030100    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-464v8/perf-node-gather-daemonset-6t7cb"
Jul 28 17:12:19.048322 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:12:19.048262    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-464v8/perf-node-gather-daemonset-6t7cb" podStartSLOduration=1.048243004 podStartE2EDuration="1.048243004s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:12:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:12:19.047708118 +0000 UTC m=+3056.186177545" watchObservedRunningTime="2026-07-28 17:12:19.048243004 +0000 UTC m=+3056.186712420"
Jul 28 17:12:19.516883 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:12:19.516852    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-wbnl5_9b7c5d20-5762-4a8d-9420-ca25360cd351/dns/0.log"
Jul 28 17:12:19.538903 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:12:19.538880    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-wbnl5_9b7c5d20-5762-4a8d-9420-ca25360cd351/kube-rbac-proxy/0.log"
Jul 28 17:12:19.624354 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:12:19.624328    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-rn2z4_b6c82e3b-ae45-41a0-b919-d4fbd8c3fc64/dns-node-resolver/0.log"
Jul 28 17:12:20.113239 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:12:20.113206    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-77fbd874ff-997dw_63f9d5f1-d1c2-4932-9cbd-46aa1d644e69/registry/0.log"
Jul 28 17:12:20.158954 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:12:20.158924    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-f8l6z_ff81c208-be53-4aff-9993-47b1965201b3/node-ca/0.log"
Jul 28 17:12:21.269970 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:12:21.269946    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-rt2ch_e958d4e8-df2b-4009-b83a-2625c110d4f9/serve-healthcheck-canary/0.log"
Jul 28 17:12:21.693390 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:12:21.693356    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-6fjcf_f59115d6-8481-469e-a651-21e7faa6c392/kube-rbac-proxy/0.log"
Jul 28 17:12:21.712366 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:12:21.712337    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-6fjcf_f59115d6-8481-469e-a651-21e7faa6c392/exporter/0.log"
Jul 28 17:12:21.733900 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:12:21.733866    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-6fjcf_f59115d6-8481-469e-a651-21e7faa6c392/extractor/0.log"
Jul 28 17:12:22.731471 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:12:22.731433    2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6rk8j_da6efa62-02e3-4cac-8881-e62018734a17/konnectivity-agent/0.log" limit=1024
Jul 28 17:12:24.080998 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:12:24.080967    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_llmisvc-controller-manager-84fc959947-2dfkr_f42f830b-f5e5-4749-a39d-77bbe83c7707/manager/0.log"
Jul 28 17:12:24.100822 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:12:24.100795    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_model-serving-api-7b954ffc95-7r22c_d1113ad5-4346-4d74-a8ef-06b1c8e48587/server/0.log"
Jul 28 17:12:24.292332 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:12:24.292303    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_seaweedfs-64568bcd49-vqrgh_489f1c36-f573-45dd-b6b3-0c13b3f8bcd3/seaweedfs/0.log"
Jul 28 17:12:25.045692 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:12:25.045662    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-464v8/perf-node-gather-daemonset-6t7cb"
Jul 28 17:12:29.726911 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:12:29.726884    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-6mmzq_228cc225-7549-41ea-928b-23d353595cd3/kube-multus/0.log"
Jul 28 17:12:30.081609 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:12:30.081577    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-x2pv7_695dbc5b-d1b4-42f3-921a-cc62010ddc14/kube-multus-additional-cni-plugins/0.log"
Jul 28 17:12:30.104594 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:12:30.104570    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-x2pv7_695dbc5b-d1b4-42f3-921a-cc62010ddc14/egress-router-binary-copy/0.log"
Jul 28 17:12:30.146983 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:12:30.146911    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-x2pv7_695dbc5b-d1b4-42f3-921a-cc62010ddc14/cni-plugins/0.log"
Jul 28 17:12:30.172096 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:12:30.172073    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-x2pv7_695dbc5b-d1b4-42f3-921a-cc62010ddc14/bond-cni-plugin/0.log"
Jul 28 17:12:30.191914 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:12:30.191893    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-x2pv7_695dbc5b-d1b4-42f3-921a-cc62010ddc14/routeoverride-cni/0.log"
Jul 28 17:12:30.211105 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:12:30.211079    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-x2pv7_695dbc5b-d1b4-42f3-921a-cc62010ddc14/whereabouts-cni-bincopy/0.log"
Jul 28 17:12:30.230746 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:12:30.230728    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-x2pv7_695dbc5b-d1b4-42f3-921a-cc62010ddc14/whereabouts-cni/0.log"
Jul 28 17:12:30.397747 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:12:30.397669    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-qm9v7_92ef4728-9e89-4494-8bdf-72dcfe9b3ff4/network-metrics-daemon/0.log"
Jul 28 17:12:30.416589 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:12:30.416560    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-qm9v7_92ef4728-9e89-4494-8bdf-72dcfe9b3ff4/kube-rbac-proxy/0.log"
Jul 28 17:12:31.323661 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:12:31.323628    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sbf8h_396c56d3-19a2-49b4-bc4a-c3dabeb144b4/ovn-controller/0.log"
Jul 28 17:12:31.343973 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:12:31.343946    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sbf8h_396c56d3-19a2-49b4-bc4a-c3dabeb144b4/ovn-acl-logging/0.log"
Jul 28 17:12:31.368012 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:12:31.367991    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sbf8h_396c56d3-19a2-49b4-bc4a-c3dabeb144b4/kube-rbac-proxy-node/0.log"
Jul 28 17:12:31.393655 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:12:31.393625    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sbf8h_396c56d3-19a2-49b4-bc4a-c3dabeb144b4/kube-rbac-proxy-ovn-metrics/0.log"
Jul 28 17:12:31.416544 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:12:31.416521    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sbf8h_396c56d3-19a2-49b4-bc4a-c3dabeb144b4/northd/0.log"
Jul 28 17:12:31.440268 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:12:31.440245    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sbf8h_396c56d3-19a2-49b4-bc4a-c3dabeb144b4/nbdb/0.log"
Jul 28 17:12:31.464960 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:12:31.464918    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sbf8h_396c56d3-19a2-49b4-bc4a-c3dabeb144b4/sbdb/0.log"
Jul 28 17:12:31.512159 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:12:31.512121    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sbf8h_396c56d3-19a2-49b4-bc4a-c3dabeb144b4/ovnkube-controller/0.log"
Jul 28 17:12:32.692743 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:12:32.692714    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-qzkm5_c687045e-f5d0-4a22-b71e-9c01463677e6/network-check-target-container/0.log"
Jul 28 17:12:33.539791 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:12:33.539763    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-2lpp7_ecff0248-22af-459d-ae9a-083ed7508975/iptables-alerter/0.log"
Jul 28 17:12:34.243953 ip-10-0-140-154 kubenswrapper[2557]: I0728 17:12:34.243926    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-node-tuning-operator_tuned-nmwkn_9af11dde-7687-459a-8857-0d41fd189315/tuned/0.log"