Jul 31 23:44:20.648384 ip-10-0-128-75 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 31 23:44:20.648391 ip-10-0-128-75 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 31 23:44:20.648401 ip-10-0-128-75 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 31 23:44:20.648721 ip-10-0-128-75 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 31 23:44:30.727182 ip-10-0-128-75 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 31 23:44:30.727199 ip-10-0-128-75 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 650b7273e3d34ddf835d2e99ca895e7c --
Jul 31 23:46:20.738719 ip-10-0-128-75 systemd[1]: Starting Kubernetes Kubelet...
Jul 31 23:46:21.174542 ip-10-0-128-75 kubenswrapper[2564]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 31 23:46:21.174542 ip-10-0-128-75 kubenswrapper[2564]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 31 23:46:21.174542 ip-10-0-128-75 kubenswrapper[2564]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 31 23:46:21.174542 ip-10-0-128-75 kubenswrapper[2564]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 31 23:46:21.174542 ip-10-0-128-75 kubenswrapper[2564]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 31 23:46:21.176452 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.176353    2564 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 31 23:46:21.179892 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.179873    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 31 23:46:21.179892 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.179892    2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 31 23:46:21.179968 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.179896    2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 31 23:46:21.179968 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.179899    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 31 23:46:21.179968 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.179902    2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 31 23:46:21.179968 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.179905    2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 31 23:46:21.179968 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.179908    2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 31 23:46:21.179968 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.179911    2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 31 23:46:21.179968 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.179914    2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 31 23:46:21.179968 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.179917    2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 31 23:46:21.179968 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.179919    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 31 23:46:21.179968 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.179922    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 31 23:46:21.179968 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.179925    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 31 23:46:21.179968 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.179928    2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 31 23:46:21.179968 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.179930    2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 31 23:46:21.179968 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.179933    2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 31 23:46:21.179968 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.179936    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 31 23:46:21.179968 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.179938    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 31 23:46:21.179968 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.179941    2564 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 31 23:46:21.179968 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.179943    2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 31 23:46:21.179968 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.179951    2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 31 23:46:21.179968 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.179954    2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 31 23:46:21.180455 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.179956    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 31 23:46:21.180455 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.179962    2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 31 23:46:21.180455 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.179966    2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 31 23:46:21.180455 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.179969    2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 31 23:46:21.180455 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.179972    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 31 23:46:21.180455 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.179975    2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 31 23:46:21.180455 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.179978    2564 feature_gate.go:328] unrecognized feature gate: Example2
Jul 31 23:46:21.180455 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.179981    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 31 23:46:21.180455 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.179984    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 31 23:46:21.180455 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.179986    2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 31 23:46:21.180455 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.179989    2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 31 23:46:21.180455 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.179991    2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 31 23:46:21.180455 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.179994    2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 31 23:46:21.180455 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.179997    2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 31 23:46:21.180455 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.180000    2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 31 23:46:21.180455 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.180003    2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 31 23:46:21.180455 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.180006    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 31 23:46:21.180455 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.180009    2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 31 23:46:21.180455 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.180011    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 31 23:46:21.180920 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.180014    2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 31 23:46:21.180920 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.180017    2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 31 23:46:21.180920 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.180019    2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 31 23:46:21.180920 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.180023    2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 31 23:46:21.180920 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.180026    2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 31 23:46:21.180920 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.180028    2564 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 31 23:46:21.180920 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.180030    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 31 23:46:21.180920 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.180033    2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 31 23:46:21.180920 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.180036    2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 31 23:46:21.180920 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.180038    2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 31 23:46:21.180920 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.180041    2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 31 23:46:21.180920 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.180043    2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 31 23:46:21.180920 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.180046    2564 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 31 23:46:21.180920 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.180048    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 31 23:46:21.180920 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.180052    2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 31 23:46:21.180920 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.180055    2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 31 23:46:21.180920 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.180058    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 31 23:46:21.180920 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.180060    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 31 23:46:21.180920 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.180090    2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 31 23:46:21.180920 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.180094    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 31 23:46:21.181411 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.180098    2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 31 23:46:21.181411 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.180100    2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 31 23:46:21.181411 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.180104    2564 feature_gate.go:328] unrecognized feature gate: Example
Jul 31 23:46:21.181411 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.180107    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 31 23:46:21.181411 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.180109    2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 31 23:46:21.181411 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.180112    2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 31 23:46:21.181411 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.180114    2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 31 23:46:21.181411 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.180117    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 31 23:46:21.181411 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.180120    2564 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 31 23:46:21.181411 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.180122    2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 31 23:46:21.181411 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.180127    2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 31 23:46:21.181411 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.180130    2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 31 23:46:21.181411 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.180133    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 31 23:46:21.181411 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.180137    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 31 23:46:21.181411 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.180139    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 31 23:46:21.181411 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.180142    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 31 23:46:21.181411 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.180145    2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 31 23:46:21.181411 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.180147    2564 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 31 23:46:21.181411 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.180150    2564 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 31 23:46:21.181411 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.180153    2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 31 23:46:21.181922 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.180158    2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 31 23:46:21.181922 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.180162    2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 31 23:46:21.181922 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.180165    2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 31 23:46:21.181922 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.180168    2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 31 23:46:21.181922 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.180171    2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 31 23:46:21.181922 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.180175    2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 31 23:46:21.181922 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.180178    2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 31 23:46:21.181922 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.180181    2564 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 31 23:46:21.181922 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.180185    2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 31 23:46:21.181922 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.180188    2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 31 23:46:21.181922 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.180191    2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 31 23:46:21.181922 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.180194    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 31 23:46:21.181922 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.180197    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 31 23:46:21.181922 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.180199    2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 31 23:46:21.181922 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.180201    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 31 23:46:21.181922 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.180204    2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 31 23:46:21.181922 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.180206    2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 31 23:46:21.181922 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.180209    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 31 23:46:21.181922 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.180961    2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 31 23:46:21.182386 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.180968    2564 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 31 23:46:21.182386 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.180972    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 31 23:46:21.182386 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.180975    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 31 23:46:21.182386 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.180978    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 31 23:46:21.182386 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.180982    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 31 23:46:21.182386 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.180985    2564 feature_gate.go:328] unrecognized feature gate: Example
Jul 31 23:46:21.182386 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.180987    2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 31 23:46:21.182386 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.180990    2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 31 23:46:21.182386 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.180993    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 31 23:46:21.182386 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.180997    2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 31 23:46:21.182386 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.180999    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 31 23:46:21.182386 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181002    2564 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 31 23:46:21.182386 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181005    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 31 23:46:21.182386 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181007    2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 31 23:46:21.182386 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181010    2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 31 23:46:21.182386 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181014    2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 31 23:46:21.182386 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181018    2564 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 31 23:46:21.182386 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181022    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 31 23:46:21.182386 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181025    2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 31 23:46:21.182386 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181028    2564 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 31 23:46:21.182894 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181030    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 31 23:46:21.182894 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181033    2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 31 23:46:21.182894 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181036    2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 31 23:46:21.182894 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181039    2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 31 23:46:21.182894 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181042    2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 31 23:46:21.182894 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181044    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 31 23:46:21.182894 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181047    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 31 23:46:21.182894 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181050    2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 31 23:46:21.182894 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181052    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 31 23:46:21.182894 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181055    2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 31 23:46:21.182894 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181058    2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 31 23:46:21.182894 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181060    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 31 23:46:21.182894 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181063    2564 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 31 23:46:21.182894 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181067    2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 31 23:46:21.182894 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181071    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 31 23:46:21.182894 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181073    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 31 23:46:21.182894 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181076    2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 31 23:46:21.182894 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181079    2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 31 23:46:21.182894 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181082    2564 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 31 23:46:21.183364 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181084    2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 31 23:46:21.183364 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181087    2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 31 23:46:21.183364 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181089    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 31 23:46:21.183364 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181092    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 31 23:46:21.183364 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181095    2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 31 23:46:21.183364 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181097    2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 31 23:46:21.183364 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181099    2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 31 23:46:21.183364 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181102    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 31 23:46:21.183364 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181105    2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 31 23:46:21.183364 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181108    2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 31 23:46:21.183364 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181111    2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 31 23:46:21.183364 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181113    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 31 23:46:21.183364 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181116    2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 31 23:46:21.183364 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181118    2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 31 23:46:21.183364 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181121    2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 31 23:46:21.183364 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181123    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 31 23:46:21.183364 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181126    2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 31 23:46:21.183364 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181128    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 31 23:46:21.183364 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181131    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 31 23:46:21.183364 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181133    2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 31 23:46:21.183881 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181136    2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 31 23:46:21.183881 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181139    2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 31 23:46:21.183881 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181141    2564 feature_gate.go:328] unrecognized feature gate: Example2
Jul 31 23:46:21.183881 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181144    2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 31 23:46:21.183881 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181146    2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 31 23:46:21.183881 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181149    2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 31 23:46:21.183881 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181152    2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 31 23:46:21.183881 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181154    2564 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 31 23:46:21.183881 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181157    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 31 23:46:21.183881 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181159    2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 31 23:46:21.183881 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181162    2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 31 23:46:21.183881 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181165    2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 31 23:46:21.183881 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181168    2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 31 23:46:21.183881 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181171    2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 31 23:46:21.183881 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181173    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 31 23:46:21.183881 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181176    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 31 23:46:21.183881 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181178    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 31 23:46:21.183881 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181181    2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 31 23:46:21.183881 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181183    2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 31 23:46:21.183881 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181186    2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 31 23:46:21.184405 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181189    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 31 23:46:21.184405 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181192    2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 31 23:46:21.184405 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181194    2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 31 23:46:21.184405 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181198    2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 31 23:46:21.184405 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181200    2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 31 23:46:21.184405 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181204    2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 31 23:46:21.184405 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181207    2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 31 23:46:21.184405 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181210    2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 31 23:46:21.184405 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181212    2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 31 23:46:21.184405 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181215    2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 31 23:46:21.184405 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181217    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 31 23:46:21.184405 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181220    2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 31 23:46:21.184405 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181222    2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 31 23:46:21.184405 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181225    2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 31 23:46:21.184405 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181227    2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 31 23:46:21.184405 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181230    2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 31 23:46:21.184405 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181233    2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 31 23:46:21.184405 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181235    2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 31 23:46:21.184405 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181238    2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 31 23:46:21.184889 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181642    2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 31 23:46:21.184889 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181647    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 31 23:46:21.184889 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181651    2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 31 23:46:21.184889 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181653    2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 31 23:46:21.184889 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181657    2564 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 31 23:46:21.184889 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181659    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 31 23:46:21.184889 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181662    2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 31 23:46:21.184889 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181665    2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 31 23:46:21.184889 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181667    2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 31 23:46:21.184889 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181670    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 31 23:46:21.184889 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181672    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 31 23:46:21.184889 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181675    2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 31 23:46:21.184889 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181678    2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 31 23:46:21.184889 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181680    2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 31 23:46:21.184889 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181684    2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 31 23:46:21.184889 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181686    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 31 23:46:21.184889 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181689    2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 31 23:46:21.184889 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181692    2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 31 23:46:21.184889 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181694    2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 31 23:46:21.184889 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181697    2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 31 23:46:21.185372 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181700    2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 31 23:46:21.185372 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181702    2564 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 31 23:46:21.185372 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181705    2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 31 23:46:21.185372 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181707    2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 31 23:46:21.185372 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181710    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 31 23:46:21.185372 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181712    2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 31 23:46:21.185372 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181715    2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 31 23:46:21.185372 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181717    2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 31 23:46:21.185372 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181720    2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 31 23:46:21.185372 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181722    2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 31 23:46:21.185372 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181725    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 31 23:46:21.185372 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181731    2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 31 23:46:21.185372 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181733    2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 31 23:46:21.185372 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181736    2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 31 23:46:21.185372 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181739    2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 31 23:46:21.185372 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181741    2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 31 23:46:21.185372 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181743    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 31 23:46:21.185372 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181747    2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 31 23:46:21.185372 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181749    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 31 23:46:21.185372 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181752    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 31 23:46:21.185893 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181755    2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 31 23:46:21.185893 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181758    2564 feature_gate.go:328] unrecognized feature gate: Example2
Jul 31 23:46:21.185893 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181760    2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 31 23:46:21.185893 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181762    2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 31 23:46:21.185893 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181765    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 31 23:46:21.185893 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181768    2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 31 23:46:21.185893 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181770    2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 31 23:46:21.185893 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181773    2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 31 23:46:21.185893 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181775    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 31 23:46:21.185893 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181778    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 31 23:46:21.185893 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181780    2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 31 23:46:21.185893 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181783    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 31 23:46:21.185893 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181785    2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 31 23:46:21.185893 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181788    2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 31 23:46:21.185893 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181791    2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 31 23:46:21.185893 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181793    2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 31 23:46:21.185893 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181796    2564 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 31 23:46:21.185893 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181798    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 31 23:46:21.185893 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181800    2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 31 23:46:21.185893 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181803    2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 31 23:46:21.186383 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181806    2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 31 23:46:21.186383 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181810    2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 31 23:46:21.186383 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181814    2564 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 31 23:46:21.186383 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181836    2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 31 23:46:21.186383 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181839    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 31 23:46:21.186383 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181842    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 31 23:46:21.186383 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181845    2564 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 31 23:46:21.186383 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181847    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 31 23:46:21.186383 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181850    2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 31 23:46:21.186383 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181853    2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 31 23:46:21.186383 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181856    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 31 23:46:21.186383 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181859    2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 31 23:46:21.186383 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181861    2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 31 23:46:21.186383 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181864    2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 31 23:46:21.186383 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181867    2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 31 23:46:21.186383 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181869    2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 31 23:46:21.186383 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181872    2564 feature_gate.go:328] unrecognized feature gate: Example
Jul 31 23:46:21.186383 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181874    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 31 23:46:21.186383 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181877    2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 31 23:46:21.186849 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181879    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 31 23:46:21.186849 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181882    2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 31 23:46:21.186849 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181884    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 31 23:46:21.186849 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181887    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 31 23:46:21.186849 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181890    2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 31 23:46:21.186849 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181892    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 31 23:46:21.186849 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181895    2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 31 23:46:21.186849 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181898    2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 31 23:46:21.186849 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181901    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 31 23:46:21.186849 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181903    2564 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 31 23:46:21.186849 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181906    2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 31 23:46:21.186849 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181911    2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 31 23:46:21.186849 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181914    2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 31 23:46:21.186849 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181918    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 31 23:46:21.186849 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181921    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 31 23:46:21.186849 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181925    2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 31 23:46:21.186849 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181930    2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 31 23:46:21.186849 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181933    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 31 23:46:21.186849 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181936    2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 31 23:46:21.187363 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.181939    2564 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 31 23:46:21.187363 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182011    2564 flags.go:64] FLAG: --address="0.0.0.0"
Jul 31 23:46:21.187363 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182019    2564 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 31 23:46:21.187363 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182025    2564 flags.go:64] FLAG: --anonymous-auth="true"
Jul 31 23:46:21.187363 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182030    2564 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 31 23:46:21.187363 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182035    2564 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 31 23:46:21.187363 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182039    2564 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 31 23:46:21.187363 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182043    2564 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 31 23:46:21.187363 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182048    2564 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 31 23:46:21.187363 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182052    2564 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 31 23:46:21.187363 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182055    2564 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 31 23:46:21.187363 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182059    2564 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 31 23:46:21.187363 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182062    2564 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 31 23:46:21.187363 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182065    2564 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 31 23:46:21.187363 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182068    2564 flags.go:64] FLAG: --cgroup-root=""
Jul 31 23:46:21.187363 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182071    2564 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 31 23:46:21.187363 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182074    2564 flags.go:64] FLAG: --client-ca-file=""
Jul 31 23:46:21.187363 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182077    2564 flags.go:64] FLAG: --cloud-provider="external"
Jul 31 23:46:21.187363 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182080    2564 flags.go:64] FLAG: --cluster-dns="[]"
Jul 31 23:46:21.187363 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182084    2564 flags.go:64] FLAG: --cluster-domain=""
Jul 31 23:46:21.187363 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182087    2564 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 31 23:46:21.187363 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182090    2564 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 31 23:46:21.187363 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182094    2564 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 31 23:46:21.187363 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182097    2564 flags.go:64] FLAG: --container-log-max-files="5"
Jul 31 23:46:21.187954 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182101    2564 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 31 23:46:21.187954 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182104    2564 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 31 23:46:21.187954 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182107    2564 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 31 23:46:21.187954 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182111    2564 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 31 23:46:21.187954 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182114    2564 flags.go:64] FLAG: --contention-profiling="false"
Jul 31 23:46:21.187954 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182118    2564 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 31 23:46:21.187954 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182122    2564 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 31 23:46:21.187954 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182125    2564 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 31 23:46:21.187954 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182130    2564 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 31 23:46:21.187954 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182134    2564 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 31 23:46:21.187954 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182137    2564 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 31 23:46:21.187954 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182140    2564 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 31 23:46:21.187954 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182143    2564 flags.go:64] FLAG: --enable-load-reader="false"
Jul 31 23:46:21.187954 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182146    2564 flags.go:64] FLAG: --enable-server="true"
Jul 31 23:46:21.187954 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182150    2564 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 31 23:46:21.187954 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182154    2564 flags.go:64] FLAG: --event-burst="100"
Jul 31 23:46:21.187954 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182158    2564 flags.go:64] FLAG: --event-qps="50"
Jul 31 23:46:21.187954 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182160    2564 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 31 23:46:21.187954 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182164    2564 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 31 23:46:21.187954 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182167    2564 flags.go:64] FLAG: --eviction-hard=""
Jul 31 23:46:21.187954 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182171    2564 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 31 23:46:21.187954 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182174    2564 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 31 23:46:21.187954 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182177    2564 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 31 23:46:21.187954 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182180    2564 flags.go:64] FLAG: --eviction-soft=""
Jul 31 23:46:21.187954 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182183    2564 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 31 23:46:21.188568 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182186    2564 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 31 23:46:21.188568 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182189    2564 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 31 23:46:21.188568 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182192    2564 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 31 23:46:21.188568 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182195    2564 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 31 23:46:21.188568 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182198    2564 flags.go:64] FLAG: --fail-swap-on="true"
Jul 31 23:46:21.188568 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182201    2564 flags.go:64] FLAG: --feature-gates=""
Jul 31 23:46:21.188568 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182205    2564 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 31 23:46:21.188568 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182209    2564 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 31 23:46:21.188568 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182212    2564 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 31 23:46:21.188568 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182215    2564 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 31 23:46:21.188568 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182218    2564 flags.go:64] FLAG: --healthz-port="10248"
Jul 31 23:46:21.188568 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182221    2564 flags.go:64] FLAG: --help="false"
Jul 31 23:46:21.188568 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182225    2564 flags.go:64] FLAG: --hostname-override="ip-10-0-128-75.ec2.internal"
Jul 31 23:46:21.188568 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182228    2564 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 31 23:46:21.188568 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182231    2564 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 31 23:46:21.188568 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182235    2564 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 31 23:46:21.188568 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182239    2564 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 31 23:46:21.188568 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182242    2564 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 31 23:46:21.188568 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182245    2564 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 31 23:46:21.188568 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182248    2564 flags.go:64] FLAG: --image-service-endpoint=""
Jul 31 23:46:21.188568 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182251    2564 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 31 23:46:21.188568 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182254    2564 flags.go:64] FLAG: --kube-api-burst="100"
Jul 31 23:46:21.188568 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182257    2564 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 31 23:46:21.189486 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182261    2564 flags.go:64] FLAG: --kube-api-qps="50"
Jul 31 23:46:21.189486 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182264    2564 flags.go:64] FLAG: --kube-reserved=""
Jul 31 23:46:21.189486 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182267    2564 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 31 23:46:21.189486 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182270    2564 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 31 23:46:21.189486 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182273    2564 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 31 23:46:21.189486 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182276    2564 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 31 23:46:21.189486 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182279    2564 flags.go:64] FLAG: --lock-file=""
Jul 31 23:46:21.189486 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182282    2564 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 31 23:46:21.189486 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182285    2564 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 31 23:46:21.189486 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182288    2564 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 31 23:46:21.189486 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182293    2564 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 31 23:46:21.189486 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182296    2564 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 31 23:46:21.189486 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182299    2564 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 31 23:46:21.189486 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182301    2564 flags.go:64] FLAG: --logging-format="text"
Jul 31 23:46:21.189486 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182305    2564 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 31 23:46:21.189486 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182308    2564 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 31 23:46:21.189486 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182311    2564 flags.go:64] FLAG: --manifest-url=""
Jul 31 23:46:21.189486 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182315    2564 flags.go:64] FLAG: --manifest-url-header=""
Jul 31 23:46:21.189486 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182319    2564 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 31 23:46:21.189486 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182322    2564 flags.go:64] FLAG: --max-open-files="1000000"
Jul 31 23:46:21.189486 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182326    2564 flags.go:64] FLAG: --max-pods="110"
Jul 31 23:46:21.189486 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182331    2564 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 31 23:46:21.189486 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182334    2564 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 31 23:46:21.189486 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182337    2564 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 31 23:46:21.189486 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182341    2564 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 31 23:46:21.190528 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182344    2564 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 31 23:46:21.190528 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182348    2564 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 31 23:46:21.190528 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182351    2564 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 31 23:46:21.190528 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182359    2564 flags.go:64] FLAG: --node-status-max-images="50"
Jul 31 23:46:21.190528 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182362    2564 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 31 23:46:21.190528 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182365    2564 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 31 23:46:21.190528 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182368    2564 flags.go:64] FLAG: --pod-cidr=""
Jul 31 23:46:21.190528 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182372    2564 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:267c8c33005f5dc3d4121af550b1c396e30e2928a13624d7362adfbfb7f440f8"
Jul 31 23:46:21.190528 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182377    2564 flags.go:64] FLAG: --pod-manifest-path=""
Jul 31 23:46:21.190528 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182380    2564 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 31 23:46:21.190528 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182384    2564 flags.go:64] FLAG: --pods-per-core="0"
Jul 31 23:46:21.190528 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182387    2564 flags.go:64] FLAG: --port="10250"
Jul 31 23:46:21.190528 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182390    2564 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 31 23:46:21.190528 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182393    2564 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0c7daa28e1afa5ee0"
Jul 31 23:46:21.190528 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182397    2564 flags.go:64] FLAG: --qos-reserved=""
Jul 31 23:46:21.190528 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182400    2564 flags.go:64] FLAG: --read-only-port="10255"
Jul 31 23:46:21.190528 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182403    2564 flags.go:64] FLAG: --register-node="true"
Jul 31 23:46:21.190528 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182406    2564 flags.go:64] FLAG: --register-with-taints=""
Jul 31 23:46:21.190528 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182410    2564 flags.go:64] FLAG: --registry-burst="10"
Jul 31 23:46:21.190528 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182413    2564 flags.go:64] FLAG: --registry-qps="5"
Jul 31 23:46:21.190528 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182416    2564 flags.go:64] FLAG: --reserved-cpus=""
Jul 31 23:46:21.190528 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182419    2564 flags.go:64] FLAG: --reserved-memory=""
Jul 31 23:46:21.190528 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182422    2564 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 31 23:46:21.190528 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182437    2564 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 31 23:46:21.190528 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182441    2564 flags.go:64] FLAG: --rotate-certificates="false"
Jul 31 23:46:21.191305 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182444    2564 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 31 23:46:21.191305 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182447    2564 flags.go:64] FLAG: --runonce="false"
Jul 31 23:46:21.191305 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182450    2564 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 31 23:46:21.191305 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182454    2564 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 31 23:46:21.191305 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182457    2564 flags.go:64] FLAG: --seccomp-default="false"
Jul 31 23:46:21.191305 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182460    2564 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 31 23:46:21.191305 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182463    2564 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 31 23:46:21.191305 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182467    2564 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 31 23:46:21.191305 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182470    2564 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 31 23:46:21.191305 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182474    2564 flags.go:64] FLAG: --storage-driver-password="root"
Jul 31 23:46:21.191305 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182477    2564 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 31 23:46:21.191305 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182479    2564 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 31 23:46:21.191305 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182483    2564 flags.go:64] FLAG: --storage-driver-user="root"
Jul 31 23:46:21.191305 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182485    2564 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 31 23:46:21.191305 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182489    2564 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 31 23:46:21.191305 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182492    2564 flags.go:64] FLAG: --system-cgroups=""
Jul 31 23:46:21.191305 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182495    2564 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 31 23:46:21.191305 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182501    2564 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 31 23:46:21.191305 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182504    2564 flags.go:64] FLAG: --tls-cert-file=""
Jul 31 23:46:21.191305 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182507    2564 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 31 23:46:21.191305 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182512    2564 flags.go:64] FLAG: --tls-min-version=""
Jul 31 23:46:21.191305 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182514    2564 flags.go:64] FLAG: --tls-private-key-file=""
Jul 31 23:46:21.191305 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182517    2564 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 31 23:46:21.191305 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182520    2564 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 31 23:46:21.191305 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182523    2564 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 31 23:46:21.192072 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182526    2564 flags.go:64] FLAG: --v="2"
Jul 31 23:46:21.192072 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182530    2564 flags.go:64] FLAG: --version="false"
Jul 31 23:46:21.192072 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182534    2564 flags.go:64] FLAG: --vmodule=""
Jul 31 23:46:21.192072 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182538    2564 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 31 23:46:21.192072 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.182542    2564 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 31 23:46:21.192072 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.182661    2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 31 23:46:21.192072 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.182665    2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 31 23:46:21.192072 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.182668    2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 31 23:46:21.192072 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.182671    2564 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 31 23:46:21.192072 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.182674    2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 31 23:46:21.192072 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.182677    2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 31 23:46:21.192072 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.182680    2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 31 23:46:21.192072 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.182683    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 31 23:46:21.192072 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.182685    2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 31 23:46:21.192072 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.182690    2564 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 31 23:46:21.192072 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.182692    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 31 23:46:21.192072 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.182695    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 31 23:46:21.192072 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.182698    2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 31 23:46:21.192072 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.182700    2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 31 23:46:21.192072 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.182703    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 31 23:46:21.192072 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.182705    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 31 23:46:21.193011 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.182708    2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 31 23:46:21.193011 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.182711    2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 31 23:46:21.193011 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.182714    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 31 23:46:21.193011 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.182716    2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 31 23:46:21.193011 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.182719    2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 31 23:46:21.193011 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.182722    2564 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 31 23:46:21.193011 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.182726    2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 31 23:46:21.193011 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.182729    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 31 23:46:21.193011 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.182732    2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 31 23:46:21.193011 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.182735    2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 31 23:46:21.193011 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.182738    2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 31 23:46:21.193011 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.182740    2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 31 23:46:21.193011 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.182743    2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 31 23:46:21.193011 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.182746    2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 31 23:46:21.193011 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.182748    2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 31 23:46:21.193011 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.182751    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 31 23:46:21.193011 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.182754    2564 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 31 23:46:21.193011 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.182756    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 31 23:46:21.193011 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.182760    2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 31 23:46:21.193717 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.182764    2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 31 23:46:21.193717 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.182767    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 31 23:46:21.193717 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.182770    2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 31 23:46:21.193717 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.182773    2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 31 23:46:21.193717 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.182775    2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 31 23:46:21.193717 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.182778    2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 31 23:46:21.193717 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.182782    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 31 23:46:21.193717 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.182785    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 31 23:46:21.193717 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.182788    2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 31 23:46:21.193717 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.182807    2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 31 23:46:21.193717 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.182811    2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 31 23:46:21.193717 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.182814    2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 31 23:46:21.193717 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.182817    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 31 23:46:21.193717 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.182821    2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 31 23:46:21.193717 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.182825    2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 31 23:46:21.193717 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.182828    2564 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 31 23:46:21.193717 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.182831    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 31 23:46:21.193717 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.182834    2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 31 23:46:21.193717 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.182837    2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 31 23:46:21.194392 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.182840    2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 31 23:46:21.194392 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.182843    2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 31 23:46:21.194392 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.182845    2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 31 23:46:21.194392 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.182848    2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 31 23:46:21.194392 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.182850    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 31 23:46:21.194392 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.182853    2564 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 31 23:46:21.194392 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.182856    2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 31 23:46:21.194392 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.182858    2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 31 23:46:21.194392 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.182861    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 31 23:46:21.194392 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.182864    2564 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 31 23:46:21.194392 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.182866    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 31 23:46:21.194392 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.182869    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 31 23:46:21.194392 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.182872    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 31 23:46:21.194392 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.182874    2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 31 23:46:21.194392 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.182877    2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 31 23:46:21.194392 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.182879    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 31 23:46:21.194392 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.182882    2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 31 23:46:21.194392 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.182885    2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 31 23:46:21.194392 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.182888    2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 31 23:46:21.194392 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.182892    2564 feature_gate.go:328] unrecognized feature gate: Example2
Jul 31 23:46:21.195204 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.182894    2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 31 23:46:21.195204 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.182897    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 31 23:46:21.195204 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.182900    2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 31 23:46:21.195204 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.182902    2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 31 23:46:21.195204 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.182905    2564 feature_gate.go:328] unrecognized feature gate: Example
Jul 31 23:46:21.195204 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.182908    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 31 23:46:21.195204 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.182910    2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 31 23:46:21.195204 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.182913    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 31 23:46:21.195204 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.182916    2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 31 23:46:21.195204 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.182918    2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 31 23:46:21.195204 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.182921    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 31 23:46:21.195204 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.182924    2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 31 23:46:21.195204 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.182926    2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 31 23:46:21.195204 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.182929    2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 31 23:46:21.195204 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.182931    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 31 23:46:21.195204 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.182934    2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 31 23:46:21.195204 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.182936    2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 31 23:46:21.195204 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.182941    2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 31 23:46:21.195204 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.182943    2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 31 23:46:21.195204 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.182946    2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 31 23:46:21.196010 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.182948    2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 31 23:46:21.196010 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.182951    2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 31 23:46:21.196010 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.182954    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 31 23:46:21.196010 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.182956    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 31 23:46:21.196010 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.182959    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 31 23:46:21.196010 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.183570    2564 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 31 23:46:21.196010 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.190882    2564 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 31 23:46:21.196010 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.190904    2564 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 31 23:46:21.196010 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.191871    2564 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 31 23:46:21.196010 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.191890    2564 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 31 23:46:21.196010 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.191982    2564 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 31 23:46:21.196010 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.191990    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 31 23:46:21.196010 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.191996    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 31 23:46:21.196010 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192003    2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 31 23:46:21.196414 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192009    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 31 23:46:21.196414 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192015    2564 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 31 23:46:21.196414 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192019    2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 31 23:46:21.196414 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192025    2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 31 23:46:21.196414 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192029    2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 31 23:46:21.196414 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192034    2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 31 23:46:21.196414 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192039    2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 31 23:46:21.196414 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192044    2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 31 23:46:21.196414 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192048    2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 31 23:46:21.196414 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192052    2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 31 23:46:21.196414 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192057    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 31 23:46:21.196414 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192061    2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 31 23:46:21.196414 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192065    2564 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 31 23:46:21.196414 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192070    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 31 23:46:21.196414 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192074    2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 31 23:46:21.196414 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192078    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 31 23:46:21.196414 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192082    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 31 23:46:21.196414 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192085    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 31 23:46:21.196414 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192089    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 31 23:46:21.196414 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192093    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 31 23:46:21.196983 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192097    2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 31 23:46:21.196983 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192102    2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 31 23:46:21.196983 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192106    2564 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 31 23:46:21.196983 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192110    2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 31 23:46:21.196983 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192115    2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 31 23:46:21.196983 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192123    2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 31 23:46:21.196983 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192128    2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 31 23:46:21.196983 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192133    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 31 23:46:21.196983 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192137    2564 feature_gate.go:328] unrecognized feature gate: Example
Jul 31 23:46:21.196983 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192144    2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 31 23:46:21.196983 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192150    2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 31 23:46:21.196983 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192155    2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 31 23:46:21.196983 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192160    2564 feature_gate.go:328] unrecognized feature gate: Example2
Jul 31 23:46:21.196983 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192165    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 31 23:46:21.196983 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192169    2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 31 23:46:21.196983 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192176    2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 31 23:46:21.196983 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192182    2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 31 23:46:21.196983 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192187    2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 31 23:46:21.196983 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192191    2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 31 23:46:21.197475 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192196    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 31 23:46:21.197475 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192202    2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 31 23:46:21.197475 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192207    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 31 23:46:21.197475 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192211    2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 31 23:46:21.197475 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192216    2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 31 23:46:21.197475 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192220    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 31 23:46:21.197475 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192224    2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 31 23:46:21.197475 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192228    2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 31 23:46:21.197475 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192233    2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 31 23:46:21.197475 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192237    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 31 23:46:21.197475 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192241    2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 31 23:46:21.197475 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192245    2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 31 23:46:21.197475 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192250    2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 31 23:46:21.197475 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192255    2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 31 23:46:21.197475 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192259    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 31 23:46:21.197475 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192263    2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 31 23:46:21.197475 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192268    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 31 23:46:21.197475 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192272    2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 31 23:46:21.197475 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192278    2564 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 31 23:46:21.197927 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192283    2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 31 23:46:21.197927 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192287    2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 31 23:46:21.197927 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192293    2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 31 23:46:21.197927 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192297    2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 31 23:46:21.197927 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192301    2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 31 23:46:21.197927 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192306    2564 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 31 23:46:21.197927 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192310    2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 31 23:46:21.197927 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192314    2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 31 23:46:21.197927 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192318    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 31 23:46:21.197927 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192324    2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 31 23:46:21.197927 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192328    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 31 23:46:21.197927 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192333    2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 31 23:46:21.197927 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192337    2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 31 23:46:21.197927 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192341    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 31 23:46:21.197927 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192346    2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 31 23:46:21.197927 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192350    2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 31 23:46:21.197927 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192354    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 31 23:46:21.197927 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192358    2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 31 23:46:21.197927 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192363    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 31 23:46:21.197927 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192367    2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 31 23:46:21.198406 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192371    2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 31 23:46:21.198406 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192375    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 31 23:46:21.198406 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192380    2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 31 23:46:21.198406 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192384    2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 31 23:46:21.198406 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192388    2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 31 23:46:21.198406 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192393    2564 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 31 23:46:21.198406 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192397    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 31 23:46:21.198406 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192401    2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 31 23:46:21.198406 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192405    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 31 23:46:21.198406 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192409    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 31 23:46:21.198406 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192414    2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 31 23:46:21.198406 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192419    2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 31 23:46:21.198406 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192424    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 31 23:46:21.198406 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192448    2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 31 23:46:21.198406 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192452    2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 31 23:46:21.198406 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192457    2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 31 23:46:21.198406 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192461    2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 31 23:46:21.198838 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.192469    2564 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 31 23:46:21.198838 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192659    2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 31 23:46:21.198838 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192668    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 31 23:46:21.198838 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192672    2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 31 23:46:21.198838 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192677    2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 31 23:46:21.198838 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192682    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 31 23:46:21.198838 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192686    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 31 23:46:21.198838 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192691    2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 31 23:46:21.198838 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192695    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 31 23:46:21.198838 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192700    2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 31 23:46:21.198838 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192704    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 31 23:46:21.198838 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192708    2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 31 23:46:21.198838 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192712    2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 31 23:46:21.198838 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192719    2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 31 23:46:21.199188 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192725    2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 31 23:46:21.199188 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192729    2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 31 23:46:21.199188 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192735    2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 31 23:46:21.199188 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192739    2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 31 23:46:21.199188 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192744    2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 31 23:46:21.199188 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192748    2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 31 23:46:21.199188 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192752    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 31 23:46:21.199188 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192756    2564 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 31 23:46:21.199188 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192760    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 31 23:46:21.199188 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192765    2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 31 23:46:21.199188 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192769    2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 31 23:46:21.199188 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192773    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 31 23:46:21.199188 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192778    2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 31 23:46:21.199188 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192782    2564 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 31 23:46:21.199188 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192787    2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 31 23:46:21.199188 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192792    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 31 23:46:21.199188 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192796    2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 31 23:46:21.199188 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192800    2564 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 31 23:46:21.199188 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192804    2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 31 23:46:21.199188 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192809    2564 feature_gate.go:328] unrecognized feature gate: Example2
Jul 31 23:46:21.199711 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192813    2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 31 23:46:21.199711 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192817    2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 31 23:46:21.199711 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192824    2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 31 23:46:21.199711 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192829    2564 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 31 23:46:21.199711 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192833    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 31 23:46:21.199711 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192837    2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 31 23:46:21.199711 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192841    2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 31 23:46:21.199711 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192845    2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 31 23:46:21.199711 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192849    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 31 23:46:21.199711 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192854    2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 31 23:46:21.199711 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192858    2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 31 23:46:21.199711 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192862    2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 31 23:46:21.199711 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192866    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 31 23:46:21.199711 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192870    2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 31 23:46:21.199711 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192875    2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 31 23:46:21.199711 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192879    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 31 23:46:21.199711 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192883    2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 31 23:46:21.199711 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192887    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 31 23:46:21.199711 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192892    2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 31 23:46:21.200187 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192896    2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 31 23:46:21.200187 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192901    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 31 23:46:21.200187 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192905    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 31 23:46:21.200187 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192909    2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 31 23:46:21.200187 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192913    2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 31 23:46:21.200187 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192918    2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 31 23:46:21.200187 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192923    2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 31 23:46:21.200187 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192927    2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 31 23:46:21.200187 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192931    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 31 23:46:21.200187 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192936    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 31 23:46:21.200187 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192941    2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 31 23:46:21.200187 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192945    2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 31 23:46:21.200187 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192949    2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 31 23:46:21.200187 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192953    2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 31 23:46:21.200187 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192957    2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 31 23:46:21.200187 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192961    2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 31 23:46:21.200187 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192965    2564 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 31 23:46:21.200187 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192969    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 31 23:46:21.200187 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192973    2564 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 31 23:46:21.200187 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192977    2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 31 23:46:21.200703 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192981    2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 31 23:46:21.200703 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192986    2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 31 23:46:21.200703 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192990    2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 31 23:46:21.200703 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192994    2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 31 23:46:21.200703 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.192998    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 31 23:46:21.200703 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.193002    2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 31 23:46:21.200703 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.193006    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 31 23:46:21.200703 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.193012    2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 31 23:46:21.200703 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.193017    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 31 23:46:21.200703 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.193021    2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 31 23:46:21.200703 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.193026    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 31 23:46:21.200703 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.193029    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 31 23:46:21.200703 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.193034    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 31 23:46:21.200703 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.193038    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 31 23:46:21.200703 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.193042    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 31 23:46:21.200703 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.193047    2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 31 23:46:21.200703 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.193051    2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 31 23:46:21.200703 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.193055    2564 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 31 23:46:21.200703 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.193060    2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 31 23:46:21.200703 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.193064    2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 31 23:46:21.201184 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.193068    2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 31 23:46:21.201184 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.193072    2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 31 23:46:21.201184 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.193077    2564 feature_gate.go:328] unrecognized feature gate: Example
Jul 31 23:46:21.201184 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.193081    2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 31 23:46:21.201184 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.193085    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 31 23:46:21.201184 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.193089    2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 31 23:46:21.201184 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:21.193093    2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 31 23:46:21.201184 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.193102    2564 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 31 23:46:21.201184 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.193289    2564 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 31 23:46:21.201184 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.197235    2564 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 31 23:46:21.201184 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.198610    2564 server.go:1019] "Starting client certificate rotation"
Jul 31 23:46:21.201184 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.198707    2564 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 31 23:46:21.201184 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.198743    2564 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 31 23:46:21.201649 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.200967    2564 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 31 23:46:21.204046 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.204031    2564 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 31 23:46:21.220604 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.220583    2564 log.go:25] "Validated CRI v1 runtime API"
Jul 31 23:46:21.221918 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.221900    2564 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 31 23:46:21.223230 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.223215    2564 log.go:25] "Validated CRI v1 image API"
Jul 31 23:46:21.225086 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.225061    2564 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 31 23:46:21.227782 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.227763    2564 fs.go:135] Filesystem UUIDs: map[6888c0fa-d5b1-404b-9d91-e6f928d78bb5:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 df681eca-e476-4475-bea8-f478f57d4d20:/dev/nvme0n1p4]
Jul 31 23:46:21.227844 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.227782    2564 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 31 23:46:21.233168 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.233054    2564 manager.go:216] Machine: {Timestamp:2026-07-31 23:46:21.231516428 +0000 UTC m=+0.359841845 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3103961 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2fa5648590edba822048cb43b33b79 SystemUUID:ec2fa564-8590-edba-8220-48cb43b33b79 BootID:650b7273-e3d3-4ddf-835d-2e99ca895e7c Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:f9:46:48:51:e5 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:f9:46:48:51:e5 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:e2:44:4f:c3:0c:6b Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 31 23:46:21.233168 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.233159    2564 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 31 23:46:21.233293 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.233245    2564 manager.go:232] Version: {KernelVersion:5.14.0-570.131.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260727-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 31 23:46:21.234506 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.234481    2564 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 31 23:46:21.234666 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.234510    2564 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-128-75.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 31 23:46:21.235187 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.235177    2564 topology_manager.go:138] "Creating topology manager with none policy"
Jul 31 23:46:21.235217 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.235190    2564 container_manager_linux.go:309] "Creating device plugin manager"
Jul 31 23:46:21.235217 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.235198    2564 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 31 23:46:21.236010 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.235998    2564 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 31 23:46:21.236795 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.236784    2564 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 31 23:46:21.240680 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.240667    2564 state_mem.go:36] "Initialized new in-memory state store"
Jul 31 23:46:21.240820 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.240810    2564 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 31 23:46:21.240859 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.240840    2564 kubelet.go:486] "Attempting to sync node with API server"
Jul 31 23:46:21.240859 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.240849    2564 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 31 23:46:21.240954 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.240864    2564 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 31 23:46:21.240954 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.240875    2564 kubelet.go:398] "Adding apiserver pod source"
Jul 31 23:46:21.240954 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.240884    2564 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 31 23:46:21.241459 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.241445    2564 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 31 23:46:21.241514 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.241467    2564 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 31 23:46:21.241974 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.241930    2564 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-m7cll"
Jul 31 23:46:21.244012 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.243995    2564 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 31 23:46:21.247520 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.247503    2564 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 31 23:46:21.247587 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.247541    2564 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 31 23:46:21.248485 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.248462    2564 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-m7cll"
Jul 31 23:46:21.248695 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.248680    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 31 23:46:21.248753 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.248700    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 31 23:46:21.248753 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.248707    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 31 23:46:21.248753 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.248716    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 31 23:46:21.248753 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.248727    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 31 23:46:21.248753 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.248734    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 31 23:46:21.248753 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.248742    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 31 23:46:21.248753 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.248748    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 31 23:46:21.248938 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.248760    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 31 23:46:21.248938 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.248766    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 31 23:46:21.248938 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.248775    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 31 23:46:21.248938 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.248781    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 31 23:46:21.249832 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:46:21.249810    2564 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 31 23:46:21.249903 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:46:21.249861    2564 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-128-75.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 31 23:46:21.250309 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.250290    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 31 23:46:21.250342 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.250313    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 31 23:46:21.253836 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.253822    2564 server.go:1268] "Started kubelet"
Jul 31 23:46:21.253893 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.253852    2564 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 31 23:46:21.253991 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.253945    2564 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 31 23:46:21.254039 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.254011    2564 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 31 23:46:21.254778 ip-10-0-128-75 systemd[1]: Started Kubernetes Kubelet.
Jul 31 23:46:21.255069 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.255050    2564 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 31 23:46:21.255420 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.255195    2564 server.go:310] "Adding debug handlers to kubelet server"
Jul 31 23:46:21.259108 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:46:21.259083    2564 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 31 23:46:21.259631 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.259617    2564 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 31 23:46:21.259685 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.259629    2564 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 31 23:46:21.259760 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:46:21.259726    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-75.ec2.internal\" not found"
Jul 31 23:46:21.260140 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.260123    2564 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 31 23:46:21.260140 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.260139    2564 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 31 23:46:21.260264 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.260123    2564 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 31 23:46:21.260264 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.260258    2564 reconstruct.go:97] "Volume reconstruction finished"
Jul 31 23:46:21.260264 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.260265    2564 reconciler.go:29] "Reconciler: start to sync state"
Jul 31 23:46:21.260417 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.260402    2564 factory.go:55] Registering systemd factory
Jul 31 23:46:21.260493 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.260421    2564 factory.go:223] Registration of the systemd container factory successfully
Jul 31 23:46:21.260693 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.260675    2564 factory.go:153] Registering CRI-O factory
Jul 31 23:46:21.260790 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.260697    2564 factory.go:223] Registration of the crio container factory successfully
Jul 31 23:46:21.260790 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.260758    2564 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 31 23:46:21.260790 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.260788    2564 factory.go:103] Registering Raw factory
Jul 31 23:46:21.260942 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.260803    2564 manager.go:1196] Started watching for new ooms in manager
Jul 31 23:46:21.261289 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.261275    2564 manager.go:349] Starting recovery of all containers
Jul 31 23:46:21.261808 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.261787    2564 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 31 23:46:21.263174 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.263152    2564 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-128-75.ec2.internal" not found
Jul 31 23:46:21.265707 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:46:21.265681    2564 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-128-75.ec2.internal\" not found" node="ip-10-0-128-75.ec2.internal"
Jul 31 23:46:21.273899 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.273857    2564 manager.go:354] Recovery completed
Jul 31 23:46:21.279667 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.279648    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 31 23:46:21.281632 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.281616    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-75.ec2.internal" event="NodeHasSufficientMemory"
Jul 31 23:46:21.281688 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.281657    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-75.ec2.internal" event="NodeHasNoDiskPressure"
Jul 31 23:46:21.281688 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.281671    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-75.ec2.internal" event="NodeHasSufficientPID"
Jul 31 23:46:21.282266 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.282251    2564 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 31 23:46:21.282331 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.282265    2564 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 31 23:46:21.282331 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.282286    2564 state_mem.go:36] "Initialized new in-memory state store"
Jul 31 23:46:21.282331 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.282309    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 31 23:46:21.282965 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.282947    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-75.ec2.internal" event="NodeHasSufficientMemory"
Jul 31 23:46:21.283041 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.282984    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-75.ec2.internal" event="NodeHasNoDiskPressure"
Jul 31 23:46:21.283041 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.282996    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-75.ec2.internal" event="NodeHasSufficientPID"
Jul 31 23:46:21.283360 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.283334    2564 policy_none.go:49] "None policy: Start"
Jul 31 23:46:21.283470 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.283366    2564 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 31 23:46:21.283470 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.283383    2564 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 31 23:46:21.283551 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.283519    2564 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-128-75.ec2.internal" not found
Jul 31 23:46:21.284376 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.284362    2564 policy_none.go:47] "Start"
Jul 31 23:46:21.315074 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.315055    2564 manager.go:337] "Starting Device Plugin manager"
Jul 31 23:46:21.315196 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:46:21.315091    2564 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 31 23:46:21.315196 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.315104    2564 server.go:87] "Starting device plugin registration server"
Jul 31 23:46:21.315403 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.315386    2564 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 31 23:46:21.315521 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.315400    2564 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 31 23:46:21.315521 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.315501    2564 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 31 23:46:21.315637 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.315610    2564 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 31 23:46:21.315637 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.315625    2564 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 31 23:46:21.316035 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:46:21.316016    2564 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 31 23:46:21.316112 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:46:21.316063    2564 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-128-75.ec2.internal\" not found"
Jul 31 23:46:21.341811 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:21.341784    2564 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-128-75.ec2.internal" not found
Jul 31 23:46:22.414511 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:22.414476    2564 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 31 23:46:22.415667 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:22.415646    2564 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 31 23:46:22.415759 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:22.415692    2564 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 31 23:46:22.415759 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:22.415715    2564 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 31 23:46:22.415759 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:46:22.415749    2564 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 31 23:46:22.418196 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:22.418178    2564 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 31 23:46:22.432199 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:22.432183    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 31 23:46:22.433057 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:22.433037    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-75.ec2.internal" event="NodeHasSufficientMemory"
Jul 31 23:46:22.433148 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:22.433075    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-75.ec2.internal" event="NodeHasNoDiskPressure"
Jul 31 23:46:22.433148 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:22.433091    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-75.ec2.internal" event="NodeHasSufficientPID"
Jul 31 23:46:22.433148 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:22.433130    2564 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-128-75.ec2.internal"
Jul 31 23:46:22.441918 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:22.441847    2564 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-128-75.ec2.internal"
Jul 31 23:46:22.441918 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:46:22.441870    2564 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-128-75.ec2.internal\": node \"ip-10-0-128-75.ec2.internal\" not found"
Jul 31 23:46:22.457261 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:46:22.457239    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-75.ec2.internal\" not found"
Jul 31 23:46:22.516705 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:22.516526    2564 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-75.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-128-75.ec2.internal"]
Jul 31 23:46:22.516826 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:22.516753    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 31 23:46:22.517755 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:22.517740    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-75.ec2.internal" event="NodeHasSufficientMemory"
Jul 31 23:46:22.517823 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:22.517769    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-75.ec2.internal" event="NodeHasNoDiskPressure"
Jul 31 23:46:22.517823 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:22.517779    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-75.ec2.internal" event="NodeHasSufficientPID"
Jul 31 23:46:22.519010 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:22.518997    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 31 23:46:22.519142 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:22.519127    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-75.ec2.internal"
Jul 31 23:46:22.519177 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:22.519160    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 31 23:46:22.519723 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:22.519707    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-75.ec2.internal" event="NodeHasSufficientMemory"
Jul 31 23:46:22.519723 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:22.519718    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-75.ec2.internal" event="NodeHasSufficientMemory"
Jul 31 23:46:22.519833 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:22.519735    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-75.ec2.internal" event="NodeHasNoDiskPressure"
Jul 31 23:46:22.519833 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:22.519742    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-75.ec2.internal" event="NodeHasNoDiskPressure"
Jul 31 23:46:22.519833 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:22.519745    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-75.ec2.internal" event="NodeHasSufficientPID"
Jul 31 23:46:22.519833 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:22.519753    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-75.ec2.internal" event="NodeHasSufficientPID"
Jul 31 23:46:22.520965 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:22.520952    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-75.ec2.internal"
Jul 31 23:46:22.521012 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:22.520978    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 31 23:46:22.521590 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:22.521572    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-75.ec2.internal" event="NodeHasSufficientMemory"
Jul 31 23:46:22.521690 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:22.521599    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-75.ec2.internal" event="NodeHasNoDiskPressure"
Jul 31 23:46:22.521690 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:22.521609    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-75.ec2.internal" event="NodeHasSufficientPID"
Jul 31 23:46:22.544494 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:46:22.544470    2564 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-75.ec2.internal\" not found" node="ip-10-0-128-75.ec2.internal"
Jul 31 23:46:22.549115 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:46:22.549093    2564 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-75.ec2.internal\" not found" node="ip-10-0-128-75.ec2.internal"
Jul 31 23:46:22.558150 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:46:22.558130    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-75.ec2.internal\" not found"
Jul 31 23:46:22.577899 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:22.577875    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ee3cb6879ff58186c26ebd09a71c7e47-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-75.ec2.internal\" (UID: \"ee3cb6879ff58186c26ebd09a71c7e47\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-75.ec2.internal"
Jul 31 23:46:22.577988 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:22.577903    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ee3cb6879ff58186c26ebd09a71c7e47-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-75.ec2.internal\" (UID: \"ee3cb6879ff58186c26ebd09a71c7e47\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-75.ec2.internal"
Jul 31 23:46:22.577988 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:22.577927    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/187e741f1e261812822fd24432933695-config\") pod \"kube-apiserver-proxy-ip-10-0-128-75.ec2.internal\" (UID: \"187e741f1e261812822fd24432933695\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-75.ec2.internal"
Jul 31 23:46:22.658661 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:46:22.658612    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-75.ec2.internal\" not found"
Jul 31 23:46:22.678493 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:22.678468    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ee3cb6879ff58186c26ebd09a71c7e47-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-75.ec2.internal\" (UID: \"ee3cb6879ff58186c26ebd09a71c7e47\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-75.ec2.internal"
Jul 31 23:46:22.678576 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:22.678498    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ee3cb6879ff58186c26ebd09a71c7e47-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-75.ec2.internal\" (UID: \"ee3cb6879ff58186c26ebd09a71c7e47\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-75.ec2.internal"
Jul 31 23:46:22.678576 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:22.678526    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/187e741f1e261812822fd24432933695-config\") pod \"kube-apiserver-proxy-ip-10-0-128-75.ec2.internal\" (UID: \"187e741f1e261812822fd24432933695\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-75.ec2.internal"
Jul 31 23:46:22.678576 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:22.678555    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/187e741f1e261812822fd24432933695-config\") pod \"kube-apiserver-proxy-ip-10-0-128-75.ec2.internal\" (UID: \"187e741f1e261812822fd24432933695\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-75.ec2.internal"
Jul 31 23:46:22.678762 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:22.678574    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ee3cb6879ff58186c26ebd09a71c7e47-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-75.ec2.internal\" (UID: \"ee3cb6879ff58186c26ebd09a71c7e47\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-75.ec2.internal"
Jul 31 23:46:22.678762 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:22.678579    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ee3cb6879ff58186c26ebd09a71c7e47-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-75.ec2.internal\" (UID: \"ee3cb6879ff58186c26ebd09a71c7e47\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-75.ec2.internal"
Jul 31 23:46:22.758944 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:46:22.758873    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-75.ec2.internal\" not found"
Jul 31 23:46:22.847410 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:22.847374    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-75.ec2.internal"
Jul 31 23:46:22.852652 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:22.852621    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-75.ec2.internal"
Jul 31 23:46:22.859155 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:46:22.859138    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-75.ec2.internal\" not found"
Jul 31 23:46:22.959739 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:46:22.959698    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-75.ec2.internal\" not found"
Jul 31 23:46:23.060259 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:46:23.060183    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-75.ec2.internal\" not found"
Jul 31 23:46:23.160953 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:46:23.160900    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-75.ec2.internal\" not found"
Jul 31 23:46:23.204431 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.204399    2564 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 31 23:46:23.214684 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.214666    2564 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 31 23:46:23.214838 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.214810    2564 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 31 23:46:23.214879 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.214813    2564 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 31 23:46:23.214879 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.214858    2564 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 31 23:46:23.258014 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.257984    2564 apiserver.go:52] "Watching apiserver"
Jul 31 23:46:23.268222 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.268185    2564 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-30 23:41:22 +0000 UTC" deadline="2027-12-31 04:37:39.381220854 +0000 UTC"
Jul 31 23:46:23.268222 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.268232    2564 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12412h51m16.113006178s"
Jul 31 23:46:23.275972 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.275940    2564 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-75.ec2.internal"
Jul 31 23:46:23.276119 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.275994    2564 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 31 23:46:23.282993 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.282972    2564 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 31 23:46:23.284114 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.284093    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ovn-kubernetes/ovnkube-node-j5545","kube-system/global-pull-secret-syncer-h45s5","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4dfd9","openshift-cluster-node-tuning-operator/tuned-h5jbv","openshift-image-registry/node-ca-ls2gt","openshift-multus/multus-jw7ld","openshift-network-diagnostics/network-check-target-7n4sw","kube-system/konnectivity-agent-x2dtl","openshift-dns/node-resolver-wxz8k","openshift-multus/multus-additional-cni-plugins-jtsl2","openshift-multus/network-metrics-daemon-j97wn","openshift-network-operator/iptables-alerter-7ttkn"]
Jul 31 23:46:23.285437 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.285416    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-j5545"
Jul 31 23:46:23.286490 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.286469    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h45s5"
Jul 31 23:46:23.286584 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:46:23.286566    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h45s5" podUID="69e78750-573b-4dbd-a585-cf0d88056f4f"
Jul 31 23:46:23.286839 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.286820    2564 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 31 23:46:23.287738 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.287718    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4dfd9"
Jul 31 23:46:23.288097 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.288054    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 31 23:46:23.288097 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.288075    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 31 23:46:23.288295 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.288135    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 31 23:46:23.288431 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.288419    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 31 23:46:23.288873 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.288861    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-h5jbv"
Jul 31 23:46:23.289269 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.289252    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 31 23:46:23.289360 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.289285    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-9jqzw\""
Jul 31 23:46:23.289360 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.289311    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 31 23:46:23.290263 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.290242    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-ls2gt"
Jul 31 23:46:23.290644 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.290612    2564 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 31 23:46:23.290725 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.290689    2564 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-75.ec2.internal"
Jul 31 23:46:23.290786 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.290748    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 31 23:46:23.290846 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.290790    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 31 23:46:23.291536 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.291332    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 31 23:46:23.291536 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.291345    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 31 23:46:23.291536 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.291384    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-qfjxw\""
Jul 31 23:46:23.291536 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.291423    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-mklcr\""
Jul 31 23:46:23.291536 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.291427    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 31 23:46:23.291831 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.291674    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-jw7ld"
Jul 31 23:46:23.292821 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.292803    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 31 23:46:23.292964 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.292948    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7n4sw"
Jul 31 23:46:23.293216 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:46:23.293003    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7n4sw" podUID="2887ad3f-73c9-4fce-8359-c86e96300a84"
Jul 31 23:46:23.293216 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.293039    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 31 23:46:23.293216 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.293097    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-cj8hb\""
Jul 31 23:46:23.293216 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.293114    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 31 23:46:23.294061 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.294045    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 31 23:46:23.294203 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.294180    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 31 23:46:23.294307 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.294234    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 31 23:46:23.294307 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.294253    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 31 23:46:23.294307 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.294290    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-vrlhf\""
Jul 31 23:46:23.294500 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.294481    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-x2dtl"
Jul 31 23:46:23.295493 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.295474    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-wxz8k"
Jul 31 23:46:23.296751 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.296728    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-jtsl2"
Jul 31 23:46:23.298041 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.298026    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j97wn"
Jul 31 23:46:23.298364 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:46:23.298303    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j97wn" podUID="815ee183-a99c-4636-9937-da826945f531"
Jul 31 23:46:23.299041 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.298918    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 31 23:46:23.299041 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.299012    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-clznw\""
Jul 31 23:46:23.299677 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.299437    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 31 23:46:23.300394 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.300372    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 31 23:46:23.300680 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.300535    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 31 23:46:23.300756 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.300683    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 31 23:46:23.301013 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.300996    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 31 23:46:23.301093 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.300999    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-znngt\""
Jul 31 23:46:23.301397 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.301189    2564 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 31 23:46:23.301397 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.301314    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-75.ec2.internal"]
Jul 31 23:46:23.301397 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.301365    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-128-75.ec2.internal"]
Jul 31 23:46:23.301653 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.301459    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-nj6rj\""
Jul 31 23:46:23.301653 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.301521    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-7ttkn"
Jul 31 23:46:23.303948 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.303929    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 31 23:46:23.304220 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.304201    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 31 23:46:23.304302 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.304251    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 31 23:46:23.304302 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.304278    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-bvnmt\""
Jul 31 23:46:23.309405 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:23.309376    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod187e741f1e261812822fd24432933695.slice/crio-a2e997673454a36319eb8170084cdb7269fbd2e638e66d912581ad3e070a7776 WatchSource:0}: Error finding container a2e997673454a36319eb8170084cdb7269fbd2e638e66d912581ad3e070a7776: Status 404 returned error can't find the container with id a2e997673454a36319eb8170084cdb7269fbd2e638e66d912581ad3e070a7776
Jul 31 23:46:23.310069 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:23.310044    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee3cb6879ff58186c26ebd09a71c7e47.slice/crio-28dfc049d7b07d6bd81783cc3c6b5fe4365fd206a1650d1f911b3dc0ef2425f5 WatchSource:0}: Error finding container 28dfc049d7b07d6bd81783cc3c6b5fe4365fd206a1650d1f911b3dc0ef2425f5: Status 404 returned error can't find the container with id 28dfc049d7b07d6bd81783cc3c6b5fe4365fd206a1650d1f911b3dc0ef2425f5
Jul 31 23:46:23.311805 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.311754    2564 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-w5g9r"
Jul 31 23:46:23.314447 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.314430    2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 31 23:46:23.320527 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.320508    2564 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-w5g9r"
Jul 31 23:46:23.378097 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.378070    2564 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 31 23:46:23.382087 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.382069    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b0541f63-78c6-4ae2-b6aa-3e7ad98eb151-host-var-lib-kubelet\") pod \"multus-jw7ld\" (UID: \"b0541f63-78c6-4ae2-b6aa-3e7ad98eb151\") " pod="openshift-multus/multus-jw7ld"
Jul 31 23:46:23.382184 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.382096    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/4e0b1a1a-3553-464e-8094-45a9dbf1cb66-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-jtsl2\" (UID: \"4e0b1a1a-3553-464e-8094-45a9dbf1cb66\") " pod="openshift-multus/multus-additional-cni-plugins-jtsl2"
Jul 31 23:46:23.382184 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.382115    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8ktfb\" (UniqueName: \"kubernetes.io/projected/4e0b1a1a-3553-464e-8094-45a9dbf1cb66-kube-api-access-8ktfb\") pod \"multus-additional-cni-plugins-jtsl2\" (UID: \"4e0b1a1a-3553-464e-8094-45a9dbf1cb66\") " pod="openshift-multus/multus-additional-cni-plugins-jtsl2"
Jul 31 23:46:23.382184 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.382131    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8nrs5\" (UniqueName: \"kubernetes.io/projected/c0bfad85-f7bf-4457-8523-6a1aa966a280-kube-api-access-8nrs5\") pod \"iptables-alerter-7ttkn\" (UID: \"c0bfad85-f7bf-4457-8523-6a1aa966a280\") " pod="openshift-network-operator/iptables-alerter-7ttkn"
Jul 31 23:46:23.382302 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.382184    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/2bd960e3-3638-49f5-bd70-96891131423e-run-ovn\") pod \"ovnkube-node-j5545\" (UID: \"2bd960e3-3638-49f5-bd70-96891131423e\") " pod="openshift-ovn-kubernetes/ovnkube-node-j5545"
Jul 31 23:46:23.382302 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.382207    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/2bd960e3-3638-49f5-bd70-96891131423e-host-cni-netd\") pod \"ovnkube-node-j5545\" (UID: \"2bd960e3-3638-49f5-bd70-96891131423e\") " pod="openshift-ovn-kubernetes/ovnkube-node-j5545"
Jul 31 23:46:23.382302 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.382234    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d997aba4-5c6e-49ad-81bd-95efee82e1ff-etc-sysconfig\") pod \"tuned-h5jbv\" (UID: \"d997aba4-5c6e-49ad-81bd-95efee82e1ff\") " pod="openshift-cluster-node-tuning-operator/tuned-h5jbv"
Jul 31 23:46:23.382302 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.382251    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b0541f63-78c6-4ae2-b6aa-3e7ad98eb151-host-run-netns\") pod \"multus-jw7ld\" (UID: \"b0541f63-78c6-4ae2-b6aa-3e7ad98eb151\") " pod="openshift-multus/multus-jw7ld"
Jul 31 23:46:23.382302 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.382265    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/2bd960e3-3638-49f5-bd70-96891131423e-node-log\") pod \"ovnkube-node-j5545\" (UID: \"2bd960e3-3638-49f5-bd70-96891131423e\") " pod="openshift-ovn-kubernetes/ovnkube-node-j5545"
Jul 31 23:46:23.382302 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.382279    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d997aba4-5c6e-49ad-81bd-95efee82e1ff-var-lib-kubelet\") pod \"tuned-h5jbv\" (UID: \"d997aba4-5c6e-49ad-81bd-95efee82e1ff\") " pod="openshift-cluster-node-tuning-operator/tuned-h5jbv"
Jul 31 23:46:23.382302 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.382293    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d997aba4-5c6e-49ad-81bd-95efee82e1ff-host\") pod \"tuned-h5jbv\" (UID: \"d997aba4-5c6e-49ad-81bd-95efee82e1ff\") " pod="openshift-cluster-node-tuning-operator/tuned-h5jbv"
Jul 31 23:46:23.382566 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.382308    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ltc7q\" (UniqueName: \"kubernetes.io/projected/d997aba4-5c6e-49ad-81bd-95efee82e1ff-kube-api-access-ltc7q\") pod \"tuned-h5jbv\" (UID: \"d997aba4-5c6e-49ad-81bd-95efee82e1ff\") " pod="openshift-cluster-node-tuning-operator/tuned-h5jbv"
Jul 31 23:46:23.382566 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.382325    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4e0b1a1a-3553-464e-8094-45a9dbf1cb66-system-cni-dir\") pod \"multus-additional-cni-plugins-jtsl2\" (UID: \"4e0b1a1a-3553-464e-8094-45a9dbf1cb66\") " pod="openshift-multus/multus-additional-cni-plugins-jtsl2"
Jul 31 23:46:23.382566 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.382340    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/2bd960e3-3638-49f5-bd70-96891131423e-log-socket\") pod \"ovnkube-node-j5545\" (UID: \"2bd960e3-3638-49f5-bd70-96891131423e\") " pod="openshift-ovn-kubernetes/ovnkube-node-j5545"
Jul 31 23:46:23.382566 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.382355    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b0541f63-78c6-4ae2-b6aa-3e7ad98eb151-host-run-k8s-cni-cncf-io\") pod \"multus-jw7ld\" (UID: \"b0541f63-78c6-4ae2-b6aa-3e7ad98eb151\") " pod="openshift-multus/multus-jw7ld"
Jul 31 23:46:23.382566 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.382368    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nxsfk\" (UniqueName: \"kubernetes.io/projected/815ee183-a99c-4636-9937-da826945f531-kube-api-access-nxsfk\") pod \"network-metrics-daemon-j97wn\" (UID: \"815ee183-a99c-4636-9937-da826945f531\") " pod="openshift-multus/network-metrics-daemon-j97wn"
Jul 31 23:46:23.382566 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.382382    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2bd960e3-3638-49f5-bd70-96891131423e-host-cni-bin\") pod \"ovnkube-node-j5545\" (UID: \"2bd960e3-3638-49f5-bd70-96891131423e\") " pod="openshift-ovn-kubernetes/ovnkube-node-j5545"
Jul 31 23:46:23.382566 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.382449    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-88qdj\" (UniqueName: \"kubernetes.io/projected/2bd960e3-3638-49f5-bd70-96891131423e-kube-api-access-88qdj\") pod \"ovnkube-node-j5545\" (UID: \"2bd960e3-3638-49f5-bd70-96891131423e\") " pod="openshift-ovn-kubernetes/ovnkube-node-j5545"
Jul 31 23:46:23.382566 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.382482    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d997aba4-5c6e-49ad-81bd-95efee82e1ff-etc-modprobe-d\") pod \"tuned-h5jbv\" (UID: \"d997aba4-5c6e-49ad-81bd-95efee82e1ff\") " pod="openshift-cluster-node-tuning-operator/tuned-h5jbv"
Jul 31 23:46:23.382566 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.382503    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b0541f63-78c6-4ae2-b6aa-3e7ad98eb151-hostroot\") pod \"multus-jw7ld\" (UID: \"b0541f63-78c6-4ae2-b6aa-3e7ad98eb151\") " pod="openshift-multus/multus-jw7ld"
Jul 31 23:46:23.382566 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.382533    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b0541f63-78c6-4ae2-b6aa-3e7ad98eb151-multus-daemon-config\") pod \"multus-jw7ld\" (UID: \"b0541f63-78c6-4ae2-b6aa-3e7ad98eb151\") " pod="openshift-multus/multus-jw7ld"
Jul 31 23:46:23.382566 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.382562    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cv7k5\" (UniqueName: \"kubernetes.io/projected/2887ad3f-73c9-4fce-8359-c86e96300a84-kube-api-access-cv7k5\") pod \"network-check-target-7n4sw\" (UID: \"2887ad3f-73c9-4fce-8359-c86e96300a84\") " pod="openshift-network-diagnostics/network-check-target-7n4sw"
Jul 31 23:46:23.382992 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.382591    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/fd82faec-581a-491a-8515-4217ac8cac14-agent-certs\") pod \"konnectivity-agent-x2dtl\" (UID: \"fd82faec-581a-491a-8515-4217ac8cac14\") " pod="kube-system/konnectivity-agent-x2dtl"
Jul 31 23:46:23.382992 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.382619    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/69e78750-573b-4dbd-a585-cf0d88056f4f-dbus\") pod \"global-pull-secret-syncer-h45s5\" (UID: \"69e78750-573b-4dbd-a585-cf0d88056f4f\") " pod="kube-system/global-pull-secret-syncer-h45s5"
Jul 31 23:46:23.382992 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.382660    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c0bfad85-f7bf-4457-8523-6a1aa966a280-host-slash\") pod \"iptables-alerter-7ttkn\" (UID: \"c0bfad85-f7bf-4457-8523-6a1aa966a280\") " pod="openshift-network-operator/iptables-alerter-7ttkn"
Jul 31 23:46:23.382992 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.382685    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/1505ec5c-3dd6-4199-8f53-a93fa90f91b7-socket-dir\") pod \"aws-ebs-csi-driver-node-4dfd9\" (UID: \"1505ec5c-3dd6-4199-8f53-a93fa90f91b7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4dfd9"
Jul 31 23:46:23.382992 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.382704    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/1505ec5c-3dd6-4199-8f53-a93fa90f91b7-sys-fs\") pod \"aws-ebs-csi-driver-node-4dfd9\" (UID: \"1505ec5c-3dd6-4199-8f53-a93fa90f91b7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4dfd9"
Jul 31 23:46:23.382992 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.382733    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tmrvx\" (UniqueName: \"kubernetes.io/projected/1505ec5c-3dd6-4199-8f53-a93fa90f91b7-kube-api-access-tmrvx\") pod \"aws-ebs-csi-driver-node-4dfd9\" (UID: \"1505ec5c-3dd6-4199-8f53-a93fa90f91b7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4dfd9"
Jul 31 23:46:23.382992 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.382771    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/fd82faec-581a-491a-8515-4217ac8cac14-konnectivity-ca\") pod \"konnectivity-agent-x2dtl\" (UID: \"fd82faec-581a-491a-8515-4217ac8cac14\") " pod="kube-system/konnectivity-agent-x2dtl"
Jul 31 23:46:23.382992 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.382791    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4e0b1a1a-3553-464e-8094-45a9dbf1cb66-cni-binary-copy\") pod \"multus-additional-cni-plugins-jtsl2\" (UID: \"4e0b1a1a-3553-464e-8094-45a9dbf1cb66\") " pod="openshift-multus/multus-additional-cni-plugins-jtsl2"
Jul 31 23:46:23.382992 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.382810    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c0bfad85-f7bf-4457-8523-6a1aa966a280-iptables-alerter-script\") pod \"iptables-alerter-7ttkn\" (UID: \"c0bfad85-f7bf-4457-8523-6a1aa966a280\") " pod="openshift-network-operator/iptables-alerter-7ttkn"
Jul 31 23:46:23.382992 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.382828    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d997aba4-5c6e-49ad-81bd-95efee82e1ff-etc-sysctl-d\") pod \"tuned-h5jbv\" (UID: \"d997aba4-5c6e-49ad-81bd-95efee82e1ff\") " pod="openshift-cluster-node-tuning-operator/tuned-h5jbv"
Jul 31 23:46:23.382992 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.382842    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b0541f63-78c6-4ae2-b6aa-3e7ad98eb151-os-release\") pod \"multus-jw7ld\" (UID: \"b0541f63-78c6-4ae2-b6aa-3e7ad98eb151\") " pod="openshift-multus/multus-jw7ld"
Jul 31 23:46:23.382992 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.382862    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2bd960e3-3638-49f5-bd70-96891131423e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-j5545\" (UID: \"2bd960e3-3638-49f5-bd70-96891131423e\") " pod="openshift-ovn-kubernetes/ovnkube-node-j5545"
Jul 31 23:46:23.382992 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.382880    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/2bd960e3-3638-49f5-bd70-96891131423e-env-overrides\") pod \"ovnkube-node-j5545\" (UID: \"2bd960e3-3638-49f5-bd70-96891131423e\") " pod="openshift-ovn-kubernetes/ovnkube-node-j5545"
Jul 31 23:46:23.382992 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.382896    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d997aba4-5c6e-49ad-81bd-95efee82e1ff-tmp\") pod \"tuned-h5jbv\" (UID: \"d997aba4-5c6e-49ad-81bd-95efee82e1ff\") " pod="openshift-cluster-node-tuning-operator/tuned-h5jbv"
Jul 31 23:46:23.382992 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.382909    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d98fe0c2-0a27-408e-8131-252e3d469281-host\") pod \"node-ca-ls2gt\" (UID: \"d98fe0c2-0a27-408e-8131-252e3d469281\") " pod="openshift-image-registry/node-ca-ls2gt"
Jul 31 23:46:23.382992 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.382929    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mpwsh\" (UniqueName: \"kubernetes.io/projected/d98fe0c2-0a27-408e-8131-252e3d469281-kube-api-access-mpwsh\") pod \"node-ca-ls2gt\" (UID: \"d98fe0c2-0a27-408e-8131-252e3d469281\") " pod="openshift-image-registry/node-ca-ls2gt"
Jul 31 23:46:23.383507 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.382944    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b0541f63-78c6-4ae2-b6aa-3e7ad98eb151-host-run-multus-certs\") pod \"multus-jw7ld\" (UID: \"b0541f63-78c6-4ae2-b6aa-3e7ad98eb151\") " pod="openshift-multus/multus-jw7ld"
Jul 31 23:46:23.383507 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.382966    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4e0b1a1a-3553-464e-8094-45a9dbf1cb66-tuning-conf-dir\") pod \"multus-additional-cni-plugins-jtsl2\" (UID: \"4e0b1a1a-3553-464e-8094-45a9dbf1cb66\") " pod="openshift-multus/multus-additional-cni-plugins-jtsl2"
Jul 31 23:46:23.383507 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.382981    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2bd960e3-3638-49f5-bd70-96891131423e-host-run-netns\") pod \"ovnkube-node-j5545\" (UID: \"2bd960e3-3638-49f5-bd70-96891131423e\") " pod="openshift-ovn-kubernetes/ovnkube-node-j5545"
Jul 31 23:46:23.383507 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.383009    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/2bd960e3-3638-49f5-bd70-96891131423e-ovnkube-script-lib\") pod \"ovnkube-node-j5545\" (UID: \"2bd960e3-3638-49f5-bd70-96891131423e\") " pod="openshift-ovn-kubernetes/ovnkube-node-j5545"
Jul 31 23:46:23.383507 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.383050    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b0541f63-78c6-4ae2-b6aa-3e7ad98eb151-multus-cni-dir\") pod \"multus-jw7ld\" (UID: \"b0541f63-78c6-4ae2-b6aa-3e7ad98eb151\") " pod="openshift-multus/multus-jw7ld"
Jul 31 23:46:23.383507 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.383065    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b0541f63-78c6-4ae2-b6aa-3e7ad98eb151-multus-socket-dir-parent\") pod \"multus-jw7ld\" (UID: \"b0541f63-78c6-4ae2-b6aa-3e7ad98eb151\") " pod="openshift-multus/multus-jw7ld"
Jul 31 23:46:23.383507 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.383092    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b0541f63-78c6-4ae2-b6aa-3e7ad98eb151-host-var-lib-cni-multus\") pod \"multus-jw7ld\" (UID: \"b0541f63-78c6-4ae2-b6aa-3e7ad98eb151\") " pod="openshift-multus/multus-jw7ld"
Jul 31 23:46:23.383507 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.383115    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b0541f63-78c6-4ae2-b6aa-3e7ad98eb151-etc-kubernetes\") pod \"multus-jw7ld\" (UID: \"b0541f63-78c6-4ae2-b6aa-3e7ad98eb151\") " pod="openshift-multus/multus-jw7ld"
Jul 31 23:46:23.383507 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.383142    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z9qrg\" (UniqueName: \"kubernetes.io/projected/b0541f63-78c6-4ae2-b6aa-3e7ad98eb151-kube-api-access-z9qrg\") pod \"multus-jw7ld\" (UID: \"b0541f63-78c6-4ae2-b6aa-3e7ad98eb151\") " pod="openshift-multus/multus-jw7ld"
Jul 31 23:46:23.383507 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.383159    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/69e78750-573b-4dbd-a585-cf0d88056f4f-kubelet-config\") pod \"global-pull-secret-syncer-h45s5\" (UID: \"69e78750-573b-4dbd-a585-cf0d88056f4f\") " pod="kube-system/global-pull-secret-syncer-h45s5"
Jul 31 23:46:23.383507 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.383178    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2bd960e3-3638-49f5-bd70-96891131423e-host-run-ovn-kubernetes\") pod \"ovnkube-node-j5545\" (UID: \"2bd960e3-3638-49f5-bd70-96891131423e\") " pod="openshift-ovn-kubernetes/ovnkube-node-j5545"
Jul 31 23:46:23.383507 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.383192    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/1505ec5c-3dd6-4199-8f53-a93fa90f91b7-kubelet-dir\") pod \"aws-ebs-csi-driver-node-4dfd9\" (UID: \"1505ec5c-3dd6-4199-8f53-a93fa90f91b7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4dfd9"
Jul 31 23:46:23.383507 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.383223    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b0541f63-78c6-4ae2-b6aa-3e7ad98eb151-cnibin\") pod \"multus-jw7ld\" (UID: \"b0541f63-78c6-4ae2-b6aa-3e7ad98eb151\") " pod="openshift-multus/multus-jw7ld"
Jul 31 23:46:23.383507 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.383250    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b0541f63-78c6-4ae2-b6aa-3e7ad98eb151-multus-conf-dir\") pod \"multus-jw7ld\" (UID: \"b0541f63-78c6-4ae2-b6aa-3e7ad98eb151\") " pod="openshift-multus/multus-jw7ld"
Jul 31 23:46:23.383507 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.383276    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/69e78750-573b-4dbd-a585-cf0d88056f4f-original-pull-secret\") pod \"global-pull-secret-syncer-h45s5\" (UID: \"69e78750-573b-4dbd-a585-cf0d88056f4f\") " pod="kube-system/global-pull-secret-syncer-h45s5"
Jul 31 23:46:23.383507 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.383296    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/2bd960e3-3638-49f5-bd70-96891131423e-run-systemd\") pod \"ovnkube-node-j5545\" (UID: \"2bd960e3-3638-49f5-bd70-96891131423e\") " pod="openshift-ovn-kubernetes/ovnkube-node-j5545"
Jul 31 23:46:23.384200 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.383317    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/1505ec5c-3dd6-4199-8f53-a93fa90f91b7-etc-selinux\") pod \"aws-ebs-csi-driver-node-4dfd9\" (UID: \"1505ec5c-3dd6-4199-8f53-a93fa90f91b7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4dfd9"
Jul 31 23:46:23.384200 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.383346    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d997aba4-5c6e-49ad-81bd-95efee82e1ff-etc-systemd\") pod \"tuned-h5jbv\" (UID: \"d997aba4-5c6e-49ad-81bd-95efee82e1ff\") " pod="openshift-cluster-node-tuning-operator/tuned-h5jbv"
Jul 31 23:46:23.384200 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.383380    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b0541f63-78c6-4ae2-b6aa-3e7ad98eb151-cni-binary-copy\") pod \"multus-jw7ld\" (UID: \"b0541f63-78c6-4ae2-b6aa-3e7ad98eb151\") " pod="openshift-multus/multus-jw7ld"
Jul 31 23:46:23.384200 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.383401    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/816ea407-9c93-4455-8202-8229340dc7db-hosts-file\") pod \"node-resolver-wxz8k\" (UID: \"816ea407-9c93-4455-8202-8229340dc7db\") " pod="openshift-dns/node-resolver-wxz8k"
Jul 31 23:46:23.384200 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.383416    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/815ee183-a99c-4636-9937-da826945f531-metrics-certs\") pod \"network-metrics-daemon-j97wn\" (UID: \"815ee183-a99c-4636-9937-da826945f531\") " pod="openshift-multus/network-metrics-daemon-j97wn"
Jul 31 23:46:23.384200 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.383435    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2bd960e3-3638-49f5-bd70-96891131423e-var-lib-openvswitch\") pod \"ovnkube-node-j5545\" (UID: \"2bd960e3-3638-49f5-bd70-96891131423e\") " pod="openshift-ovn-kubernetes/ovnkube-node-j5545"
Jul 31 23:46:23.384200 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.383451    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2bd960e3-3638-49f5-bd70-96891131423e-run-openvswitch\") pod \"ovnkube-node-j5545\" (UID: \"2bd960e3-3638-49f5-bd70-96891131423e\") " pod="openshift-ovn-kubernetes/ovnkube-node-j5545"
Jul 31 23:46:23.384200 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.383469    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d997aba4-5c6e-49ad-81bd-95efee82e1ff-etc-sysctl-conf\") pod \"tuned-h5jbv\" (UID: \"d997aba4-5c6e-49ad-81bd-95efee82e1ff\") " pod="openshift-cluster-node-tuning-operator/tuned-h5jbv"
Jul 31 23:46:23.384200 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.383483    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4e0b1a1a-3553-464e-8094-45a9dbf1cb66-os-release\") pod \"multus-additional-cni-plugins-jtsl2\" (UID: \"4e0b1a1a-3553-464e-8094-45a9dbf1cb66\") " pod="openshift-multus/multus-additional-cni-plugins-jtsl2"
Jul 31 23:46:23.384200 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.383500    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/2bd960e3-3638-49f5-bd70-96891131423e-host-kubelet\") pod \"ovnkube-node-j5545\" (UID: \"2bd960e3-3638-49f5-bd70-96891131423e\") " pod="openshift-ovn-kubernetes/ovnkube-node-j5545"
Jul 31 23:46:23.384200 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.383515    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/2bd960e3-3638-49f5-bd70-96891131423e-systemd-units\") pod \"ovnkube-node-j5545\" (UID: \"2bd960e3-3638-49f5-bd70-96891131423e\") " pod="openshift-ovn-kubernetes/ovnkube-node-j5545"
Jul 31 23:46:23.384200 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.383543    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d997aba4-5c6e-49ad-81bd-95efee82e1ff-etc-kubernetes\") pod \"tuned-h5jbv\" (UID: \"d997aba4-5c6e-49ad-81bd-95efee82e1ff\") " pod="openshift-cluster-node-tuning-operator/tuned-h5jbv"
Jul 31 23:46:23.384200 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.383576    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d997aba4-5c6e-49ad-81bd-95efee82e1ff-lib-modules\") pod \"tuned-h5jbv\" (UID: \"d997aba4-5c6e-49ad-81bd-95efee82e1ff\") " pod="openshift-cluster-node-tuning-operator/tuned-h5jbv"
Jul 31 23:46:23.384200 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.383599    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d997aba4-5c6e-49ad-81bd-95efee82e1ff-etc-tuned\") pod \"tuned-h5jbv\" (UID: \"d997aba4-5c6e-49ad-81bd-95efee82e1ff\") " pod="openshift-cluster-node-tuning-operator/tuned-h5jbv"
Jul 31 23:46:23.384200 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.383640    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d98fe0c2-0a27-408e-8131-252e3d469281-serviceca\") pod \"node-ca-ls2gt\" (UID: \"d98fe0c2-0a27-408e-8131-252e3d469281\") " pod="openshift-image-registry/node-ca-ls2gt"
Jul 31 23:46:23.384200 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.383689    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/816ea407-9c93-4455-8202-8229340dc7db-tmp-dir\") pod \"node-resolver-wxz8k\" (UID: \"816ea407-9c93-4455-8202-8229340dc7db\") " pod="openshift-dns/node-resolver-wxz8k"
Jul 31 23:46:23.384681 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.383714    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/1505ec5c-3dd6-4199-8f53-a93fa90f91b7-registration-dir\") pod \"aws-ebs-csi-driver-node-4dfd9\" (UID: \"1505ec5c-3dd6-4199-8f53-a93fa90f91b7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4dfd9"
Jul 31 23:46:23.384681 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.383740    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4e0b1a1a-3553-464e-8094-45a9dbf1cb66-cnibin\") pod \"multus-additional-cni-plugins-jtsl2\" (UID: \"4e0b1a1a-3553-464e-8094-45a9dbf1cb66\") " pod="openshift-multus/multus-additional-cni-plugins-jtsl2"
Jul 31 23:46:23.384681 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.383765    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2bd960e3-3638-49f5-bd70-96891131423e-host-slash\") pod \"ovnkube-node-j5545\" (UID: \"2bd960e3-3638-49f5-bd70-96891131423e\") " pod="openshift-ovn-kubernetes/ovnkube-node-j5545"
Jul 31 23:46:23.384681 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.383788    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/1505ec5c-3dd6-4199-8f53-a93fa90f91b7-device-dir\") pod \"aws-ebs-csi-driver-node-4dfd9\" (UID: \"1505ec5c-3dd6-4199-8f53-a93fa90f91b7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4dfd9"
Jul 31 23:46:23.384681 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.383821    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d997aba4-5c6e-49ad-81bd-95efee82e1ff-run\") pod \"tuned-h5jbv\" (UID: \"d997aba4-5c6e-49ad-81bd-95efee82e1ff\") " pod="openshift-cluster-node-tuning-operator/tuned-h5jbv"
Jul 31 23:46:23.384681 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.383844    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b0541f63-78c6-4ae2-b6aa-3e7ad98eb151-system-cni-dir\") pod \"multus-jw7ld\" (UID: \"b0541f63-78c6-4ae2-b6aa-3e7ad98eb151\") " pod="openshift-multus/multus-jw7ld"
Jul 31 23:46:23.384681 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.383868    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/2bd960e3-3638-49f5-bd70-96891131423e-ovn-node-metrics-cert\") pod \"ovnkube-node-j5545\" (UID: \"2bd960e3-3638-49f5-bd70-96891131423e\") " pod="openshift-ovn-kubernetes/ovnkube-node-j5545"
Jul 31 23:46:23.384681 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.383890    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d997aba4-5c6e-49ad-81bd-95efee82e1ff-sys\") pod \"tuned-h5jbv\" (UID: \"d997aba4-5c6e-49ad-81bd-95efee82e1ff\") " pod="openshift-cluster-node-tuning-operator/tuned-h5jbv"
Jul 31 23:46:23.384681 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.383912    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j28hp\" (UniqueName: \"kubernetes.io/projected/816ea407-9c93-4455-8202-8229340dc7db-kube-api-access-j28hp\") pod \"node-resolver-wxz8k\" (UID: \"816ea407-9c93-4455-8202-8229340dc7db\") " pod="openshift-dns/node-resolver-wxz8k"
Jul 31 23:46:23.384681 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.383938    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/4e0b1a1a-3553-464e-8094-45a9dbf1cb66-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-jtsl2\" (UID: \"4e0b1a1a-3553-464e-8094-45a9dbf1cb66\") " pod="openshift-multus/multus-additional-cni-plugins-jtsl2"
Jul 31 23:46:23.384681 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.383964    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2bd960e3-3638-49f5-bd70-96891131423e-etc-openvswitch\") pod \"ovnkube-node-j5545\" (UID: \"2bd960e3-3638-49f5-bd70-96891131423e\") " pod="openshift-ovn-kubernetes/ovnkube-node-j5545"
Jul 31 23:46:23.384681 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.383987    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/2bd960e3-3638-49f5-bd70-96891131423e-ovnkube-config\") pod \"ovnkube-node-j5545\" (UID: \"2bd960e3-3638-49f5-bd70-96891131423e\") " pod="openshift-ovn-kubernetes/ovnkube-node-j5545"
Jul 31 23:46:23.384681 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.384009    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b0541f63-78c6-4ae2-b6aa-3e7ad98eb151-host-var-lib-cni-bin\") pod \"multus-jw7ld\" (UID: \"b0541f63-78c6-4ae2-b6aa-3e7ad98eb151\") " pod="openshift-multus/multus-jw7ld"
Jul 31 23:46:23.420221 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.420182    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-75.ec2.internal" event={"ID":"ee3cb6879ff58186c26ebd09a71c7e47","Type":"ContainerStarted","Data":"28dfc049d7b07d6bd81783cc3c6b5fe4365fd206a1650d1f911b3dc0ef2425f5"}
Jul 31 23:46:23.421148 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.421129    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-75.ec2.internal" event={"ID":"187e741f1e261812822fd24432933695","Type":"ContainerStarted","Data":"a2e997673454a36319eb8170084cdb7269fbd2e638e66d912581ad3e070a7776"}
Jul 31 23:46:23.484946 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.484916    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/2bd960e3-3638-49f5-bd70-96891131423e-ovnkube-config\") pod \"ovnkube-node-j5545\" (UID: \"2bd960e3-3638-49f5-bd70-96891131423e\") " pod="openshift-ovn-kubernetes/ovnkube-node-j5545"
Jul 31 23:46:23.484946 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.484948    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b0541f63-78c6-4ae2-b6aa-3e7ad98eb151-host-var-lib-cni-bin\") pod \"multus-jw7ld\" (UID: \"b0541f63-78c6-4ae2-b6aa-3e7ad98eb151\") " pod="openshift-multus/multus-jw7ld"
Jul 31 23:46:23.484946 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.484964    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b0541f63-78c6-4ae2-b6aa-3e7ad98eb151-host-var-lib-kubelet\") pod \"multus-jw7ld\" (UID: \"b0541f63-78c6-4ae2-b6aa-3e7ad98eb151\") " pod="openshift-multus/multus-jw7ld"
Jul 31 23:46:23.485186 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.484985    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/4e0b1a1a-3553-464e-8094-45a9dbf1cb66-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-jtsl2\" (UID: \"4e0b1a1a-3553-464e-8094-45a9dbf1cb66\") " pod="openshift-multus/multus-additional-cni-plugins-jtsl2"
Jul 31 23:46:23.485186 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.485011    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8ktfb\" (UniqueName: \"kubernetes.io/projected/4e0b1a1a-3553-464e-8094-45a9dbf1cb66-kube-api-access-8ktfb\") pod \"multus-additional-cni-plugins-jtsl2\" (UID: \"4e0b1a1a-3553-464e-8094-45a9dbf1cb66\") " pod="openshift-multus/multus-additional-cni-plugins-jtsl2"
Jul 31 23:46:23.485186 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.485041    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b0541f63-78c6-4ae2-b6aa-3e7ad98eb151-host-var-lib-cni-bin\") pod \"multus-jw7ld\" (UID: \"b0541f63-78c6-4ae2-b6aa-3e7ad98eb151\") " pod="openshift-multus/multus-jw7ld"
Jul 31 23:46:23.485186 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.485069    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8nrs5\" (UniqueName: \"kubernetes.io/projected/c0bfad85-f7bf-4457-8523-6a1aa966a280-kube-api-access-8nrs5\") pod \"iptables-alerter-7ttkn\" (UID: \"c0bfad85-f7bf-4457-8523-6a1aa966a280\") " pod="openshift-network-operator/iptables-alerter-7ttkn"
Jul 31 23:46:23.485186 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.485112    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/2bd960e3-3638-49f5-bd70-96891131423e-run-ovn\") pod \"ovnkube-node-j5545\" (UID: \"2bd960e3-3638-49f5-bd70-96891131423e\") " pod="openshift-ovn-kubernetes/ovnkube-node-j5545"
Jul 31 23:46:23.485186 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.485141    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/2bd960e3-3638-49f5-bd70-96891131423e-host-cni-netd\") pod \"ovnkube-node-j5545\" (UID: \"2bd960e3-3638-49f5-bd70-96891131423e\") " pod="openshift-ovn-kubernetes/ovnkube-node-j5545"
Jul 31 23:46:23.485186 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.485164    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d997aba4-5c6e-49ad-81bd-95efee82e1ff-etc-sysconfig\") pod \"tuned-h5jbv\" (UID: \"d997aba4-5c6e-49ad-81bd-95efee82e1ff\") " pod="openshift-cluster-node-tuning-operator/tuned-h5jbv"
Jul 31 23:46:23.485186 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.485187    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b0541f63-78c6-4ae2-b6aa-3e7ad98eb151-host-run-netns\") pod \"multus-jw7ld\" (UID: \"b0541f63-78c6-4ae2-b6aa-3e7ad98eb151\") " pod="openshift-multus/multus-jw7ld"
Jul 31 23:46:23.485546 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.485208    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/2bd960e3-3638-49f5-bd70-96891131423e-node-log\") pod \"ovnkube-node-j5545\" (UID: \"2bd960e3-3638-49f5-bd70-96891131423e\") " pod="openshift-ovn-kubernetes/ovnkube-node-j5545"
Jul 31 23:46:23.485546 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.485215    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/2bd960e3-3638-49f5-bd70-96891131423e-host-cni-netd\") pod \"ovnkube-node-j5545\" (UID: \"2bd960e3-3638-49f5-bd70-96891131423e\") " pod="openshift-ovn-kubernetes/ovnkube-node-j5545"
Jul 31 23:46:23.485546 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.485250    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d997aba4-5c6e-49ad-81bd-95efee82e1ff-var-lib-kubelet\") pod \"tuned-h5jbv\" (UID: \"d997aba4-5c6e-49ad-81bd-95efee82e1ff\") " pod="openshift-cluster-node-tuning-operator/tuned-h5jbv"
Jul 31 23:46:23.485546 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.485293    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/2bd960e3-3638-49f5-bd70-96891131423e-node-log\") pod \"ovnkube-node-j5545\" (UID: \"2bd960e3-3638-49f5-bd70-96891131423e\") " pod="openshift-ovn-kubernetes/ovnkube-node-j5545"
Jul 31 23:46:23.485546 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.485307    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d997aba4-5c6e-49ad-81bd-95efee82e1ff-host\") pod \"tuned-h5jbv\" (UID: \"d997aba4-5c6e-49ad-81bd-95efee82e1ff\") " pod="openshift-cluster-node-tuning-operator/tuned-h5jbv"
Jul 31 23:46:23.485546 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.485259    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/2bd960e3-3638-49f5-bd70-96891131423e-run-ovn\") pod \"ovnkube-node-j5545\" (UID: \"2bd960e3-3638-49f5-bd70-96891131423e\") " pod="openshift-ovn-kubernetes/ovnkube-node-j5545"
Jul 31 23:46:23.485546 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.485315    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d997aba4-5c6e-49ad-81bd-95efee82e1ff-var-lib-kubelet\") pod \"tuned-h5jbv\" (UID: \"d997aba4-5c6e-49ad-81bd-95efee82e1ff\") " pod="openshift-cluster-node-tuning-operator/tuned-h5jbv"
Jul 31 23:46:23.485546 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.485251    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d997aba4-5c6e-49ad-81bd-95efee82e1ff-etc-sysconfig\") pod \"tuned-h5jbv\" (UID: \"d997aba4-5c6e-49ad-81bd-95efee82e1ff\") " pod="openshift-cluster-node-tuning-operator/tuned-h5jbv"
Jul 31 23:46:23.485546 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.485338    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ltc7q\" (UniqueName: \"kubernetes.io/projected/d997aba4-5c6e-49ad-81bd-95efee82e1ff-kube-api-access-ltc7q\") pod \"tuned-h5jbv\" (UID: \"d997aba4-5c6e-49ad-81bd-95efee82e1ff\") " pod="openshift-cluster-node-tuning-operator/tuned-h5jbv"
Jul 31 23:46:23.485546 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.485334    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b0541f63-78c6-4ae2-b6aa-3e7ad98eb151-host-run-netns\") pod \"multus-jw7ld\" (UID: \"b0541f63-78c6-4ae2-b6aa-3e7ad98eb151\") " pod="openshift-multus/multus-jw7ld"
Jul 31 23:46:23.485546 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.485378    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d997aba4-5c6e-49ad-81bd-95efee82e1ff-host\") pod \"tuned-h5jbv\" (UID: \"d997aba4-5c6e-49ad-81bd-95efee82e1ff\") " pod="openshift-cluster-node-tuning-operator/tuned-h5jbv"
Jul 31 23:46:23.485546 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.485388    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b0541f63-78c6-4ae2-b6aa-3e7ad98eb151-host-var-lib-kubelet\") pod \"multus-jw7ld\" (UID: \"b0541f63-78c6-4ae2-b6aa-3e7ad98eb151\") " pod="openshift-multus/multus-jw7ld"
Jul 31 23:46:23.485546 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.485379    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4e0b1a1a-3553-464e-8094-45a9dbf1cb66-system-cni-dir\") pod \"multus-additional-cni-plugins-jtsl2\" (UID: \"4e0b1a1a-3553-464e-8094-45a9dbf1cb66\") " pod="openshift-multus/multus-additional-cni-plugins-jtsl2"
Jul 31 23:46:23.485546 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.485455    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/2bd960e3-3638-49f5-bd70-96891131423e-log-socket\") pod \"ovnkube-node-j5545\" (UID: \"2bd960e3-3638-49f5-bd70-96891131423e\") " pod="openshift-ovn-kubernetes/ovnkube-node-j5545"
Jul 31 23:46:23.485546 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.485496    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/2bd960e3-3638-49f5-bd70-96891131423e-log-socket\") pod \"ovnkube-node-j5545\" (UID: \"2bd960e3-3638-49f5-bd70-96891131423e\") " pod="openshift-ovn-kubernetes/ovnkube-node-j5545"
Jul 31 23:46:23.485546 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.485495    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4e0b1a1a-3553-464e-8094-45a9dbf1cb66-system-cni-dir\") pod \"multus-additional-cni-plugins-jtsl2\" (UID: \"4e0b1a1a-3553-464e-8094-45a9dbf1cb66\") " pod="openshift-multus/multus-additional-cni-plugins-jtsl2"
Jul 31 23:46:23.485546 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.485531    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b0541f63-78c6-4ae2-b6aa-3e7ad98eb151-host-run-k8s-cni-cncf-io\") pod \"multus-jw7ld\" (UID: \"b0541f63-78c6-4ae2-b6aa-3e7ad98eb151\") " pod="openshift-multus/multus-jw7ld"
Jul 31 23:46:23.485546 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.485557    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nxsfk\" (UniqueName: \"kubernetes.io/projected/815ee183-a99c-4636-9937-da826945f531-kube-api-access-nxsfk\") pod \"network-metrics-daemon-j97wn\" (UID: \"815ee183-a99c-4636-9937-da826945f531\") " pod="openshift-multus/network-metrics-daemon-j97wn"
Jul 31 23:46:23.486354 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.485582    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2bd960e3-3638-49f5-bd70-96891131423e-host-cni-bin\") pod \"ovnkube-node-j5545\" (UID: \"2bd960e3-3638-49f5-bd70-96891131423e\") " pod="openshift-ovn-kubernetes/ovnkube-node-j5545"
Jul 31 23:46:23.486354 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.485607    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b0541f63-78c6-4ae2-b6aa-3e7ad98eb151-host-run-k8s-cni-cncf-io\") pod \"multus-jw7ld\" (UID: \"b0541f63-78c6-4ae2-b6aa-3e7ad98eb151\") " pod="openshift-multus/multus-jw7ld"
Jul 31 23:46:23.486354 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.485609    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-88qdj\" (UniqueName: \"kubernetes.io/projected/2bd960e3-3638-49f5-bd70-96891131423e-kube-api-access-88qdj\") pod \"ovnkube-node-j5545\" (UID: \"2bd960e3-3638-49f5-bd70-96891131423e\") " pod="openshift-ovn-kubernetes/ovnkube-node-j5545"
Jul 31 23:46:23.486354 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.485659    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2bd960e3-3638-49f5-bd70-96891131423e-host-cni-bin\") pod \"ovnkube-node-j5545\" (UID: \"2bd960e3-3638-49f5-bd70-96891131423e\") " pod="openshift-ovn-kubernetes/ovnkube-node-j5545"
Jul 31 23:46:23.486354 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.485667    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d997aba4-5c6e-49ad-81bd-95efee82e1ff-etc-modprobe-d\") pod \"tuned-h5jbv\" (UID: \"d997aba4-5c6e-49ad-81bd-95efee82e1ff\") " pod="openshift-cluster-node-tuning-operator/tuned-h5jbv"
Jul 31 23:46:23.486354 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.485694    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b0541f63-78c6-4ae2-b6aa-3e7ad98eb151-hostroot\") pod \"multus-jw7ld\" (UID: \"b0541f63-78c6-4ae2-b6aa-3e7ad98eb151\") " pod="openshift-multus/multus-jw7ld"
Jul 31 23:46:23.486354 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.485692    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/4e0b1a1a-3553-464e-8094-45a9dbf1cb66-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-jtsl2\" (UID: \"4e0b1a1a-3553-464e-8094-45a9dbf1cb66\") " pod="openshift-multus/multus-additional-cni-plugins-jtsl2"
Jul 31 23:46:23.486354 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.485711    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/2bd960e3-3638-49f5-bd70-96891131423e-ovnkube-config\") pod \"ovnkube-node-j5545\" (UID: \"2bd960e3-3638-49f5-bd70-96891131423e\") " pod="openshift-ovn-kubernetes/ovnkube-node-j5545"
Jul 31 23:46:23.486354 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.485718    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b0541f63-78c6-4ae2-b6aa-3e7ad98eb151-multus-daemon-config\") pod \"multus-jw7ld\" (UID: \"b0541f63-78c6-4ae2-b6aa-3e7ad98eb151\") " pod="openshift-multus/multus-jw7ld"
Jul 31 23:46:23.486354 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.485752    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cv7k5\" (UniqueName: \"kubernetes.io/projected/2887ad3f-73c9-4fce-8359-c86e96300a84-kube-api-access-cv7k5\") pod \"network-check-target-7n4sw\" (UID: \"2887ad3f-73c9-4fce-8359-c86e96300a84\") " pod="openshift-network-diagnostics/network-check-target-7n4sw"
Jul 31 23:46:23.486354 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.485769    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b0541f63-78c6-4ae2-b6aa-3e7ad98eb151-hostroot\") pod \"multus-jw7ld\" (UID: \"b0541f63-78c6-4ae2-b6aa-3e7ad98eb151\") " pod="openshift-multus/multus-jw7ld"
Jul 31 23:46:23.486354 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.485780    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/fd82faec-581a-491a-8515-4217ac8cac14-agent-certs\") pod \"konnectivity-agent-x2dtl\" (UID: \"fd82faec-581a-491a-8515-4217ac8cac14\") " pod="kube-system/konnectivity-agent-x2dtl"
Jul 31 23:46:23.486354 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.485794    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d997aba4-5c6e-49ad-81bd-95efee82e1ff-etc-modprobe-d\") pod \"tuned-h5jbv\" (UID: \"d997aba4-5c6e-49ad-81bd-95efee82e1ff\") " pod="openshift-cluster-node-tuning-operator/tuned-h5jbv"
Jul 31 23:46:23.486354 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.485807    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/69e78750-573b-4dbd-a585-cf0d88056f4f-dbus\") pod \"global-pull-secret-syncer-h45s5\" (UID: \"69e78750-573b-4dbd-a585-cf0d88056f4f\") " pod="kube-system/global-pull-secret-syncer-h45s5"
Jul 31 23:46:23.486354 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.485830    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c0bfad85-f7bf-4457-8523-6a1aa966a280-host-slash\") pod \"iptables-alerter-7ttkn\" (UID: \"c0bfad85-f7bf-4457-8523-6a1aa966a280\") " pod="openshift-network-operator/iptables-alerter-7ttkn"
Jul 31 23:46:23.486354 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.485854    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/1505ec5c-3dd6-4199-8f53-a93fa90f91b7-socket-dir\") pod \"aws-ebs-csi-driver-node-4dfd9\" (UID: \"1505ec5c-3dd6-4199-8f53-a93fa90f91b7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4dfd9"
Jul 31 23:46:23.486354 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.485878    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/1505ec5c-3dd6-4199-8f53-a93fa90f91b7-sys-fs\") pod \"aws-ebs-csi-driver-node-4dfd9\" (UID: \"1505ec5c-3dd6-4199-8f53-a93fa90f91b7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4dfd9"
Jul 31 23:46:23.487150 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.485901    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tmrvx\" (UniqueName: \"kubernetes.io/projected/1505ec5c-3dd6-4199-8f53-a93fa90f91b7-kube-api-access-tmrvx\") pod \"aws-ebs-csi-driver-node-4dfd9\" (UID: \"1505ec5c-3dd6-4199-8f53-a93fa90f91b7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4dfd9"
Jul 31 23:46:23.487150 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.485913    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c0bfad85-f7bf-4457-8523-6a1aa966a280-host-slash\") pod \"iptables-alerter-7ttkn\" (UID: \"c0bfad85-f7bf-4457-8523-6a1aa966a280\") " pod="openshift-network-operator/iptables-alerter-7ttkn"
Jul 31 23:46:23.487150 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.485925    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/fd82faec-581a-491a-8515-4217ac8cac14-konnectivity-ca\") pod \"konnectivity-agent-x2dtl\" (UID: \"fd82faec-581a-491a-8515-4217ac8cac14\") " pod="kube-system/konnectivity-agent-x2dtl"
Jul 31 23:46:23.487150 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.485940    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/69e78750-573b-4dbd-a585-cf0d88056f4f-dbus\") pod \"global-pull-secret-syncer-h45s5\" (UID: \"69e78750-573b-4dbd-a585-cf0d88056f4f\") " pod="kube-system/global-pull-secret-syncer-h45s5"
Jul 31 23:46:23.487150 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.485947    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/1505ec5c-3dd6-4199-8f53-a93fa90f91b7-sys-fs\") pod \"aws-ebs-csi-driver-node-4dfd9\" (UID: \"1505ec5c-3dd6-4199-8f53-a93fa90f91b7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4dfd9"
Jul 31 23:46:23.487150 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.486018    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/1505ec5c-3dd6-4199-8f53-a93fa90f91b7-socket-dir\") pod \"aws-ebs-csi-driver-node-4dfd9\" (UID: \"1505ec5c-3dd6-4199-8f53-a93fa90f91b7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4dfd9"
Jul 31 23:46:23.487150 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.486045    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4e0b1a1a-3553-464e-8094-45a9dbf1cb66-cni-binary-copy\") pod \"multus-additional-cni-plugins-jtsl2\" (UID: \"4e0b1a1a-3553-464e-8094-45a9dbf1cb66\") " pod="openshift-multus/multus-additional-cni-plugins-jtsl2"
Jul 31 23:46:23.487150 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.486078    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c0bfad85-f7bf-4457-8523-6a1aa966a280-iptables-alerter-script\") pod \"iptables-alerter-7ttkn\" (UID: \"c0bfad85-f7bf-4457-8523-6a1aa966a280\") " pod="openshift-network-operator/iptables-alerter-7ttkn"
Jul 31 23:46:23.487150 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.486104    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d997aba4-5c6e-49ad-81bd-95efee82e1ff-etc-sysctl-d\") pod \"tuned-h5jbv\" (UID: \"d997aba4-5c6e-49ad-81bd-95efee82e1ff\") " pod="openshift-cluster-node-tuning-operator/tuned-h5jbv"
Jul 31 23:46:23.487150 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.486131    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b0541f63-78c6-4ae2-b6aa-3e7ad98eb151-os-release\") pod \"multus-jw7ld\" (UID: \"b0541f63-78c6-4ae2-b6aa-3e7ad98eb151\") " pod="openshift-multus/multus-jw7ld"
Jul 31 23:46:23.487150 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.486157    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2bd960e3-3638-49f5-bd70-96891131423e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-j5545\" (UID: \"2bd960e3-3638-49f5-bd70-96891131423e\") " pod="openshift-ovn-kubernetes/ovnkube-node-j5545"
Jul 31 23:46:23.487150 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.486145    2564 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 31 23:46:23.487150 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.486185    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/2bd960e3-3638-49f5-bd70-96891131423e-env-overrides\") pod \"ovnkube-node-j5545\" (UID: \"2bd960e3-3638-49f5-bd70-96891131423e\") " pod="openshift-ovn-kubernetes/ovnkube-node-j5545"
Jul 31 23:46:23.487150 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.486207    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d997aba4-5c6e-49ad-81bd-95efee82e1ff-tmp\") pod \"tuned-h5jbv\" (UID: \"d997aba4-5c6e-49ad-81bd-95efee82e1ff\") " pod="openshift-cluster-node-tuning-operator/tuned-h5jbv"
Jul 31 23:46:23.487150 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.486212    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b0541f63-78c6-4ae2-b6aa-3e7ad98eb151-os-release\") pod \"multus-jw7ld\" (UID: \"b0541f63-78c6-4ae2-b6aa-3e7ad98eb151\") " pod="openshift-multus/multus-jw7ld"
Jul 31 23:46:23.487150 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.486220    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b0541f63-78c6-4ae2-b6aa-3e7ad98eb151-multus-daemon-config\") pod \"multus-jw7ld\" (UID: \"b0541f63-78c6-4ae2-b6aa-3e7ad98eb151\") " pod="openshift-multus/multus-jw7ld"
Jul 31 23:46:23.487150 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.486234    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d997aba4-5c6e-49ad-81bd-95efee82e1ff-etc-sysctl-d\") pod \"tuned-h5jbv\" (UID: \"d997aba4-5c6e-49ad-81bd-95efee82e1ff\") " pod="openshift-cluster-node-tuning-operator/tuned-h5jbv"
Jul 31 23:46:23.487150 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.486228    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d98fe0c2-0a27-408e-8131-252e3d469281-host\") pod \"node-ca-ls2gt\" (UID: \"d98fe0c2-0a27-408e-8131-252e3d469281\") " pod="openshift-image-registry/node-ca-ls2gt"
Jul 31 23:46:23.488053 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.486263    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d98fe0c2-0a27-408e-8131-252e3d469281-host\") pod \"node-ca-ls2gt\" (UID: \"d98fe0c2-0a27-408e-8131-252e3d469281\") " pod="openshift-image-registry/node-ca-ls2gt"
Jul 31 23:46:23.488053 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.486275    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mpwsh\" (UniqueName: \"kubernetes.io/projected/d98fe0c2-0a27-408e-8131-252e3d469281-kube-api-access-mpwsh\") pod \"node-ca-ls2gt\" (UID: \"d98fe0c2-0a27-408e-8131-252e3d469281\") " pod="openshift-image-registry/node-ca-ls2gt"
Jul 31 23:46:23.488053 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.486283    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2bd960e3-3638-49f5-bd70-96891131423e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-j5545\" (UID: \"2bd960e3-3638-49f5-bd70-96891131423e\") " pod="openshift-ovn-kubernetes/ovnkube-node-j5545"
Jul 31 23:46:23.488053 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.486303    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b0541f63-78c6-4ae2-b6aa-3e7ad98eb151-host-run-multus-certs\") pod \"multus-jw7ld\" (UID: \"b0541f63-78c6-4ae2-b6aa-3e7ad98eb151\") " pod="openshift-multus/multus-jw7ld"
Jul 31 23:46:23.488053 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.486328    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4e0b1a1a-3553-464e-8094-45a9dbf1cb66-tuning-conf-dir\") pod \"multus-additional-cni-plugins-jtsl2\" (UID: \"4e0b1a1a-3553-464e-8094-45a9dbf1cb66\") " pod="openshift-multus/multus-additional-cni-plugins-jtsl2"
Jul 31 23:46:23.488053 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.486354    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2bd960e3-3638-49f5-bd70-96891131423e-host-run-netns\") pod \"ovnkube-node-j5545\" (UID: \"2bd960e3-3638-49f5-bd70-96891131423e\") " pod="openshift-ovn-kubernetes/ovnkube-node-j5545"
Jul 31 23:46:23.488053 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.486380    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/2bd960e3-3638-49f5-bd70-96891131423e-ovnkube-script-lib\") pod \"ovnkube-node-j5545\" (UID: \"2bd960e3-3638-49f5-bd70-96891131423e\") " pod="openshift-ovn-kubernetes/ovnkube-node-j5545"
Jul 31 23:46:23.488053 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.486396    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/fd82faec-581a-491a-8515-4217ac8cac14-konnectivity-ca\") pod \"konnectivity-agent-x2dtl\" (UID: \"fd82faec-581a-491a-8515-4217ac8cac14\") " pod="kube-system/konnectivity-agent-x2dtl"
Jul 31 23:46:23.488053 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.486405    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b0541f63-78c6-4ae2-b6aa-3e7ad98eb151-multus-cni-dir\") pod \"multus-jw7ld\" (UID: \"b0541f63-78c6-4ae2-b6aa-3e7ad98eb151\") " pod="openshift-multus/multus-jw7ld"
Jul 31 23:46:23.488053 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.486465    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b0541f63-78c6-4ae2-b6aa-3e7ad98eb151-multus-socket-dir-parent\") pod \"multus-jw7ld\" (UID: \"b0541f63-78c6-4ae2-b6aa-3e7ad98eb151\") " pod="openshift-multus/multus-jw7ld"
Jul 31 23:46:23.488053 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.486494    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b0541f63-78c6-4ae2-b6aa-3e7ad98eb151-host-var-lib-cni-multus\") pod \"multus-jw7ld\" (UID: \"b0541f63-78c6-4ae2-b6aa-3e7ad98eb151\") " pod="openshift-multus/multus-jw7ld"
Jul 31 23:46:23.488053 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.486509    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b0541f63-78c6-4ae2-b6aa-3e7ad98eb151-multus-cni-dir\") pod \"multus-jw7ld\" (UID: \"b0541f63-78c6-4ae2-b6aa-3e7ad98eb151\") " pod="openshift-multus/multus-jw7ld"
Jul 31 23:46:23.488053 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.486519    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b0541f63-78c6-4ae2-b6aa-3e7ad98eb151-etc-kubernetes\") pod \"multus-jw7ld\" (UID: \"b0541f63-78c6-4ae2-b6aa-3e7ad98eb151\") " pod="openshift-multus/multus-jw7ld"
Jul 31 23:46:23.488053 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.486541    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2bd960e3-3638-49f5-bd70-96891131423e-host-run-netns\") pod \"ovnkube-node-j5545\" (UID: \"2bd960e3-3638-49f5-bd70-96891131423e\") " pod="openshift-ovn-kubernetes/ovnkube-node-j5545"
Jul 31 23:46:23.488053 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.486545    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z9qrg\" (UniqueName: \"kubernetes.io/projected/b0541f63-78c6-4ae2-b6aa-3e7ad98eb151-kube-api-access-z9qrg\") pod \"multus-jw7ld\" (UID: \"b0541f63-78c6-4ae2-b6aa-3e7ad98eb151\") " pod="openshift-multus/multus-jw7ld"
Jul 31 23:46:23.488053 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.486576    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b0541f63-78c6-4ae2-b6aa-3e7ad98eb151-host-run-multus-certs\") pod \"multus-jw7ld\" (UID: \"b0541f63-78c6-4ae2-b6aa-3e7ad98eb151\") " pod="openshift-multus/multus-jw7ld"
Jul 31 23:46:23.488053 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.486595    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b0541f63-78c6-4ae2-b6aa-3e7ad98eb151-multus-socket-dir-parent\") pod \"multus-jw7ld\" (UID: \"b0541f63-78c6-4ae2-b6aa-3e7ad98eb151\") " pod="openshift-multus/multus-jw7ld"
Jul 31 23:46:23.488857 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.486610    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/69e78750-573b-4dbd-a585-cf0d88056f4f-kubelet-config\") pod \"global-pull-secret-syncer-h45s5\" (UID: \"69e78750-573b-4dbd-a585-cf0d88056f4f\") " pod="kube-system/global-pull-secret-syncer-h45s5"
Jul 31 23:46:23.488857 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.486614    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/2bd960e3-3638-49f5-bd70-96891131423e-env-overrides\") pod \"ovnkube-node-j5545\" (UID: \"2bd960e3-3638-49f5-bd70-96891131423e\") " pod="openshift-ovn-kubernetes/ovnkube-node-j5545"
Jul 31 23:46:23.488857 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.486667    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2bd960e3-3638-49f5-bd70-96891131423e-host-run-ovn-kubernetes\") pod \"ovnkube-node-j5545\" (UID: \"2bd960e3-3638-49f5-bd70-96891131423e\") " pod="openshift-ovn-kubernetes/ovnkube-node-j5545"
Jul 31 23:46:23.488857 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.486681    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b0541f63-78c6-4ae2-b6aa-3e7ad98eb151-etc-kubernetes\") pod \"multus-jw7ld\" (UID: \"b0541f63-78c6-4ae2-b6aa-3e7ad98eb151\") " pod="openshift-multus/multus-jw7ld"
Jul 31 23:46:23.488857 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.486694    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/1505ec5c-3dd6-4199-8f53-a93fa90f91b7-kubelet-dir\") pod \"aws-ebs-csi-driver-node-4dfd9\" (UID: \"1505ec5c-3dd6-4199-8f53-a93fa90f91b7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4dfd9"
Jul 31 23:46:23.488857 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.486681    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c0bfad85-f7bf-4457-8523-6a1aa966a280-iptables-alerter-script\") pod \"iptables-alerter-7ttkn\" (UID: \"c0bfad85-f7bf-4457-8523-6a1aa966a280\") " pod="openshift-network-operator/iptables-alerter-7ttkn"
Jul 31 23:46:23.488857 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.486722    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/69e78750-573b-4dbd-a585-cf0d88056f4f-kubelet-config\") pod \"global-pull-secret-syncer-h45s5\" (UID: \"69e78750-573b-4dbd-a585-cf0d88056f4f\") " pod="kube-system/global-pull-secret-syncer-h45s5"
Jul 31 23:46:23.488857 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.486725    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4e0b1a1a-3553-464e-8094-45a9dbf1cb66-tuning-conf-dir\") pod \"multus-additional-cni-plugins-jtsl2\" (UID: \"4e0b1a1a-3553-464e-8094-45a9dbf1cb66\") " pod="openshift-multus/multus-additional-cni-plugins-jtsl2"
Jul 31 23:46:23.488857 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.486730    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b0541f63-78c6-4ae2-b6aa-3e7ad98eb151-cnibin\") pod \"multus-jw7ld\" (UID: \"b0541f63-78c6-4ae2-b6aa-3e7ad98eb151\") " pod="openshift-multus/multus-jw7ld"
Jul 31 23:46:23.488857 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.486765    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b0541f63-78c6-4ae2-b6aa-3e7ad98eb151-multus-conf-dir\") pod \"multus-jw7ld\" (UID: \"b0541f63-78c6-4ae2-b6aa-3e7ad98eb151\") " pod="openshift-multus/multus-jw7ld"
Jul 31 23:46:23.488857 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.486769    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b0541f63-78c6-4ae2-b6aa-3e7ad98eb151-host-var-lib-cni-multus\") pod \"multus-jw7ld\" (UID: \"b0541f63-78c6-4ae2-b6aa-3e7ad98eb151\") " pod="openshift-multus/multus-jw7ld"
Jul 31 23:46:23.488857 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.486767    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/1505ec5c-3dd6-4199-8f53-a93fa90f91b7-kubelet-dir\") pod \"aws-ebs-csi-driver-node-4dfd9\" (UID: \"1505ec5c-3dd6-4199-8f53-a93fa90f91b7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4dfd9"
Jul 31 23:46:23.488857 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.486804    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b0541f63-78c6-4ae2-b6aa-3e7ad98eb151-cnibin\") pod \"multus-jw7ld\" (UID: \"b0541f63-78c6-4ae2-b6aa-3e7ad98eb151\") " pod="openshift-multus/multus-jw7ld"
Jul 31 23:46:23.488857 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.486807    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b0541f63-78c6-4ae2-b6aa-3e7ad98eb151-multus-conf-dir\") pod \"multus-jw7ld\" (UID: \"b0541f63-78c6-4ae2-b6aa-3e7ad98eb151\") " pod="openshift-multus/multus-jw7ld"
Jul 31 23:46:23.488857 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.486805    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2bd960e3-3638-49f5-bd70-96891131423e-host-run-ovn-kubernetes\") pod \"ovnkube-node-j5545\" (UID: \"2bd960e3-3638-49f5-bd70-96891131423e\") " pod="openshift-ovn-kubernetes/ovnkube-node-j5545"
Jul 31 23:46:23.488857 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.486896    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/69e78750-573b-4dbd-a585-cf0d88056f4f-original-pull-secret\") pod \"global-pull-secret-syncer-h45s5\" (UID: \"69e78750-573b-4dbd-a585-cf0d88056f4f\") " pod="kube-system/global-pull-secret-syncer-h45s5"
Jul 31 23:46:23.488857 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.486945    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/2bd960e3-3638-49f5-bd70-96891131423e-run-systemd\") pod \"ovnkube-node-j5545\" (UID: \"2bd960e3-3638-49f5-bd70-96891131423e\") " pod="openshift-ovn-kubernetes/ovnkube-node-j5545"
Jul 31 23:46:23.488857 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:46:23.486994    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 31 23:46:23.489740 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.487031    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/1505ec5c-3dd6-4199-8f53-a93fa90f91b7-etc-selinux\") pod \"aws-ebs-csi-driver-node-4dfd9\" (UID: \"1505ec5c-3dd6-4199-8f53-a93fa90f91b7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4dfd9"
Jul 31 23:46:23.489740 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.487050    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/2bd960e3-3638-49f5-bd70-96891131423e-run-systemd\") pod \"ovnkube-node-j5545\" (UID: \"2bd960e3-3638-49f5-bd70-96891131423e\") " pod="openshift-ovn-kubernetes/ovnkube-node-j5545"
Jul 31 23:46:23.489740 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.487090    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d997aba4-5c6e-49ad-81bd-95efee82e1ff-etc-systemd\") pod \"tuned-h5jbv\" (UID: \"d997aba4-5c6e-49ad-81bd-95efee82e1ff\") " pod="openshift-cluster-node-tuning-operator/tuned-h5jbv"
Jul 31 23:46:23.489740 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:46:23.487131    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/69e78750-573b-4dbd-a585-cf0d88056f4f-original-pull-secret podName:69e78750-573b-4dbd-a585-cf0d88056f4f nodeName:}" failed. No retries permitted until 2026-07-31 23:46:23.987111599 +0000 UTC m=+2.099239269 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/69e78750-573b-4dbd-a585-cf0d88056f4f-original-pull-secret") pod "global-pull-secret-syncer-h45s5" (UID: "69e78750-573b-4dbd-a585-cf0d88056f4f") : object "kube-system"/"original-pull-secret" not registered
Jul 31 23:46:23.489740 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.487143    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d997aba4-5c6e-49ad-81bd-95efee82e1ff-etc-systemd\") pod \"tuned-h5jbv\" (UID: \"d997aba4-5c6e-49ad-81bd-95efee82e1ff\") " pod="openshift-cluster-node-tuning-operator/tuned-h5jbv"
Jul 31 23:46:23.489740 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.487152    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b0541f63-78c6-4ae2-b6aa-3e7ad98eb151-cni-binary-copy\") pod \"multus-jw7ld\" (UID: \"b0541f63-78c6-4ae2-b6aa-3e7ad98eb151\") " pod="openshift-multus/multus-jw7ld"
Jul 31 23:46:23.489740 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.487179    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/816ea407-9c93-4455-8202-8229340dc7db-hosts-file\") pod \"node-resolver-wxz8k\" (UID: \"816ea407-9c93-4455-8202-8229340dc7db\") " pod="openshift-dns/node-resolver-wxz8k"
Jul 31 23:46:23.489740 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.487210    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/1505ec5c-3dd6-4199-8f53-a93fa90f91b7-etc-selinux\") pod \"aws-ebs-csi-driver-node-4dfd9\" (UID: \"1505ec5c-3dd6-4199-8f53-a93fa90f91b7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4dfd9"
Jul 31 23:46:23.489740 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.487281    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/815ee183-a99c-4636-9937-da826945f531-metrics-certs\") pod \"network-metrics-daemon-j97wn\" (UID: \"815ee183-a99c-4636-9937-da826945f531\") " pod="openshift-multus/network-metrics-daemon-j97wn"
Jul 31 23:46:23.489740 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.487322    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/816ea407-9c93-4455-8202-8229340dc7db-hosts-file\") pod \"node-resolver-wxz8k\" (UID: \"816ea407-9c93-4455-8202-8229340dc7db\") " pod="openshift-dns/node-resolver-wxz8k"
Jul 31 23:46:23.489740 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.487349    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2bd960e3-3638-49f5-bd70-96891131423e-var-lib-openvswitch\") pod \"ovnkube-node-j5545\" (UID: \"2bd960e3-3638-49f5-bd70-96891131423e\") " pod="openshift-ovn-kubernetes/ovnkube-node-j5545"
Jul 31 23:46:23.489740 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.487366    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/2bd960e3-3638-49f5-bd70-96891131423e-ovnkube-script-lib\") pod \"ovnkube-node-j5545\" (UID: \"2bd960e3-3638-49f5-bd70-96891131423e\") " pod="openshift-ovn-kubernetes/ovnkube-node-j5545"
Jul 31 23:46:23.489740 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.487371    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2bd960e3-3638-49f5-bd70-96891131423e-run-openvswitch\") pod \"ovnkube-node-j5545\" (UID: \"2bd960e3-3638-49f5-bd70-96891131423e\") " pod="openshift-ovn-kubernetes/ovnkube-node-j5545"
Jul 31 23:46:23.489740 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.487401    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2bd960e3-3638-49f5-bd70-96891131423e-run-openvswitch\") pod \"ovnkube-node-j5545\" (UID: \"2bd960e3-3638-49f5-bd70-96891131423e\") " pod="openshift-ovn-kubernetes/ovnkube-node-j5545"
Jul 31 23:46:23.489740 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.487411    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d997aba4-5c6e-49ad-81bd-95efee82e1ff-etc-sysctl-conf\") pod \"tuned-h5jbv\" (UID: \"d997aba4-5c6e-49ad-81bd-95efee82e1ff\") " pod="openshift-cluster-node-tuning-operator/tuned-h5jbv"
Jul 31 23:46:23.489740 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.487437    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2bd960e3-3638-49f5-bd70-96891131423e-var-lib-openvswitch\") pod \"ovnkube-node-j5545\" (UID: \"2bd960e3-3638-49f5-bd70-96891131423e\") " pod="openshift-ovn-kubernetes/ovnkube-node-j5545"
Jul 31 23:46:23.489740 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.487446    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4e0b1a1a-3553-464e-8094-45a9dbf1cb66-os-release\") pod \"multus-additional-cni-plugins-jtsl2\" (UID: \"4e0b1a1a-3553-464e-8094-45a9dbf1cb66\") " pod="openshift-multus/multus-additional-cni-plugins-jtsl2"
Jul 31 23:46:23.490467 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:46:23.487451    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 31 23:46:23.490467 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.487471    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/2bd960e3-3638-49f5-bd70-96891131423e-host-kubelet\") pod \"ovnkube-node-j5545\" (UID: \"2bd960e3-3638-49f5-bd70-96891131423e\") " pod="openshift-ovn-kubernetes/ovnkube-node-j5545"
Jul 31 23:46:23.490467 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:46:23.487501    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/815ee183-a99c-4636-9937-da826945f531-metrics-certs podName:815ee183-a99c-4636-9937-da826945f531 nodeName:}" failed. No retries permitted until 2026-07-31 23:46:23.987484339 +0000 UTC m=+2.099612004 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/815ee183-a99c-4636-9937-da826945f531-metrics-certs") pod "network-metrics-daemon-j97wn" (UID: "815ee183-a99c-4636-9937-da826945f531") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 31 23:46:23.490467 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.487505    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/2bd960e3-3638-49f5-bd70-96891131423e-host-kubelet\") pod \"ovnkube-node-j5545\" (UID: \"2bd960e3-3638-49f5-bd70-96891131423e\") " pod="openshift-ovn-kubernetes/ovnkube-node-j5545"
Jul 31 23:46:23.490467 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.487521    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/2bd960e3-3638-49f5-bd70-96891131423e-systemd-units\") pod \"ovnkube-node-j5545\" (UID: \"2bd960e3-3638-49f5-bd70-96891131423e\") " pod="openshift-ovn-kubernetes/ovnkube-node-j5545"
Jul 31 23:46:23.490467 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.487534    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d997aba4-5c6e-49ad-81bd-95efee82e1ff-etc-sysctl-conf\") pod \"tuned-h5jbv\" (UID: \"d997aba4-5c6e-49ad-81bd-95efee82e1ff\") " pod="openshift-cluster-node-tuning-operator/tuned-h5jbv"
Jul 31 23:46:23.490467 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.487557    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d997aba4-5c6e-49ad-81bd-95efee82e1ff-etc-kubernetes\") pod \"tuned-h5jbv\" (UID: \"d997aba4-5c6e-49ad-81bd-95efee82e1ff\") " pod="openshift-cluster-node-tuning-operator/tuned-h5jbv"
Jul 31 23:46:23.490467 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.487568    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4e0b1a1a-3553-464e-8094-45a9dbf1cb66-os-release\") pod \"multus-additional-cni-plugins-jtsl2\" (UID: \"4e0b1a1a-3553-464e-8094-45a9dbf1cb66\") " pod="openshift-multus/multus-additional-cni-plugins-jtsl2"
Jul 31 23:46:23.490467 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.487575    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b0541f63-78c6-4ae2-b6aa-3e7ad98eb151-cni-binary-copy\") pod \"multus-jw7ld\" (UID: \"b0541f63-78c6-4ae2-b6aa-3e7ad98eb151\") " pod="openshift-multus/multus-jw7ld"
Jul 31 23:46:23.490467 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.487584    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d997aba4-5c6e-49ad-81bd-95efee82e1ff-lib-modules\") pod \"tuned-h5jbv\" (UID: \"d997aba4-5c6e-49ad-81bd-95efee82e1ff\") " pod="openshift-cluster-node-tuning-operator/tuned-h5jbv"
Jul 31 23:46:23.490467 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.487588    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/2bd960e3-3638-49f5-bd70-96891131423e-systemd-units\") pod \"ovnkube-node-j5545\" (UID: \"2bd960e3-3638-49f5-bd70-96891131423e\") " pod="openshift-ovn-kubernetes/ovnkube-node-j5545"
Jul 31 23:46:23.490467 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.487618    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d997aba4-5c6e-49ad-81bd-95efee82e1ff-etc-kubernetes\") pod \"tuned-h5jbv\" (UID: \"d997aba4-5c6e-49ad-81bd-95efee82e1ff\") " pod="openshift-cluster-node-tuning-operator/tuned-h5jbv"
Jul 31 23:46:23.490467 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.487649    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d997aba4-5c6e-49ad-81bd-95efee82e1ff-etc-tuned\") pod \"tuned-h5jbv\" (UID: \"d997aba4-5c6e-49ad-81bd-95efee82e1ff\") " pod="openshift-cluster-node-tuning-operator/tuned-h5jbv"
Jul 31 23:46:23.490467 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.487674    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d98fe0c2-0a27-408e-8131-252e3d469281-serviceca\") pod \"node-ca-ls2gt\" (UID: \"d98fe0c2-0a27-408e-8131-252e3d469281\") " pod="openshift-image-registry/node-ca-ls2gt"
Jul 31 23:46:23.490467 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.487699    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/816ea407-9c93-4455-8202-8229340dc7db-tmp-dir\") pod \"node-resolver-wxz8k\" (UID: \"816ea407-9c93-4455-8202-8229340dc7db\") " pod="openshift-dns/node-resolver-wxz8k"
Jul 31 23:46:23.490467 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.487716    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d997aba4-5c6e-49ad-81bd-95efee82e1ff-lib-modules\") pod \"tuned-h5jbv\" (UID: \"d997aba4-5c6e-49ad-81bd-95efee82e1ff\") " pod="openshift-cluster-node-tuning-operator/tuned-h5jbv"
Jul 31 23:46:23.490467 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.487741    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/1505ec5c-3dd6-4199-8f53-a93fa90f91b7-registration-dir\") pod \"aws-ebs-csi-driver-node-4dfd9\" (UID: \"1505ec5c-3dd6-4199-8f53-a93fa90f91b7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4dfd9"
Jul 31 23:46:23.490967 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.487768    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4e0b1a1a-3553-464e-8094-45a9dbf1cb66-cnibin\") pod \"multus-additional-cni-plugins-jtsl2\" (UID: \"4e0b1a1a-3553-464e-8094-45a9dbf1cb66\") " pod="openshift-multus/multus-additional-cni-plugins-jtsl2"
Jul 31 23:46:23.490967 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.487791    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2bd960e3-3638-49f5-bd70-96891131423e-host-slash\") pod \"ovnkube-node-j5545\" (UID: \"2bd960e3-3638-49f5-bd70-96891131423e\") " pod="openshift-ovn-kubernetes/ovnkube-node-j5545"
Jul 31 23:46:23.490967 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.487814    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/1505ec5c-3dd6-4199-8f53-a93fa90f91b7-device-dir\") pod \"aws-ebs-csi-driver-node-4dfd9\" (UID: \"1505ec5c-3dd6-4199-8f53-a93fa90f91b7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4dfd9"
Jul 31 23:46:23.490967 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.487836    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d997aba4-5c6e-49ad-81bd-95efee82e1ff-run\") pod \"tuned-h5jbv\" (UID: \"d997aba4-5c6e-49ad-81bd-95efee82e1ff\") " pod="openshift-cluster-node-tuning-operator/tuned-h5jbv"
Jul 31 23:46:23.490967 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.487859    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b0541f63-78c6-4ae2-b6aa-3e7ad98eb151-system-cni-dir\") pod \"multus-jw7ld\" (UID: \"b0541f63-78c6-4ae2-b6aa-3e7ad98eb151\") " pod="openshift-multus/multus-jw7ld"
Jul 31 23:46:23.490967 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.487886    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4e0b1a1a-3553-464e-8094-45a9dbf1cb66-cnibin\") pod \"multus-additional-cni-plugins-jtsl2\" (UID: \"4e0b1a1a-3553-464e-8094-45a9dbf1cb66\") " pod="openshift-multus/multus-additional-cni-plugins-jtsl2"
Jul 31 23:46:23.490967 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.487905    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/2bd960e3-3638-49f5-bd70-96891131423e-ovn-node-metrics-cert\") pod \"ovnkube-node-j5545\" (UID: \"2bd960e3-3638-49f5-bd70-96891131423e\") " pod="openshift-ovn-kubernetes/ovnkube-node-j5545"
Jul 31 23:46:23.490967 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.487930    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d997aba4-5c6e-49ad-81bd-95efee82e1ff-sys\") pod \"tuned-h5jbv\" (UID: \"d997aba4-5c6e-49ad-81bd-95efee82e1ff\") " pod="openshift-cluster-node-tuning-operator/tuned-h5jbv"
Jul 31 23:46:23.490967 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.487955    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j28hp\" (UniqueName: \"kubernetes.io/projected/816ea407-9c93-4455-8202-8229340dc7db-kube-api-access-j28hp\") pod \"node-resolver-wxz8k\" (UID: \"816ea407-9c93-4455-8202-8229340dc7db\") " pod="openshift-dns/node-resolver-wxz8k"
Jul 31 23:46:23.490967 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.488000    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/4e0b1a1a-3553-464e-8094-45a9dbf1cb66-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-jtsl2\" (UID: \"4e0b1a1a-3553-464e-8094-45a9dbf1cb66\") " pod="openshift-multus/multus-additional-cni-plugins-jtsl2"
Jul 31 23:46:23.490967 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.488027    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2bd960e3-3638-49f5-bd70-96891131423e-host-slash\") pod \"ovnkube-node-j5545\" (UID: \"2bd960e3-3638-49f5-bd70-96891131423e\") " pod="openshift-ovn-kubernetes/ovnkube-node-j5545"
Jul 31 23:46:23.490967 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.488037    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2bd960e3-3638-49f5-bd70-96891131423e-etc-openvswitch\") pod \"ovnkube-node-j5545\" (UID: \"2bd960e3-3638-49f5-bd70-96891131423e\") " pod="openshift-ovn-kubernetes/ovnkube-node-j5545"
Jul 31 23:46:23.490967 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.488115    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d997aba4-5c6e-49ad-81bd-95efee82e1ff-run\") pod \"tuned-h5jbv\" (UID: \"d997aba4-5c6e-49ad-81bd-95efee82e1ff\") " pod="openshift-cluster-node-tuning-operator/tuned-h5jbv"
Jul 31 23:46:23.490967 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.488132    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d98fe0c2-0a27-408e-8131-252e3d469281-serviceca\") pod \"node-ca-ls2gt\" (UID: \"d98fe0c2-0a27-408e-8131-252e3d469281\") " pod="openshift-image-registry/node-ca-ls2gt"
Jul 31 23:46:23.490967 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.488141    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/816ea407-9c93-4455-8202-8229340dc7db-tmp-dir\") pod \"node-resolver-wxz8k\" (UID: \"816ea407-9c93-4455-8202-8229340dc7db\") " pod="openshift-dns/node-resolver-wxz8k"
Jul 31 23:46:23.490967 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.488188    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/1505ec5c-3dd6-4199-8f53-a93fa90f91b7-device-dir\") pod \"aws-ebs-csi-driver-node-4dfd9\" (UID: \"1505ec5c-3dd6-4199-8f53-a93fa90f91b7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4dfd9"
Jul 31 23:46:23.490967 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.488194    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/1505ec5c-3dd6-4199-8f53-a93fa90f91b7-registration-dir\") pod \"aws-ebs-csi-driver-node-4dfd9\" (UID: \"1505ec5c-3dd6-4199-8f53-a93fa90f91b7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4dfd9"
Jul 31 23:46:23.490967 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.488243    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d997aba4-5c6e-49ad-81bd-95efee82e1ff-sys\") pod \"tuned-h5jbv\" (UID: \"d997aba4-5c6e-49ad-81bd-95efee82e1ff\") " pod="openshift-cluster-node-tuning-operator/tuned-h5jbv"
Jul 31 23:46:23.491476 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.488252    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2bd960e3-3638-49f5-bd70-96891131423e-etc-openvswitch\") pod \"ovnkube-node-j5545\" (UID: \"2bd960e3-3638-49f5-bd70-96891131423e\") " pod="openshift-ovn-kubernetes/ovnkube-node-j5545"
Jul 31 23:46:23.491476 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.488293    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b0541f63-78c6-4ae2-b6aa-3e7ad98eb151-system-cni-dir\") pod \"multus-jw7ld\" (UID: \"b0541f63-78c6-4ae2-b6aa-3e7ad98eb151\") " pod="openshift-multus/multus-jw7ld"
Jul 31 23:46:23.491476 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.488680    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/4e0b1a1a-3553-464e-8094-45a9dbf1cb66-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-jtsl2\" (UID: \"4e0b1a1a-3553-464e-8094-45a9dbf1cb66\") " pod="openshift-multus/multus-additional-cni-plugins-jtsl2"
Jul 31 23:46:23.491476 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.488773    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4e0b1a1a-3553-464e-8094-45a9dbf1cb66-cni-binary-copy\") pod \"multus-additional-cni-plugins-jtsl2\" (UID: \"4e0b1a1a-3553-464e-8094-45a9dbf1cb66\") " pod="openshift-multus/multus-additional-cni-plugins-jtsl2"
Jul 31 23:46:23.491476 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.489696    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d997aba4-5c6e-49ad-81bd-95efee82e1ff-tmp\") pod \"tuned-h5jbv\" (UID: \"d997aba4-5c6e-49ad-81bd-95efee82e1ff\") " pod="openshift-cluster-node-tuning-operator/tuned-h5jbv"
Jul 31 23:46:23.491476 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.489782    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d997aba4-5c6e-49ad-81bd-95efee82e1ff-etc-tuned\") pod \"tuned-h5jbv\" (UID: \"d997aba4-5c6e-49ad-81bd-95efee82e1ff\") " pod="openshift-cluster-node-tuning-operator/tuned-h5jbv"
Jul 31 23:46:23.491476 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.489999    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/fd82faec-581a-491a-8515-4217ac8cac14-agent-certs\") pod \"konnectivity-agent-x2dtl\" (UID: \"fd82faec-581a-491a-8515-4217ac8cac14\") " pod="kube-system/konnectivity-agent-x2dtl"
Jul 31 23:46:23.491476 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.490479    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/2bd960e3-3638-49f5-bd70-96891131423e-ovn-node-metrics-cert\") pod \"ovnkube-node-j5545\" (UID: \"2bd960e3-3638-49f5-bd70-96891131423e\") " pod="openshift-ovn-kubernetes/ovnkube-node-j5545"
Jul 31 23:46:23.497655 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.497615    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8ktfb\" (UniqueName: \"kubernetes.io/projected/4e0b1a1a-3553-464e-8094-45a9dbf1cb66-kube-api-access-8ktfb\") pod \"multus-additional-cni-plugins-jtsl2\" (UID: \"4e0b1a1a-3553-464e-8094-45a9dbf1cb66\") " pod="openshift-multus/multus-additional-cni-plugins-jtsl2"
Jul 31 23:46:23.500223 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.500187    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nxsfk\" (UniqueName: \"kubernetes.io/projected/815ee183-a99c-4636-9937-da826945f531-kube-api-access-nxsfk\") pod \"network-metrics-daemon-j97wn\" (UID: \"815ee183-a99c-4636-9937-da826945f531\") " pod="openshift-multus/network-metrics-daemon-j97wn"
Jul 31 23:46:23.500589 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.500487    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ltc7q\" (UniqueName: \"kubernetes.io/projected/d997aba4-5c6e-49ad-81bd-95efee82e1ff-kube-api-access-ltc7q\") pod \"tuned-h5jbv\" (UID: \"d997aba4-5c6e-49ad-81bd-95efee82e1ff\") " pod="openshift-cluster-node-tuning-operator/tuned-h5jbv"
Jul 31 23:46:23.500589 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.500540    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8nrs5\" (UniqueName: \"kubernetes.io/projected/c0bfad85-f7bf-4457-8523-6a1aa966a280-kube-api-access-8nrs5\") pod \"iptables-alerter-7ttkn\" (UID: \"c0bfad85-f7bf-4457-8523-6a1aa966a280\") " pod="openshift-network-operator/iptables-alerter-7ttkn"
Jul 31 23:46:23.500759 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:46:23.500746    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 31 23:46:23.500810 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:46:23.500765    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 31 23:46:23.500810 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:46:23.500778    2564 projected.go:196] Error preparing data for projected volume kube-api-access-cv7k5 for pod openshift-network-diagnostics/network-check-target-7n4sw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 31 23:46:23.500902 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:46:23.500845    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2887ad3f-73c9-4fce-8359-c86e96300a84-kube-api-access-cv7k5 podName:2887ad3f-73c9-4fce-8359-c86e96300a84 nodeName:}" failed. No retries permitted until 2026-07-31 23:46:24.000828654 +0000 UTC m=+2.112956329 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-cv7k5" (UniqueName: "kubernetes.io/projected/2887ad3f-73c9-4fce-8359-c86e96300a84-kube-api-access-cv7k5") pod "network-check-target-7n4sw" (UID: "2887ad3f-73c9-4fce-8359-c86e96300a84") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 31 23:46:23.502141 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.502113    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j28hp\" (UniqueName: \"kubernetes.io/projected/816ea407-9c93-4455-8202-8229340dc7db-kube-api-access-j28hp\") pod \"node-resolver-wxz8k\" (UID: \"816ea407-9c93-4455-8202-8229340dc7db\") " pod="openshift-dns/node-resolver-wxz8k"
Jul 31 23:46:23.502401 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.502377    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-88qdj\" (UniqueName: \"kubernetes.io/projected/2bd960e3-3638-49f5-bd70-96891131423e-kube-api-access-88qdj\") pod \"ovnkube-node-j5545\" (UID: \"2bd960e3-3638-49f5-bd70-96891131423e\") " pod="openshift-ovn-kubernetes/ovnkube-node-j5545"
Jul 31 23:46:23.502700 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.502683    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z9qrg\" (UniqueName: \"kubernetes.io/projected/b0541f63-78c6-4ae2-b6aa-3e7ad98eb151-kube-api-access-z9qrg\") pod \"multus-jw7ld\" (UID: \"b0541f63-78c6-4ae2-b6aa-3e7ad98eb151\") " pod="openshift-multus/multus-jw7ld"
Jul 31 23:46:23.503844 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.503817    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mpwsh\" (UniqueName: \"kubernetes.io/projected/d98fe0c2-0a27-408e-8131-252e3d469281-kube-api-access-mpwsh\") pod \"node-ca-ls2gt\" (UID: \"d98fe0c2-0a27-408e-8131-252e3d469281\") " pod="openshift-image-registry/node-ca-ls2gt"
Jul 31 23:46:23.504008 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.503990    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tmrvx\" (UniqueName: \"kubernetes.io/projected/1505ec5c-3dd6-4199-8f53-a93fa90f91b7-kube-api-access-tmrvx\") pod \"aws-ebs-csi-driver-node-4dfd9\" (UID: \"1505ec5c-3dd6-4199-8f53-a93fa90f91b7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4dfd9"
Jul 31 23:46:23.612805 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.612711    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-j5545"
Jul 31 23:46:23.617783 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.617764    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4dfd9"
Jul 31 23:46:23.621576 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:23.621549    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2bd960e3_3638_49f5_bd70_96891131423e.slice/crio-4a7304934dc1172f025e4c228048d53f7fa12421a2b770f16b5b1d0549ce070a WatchSource:0}: Error finding container 4a7304934dc1172f025e4c228048d53f7fa12421a2b770f16b5b1d0549ce070a: Status 404 returned error can't find the container with id 4a7304934dc1172f025e4c228048d53f7fa12421a2b770f16b5b1d0549ce070a
Jul 31 23:46:23.623117 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.623092    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-h5jbv"
Jul 31 23:46:23.625072 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:23.625053    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1505ec5c_3dd6_4199_8f53_a93fa90f91b7.slice/crio-5baadcc65af2a0d8785968c150b12a2f9656ddb0326093e560d6cff556c3ade8 WatchSource:0}: Error finding container 5baadcc65af2a0d8785968c150b12a2f9656ddb0326093e560d6cff556c3ade8: Status 404 returned error can't find the container with id 5baadcc65af2a0d8785968c150b12a2f9656ddb0326093e560d6cff556c3ade8
Jul 31 23:46:23.629200 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.629181    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-ls2gt"
Jul 31 23:46:23.630490 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:23.630473    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd997aba4_5c6e_49ad_81bd_95efee82e1ff.slice/crio-9b3bd46b41e415f316eaf72316ea4b2e17de0ba4d1b0db73a570c1cb1009ed06 WatchSource:0}: Error finding container 9b3bd46b41e415f316eaf72316ea4b2e17de0ba4d1b0db73a570c1cb1009ed06: Status 404 returned error can't find the container with id 9b3bd46b41e415f316eaf72316ea4b2e17de0ba4d1b0db73a570c1cb1009ed06
Jul 31 23:46:23.635180 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.635155    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-jw7ld"
Jul 31 23:46:23.640188 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.640166    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-x2dtl"
Jul 31 23:46:23.643410 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:23.643382    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0541f63_78c6_4ae2_b6aa_3e7ad98eb151.slice/crio-b0df61517bc16f059c00792ded4627118769db16d0987d1fea1f6fc9da94ed86 WatchSource:0}: Error finding container b0df61517bc16f059c00792ded4627118769db16d0987d1fea1f6fc9da94ed86: Status 404 returned error can't find the container with id b0df61517bc16f059c00792ded4627118769db16d0987d1fea1f6fc9da94ed86
Jul 31 23:46:23.645512 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.645490    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-wxz8k"
Jul 31 23:46:23.649721 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:23.649666    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd82faec_581a_491a_8515_4217ac8cac14.slice/crio-860219f49f8a955c2c24f15eba6b15a28c54edc246d995a0e504abd06a61f9db WatchSource:0}: Error finding container 860219f49f8a955c2c24f15eba6b15a28c54edc246d995a0e504abd06a61f9db: Status 404 returned error can't find the container with id 860219f49f8a955c2c24f15eba6b15a28c54edc246d995a0e504abd06a61f9db
Jul 31 23:46:23.650876 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.650858    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-jtsl2"
Jul 31 23:46:23.653499 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:23.653478    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod816ea407_9c93_4455_8202_8229340dc7db.slice/crio-d0fa1a6f90706b28cb862326d67fe65ed0939469ba072ca811d51ef01392447f WatchSource:0}: Error finding container d0fa1a6f90706b28cb862326d67fe65ed0939469ba072ca811d51ef01392447f: Status 404 returned error can't find the container with id d0fa1a6f90706b28cb862326d67fe65ed0939469ba072ca811d51ef01392447f
Jul 31 23:46:23.656707 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.656692    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-7ttkn"
Jul 31 23:46:23.658941 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:23.658828    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e0b1a1a_3553_464e_8094_45a9dbf1cb66.slice/crio-8bbca26c50a23e71dcaf1e592be31820e2d8614ea0d0ce7404539f2eab341eb8 WatchSource:0}: Error finding container 8bbca26c50a23e71dcaf1e592be31820e2d8614ea0d0ce7404539f2eab341eb8: Status 404 returned error can't find the container with id 8bbca26c50a23e71dcaf1e592be31820e2d8614ea0d0ce7404539f2eab341eb8
Jul 31 23:46:23.664976 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:46:23.664944    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0bfad85_f7bf_4457_8523_6a1aa966a280.slice/crio-f632c91dd23e11a71a990c58fa1d9af67cd3e4541711d0363294178d83a42d85 WatchSource:0}: Error finding container f632c91dd23e11a71a990c58fa1d9af67cd3e4541711d0363294178d83a42d85: Status 404 returned error can't find the container with id f632c91dd23e11a71a990c58fa1d9af67cd3e4541711d0363294178d83a42d85
Jul 31 23:46:23.824082 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.823867    2564 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 31 23:46:23.992199 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.991856    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/69e78750-573b-4dbd-a585-cf0d88056f4f-original-pull-secret\") pod \"global-pull-secret-syncer-h45s5\" (UID: \"69e78750-573b-4dbd-a585-cf0d88056f4f\") " pod="kube-system/global-pull-secret-syncer-h45s5"
Jul 31 23:46:23.992199 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:23.991906    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/815ee183-a99c-4636-9937-da826945f531-metrics-certs\") pod \"network-metrics-daemon-j97wn\" (UID: \"815ee183-a99c-4636-9937-da826945f531\") " pod="openshift-multus/network-metrics-daemon-j97wn"
Jul 31 23:46:23.992199 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:46:23.992054    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 31 23:46:23.992199 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:46:23.992091    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 31 23:46:23.992199 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:46:23.992129    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/69e78750-573b-4dbd-a585-cf0d88056f4f-original-pull-secret podName:69e78750-573b-4dbd-a585-cf0d88056f4f nodeName:}" failed. No retries permitted until 2026-07-31 23:46:24.992107229 +0000 UTC m=+3.104234885 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/69e78750-573b-4dbd-a585-cf0d88056f4f-original-pull-secret") pod "global-pull-secret-syncer-h45s5" (UID: "69e78750-573b-4dbd-a585-cf0d88056f4f") : object "kube-system"/"original-pull-secret" not registered
Jul 31 23:46:23.992199 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:46:23.992150    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/815ee183-a99c-4636-9937-da826945f531-metrics-certs podName:815ee183-a99c-4636-9937-da826945f531 nodeName:}" failed. No retries permitted until 2026-07-31 23:46:24.992140259 +0000 UTC m=+3.104267916 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/815ee183-a99c-4636-9937-da826945f531-metrics-certs") pod "network-metrics-daemon-j97wn" (UID: "815ee183-a99c-4636-9937-da826945f531") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 31 23:46:24.093141 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:24.092446    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cv7k5\" (UniqueName: \"kubernetes.io/projected/2887ad3f-73c9-4fce-8359-c86e96300a84-kube-api-access-cv7k5\") pod \"network-check-target-7n4sw\" (UID: \"2887ad3f-73c9-4fce-8359-c86e96300a84\") " pod="openshift-network-diagnostics/network-check-target-7n4sw"
Jul 31 23:46:24.093141 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:46:24.092660    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 31 23:46:24.093141 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:46:24.092685    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 31 23:46:24.093141 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:46:24.092698    2564 projected.go:196] Error preparing data for projected volume kube-api-access-cv7k5 for pod openshift-network-diagnostics/network-check-target-7n4sw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 31 23:46:24.093141 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:46:24.092758    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2887ad3f-73c9-4fce-8359-c86e96300a84-kube-api-access-cv7k5 podName:2887ad3f-73c9-4fce-8359-c86e96300a84 nodeName:}" failed. No retries permitted until 2026-07-31 23:46:25.092738603 +0000 UTC m=+3.204866258 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-cv7k5" (UniqueName: "kubernetes.io/projected/2887ad3f-73c9-4fce-8359-c86e96300a84-kube-api-access-cv7k5") pod "network-check-target-7n4sw" (UID: "2887ad3f-73c9-4fce-8359-c86e96300a84") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 31 23:46:24.275721 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:24.275439    2564 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 31 23:46:24.323095 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:24.323024    2564 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-30 23:41:23 +0000 UTC" deadline="2028-04-07 19:06:57.01736366 +0000 UTC"
Jul 31 23:46:24.323095 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:24.323091    2564 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14779h20m32.694276878s"
Jul 31 23:46:24.416784 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:24.416754    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j97wn"
Jul 31 23:46:24.416967 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:46:24.416901    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j97wn" podUID="815ee183-a99c-4636-9937-da826945f531"
Jul 31 23:46:24.442809 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:24.442743    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-x2dtl" event={"ID":"fd82faec-581a-491a-8515-4217ac8cac14","Type":"ContainerStarted","Data":"860219f49f8a955c2c24f15eba6b15a28c54edc246d995a0e504abd06a61f9db"}
Jul 31 23:46:24.448856 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:24.448614    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-ls2gt" event={"ID":"d98fe0c2-0a27-408e-8131-252e3d469281","Type":"ContainerStarted","Data":"ce69cf088ce010dfaa91d720aefd7b432e5fc0f396eebc151b90c4e7a9f85aa5"}
Jul 31 23:46:24.461666 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:24.461618    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-h5jbv" event={"ID":"d997aba4-5c6e-49ad-81bd-95efee82e1ff","Type":"ContainerStarted","Data":"9b3bd46b41e415f316eaf72316ea4b2e17de0ba4d1b0db73a570c1cb1009ed06"}
Jul 31 23:46:24.466469 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:24.466436    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4dfd9" event={"ID":"1505ec5c-3dd6-4199-8f53-a93fa90f91b7","Type":"ContainerStarted","Data":"5baadcc65af2a0d8785968c150b12a2f9656ddb0326093e560d6cff556c3ade8"}
Jul 31 23:46:24.476396 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:24.476362    2564 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 31 23:46:24.476730 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:24.476707    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j5545" event={"ID":"2bd960e3-3638-49f5-bd70-96891131423e","Type":"ContainerStarted","Data":"4a7304934dc1172f025e4c228048d53f7fa12421a2b770f16b5b1d0549ce070a"}
Jul 31 23:46:24.486063 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:24.485979    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-7ttkn" event={"ID":"c0bfad85-f7bf-4457-8523-6a1aa966a280","Type":"ContainerStarted","Data":"f632c91dd23e11a71a990c58fa1d9af67cd3e4541711d0363294178d83a42d85"}
Jul 31 23:46:24.489538 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:24.489508    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jtsl2" event={"ID":"4e0b1a1a-3553-464e-8094-45a9dbf1cb66","Type":"ContainerStarted","Data":"8bbca26c50a23e71dcaf1e592be31820e2d8614ea0d0ce7404539f2eab341eb8"}
Jul 31 23:46:24.506079 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:24.506043    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-wxz8k" event={"ID":"816ea407-9c93-4455-8202-8229340dc7db","Type":"ContainerStarted","Data":"d0fa1a6f90706b28cb862326d67fe65ed0939469ba072ca811d51ef01392447f"}
Jul 31 23:46:24.519745 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:24.519708    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-jw7ld" event={"ID":"b0541f63-78c6-4ae2-b6aa-3e7ad98eb151","Type":"ContainerStarted","Data":"b0df61517bc16f059c00792ded4627118769db16d0987d1fea1f6fc9da94ed86"}
Jul 31 23:46:24.999681 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:24.999641    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/69e78750-573b-4dbd-a585-cf0d88056f4f-original-pull-secret\") pod \"global-pull-secret-syncer-h45s5\" (UID: \"69e78750-573b-4dbd-a585-cf0d88056f4f\") " pod="kube-system/global-pull-secret-syncer-h45s5"
Jul 31 23:46:24.999874 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:24.999697    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/815ee183-a99c-4636-9937-da826945f531-metrics-certs\") pod \"network-metrics-daemon-j97wn\" (UID: \"815ee183-a99c-4636-9937-da826945f531\") " pod="openshift-multus/network-metrics-daemon-j97wn"
Jul 31 23:46:24.999874 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:46:24.999864    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 31 23:46:24.999985 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:46:24.999929    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/815ee183-a99c-4636-9937-da826945f531-metrics-certs podName:815ee183-a99c-4636-9937-da826945f531 nodeName:}" failed. No retries permitted until 2026-07-31 23:46:26.999911129 +0000 UTC m=+5.112038786 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/815ee183-a99c-4636-9937-da826945f531-metrics-certs") pod "network-metrics-daemon-j97wn" (UID: "815ee183-a99c-4636-9937-da826945f531") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 31 23:46:25.000329 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:46:25.000308    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 31 23:46:25.000405 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:46:25.000374    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/69e78750-573b-4dbd-a585-cf0d88056f4f-original-pull-secret podName:69e78750-573b-4dbd-a585-cf0d88056f4f nodeName:}" failed. No retries permitted until 2026-07-31 23:46:27.000350416 +0000 UTC m=+5.112478084 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/69e78750-573b-4dbd-a585-cf0d88056f4f-original-pull-secret") pod "global-pull-secret-syncer-h45s5" (UID: "69e78750-573b-4dbd-a585-cf0d88056f4f") : object "kube-system"/"original-pull-secret" not registered
Jul 31 23:46:25.100654 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:25.100221    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cv7k5\" (UniqueName: \"kubernetes.io/projected/2887ad3f-73c9-4fce-8359-c86e96300a84-kube-api-access-cv7k5\") pod \"network-check-target-7n4sw\" (UID: \"2887ad3f-73c9-4fce-8359-c86e96300a84\") " pod="openshift-network-diagnostics/network-check-target-7n4sw"
Jul 31 23:46:25.100654 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:46:25.100396    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 31 23:46:25.100654 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:46:25.100420    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 31 23:46:25.100654 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:46:25.100433    2564 projected.go:196] Error preparing data for projected volume kube-api-access-cv7k5 for pod openshift-network-diagnostics/network-check-target-7n4sw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 31 23:46:25.100654 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:46:25.100497    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2887ad3f-73c9-4fce-8359-c86e96300a84-kube-api-access-cv7k5 podName:2887ad3f-73c9-4fce-8359-c86e96300a84 nodeName:}" failed. No retries permitted until 2026-07-31 23:46:27.100476342 +0000 UTC m=+5.212604016 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-cv7k5" (UniqueName: "kubernetes.io/projected/2887ad3f-73c9-4fce-8359-c86e96300a84-kube-api-access-cv7k5") pod "network-check-target-7n4sw" (UID: "2887ad3f-73c9-4fce-8359-c86e96300a84") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 31 23:46:25.323944 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:25.323729    2564 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-30 23:41:23 +0000 UTC" deadline="2028-02-14 14:06:45.038064412 +0000 UTC"
Jul 31 23:46:25.323944 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:25.323778    2564 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13502h20m19.714290887s"
Jul 31 23:46:25.416770 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:25.416726    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h45s5"
Jul 31 23:46:25.416957 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:46:25.416870    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h45s5" podUID="69e78750-573b-4dbd-a585-cf0d88056f4f"
Jul 31 23:46:25.417445 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:25.417272    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7n4sw"
Jul 31 23:46:25.417445 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:46:25.417374    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7n4sw" podUID="2887ad3f-73c9-4fce-8359-c86e96300a84"
Jul 31 23:46:26.242093 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:26.242060    2564 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 31 23:46:26.419428 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:26.419394    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j97wn"
Jul 31 23:46:26.419972 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:46:26.419552    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j97wn" podUID="815ee183-a99c-4636-9937-da826945f531"
Jul 31 23:46:27.018436 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:27.018400    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/69e78750-573b-4dbd-a585-cf0d88056f4f-original-pull-secret\") pod \"global-pull-secret-syncer-h45s5\" (UID: \"69e78750-573b-4dbd-a585-cf0d88056f4f\") " pod="kube-system/global-pull-secret-syncer-h45s5"
Jul 31 23:46:27.018588 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:27.018450    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/815ee183-a99c-4636-9937-da826945f531-metrics-certs\") pod \"network-metrics-daemon-j97wn\" (UID: \"815ee183-a99c-4636-9937-da826945f531\") " pod="openshift-multus/network-metrics-daemon-j97wn"
Jul 31 23:46:27.018664 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:46:27.018606    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 31 23:46:27.018710 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:46:27.018688    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/69e78750-573b-4dbd-a585-cf0d88056f4f-original-pull-secret podName:69e78750-573b-4dbd-a585-cf0d88056f4f nodeName:}" failed. No retries permitted until 2026-07-31 23:46:31.018667492 +0000 UTC m=+9.130795156 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/69e78750-573b-4dbd-a585-cf0d88056f4f-original-pull-secret") pod "global-pull-secret-syncer-h45s5" (UID: "69e78750-573b-4dbd-a585-cf0d88056f4f") : object "kube-system"/"original-pull-secret" not registered
Jul 31 23:46:27.018710 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:46:27.018606    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 31 23:46:27.018795 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:46:27.018744    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/815ee183-a99c-4636-9937-da826945f531-metrics-certs podName:815ee183-a99c-4636-9937-da826945f531 nodeName:}" failed. No retries permitted until 2026-07-31 23:46:31.01873084 +0000 UTC m=+9.130858493 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/815ee183-a99c-4636-9937-da826945f531-metrics-certs") pod "network-metrics-daemon-j97wn" (UID: "815ee183-a99c-4636-9937-da826945f531") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 31 23:46:27.118889 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:27.118851    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cv7k5\" (UniqueName: \"kubernetes.io/projected/2887ad3f-73c9-4fce-8359-c86e96300a84-kube-api-access-cv7k5\") pod \"network-check-target-7n4sw\" (UID: \"2887ad3f-73c9-4fce-8359-c86e96300a84\") " pod="openshift-network-diagnostics/network-check-target-7n4sw"
Jul 31 23:46:27.119131 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:46:27.119015    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 31 23:46:27.119131 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:46:27.119036    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 31 23:46:27.119131 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:46:27.119051    2564 projected.go:196] Error preparing data for projected volume kube-api-access-cv7k5 for pod openshift-network-diagnostics/network-check-target-7n4sw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 31 23:46:27.119131 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:46:27.119114    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2887ad3f-73c9-4fce-8359-c86e96300a84-kube-api-access-cv7k5 podName:2887ad3f-73c9-4fce-8359-c86e96300a84 nodeName:}" failed. No retries permitted until 2026-07-31 23:46:31.119096591 +0000 UTC m=+9.231224257 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-cv7k5" (UniqueName: "kubernetes.io/projected/2887ad3f-73c9-4fce-8359-c86e96300a84-kube-api-access-cv7k5") pod "network-check-target-7n4sw" (UID: "2887ad3f-73c9-4fce-8359-c86e96300a84") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 31 23:46:27.416703 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:27.416610    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h45s5"
Jul 31 23:46:27.416957 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:46:27.416768    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h45s5" podUID="69e78750-573b-4dbd-a585-cf0d88056f4f"
Jul 31 23:46:27.416957 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:27.416913    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7n4sw"
Jul 31 23:46:27.420223 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:46:27.417794    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7n4sw" podUID="2887ad3f-73c9-4fce-8359-c86e96300a84"
Jul 31 23:46:28.416136 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:28.416107    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j97wn"
Jul 31 23:46:28.416308 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:46:28.416249    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j97wn" podUID="815ee183-a99c-4636-9937-da826945f531"
Jul 31 23:46:29.416866 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:29.416826    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h45s5"
Jul 31 23:46:29.417324 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:46:29.416954    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h45s5" podUID="69e78750-573b-4dbd-a585-cf0d88056f4f"
Jul 31 23:46:29.417324 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:29.416826    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7n4sw"
Jul 31 23:46:29.417324 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:46:29.417091    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7n4sw" podUID="2887ad3f-73c9-4fce-8359-c86e96300a84"
Jul 31 23:46:30.416740 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:30.416706    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j97wn"
Jul 31 23:46:30.416966 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:46:30.416861    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j97wn" podUID="815ee183-a99c-4636-9937-da826945f531"
Jul 31 23:46:31.052668 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:31.051912    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/815ee183-a99c-4636-9937-da826945f531-metrics-certs\") pod \"network-metrics-daemon-j97wn\" (UID: \"815ee183-a99c-4636-9937-da826945f531\") " pod="openshift-multus/network-metrics-daemon-j97wn"
Jul 31 23:46:31.052668 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:31.052013    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/69e78750-573b-4dbd-a585-cf0d88056f4f-original-pull-secret\") pod \"global-pull-secret-syncer-h45s5\" (UID: \"69e78750-573b-4dbd-a585-cf0d88056f4f\") " pod="kube-system/global-pull-secret-syncer-h45s5"
Jul 31 23:46:31.052668 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:46:31.052137    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 31 23:46:31.052668 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:46:31.052199    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/69e78750-573b-4dbd-a585-cf0d88056f4f-original-pull-secret podName:69e78750-573b-4dbd-a585-cf0d88056f4f nodeName:}" failed. No retries permitted until 2026-07-31 23:46:39.05217948 +0000 UTC m=+17.164307137 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/69e78750-573b-4dbd-a585-cf0d88056f4f-original-pull-secret") pod "global-pull-secret-syncer-h45s5" (UID: "69e78750-573b-4dbd-a585-cf0d88056f4f") : object "kube-system"/"original-pull-secret" not registered
Jul 31 23:46:31.052668 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:46:31.052568    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 31 23:46:31.052668 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:46:31.052615    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/815ee183-a99c-4636-9937-da826945f531-metrics-certs podName:815ee183-a99c-4636-9937-da826945f531 nodeName:}" failed. No retries permitted until 2026-07-31 23:46:39.052599596 +0000 UTC m=+17.164727256 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/815ee183-a99c-4636-9937-da826945f531-metrics-certs") pod "network-metrics-daemon-j97wn" (UID: "815ee183-a99c-4636-9937-da826945f531") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 31 23:46:31.153272 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:31.153148    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cv7k5\" (UniqueName: \"kubernetes.io/projected/2887ad3f-73c9-4fce-8359-c86e96300a84-kube-api-access-cv7k5\") pod \"network-check-target-7n4sw\" (UID: \"2887ad3f-73c9-4fce-8359-c86e96300a84\") " pod="openshift-network-diagnostics/network-check-target-7n4sw"
Jul 31 23:46:31.153435 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:46:31.153304    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 31 23:46:31.153435 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:46:31.153324    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 31 23:46:31.153435 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:46:31.153336    2564 projected.go:196] Error preparing data for projected volume kube-api-access-cv7k5 for pod openshift-network-diagnostics/network-check-target-7n4sw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 31 23:46:31.153435 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:46:31.153394    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2887ad3f-73c9-4fce-8359-c86e96300a84-kube-api-access-cv7k5 podName:2887ad3f-73c9-4fce-8359-c86e96300a84 nodeName:}" failed. No retries permitted until 2026-07-31 23:46:39.15337708 +0000 UTC m=+17.265504737 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-cv7k5" (UniqueName: "kubernetes.io/projected/2887ad3f-73c9-4fce-8359-c86e96300a84-kube-api-access-cv7k5") pod "network-check-target-7n4sw" (UID: "2887ad3f-73c9-4fce-8359-c86e96300a84") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 31 23:46:31.417275 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:31.416619    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7n4sw"
Jul 31 23:46:31.417275 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:46:31.416768    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7n4sw" podUID="2887ad3f-73c9-4fce-8359-c86e96300a84"
Jul 31 23:46:31.417275 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:31.417113    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h45s5"
Jul 31 23:46:31.417275 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:46:31.417199    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h45s5" podUID="69e78750-573b-4dbd-a585-cf0d88056f4f"
Jul 31 23:46:32.417648 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:32.417601    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j97wn"
Jul 31 23:46:32.418121 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:46:32.417745    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j97wn" podUID="815ee183-a99c-4636-9937-da826945f531"
Jul 31 23:46:33.416954 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:33.416920    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h45s5"
Jul 31 23:46:33.417124 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:46:33.417046    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h45s5" podUID="69e78750-573b-4dbd-a585-cf0d88056f4f"
Jul 31 23:46:33.417124 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:33.417101    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7n4sw"
Jul 31 23:46:33.417269 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:46:33.417244    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7n4sw" podUID="2887ad3f-73c9-4fce-8359-c86e96300a84"
Jul 31 23:46:34.416990 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:34.416960    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j97wn"
Jul 31 23:46:34.417360 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:46:34.417088    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j97wn" podUID="815ee183-a99c-4636-9937-da826945f531"
Jul 31 23:46:35.416932 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:35.416894    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h45s5"
Jul 31 23:46:35.417115 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:35.416906    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7n4sw"
Jul 31 23:46:35.417115 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:46:35.417101    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h45s5" podUID="69e78750-573b-4dbd-a585-cf0d88056f4f"
Jul 31 23:46:35.417474 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:46:35.417237    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7n4sw" podUID="2887ad3f-73c9-4fce-8359-c86e96300a84"
Jul 31 23:46:36.417029 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:36.416987    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j97wn"
Jul 31 23:46:36.417192 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:46:36.417136    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j97wn" podUID="815ee183-a99c-4636-9937-da826945f531"
Jul 31 23:46:37.416434 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:37.416396    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h45s5"
Jul 31 23:46:37.416616 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:37.416441    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7n4sw"
Jul 31 23:46:37.416616 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:46:37.416550    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h45s5" podUID="69e78750-573b-4dbd-a585-cf0d88056f4f"
Jul 31 23:46:37.416756 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:46:37.416670    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7n4sw" podUID="2887ad3f-73c9-4fce-8359-c86e96300a84"
Jul 31 23:46:38.416594 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:38.416561    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j97wn"
Jul 31 23:46:38.417066 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:46:38.416713    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j97wn" podUID="815ee183-a99c-4636-9937-da826945f531"
Jul 31 23:46:39.109072 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:39.108987    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/69e78750-573b-4dbd-a585-cf0d88056f4f-original-pull-secret\") pod \"global-pull-secret-syncer-h45s5\" (UID: \"69e78750-573b-4dbd-a585-cf0d88056f4f\") " pod="kube-system/global-pull-secret-syncer-h45s5"
Jul 31 23:46:39.109072 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:39.109036    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/815ee183-a99c-4636-9937-da826945f531-metrics-certs\") pod \"network-metrics-daemon-j97wn\" (UID: \"815ee183-a99c-4636-9937-da826945f531\") " pod="openshift-multus/network-metrics-daemon-j97wn"
Jul 31 23:46:39.109345 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:46:39.109152    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 31 23:46:39.109345 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:46:39.109228    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/69e78750-573b-4dbd-a585-cf0d88056f4f-original-pull-secret podName:69e78750-573b-4dbd-a585-cf0d88056f4f nodeName:}" failed. No retries permitted until 2026-07-31 23:46:55.109206769 +0000 UTC m=+33.221334422 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/69e78750-573b-4dbd-a585-cf0d88056f4f-original-pull-secret") pod "global-pull-secret-syncer-h45s5" (UID: "69e78750-573b-4dbd-a585-cf0d88056f4f") : object "kube-system"/"original-pull-secret" not registered
Jul 31 23:46:39.109345 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:46:39.109156    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 31 23:46:39.109345 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:46:39.109303    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/815ee183-a99c-4636-9937-da826945f531-metrics-certs podName:815ee183-a99c-4636-9937-da826945f531 nodeName:}" failed. No retries permitted until 2026-07-31 23:46:55.109285171 +0000 UTC m=+33.221412829 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/815ee183-a99c-4636-9937-da826945f531-metrics-certs") pod "network-metrics-daemon-j97wn" (UID: "815ee183-a99c-4636-9937-da826945f531") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 31 23:46:39.209546 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:39.209506    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cv7k5\" (UniqueName: \"kubernetes.io/projected/2887ad3f-73c9-4fce-8359-c86e96300a84-kube-api-access-cv7k5\") pod \"network-check-target-7n4sw\" (UID: \"2887ad3f-73c9-4fce-8359-c86e96300a84\") " pod="openshift-network-diagnostics/network-check-target-7n4sw"
Jul 31 23:46:39.209741 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:46:39.209691    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 31 23:46:39.209741 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:46:39.209720    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 31 23:46:39.209741 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:46:39.209732    2564 projected.go:196] Error preparing data for projected volume kube-api-access-cv7k5 for pod openshift-network-diagnostics/network-check-target-7n4sw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 31 23:46:39.209912 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:46:39.209786    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2887ad3f-73c9-4fce-8359-c86e96300a84-kube-api-access-cv7k5 podName:2887ad3f-73c9-4fce-8359-c86e96300a84 nodeName:}" failed. No retries permitted until 2026-07-31 23:46:55.209768562 +0000 UTC m=+33.321896243 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-cv7k5" (UniqueName: "kubernetes.io/projected/2887ad3f-73c9-4fce-8359-c86e96300a84-kube-api-access-cv7k5") pod "network-check-target-7n4sw" (UID: "2887ad3f-73c9-4fce-8359-c86e96300a84") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 31 23:46:39.416340 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:39.416263    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h45s5"
Jul 31 23:46:39.416505 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:46:39.416395    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h45s5" podUID="69e78750-573b-4dbd-a585-cf0d88056f4f"
Jul 31 23:46:39.416505 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:39.416455    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7n4sw"
Jul 31 23:46:39.416608 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:46:39.416573    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7n4sw" podUID="2887ad3f-73c9-4fce-8359-c86e96300a84"
Jul 31 23:46:40.416818 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:40.416680    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j97wn"
Jul 31 23:46:40.417224 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:46:40.416820    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j97wn" podUID="815ee183-a99c-4636-9937-da826945f531"
Jul 31 23:46:41.416377 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:41.416340    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h45s5"
Jul 31 23:46:41.416557 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:41.416345    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7n4sw"
Jul 31 23:46:41.416557 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:46:41.416482    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h45s5" podUID="69e78750-573b-4dbd-a585-cf0d88056f4f"
Jul 31 23:46:41.416557 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:46:41.416545    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7n4sw" podUID="2887ad3f-73c9-4fce-8359-c86e96300a84"
Jul 31 23:46:42.417271 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:42.417235    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j97wn"
Jul 31 23:46:42.417729 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:46:42.417342    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j97wn" podUID="815ee183-a99c-4636-9937-da826945f531"
Jul 31 23:46:43.416669 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:43.416417    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h45s5"
Jul 31 23:46:43.416669 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:43.416470    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7n4sw"
Jul 31 23:46:43.416988 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:46:43.416690    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h45s5" podUID="69e78750-573b-4dbd-a585-cf0d88056f4f"
Jul 31 23:46:43.416988 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:46:43.416751    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7n4sw" podUID="2887ad3f-73c9-4fce-8359-c86e96300a84"
Jul 31 23:46:43.567425 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:43.567387    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-h5jbv" event={"ID":"d997aba4-5c6e-49ad-81bd-95efee82e1ff","Type":"ContainerStarted","Data":"636ccd517c510527bf49df327f970d226294481b5b27b4e77578c2c6796de3a9"}
Jul 31 23:46:43.572553 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:43.572525    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-75.ec2.internal" event={"ID":"187e741f1e261812822fd24432933695","Type":"ContainerStarted","Data":"f5abe2fdd6cbd15e674b4aafbf14e479b4e4c02e50293e059c3944451a584100"}
Jul 31 23:46:43.574459 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:43.574434    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-jw7ld" event={"ID":"b0541f63-78c6-4ae2-b6aa-3e7ad98eb151","Type":"ContainerStarted","Data":"9789b0b0785f66019740f3733d7f4a433042a95d32432c828d1e1cf67f6a7164"}
Jul 31 23:46:43.602352 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:43.602306    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-h5jbv" podStartSLOduration=2.156214957 podStartE2EDuration="21.602289201s" totalImagesPullingTime="19.446074244s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-31 23:46:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:46:43.585759866 +0000 UTC m=+21.697887565" watchObservedRunningTime="2026-07-31 23:46:43.602289201 +0000 UTC m=+21.714416874"
Jul 31 23:46:43.602472 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:43.602437    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-jw7ld" podStartSLOduration=2.160021464 podStartE2EDuration="21.602430834s" totalImagesPullingTime="19.44240937s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-31 23:46:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:46:43.601354864 +0000 UTC m=+21.713482538" watchObservedRunningTime="2026-07-31 23:46:43.602430834 +0000 UTC m=+21.714558508"
Jul 31 23:46:43.615731 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:43.614693    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-75.ec2.internal" podStartSLOduration=20.614672245 podStartE2EDuration="20.614672245s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-31 23:46:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:46:43.614307761 +0000 UTC m=+21.726435436" watchObservedRunningTime="2026-07-31 23:46:43.614672245 +0000 UTC m=+21.726799922"
Jul 31 23:46:44.416286 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:44.416027    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j97wn"
Jul 31 23:46:44.416477 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:46:44.416444    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j97wn" podUID="815ee183-a99c-4636-9937-da826945f531"
Jul 31 23:46:44.579650 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:44.579521    2564 generic.go:356] "Generic (PLEG): container finished" podID="4e0b1a1a-3553-464e-8094-45a9dbf1cb66" containerID="95592637633e0fba88478c77751289d0d71e76b53d78c1eb15ff46a11dd2315d" exitCode=0
Jul 31 23:46:44.579650 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:44.579576    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jtsl2" event={"ID":"4e0b1a1a-3553-464e-8094-45a9dbf1cb66","Type":"ContainerDied","Data":"95592637633e0fba88478c77751289d0d71e76b53d78c1eb15ff46a11dd2315d"}
Jul 31 23:46:44.581458 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:44.581420    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-wxz8k" event={"ID":"816ea407-9c93-4455-8202-8229340dc7db","Type":"ContainerStarted","Data":"b4e6df10e564acce54bc47beeea1aa2ff0f46c06a9849c306ada86193b39ab71"}
Jul 31 23:46:44.583167 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:44.583139    2564 generic.go:356] "Generic (PLEG): container finished" podID="ee3cb6879ff58186c26ebd09a71c7e47" containerID="54179d5436d660412ab04cfa45a233276c7057da91d1c4a996f40756296f4eb1" exitCode=0
Jul 31 23:46:44.583289 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:44.583234    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-75.ec2.internal" event={"ID":"ee3cb6879ff58186c26ebd09a71c7e47","Type":"ContainerDied","Data":"54179d5436d660412ab04cfa45a233276c7057da91d1c4a996f40756296f4eb1"}
Jul 31 23:46:44.585531 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:44.585508    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-x2dtl" event={"ID":"fd82faec-581a-491a-8515-4217ac8cac14","Type":"ContainerStarted","Data":"9c175edc7dd1c9ef589a9d6e335f34a2f4345b0305d8e44b35ed813c342a2826"}
Jul 31 23:46:44.587215 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:44.587115    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-ls2gt" event={"ID":"d98fe0c2-0a27-408e-8131-252e3d469281","Type":"ContainerStarted","Data":"a76a2c627b8a375c9f87c0df3a640d7cb9d9c417ace7445a81a29e58403520f7"}
Jul 31 23:46:44.588759 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:44.588737    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4dfd9" event={"ID":"1505ec5c-3dd6-4199-8f53-a93fa90f91b7","Type":"ContainerStarted","Data":"a75fcf5ec21bd5a06cb48b86745c32a96672fb183c436d01c4d4f81b848b92b5"}
Jul 31 23:46:44.592002 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:44.591974    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j5545" event={"ID":"2bd960e3-3638-49f5-bd70-96891131423e","Type":"ContainerStarted","Data":"3f7e16ac8fe32348e1a2f6d099607c918aebd0b75f09528b874c1e974e9001e7"}
Jul 31 23:46:44.592104 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:44.592054    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j5545" event={"ID":"2bd960e3-3638-49f5-bd70-96891131423e","Type":"ContainerStarted","Data":"d8056b79751d7696c9d32c3bf1d0fd8dc9da9f84a39f466e512143496fff83c0"}
Jul 31 23:46:44.592166 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:44.592102    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j5545" event={"ID":"2bd960e3-3638-49f5-bd70-96891131423e","Type":"ContainerStarted","Data":"11b90de5653e26c170883a12cd2301cea24b7994e4e56d1b876a5860e7ca0821"}
Jul 31 23:46:44.592166 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:44.592145    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j5545" event={"ID":"2bd960e3-3638-49f5-bd70-96891131423e","Type":"ContainerStarted","Data":"d8c6042544d538c8d9529b9b312b91a3674dd89ad07fa555c9e27ef747f6e893"}
Jul 31 23:46:44.592268 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:44.592194    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j5545" event={"ID":"2bd960e3-3638-49f5-bd70-96891131423e","Type":"ContainerStarted","Data":"995bdac6d078f0bbe80d34f3ee98b15c7dee6cabeb53c322d469d0c6c8db19bb"}
Jul 31 23:46:44.592268 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:44.592240    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j5545" event={"ID":"2bd960e3-3638-49f5-bd70-96891131423e","Type":"ContainerStarted","Data":"0f2562aea9868f85cd437fc92923dde45e871cef26b9ae8a1e081b796c87dfd7"}
Jul 31 23:46:44.593473 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:44.593453    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-7ttkn" event={"ID":"c0bfad85-f7bf-4457-8523-6a1aa966a280","Type":"ContainerStarted","Data":"aea72ed381197e2b14f82fc2b33e1c0abb6ed51e3cf9e5bd2167f6458d56d0dc"}
Jul 31 23:46:44.617544 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:44.617487    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-wxz8k" podStartSLOduration=3.279140265 podStartE2EDuration="22.617467483s" totalImagesPullingTime="19.338327218s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-31 23:46:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:46:44.617022007 +0000 UTC m=+22.729149702" watchObservedRunningTime="2026-07-31 23:46:44.617467483 +0000 UTC m=+22.729595159"
Jul 31 23:46:44.633783 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:44.633737    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-7ttkn" podStartSLOduration=3.223987939 podStartE2EDuration="22.633725448s" totalImagesPullingTime="19.409737509s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-31 23:46:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:46:44.633550444 +0000 UTC m=+22.745678168" watchObservedRunningTime="2026-07-31 23:46:44.633725448 +0000 UTC m=+22.745853121"
Jul 31 23:46:44.648149 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:44.648104    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-ls2gt" podStartSLOduration=3.267273178 podStartE2EDuration="22.648086357s" totalImagesPullingTime="19.380813179s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-31 23:46:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:46:44.647822835 +0000 UTC m=+22.759950532" watchObservedRunningTime="2026-07-31 23:46:44.648086357 +0000 UTC m=+22.760214033"
Jul 31 23:46:44.690673 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:44.690575    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-x2dtl" podStartSLOduration=3.265699536 podStartE2EDuration="22.690559228s" totalImagesPullingTime="19.424859692s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:46:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:46:44.665358579 +0000 UTC m=+22.777486254" watchObservedRunningTime="2026-07-31 23:46:44.690559228 +0000 UTC m=+22.802686902"
Jul 31 23:46:45.159261 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:45.158963    2564 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 31 23:46:45.346901 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:45.346756    2564 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-31T23:46:45.159186807Z","UUID":"768bb0a3-7bff-4f37-9dc1-88fc6f67aaa7","Handler":null,"Name":"","Endpoint":""}
Jul 31 23:46:45.348383 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:45.348356    2564 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 31 23:46:45.348511 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:45.348408    2564 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 31 23:46:45.416957 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:45.416933    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7n4sw"
Jul 31 23:46:45.417174 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:45.416933    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h45s5"
Jul 31 23:46:45.417174 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:46:45.417054    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7n4sw" podUID="2887ad3f-73c9-4fce-8359-c86e96300a84"
Jul 31 23:46:45.417174 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:46:45.417124    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h45s5" podUID="69e78750-573b-4dbd-a585-cf0d88056f4f"
Jul 31 23:46:45.598721 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:45.598189    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-75.ec2.internal" event={"ID":"ee3cb6879ff58186c26ebd09a71c7e47","Type":"ContainerStarted","Data":"f5242b1f5fa095e961f555cb496db3cb256e550928bbae47e57a2250e9b8cc26"}
Jul 31 23:46:45.600384 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:45.600352    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4dfd9" event={"ID":"1505ec5c-3dd6-4199-8f53-a93fa90f91b7","Type":"ContainerStarted","Data":"f3718ec080b8f3368c4f5784d0a0069431e3e010fe2a7e03c1a88c4d5e6af623"}
Jul 31 23:46:45.613410 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:45.613339    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-75.ec2.internal" podStartSLOduration=22.613324186 podStartE2EDuration="22.613324186s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-31 23:46:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:46:45.612960921 +0000 UTC m=+23.725088595" watchObservedRunningTime="2026-07-31 23:46:45.613324186 +0000 UTC m=+23.725451861"
Jul 31 23:46:46.416708 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:46.416665    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j97wn"
Jul 31 23:46:46.416841 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:46:46.416802    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j97wn" podUID="815ee183-a99c-4636-9937-da826945f531"
Jul 31 23:46:46.604496 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:46.604445    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4dfd9" event={"ID":"1505ec5c-3dd6-4199-8f53-a93fa90f91b7","Type":"ContainerStarted","Data":"e526556e1c1c520078843986a61e160b41eeb2bf80caea5d15c5c9fe73b8b4f5"}
Jul 31 23:46:46.608280 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:46.608255    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j5545" event={"ID":"2bd960e3-3638-49f5-bd70-96891131423e","Type":"ContainerStarted","Data":"85afced143eb39b7e87528e4207ae844e853dfe0adabc2471190fbd99ece96d2"}
Jul 31 23:46:46.625621 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:46.625566    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4dfd9" podStartSLOduration=2.99941064 podStartE2EDuration="24.625545989s" totalImagesPullingTime="21.626135349s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-31 23:46:22 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:46:46.625417747 +0000 UTC m=+24.737545439" watchObservedRunningTime="2026-07-31 23:46:46.625545989 +0000 UTC m=+24.737673680"
Jul 31 23:46:47.415906 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:47.415872    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7n4sw"
Jul 31 23:46:47.416102 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:47.415886    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h45s5"
Jul 31 23:46:47.416102 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:46:47.415997    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7n4sw" podUID="2887ad3f-73c9-4fce-8359-c86e96300a84"
Jul 31 23:46:47.416102 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:46:47.416071    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h45s5" podUID="69e78750-573b-4dbd-a585-cf0d88056f4f"
Jul 31 23:46:47.893609 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:47.893572    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-x2dtl"
Jul 31 23:46:47.894194 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:47.894175    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-x2dtl"
Jul 31 23:46:47.969615 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:47.969584    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-wxz8k_816ea407-9c93-4455-8202-8229340dc7db/dns-node-resolver/0.log"
Jul 31 23:46:48.416957 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:48.416924    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j97wn"
Jul 31 23:46:48.417120 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:46:48.417059    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j97wn" podUID="815ee183-a99c-4636-9937-da826945f531"
Jul 31 23:46:48.613068 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:48.613024    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-x2dtl"
Jul 31 23:46:48.613609 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:48.613587    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-x2dtl"
Jul 31 23:46:48.748766 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:48.748416    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-ls2gt_d98fe0c2-0a27-408e-8131-252e3d469281/node-ca/0.log"
Jul 31 23:46:49.416984 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:49.416819    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7n4sw"
Jul 31 23:46:49.417408 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:49.416819    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h45s5"
Jul 31 23:46:49.417408 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:46:49.417067    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7n4sw" podUID="2887ad3f-73c9-4fce-8359-c86e96300a84"
Jul 31 23:46:49.417408 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:46:49.417117    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h45s5" podUID="69e78750-573b-4dbd-a585-cf0d88056f4f"
Jul 31 23:46:49.616905 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:49.616873    2564 generic.go:356] "Generic (PLEG): container finished" podID="4e0b1a1a-3553-464e-8094-45a9dbf1cb66" containerID="6e45a66ccf43dd7b1779571f32aab53910a1f904a8fd66460f0914c30c0833b5" exitCode=0
Jul 31 23:46:49.617086 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:49.616958    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jtsl2" event={"ID":"4e0b1a1a-3553-464e-8094-45a9dbf1cb66","Type":"ContainerDied","Data":"6e45a66ccf43dd7b1779571f32aab53910a1f904a8fd66460f0914c30c0833b5"}
Jul 31 23:46:49.620962 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:49.620936    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j5545" event={"ID":"2bd960e3-3638-49f5-bd70-96891131423e","Type":"ContainerStarted","Data":"2dec74cd950e0d271a1a986b98858a099333ad3a84a6dae679a9b497dad86667"}
Jul 31 23:46:49.621225 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:49.621203    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-j5545"
Jul 31 23:46:49.621324 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:49.621277    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-j5545"
Jul 31 23:46:49.636768 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:49.636744    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-j5545"
Jul 31 23:46:49.668482 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:49.668392    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-j5545" podStartSLOduration=7.714951079 podStartE2EDuration="27.668375351s" totalImagesPullingTime="19.953424272s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-31 23:46:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:46:49.666839868 +0000 UTC m=+27.778967563" watchObservedRunningTime="2026-07-31 23:46:49.668375351 +0000 UTC m=+27.780503029"
Jul 31 23:46:50.416759 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:50.416725    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j97wn"
Jul 31 23:46:50.416962 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:46:50.416874    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j97wn" podUID="815ee183-a99c-4636-9937-da826945f531"
Jul 31 23:46:50.623780 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:50.623748    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-j5545"
Jul 31 23:46:50.638468 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:50.638445    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-j5545"
Jul 31 23:46:50.962790 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:50.962542    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-h45s5"]
Jul 31 23:46:50.962943 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:50.962913    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h45s5"
Jul 31 23:46:50.963020 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:46:50.963002    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h45s5" podUID="69e78750-573b-4dbd-a585-cf0d88056f4f"
Jul 31 23:46:50.965321 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:50.965284    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-j97wn"]
Jul 31 23:46:50.965482 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:50.965439    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j97wn"
Jul 31 23:46:50.965583 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:46:50.965556    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j97wn" podUID="815ee183-a99c-4636-9937-da826945f531"
Jul 31 23:46:50.965928 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:50.965909    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-7n4sw"]
Jul 31 23:46:50.966013 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:50.966000    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7n4sw"
Jul 31 23:46:50.966098 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:46:50.966079    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7n4sw" podUID="2887ad3f-73c9-4fce-8359-c86e96300a84"
Jul 31 23:46:51.627202 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:51.627172    2564 generic.go:356] "Generic (PLEG): container finished" podID="4e0b1a1a-3553-464e-8094-45a9dbf1cb66" containerID="93f6c3ef2b62ba0b2575b16c419bfd8d8c891ca752ec0317d1c67ecf2025fb33" exitCode=0
Jul 31 23:46:51.627606 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:51.627259    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jtsl2" event={"ID":"4e0b1a1a-3553-464e-8094-45a9dbf1cb66","Type":"ContainerDied","Data":"93f6c3ef2b62ba0b2575b16c419bfd8d8c891ca752ec0317d1c67ecf2025fb33"}
Jul 31 23:46:52.417109 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:52.417073    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7n4sw"
Jul 31 23:46:52.417328 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:46:52.417169    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7n4sw" podUID="2887ad3f-73c9-4fce-8359-c86e96300a84"
Jul 31 23:46:52.417328 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:52.417258    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j97wn"
Jul 31 23:46:52.417452 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:46:52.417419    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j97wn" podUID="815ee183-a99c-4636-9937-da826945f531"
Jul 31 23:46:52.417511 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:52.417459    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h45s5"
Jul 31 23:46:52.417557 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:46:52.417525    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h45s5" podUID="69e78750-573b-4dbd-a585-cf0d88056f4f"
Jul 31 23:46:53.633851 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:53.633822    2564 generic.go:356] "Generic (PLEG): container finished" podID="4e0b1a1a-3553-464e-8094-45a9dbf1cb66" containerID="031f4dd1fd2d16de50be03b6e0c1fe9607d88783c4749b892fdd332b94b18ace" exitCode=0
Jul 31 23:46:53.633851 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:53.633859    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jtsl2" event={"ID":"4e0b1a1a-3553-464e-8094-45a9dbf1cb66","Type":"ContainerDied","Data":"031f4dd1fd2d16de50be03b6e0c1fe9607d88783c4749b892fdd332b94b18ace"}
Jul 31 23:46:54.416266 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:54.416176    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j97wn"
Jul 31 23:46:54.416266 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:54.416235    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7n4sw"
Jul 31 23:46:54.416266 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:54.416176    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h45s5"
Jul 31 23:46:54.416523 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:46:54.416331    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j97wn" podUID="815ee183-a99c-4636-9937-da826945f531"
Jul 31 23:46:54.416523 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:46:54.416448    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7n4sw" podUID="2887ad3f-73c9-4fce-8359-c86e96300a84"
Jul 31 23:46:54.416607 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:46:54.416530    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h45s5" podUID="69e78750-573b-4dbd-a585-cf0d88056f4f"
Jul 31 23:46:55.124460 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:55.124419    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/69e78750-573b-4dbd-a585-cf0d88056f4f-original-pull-secret\") pod \"global-pull-secret-syncer-h45s5\" (UID: \"69e78750-573b-4dbd-a585-cf0d88056f4f\") " pod="kube-system/global-pull-secret-syncer-h45s5"
Jul 31 23:46:55.124460 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:55.124472    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/815ee183-a99c-4636-9937-da826945f531-metrics-certs\") pod \"network-metrics-daemon-j97wn\" (UID: \"815ee183-a99c-4636-9937-da826945f531\") " pod="openshift-multus/network-metrics-daemon-j97wn"
Jul 31 23:46:55.124989 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:46:55.124586    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 31 23:46:55.124989 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:46:55.124593    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 31 23:46:55.124989 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:46:55.124677    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/815ee183-a99c-4636-9937-da826945f531-metrics-certs podName:815ee183-a99c-4636-9937-da826945f531 nodeName:}" failed. No retries permitted until 2026-07-31 23:47:27.124656228 +0000 UTC m=+65.236783883 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/815ee183-a99c-4636-9937-da826945f531-metrics-certs") pod "network-metrics-daemon-j97wn" (UID: "815ee183-a99c-4636-9937-da826945f531") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 31 23:46:55.124989 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:46:55.124698    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/69e78750-573b-4dbd-a585-cf0d88056f4f-original-pull-secret podName:69e78750-573b-4dbd-a585-cf0d88056f4f nodeName:}" failed. No retries permitted until 2026-07-31 23:47:27.124688396 +0000 UTC m=+65.236816053 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/69e78750-573b-4dbd-a585-cf0d88056f4f-original-pull-secret") pod "global-pull-secret-syncer-h45s5" (UID: "69e78750-573b-4dbd-a585-cf0d88056f4f") : object "kube-system"/"original-pull-secret" not registered
Jul 31 23:46:55.225752 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:55.225715    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cv7k5\" (UniqueName: \"kubernetes.io/projected/2887ad3f-73c9-4fce-8359-c86e96300a84-kube-api-access-cv7k5\") pod \"network-check-target-7n4sw\" (UID: \"2887ad3f-73c9-4fce-8359-c86e96300a84\") " pod="openshift-network-diagnostics/network-check-target-7n4sw"
Jul 31 23:46:55.225961 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:46:55.225941    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 31 23:46:55.226014 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:46:55.225965    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 31 23:46:55.226014 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:46:55.225979    2564 projected.go:196] Error preparing data for projected volume kube-api-access-cv7k5 for pod openshift-network-diagnostics/network-check-target-7n4sw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 31 23:46:55.226120 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:46:55.226047    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2887ad3f-73c9-4fce-8359-c86e96300a84-kube-api-access-cv7k5 podName:2887ad3f-73c9-4fce-8359-c86e96300a84 nodeName:}" failed. No retries permitted until 2026-07-31 23:47:27.226029629 +0000 UTC m=+65.338157297 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-cv7k5" (UniqueName: "kubernetes.io/projected/2887ad3f-73c9-4fce-8359-c86e96300a84-kube-api-access-cv7k5") pod "network-check-target-7n4sw" (UID: "2887ad3f-73c9-4fce-8359-c86e96300a84") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 31 23:46:56.416669 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:56.416638    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h45s5"
Jul 31 23:46:56.417147 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:56.416637    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7n4sw"
Jul 31 23:46:56.417147 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:46:56.416782    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h45s5" podUID="69e78750-573b-4dbd-a585-cf0d88056f4f"
Jul 31 23:46:56.417147 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:56.416824    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j97wn"
Jul 31 23:46:56.417147 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:46:56.416913    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7n4sw" podUID="2887ad3f-73c9-4fce-8359-c86e96300a84"
Jul 31 23:46:56.417147 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:46:56.417017    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j97wn" podUID="815ee183-a99c-4636-9937-da826945f531"
Jul 31 23:46:57.646084 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:57.646008    2564 generic.go:356] "Generic (PLEG): container finished" podID="4e0b1a1a-3553-464e-8094-45a9dbf1cb66" containerID="ea76d9f5b4d4a265973830c0467ac3a4c11d8cc86a2637c25a5e8c1a0d061d8d" exitCode=0
Jul 31 23:46:57.646084 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:57.646048    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jtsl2" event={"ID":"4e0b1a1a-3553-464e-8094-45a9dbf1cb66","Type":"ContainerDied","Data":"ea76d9f5b4d4a265973830c0467ac3a4c11d8cc86a2637c25a5e8c1a0d061d8d"}
Jul 31 23:46:58.416161 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:58.415990    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7n4sw"
Jul 31 23:46:58.416346 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:58.416007    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h45s5"
Jul 31 23:46:58.416346 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:46:58.416248    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7n4sw" podUID="2887ad3f-73c9-4fce-8359-c86e96300a84"
Jul 31 23:46:58.416346 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:58.416067    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j97wn"
Jul 31 23:46:58.416346 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:46:58.416294    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h45s5" podUID="69e78750-573b-4dbd-a585-cf0d88056f4f"
Jul 31 23:46:58.416573 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:46:58.416384    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j97wn" podUID="815ee183-a99c-4636-9937-da826945f531"
Jul 31 23:46:58.650951 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:58.650917    2564 generic.go:356] "Generic (PLEG): container finished" podID="4e0b1a1a-3553-464e-8094-45a9dbf1cb66" containerID="e2f8f4e998069bf853681b4145b1ef391efade73e9bb31ddf1eac4fcfc02d510" exitCode=0
Jul 31 23:46:58.651281 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:58.650960    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jtsl2" event={"ID":"4e0b1a1a-3553-464e-8094-45a9dbf1cb66","Type":"ContainerDied","Data":"e2f8f4e998069bf853681b4145b1ef391efade73e9bb31ddf1eac4fcfc02d510"}
Jul 31 23:46:59.656021 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:59.655986    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jtsl2" event={"ID":"4e0b1a1a-3553-464e-8094-45a9dbf1cb66","Type":"ContainerStarted","Data":"a8f581e0798a6a649680e27442d207b8a4a8f18b1cee235e35db20e074712dab"}
Jul 31 23:46:59.680058 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:46:59.680012    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-jtsl2" podStartSLOduration=8.406270641 podStartE2EDuration="37.679997522s" totalImagesPullingTime="29.273726881s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-31 23:46:22 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:46:59.679798532 +0000 UTC m=+37.791926211" watchObservedRunningTime="2026-07-31 23:46:59.679997522 +0000 UTC m=+37.792125196"
Jul 31 23:47:00.416164 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:00.416129    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7n4sw"
Jul 31 23:47:00.416334 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:00.416129    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h45s5"
Jul 31 23:47:00.416334 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:47:00.416235    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7n4sw" podUID="2887ad3f-73c9-4fce-8359-c86e96300a84"
Jul 31 23:47:00.416461 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:47:00.416335    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h45s5" podUID="69e78750-573b-4dbd-a585-cf0d88056f4f"
Jul 31 23:47:00.416461 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:00.416136    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j97wn"
Jul 31 23:47:00.416561 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:47:00.416456    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j97wn" podUID="815ee183-a99c-4636-9937-da826945f531"
Jul 31 23:47:02.417740 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:02.417705    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j97wn"
Jul 31 23:47:02.418185 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:02.417815    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7n4sw"
Jul 31 23:47:02.418185 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:47:02.417809    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j97wn" podUID="815ee183-a99c-4636-9937-da826945f531"
Jul 31 23:47:02.418185 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:02.417852    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h45s5"
Jul 31 23:47:02.418185 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:47:02.417919    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7n4sw" podUID="2887ad3f-73c9-4fce-8359-c86e96300a84"
Jul 31 23:47:02.418185 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:47:02.417997    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h45s5" podUID="69e78750-573b-4dbd-a585-cf0d88056f4f"
Jul 31 23:47:04.416641 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:04.416589    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j97wn"
Jul 31 23:47:04.417119 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:04.416589    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h45s5"
Jul 31 23:47:04.417119 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:47:04.416713    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j97wn" podUID="815ee183-a99c-4636-9937-da826945f531"
Jul 31 23:47:04.417119 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:04.416594    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7n4sw"
Jul 31 23:47:04.417119 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:47:04.416760    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h45s5" podUID="69e78750-573b-4dbd-a585-cf0d88056f4f"
Jul 31 23:47:04.417119 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:47:04.416838    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7n4sw" podUID="2887ad3f-73c9-4fce-8359-c86e96300a84"
Jul 31 23:47:06.416800 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:06.416757    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j97wn"
Jul 31 23:47:06.417293 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:06.416757    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h45s5"
Jul 31 23:47:06.417293 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:06.416908    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7n4sw"
Jul 31 23:47:06.417293 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:47:06.416977    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h45s5" podUID="69e78750-573b-4dbd-a585-cf0d88056f4f"
Jul 31 23:47:06.417293 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:47:06.416882    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j97wn" podUID="815ee183-a99c-4636-9937-da826945f531"
Jul 31 23:47:06.417293 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:47:06.417063    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7n4sw" podUID="2887ad3f-73c9-4fce-8359-c86e96300a84"
Jul 31 23:47:08.416041 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:08.416007    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7n4sw"
Jul 31 23:47:08.416041 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:08.416032    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h45s5"
Jul 31 23:47:08.416545 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:08.416006    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j97wn"
Jul 31 23:47:08.416545 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:47:08.416106    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7n4sw" podUID="2887ad3f-73c9-4fce-8359-c86e96300a84"
Jul 31 23:47:08.416545 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:47:08.416212    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h45s5" podUID="69e78750-573b-4dbd-a585-cf0d88056f4f"
Jul 31 23:47:08.416545 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:47:08.416259    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j97wn" podUID="815ee183-a99c-4636-9937-da826945f531"
Jul 31 23:47:10.416062 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:10.416027    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h45s5"
Jul 31 23:47:10.416062 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:10.416046    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7n4sw"
Jul 31 23:47:10.416500 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:10.416109    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j97wn"
Jul 31 23:47:10.416500 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:47:10.416190    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h45s5" podUID="69e78750-573b-4dbd-a585-cf0d88056f4f"
Jul 31 23:47:10.416500 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:47:10.416317    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j97wn" podUID="815ee183-a99c-4636-9937-da826945f531"
Jul 31 23:47:10.416500 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:47:10.416370    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7n4sw" podUID="2887ad3f-73c9-4fce-8359-c86e96300a84"
Jul 31 23:47:12.417456 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:12.417425    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j97wn"
Jul 31 23:47:12.417864 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:47:12.417511    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j97wn" podUID="815ee183-a99c-4636-9937-da826945f531"
Jul 31 23:47:12.417864 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:12.417599    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7n4sw"
Jul 31 23:47:12.417864 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:12.417703    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h45s5"
Jul 31 23:47:12.417864 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:47:12.417715    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7n4sw" podUID="2887ad3f-73c9-4fce-8359-c86e96300a84"
Jul 31 23:47:12.417864 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:47:12.417780    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h45s5" podUID="69e78750-573b-4dbd-a585-cf0d88056f4f"
Jul 31 23:47:13.223205 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:13.223131    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-75.ec2.internal" event="NodeReady"
Jul 31 23:47:13.223362 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:13.223247    2564 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 31 23:47:13.260760 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:13.260720    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-774c7c8d4c-hxx2p"]
Jul 31 23:47:13.274530 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:13.274499    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-8gxhj"]
Jul 31 23:47:13.274700 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:13.274643    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-774c7c8d4c-hxx2p"
Jul 31 23:47:13.277652 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:13.277606    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 31 23:47:13.277830 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:13.277719    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-xlm7w\""
Jul 31 23:47:13.277830 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:13.277719    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 31 23:47:13.278003 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:13.277985    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 31 23:47:13.287883 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:13.287861    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 31 23:47:13.288414 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:13.288396    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-774c7c8d4c-hxx2p"]
Jul 31 23:47:13.288516 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:13.288431    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-x5njs"]
Jul 31 23:47:13.288516 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:13.288487    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-8gxhj"
Jul 31 23:47:13.291675 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:13.291618    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 31 23:47:13.291793 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:13.291708    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-x95hq\""
Jul 31 23:47:13.292674 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:13.292657    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 31 23:47:13.310694 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:13.310672    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-8gxhj"]
Jul 31 23:47:13.310818 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:13.310704    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-x5njs"]
Jul 31 23:47:13.310818 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:13.310803    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-x5njs"
Jul 31 23:47:13.317975 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:13.317957    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 31 23:47:13.317975 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:13.317966    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 31 23:47:13.318123 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:13.318040    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-g252s\""
Jul 31 23:47:13.318202 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:13.318191    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 31 23:47:13.318242 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:13.318230    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 31 23:47:13.354825 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:13.354801    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/219c8401-f8b7-474e-ac64-98a06039633f-registry-certificates\") pod \"image-registry-774c7c8d4c-hxx2p\" (UID: \"219c8401-f8b7-474e-ac64-98a06039633f\") " pod="openshift-image-registry/image-registry-774c7c8d4c-hxx2p"
Jul 31 23:47:13.354946 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:13.354828    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/219c8401-f8b7-474e-ac64-98a06039633f-bound-sa-token\") pod \"image-registry-774c7c8d4c-hxx2p\" (UID: \"219c8401-f8b7-474e-ac64-98a06039633f\") " pod="openshift-image-registry/image-registry-774c7c8d4c-hxx2p"
Jul 31 23:47:13.354946 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:13.354858    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/219c8401-f8b7-474e-ac64-98a06039633f-trusted-ca\") pod \"image-registry-774c7c8d4c-hxx2p\" (UID: \"219c8401-f8b7-474e-ac64-98a06039633f\") " pod="openshift-image-registry/image-registry-774c7c8d4c-hxx2p"
Jul 31 23:47:13.354946 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:13.354889    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gtvcr\" (UniqueName: \"kubernetes.io/projected/219c8401-f8b7-474e-ac64-98a06039633f-kube-api-access-gtvcr\") pod \"image-registry-774c7c8d4c-hxx2p\" (UID: \"219c8401-f8b7-474e-ac64-98a06039633f\") " pod="openshift-image-registry/image-registry-774c7c8d4c-hxx2p"
Jul 31 23:47:13.354946 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:13.354915    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/219c8401-f8b7-474e-ac64-98a06039633f-registry-tls\") pod \"image-registry-774c7c8d4c-hxx2p\" (UID: \"219c8401-f8b7-474e-ac64-98a06039633f\") " pod="openshift-image-registry/image-registry-774c7c8d4c-hxx2p"
Jul 31 23:47:13.354946 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:13.354929    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/219c8401-f8b7-474e-ac64-98a06039633f-installation-pull-secrets\") pod \"image-registry-774c7c8d4c-hxx2p\" (UID: \"219c8401-f8b7-474e-ac64-98a06039633f\") " pod="openshift-image-registry/image-registry-774c7c8d4c-hxx2p"
Jul 31 23:47:13.355097 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:13.354986    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/219c8401-f8b7-474e-ac64-98a06039633f-image-registry-private-configuration\") pod \"image-registry-774c7c8d4c-hxx2p\" (UID: \"219c8401-f8b7-474e-ac64-98a06039633f\") " pod="openshift-image-registry/image-registry-774c7c8d4c-hxx2p"
Jul 31 23:47:13.355097 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:13.355012    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/219c8401-f8b7-474e-ac64-98a06039633f-ca-trust-extracted\") pod \"image-registry-774c7c8d4c-hxx2p\" (UID: \"219c8401-f8b7-474e-ac64-98a06039633f\") " pod="openshift-image-registry/image-registry-774c7c8d4c-hxx2p"
Jul 31 23:47:13.394370 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:13.394339    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-j4bjx"]
Jul 31 23:47:13.410295 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:13.410269    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-j4bjx"]
Jul 31 23:47:13.410414 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:13.410358    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-j4bjx"
Jul 31 23:47:13.413480 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:13.413462    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 31 23:47:13.414030 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:13.414018    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-srbwg\""
Jul 31 23:47:13.414095 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:13.414019    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 31 23:47:13.414399 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:13.414383    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 31 23:47:13.456189 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:13.456163    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/22c17973-7031-4c9a-bbea-39d64598f867-metrics-tls\") pod \"dns-default-8gxhj\" (UID: \"22c17973-7031-4c9a-bbea-39d64598f867\") " pod="openshift-dns/dns-default-8gxhj"
Jul 31 23:47:13.456189 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:13.456190    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-krmfk\" (UniqueName: \"kubernetes.io/projected/22c17973-7031-4c9a-bbea-39d64598f867-kube-api-access-krmfk\") pod \"dns-default-8gxhj\" (UID: \"22c17973-7031-4c9a-bbea-39d64598f867\") " pod="openshift-dns/dns-default-8gxhj"
Jul 31 23:47:13.456665 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:13.456219    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/219c8401-f8b7-474e-ac64-98a06039633f-registry-tls\") pod \"image-registry-774c7c8d4c-hxx2p\" (UID: \"219c8401-f8b7-474e-ac64-98a06039633f\") " pod="openshift-image-registry/image-registry-774c7c8d4c-hxx2p"
Jul 31 23:47:13.456665 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:13.456236    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/219c8401-f8b7-474e-ac64-98a06039633f-installation-pull-secrets\") pod \"image-registry-774c7c8d4c-hxx2p\" (UID: \"219c8401-f8b7-474e-ac64-98a06039633f\") " pod="openshift-image-registry/image-registry-774c7c8d4c-hxx2p"
Jul 31 23:47:13.456665 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:13.456275    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/22c17973-7031-4c9a-bbea-39d64598f867-config-volume\") pod \"dns-default-8gxhj\" (UID: \"22c17973-7031-4c9a-bbea-39d64598f867\") " pod="openshift-dns/dns-default-8gxhj"
Jul 31 23:47:13.456665 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:13.456338    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/219c8401-f8b7-474e-ac64-98a06039633f-image-registry-private-configuration\") pod \"image-registry-774c7c8d4c-hxx2p\" (UID: \"219c8401-f8b7-474e-ac64-98a06039633f\") " pod="openshift-image-registry/image-registry-774c7c8d4c-hxx2p"
Jul 31 23:47:13.456665 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:13.456362    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zkxz7\" (UniqueName: \"kubernetes.io/projected/9e3db559-1129-4c9d-b1b8-3861fba9a115-kube-api-access-zkxz7\") pod \"insights-runtime-extractor-x5njs\" (UID: \"9e3db559-1129-4c9d-b1b8-3861fba9a115\") " pod="openshift-insights/insights-runtime-extractor-x5njs"
Jul 31 23:47:13.456665 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:13.456384    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/219c8401-f8b7-474e-ac64-98a06039633f-ca-trust-extracted\") pod \"image-registry-774c7c8d4c-hxx2p\" (UID: \"219c8401-f8b7-474e-ac64-98a06039633f\") " pod="openshift-image-registry/image-registry-774c7c8d4c-hxx2p"
Jul 31 23:47:13.456665 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:13.456407    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/22c17973-7031-4c9a-bbea-39d64598f867-tmp-dir\") pod \"dns-default-8gxhj\" (UID: \"22c17973-7031-4c9a-bbea-39d64598f867\") " pod="openshift-dns/dns-default-8gxhj"
Jul 31 23:47:13.456665 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:13.456444    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/219c8401-f8b7-474e-ac64-98a06039633f-registry-certificates\") pod \"image-registry-774c7c8d4c-hxx2p\" (UID: \"219c8401-f8b7-474e-ac64-98a06039633f\") " pod="openshift-image-registry/image-registry-774c7c8d4c-hxx2p"
Jul 31 23:47:13.456665 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:13.456461    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/219c8401-f8b7-474e-ac64-98a06039633f-bound-sa-token\") pod \"image-registry-774c7c8d4c-hxx2p\" (UID: \"219c8401-f8b7-474e-ac64-98a06039633f\") " pod="openshift-image-registry/image-registry-774c7c8d4c-hxx2p"
Jul 31 23:47:13.456665 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:13.456481    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9e3db559-1129-4c9d-b1b8-3861fba9a115-data-volume\") pod \"insights-runtime-extractor-x5njs\" (UID: \"9e3db559-1129-4c9d-b1b8-3861fba9a115\") " pod="openshift-insights/insights-runtime-extractor-x5njs"
Jul 31 23:47:13.456665 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:13.456521    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/219c8401-f8b7-474e-ac64-98a06039633f-trusted-ca\") pod \"image-registry-774c7c8d4c-hxx2p\" (UID: \"219c8401-f8b7-474e-ac64-98a06039633f\") " pod="openshift-image-registry/image-registry-774c7c8d4c-hxx2p"
Jul 31 23:47:13.456665 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:13.456545    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9e3db559-1129-4c9d-b1b8-3861fba9a115-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-x5njs\" (UID: \"9e3db559-1129-4c9d-b1b8-3861fba9a115\") " pod="openshift-insights/insights-runtime-extractor-x5njs"
Jul 31 23:47:13.456665 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:13.456584    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9e3db559-1129-4c9d-b1b8-3861fba9a115-crio-socket\") pod \"insights-runtime-extractor-x5njs\" (UID: \"9e3db559-1129-4c9d-b1b8-3861fba9a115\") " pod="openshift-insights/insights-runtime-extractor-x5njs"
Jul 31 23:47:13.456665 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:13.456608    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9e3db559-1129-4c9d-b1b8-3861fba9a115-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-x5njs\" (UID: \"9e3db559-1129-4c9d-b1b8-3861fba9a115\") " pod="openshift-insights/insights-runtime-extractor-x5njs"
Jul 31 23:47:13.456665 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:13.456668    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gtvcr\" (UniqueName: \"kubernetes.io/projected/219c8401-f8b7-474e-ac64-98a06039633f-kube-api-access-gtvcr\") pod \"image-registry-774c7c8d4c-hxx2p\" (UID: \"219c8401-f8b7-474e-ac64-98a06039633f\") " pod="openshift-image-registry/image-registry-774c7c8d4c-hxx2p"
Jul 31 23:47:13.457227 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:13.456886    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/219c8401-f8b7-474e-ac64-98a06039633f-ca-trust-extracted\") pod \"image-registry-774c7c8d4c-hxx2p\" (UID: \"219c8401-f8b7-474e-ac64-98a06039633f\") " pod="openshift-image-registry/image-registry-774c7c8d4c-hxx2p"
Jul 31 23:47:13.457378 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:13.457362    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/219c8401-f8b7-474e-ac64-98a06039633f-registry-certificates\") pod \"image-registry-774c7c8d4c-hxx2p\" (UID: \"219c8401-f8b7-474e-ac64-98a06039633f\") " pod="openshift-image-registry/image-registry-774c7c8d4c-hxx2p"
Jul 31 23:47:13.458076 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:13.458055    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/219c8401-f8b7-474e-ac64-98a06039633f-trusted-ca\") pod \"image-registry-774c7c8d4c-hxx2p\" (UID: \"219c8401-f8b7-474e-ac64-98a06039633f\") " pod="openshift-image-registry/image-registry-774c7c8d4c-hxx2p"
Jul 31 23:47:13.460543 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:13.460524    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/219c8401-f8b7-474e-ac64-98a06039633f-image-registry-private-configuration\") pod \"image-registry-774c7c8d4c-hxx2p\" (UID: \"219c8401-f8b7-474e-ac64-98a06039633f\") " pod="openshift-image-registry/image-registry-774c7c8d4c-hxx2p"
Jul 31 23:47:13.460543 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:13.460535    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/219c8401-f8b7-474e-ac64-98a06039633f-registry-tls\") pod \"image-registry-774c7c8d4c-hxx2p\" (UID: \"219c8401-f8b7-474e-ac64-98a06039633f\") " pod="openshift-image-registry/image-registry-774c7c8d4c-hxx2p"
Jul 31 23:47:13.460683 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:13.460553    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/219c8401-f8b7-474e-ac64-98a06039633f-installation-pull-secrets\") pod \"image-registry-774c7c8d4c-hxx2p\" (UID: \"219c8401-f8b7-474e-ac64-98a06039633f\") " pod="openshift-image-registry/image-registry-774c7c8d4c-hxx2p"
Jul 31 23:47:13.468441 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:13.468397    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gtvcr\" (UniqueName: \"kubernetes.io/projected/219c8401-f8b7-474e-ac64-98a06039633f-kube-api-access-gtvcr\") pod \"image-registry-774c7c8d4c-hxx2p\" (UID: \"219c8401-f8b7-474e-ac64-98a06039633f\") " pod="openshift-image-registry/image-registry-774c7c8d4c-hxx2p"
Jul 31 23:47:13.469178 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:13.469120    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/219c8401-f8b7-474e-ac64-98a06039633f-bound-sa-token\") pod \"image-registry-774c7c8d4c-hxx2p\" (UID: \"219c8401-f8b7-474e-ac64-98a06039633f\") " pod="openshift-image-registry/image-registry-774c7c8d4c-hxx2p"
Jul 31 23:47:13.557459 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:13.557376    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9e3db559-1129-4c9d-b1b8-3861fba9a115-data-volume\") pod \"insights-runtime-extractor-x5njs\" (UID: \"9e3db559-1129-4c9d-b1b8-3861fba9a115\") " pod="openshift-insights/insights-runtime-extractor-x5njs"
Jul 31 23:47:13.557459 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:13.557422    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9e3db559-1129-4c9d-b1b8-3861fba9a115-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-x5njs\" (UID: \"9e3db559-1129-4c9d-b1b8-3861fba9a115\") " pod="openshift-insights/insights-runtime-extractor-x5njs"
Jul 31 23:47:13.557459 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:13.557448    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9e3db559-1129-4c9d-b1b8-3861fba9a115-crio-socket\") pod \"insights-runtime-extractor-x5njs\" (UID: \"9e3db559-1129-4c9d-b1b8-3861fba9a115\") " pod="openshift-insights/insights-runtime-extractor-x5njs"
Jul 31 23:47:13.557744 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:13.557465    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9e3db559-1129-4c9d-b1b8-3861fba9a115-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-x5njs\" (UID: \"9e3db559-1129-4c9d-b1b8-3861fba9a115\") " pod="openshift-insights/insights-runtime-extractor-x5njs"
Jul 31 23:47:13.557744 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:13.557486    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bkl2s\" (UniqueName: \"kubernetes.io/projected/3e93b652-caf7-4e5f-9c0a-ef498fa18697-kube-api-access-bkl2s\") pod \"ingress-canary-j4bjx\" (UID: \"3e93b652-caf7-4e5f-9c0a-ef498fa18697\") " pod="openshift-ingress-canary/ingress-canary-j4bjx"
Jul 31 23:47:13.557744 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:13.557522    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3e93b652-caf7-4e5f-9c0a-ef498fa18697-cert\") pod \"ingress-canary-j4bjx\" (UID: \"3e93b652-caf7-4e5f-9c0a-ef498fa18697\") " pod="openshift-ingress-canary/ingress-canary-j4bjx"
Jul 31 23:47:13.557744 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:13.557550    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/22c17973-7031-4c9a-bbea-39d64598f867-metrics-tls\") pod \"dns-default-8gxhj\" (UID: \"22c17973-7031-4c9a-bbea-39d64598f867\") " pod="openshift-dns/dns-default-8gxhj"
Jul 31 23:47:13.557744 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:13.557553    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9e3db559-1129-4c9d-b1b8-3861fba9a115-crio-socket\") pod \"insights-runtime-extractor-x5njs\" (UID: \"9e3db559-1129-4c9d-b1b8-3861fba9a115\") " pod="openshift-insights/insights-runtime-extractor-x5njs"
Jul 31 23:47:13.557744 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:13.557571    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-krmfk\" (UniqueName: \"kubernetes.io/projected/22c17973-7031-4c9a-bbea-39d64598f867-kube-api-access-krmfk\") pod \"dns-default-8gxhj\" (UID: \"22c17973-7031-4c9a-bbea-39d64598f867\") " pod="openshift-dns/dns-default-8gxhj"
Jul 31 23:47:13.557744 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:13.557658    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/22c17973-7031-4c9a-bbea-39d64598f867-config-volume\") pod \"dns-default-8gxhj\" (UID: \"22c17973-7031-4c9a-bbea-39d64598f867\") " pod="openshift-dns/dns-default-8gxhj"
Jul 31 23:47:13.557744 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:13.557728    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zkxz7\" (UniqueName: \"kubernetes.io/projected/9e3db559-1129-4c9d-b1b8-3861fba9a115-kube-api-access-zkxz7\") pod \"insights-runtime-extractor-x5njs\" (UID: \"9e3db559-1129-4c9d-b1b8-3861fba9a115\") " pod="openshift-insights/insights-runtime-extractor-x5njs"
Jul 31 23:47:13.558115 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:13.557763    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/22c17973-7031-4c9a-bbea-39d64598f867-tmp-dir\") pod \"dns-default-8gxhj\" (UID: \"22c17973-7031-4c9a-bbea-39d64598f867\") " pod="openshift-dns/dns-default-8gxhj"
Jul 31 23:47:13.558115 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:13.557819    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9e3db559-1129-4c9d-b1b8-3861fba9a115-data-volume\") pod \"insights-runtime-extractor-x5njs\" (UID: \"9e3db559-1129-4c9d-b1b8-3861fba9a115\") " pod="openshift-insights/insights-runtime-extractor-x5njs"
Jul 31 23:47:13.558115 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:13.558081    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/22c17973-7031-4c9a-bbea-39d64598f867-tmp-dir\") pod \"dns-default-8gxhj\" (UID: \"22c17973-7031-4c9a-bbea-39d64598f867\") " pod="openshift-dns/dns-default-8gxhj"
Jul 31 23:47:13.558115 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:13.558102    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9e3db559-1129-4c9d-b1b8-3861fba9a115-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-x5njs\" (UID: \"9e3db559-1129-4c9d-b1b8-3861fba9a115\") " pod="openshift-insights/insights-runtime-extractor-x5njs"
Jul 31 23:47:13.558255 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:13.558161    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/22c17973-7031-4c9a-bbea-39d64598f867-config-volume\") pod \"dns-default-8gxhj\" (UID: \"22c17973-7031-4c9a-bbea-39d64598f867\") " pod="openshift-dns/dns-default-8gxhj"
Jul 31 23:47:13.559840 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:13.559820    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9e3db559-1129-4c9d-b1b8-3861fba9a115-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-x5njs\" (UID: \"9e3db559-1129-4c9d-b1b8-3861fba9a115\") " pod="openshift-insights/insights-runtime-extractor-x5njs"
Jul 31 23:47:13.559939 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:13.559875    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/22c17973-7031-4c9a-bbea-39d64598f867-metrics-tls\") pod \"dns-default-8gxhj\" (UID: \"22c17973-7031-4c9a-bbea-39d64598f867\") " pod="openshift-dns/dns-default-8gxhj"
Jul 31 23:47:13.566695 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:13.566674    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zkxz7\" (UniqueName: \"kubernetes.io/projected/9e3db559-1129-4c9d-b1b8-3861fba9a115-kube-api-access-zkxz7\") pod \"insights-runtime-extractor-x5njs\" (UID: \"9e3db559-1129-4c9d-b1b8-3861fba9a115\") " pod="openshift-insights/insights-runtime-extractor-x5njs"
Jul 31 23:47:13.566801 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:13.566673    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-krmfk\" (UniqueName: \"kubernetes.io/projected/22c17973-7031-4c9a-bbea-39d64598f867-kube-api-access-krmfk\") pod \"dns-default-8gxhj\" (UID: \"22c17973-7031-4c9a-bbea-39d64598f867\") " pod="openshift-dns/dns-default-8gxhj"
Jul 31 23:47:13.584780 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:13.584756    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-774c7c8d4c-hxx2p"
Jul 31 23:47:13.597919 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:13.597898    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-8gxhj"
Jul 31 23:47:13.618724 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:13.618704    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-x5njs"
Jul 31 23:47:13.659025 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:13.658990    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bkl2s\" (UniqueName: \"kubernetes.io/projected/3e93b652-caf7-4e5f-9c0a-ef498fa18697-kube-api-access-bkl2s\") pod \"ingress-canary-j4bjx\" (UID: \"3e93b652-caf7-4e5f-9c0a-ef498fa18697\") " pod="openshift-ingress-canary/ingress-canary-j4bjx"
Jul 31 23:47:13.659925 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:13.659045    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3e93b652-caf7-4e5f-9c0a-ef498fa18697-cert\") pod \"ingress-canary-j4bjx\" (UID: \"3e93b652-caf7-4e5f-9c0a-ef498fa18697\") " pod="openshift-ingress-canary/ingress-canary-j4bjx"
Jul 31 23:47:13.662431 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:13.662399    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3e93b652-caf7-4e5f-9c0a-ef498fa18697-cert\") pod \"ingress-canary-j4bjx\" (UID: \"3e93b652-caf7-4e5f-9c0a-ef498fa18697\") " pod="openshift-ingress-canary/ingress-canary-j4bjx"
Jul 31 23:47:13.668254 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:13.668223    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bkl2s\" (UniqueName: \"kubernetes.io/projected/3e93b652-caf7-4e5f-9c0a-ef498fa18697-kube-api-access-bkl2s\") pod \"ingress-canary-j4bjx\" (UID: \"3e93b652-caf7-4e5f-9c0a-ef498fa18697\") " pod="openshift-ingress-canary/ingress-canary-j4bjx"
Jul 31 23:47:13.719093 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:13.718814    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-j4bjx"
Jul 31 23:47:13.811672 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:13.811568    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-774c7c8d4c-hxx2p"]
Jul 31 23:47:13.816385 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:47:13.816352    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod219c8401_f8b7_474e_ac64_98a06039633f.slice/crio-84ea139fa880ceec3453ffc2345efba7106a05108fe19465e2cc827b16b71057 WatchSource:0}: Error finding container 84ea139fa880ceec3453ffc2345efba7106a05108fe19465e2cc827b16b71057: Status 404 returned error can't find the container with id 84ea139fa880ceec3453ffc2345efba7106a05108fe19465e2cc827b16b71057
Jul 31 23:47:13.825063 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:13.825025    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-8gxhj"]
Jul 31 23:47:13.828568 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:47:13.828547    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22c17973_7031_4c9a_bbea_39d64598f867.slice/crio-fd373da21844eeaae974f64d288c39473a36a338e79a6c81daa617f9b5754110 WatchSource:0}: Error finding container fd373da21844eeaae974f64d288c39473a36a338e79a6c81daa617f9b5754110: Status 404 returned error can't find the container with id fd373da21844eeaae974f64d288c39473a36a338e79a6c81daa617f9b5754110
Jul 31 23:47:13.881061 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:13.881034    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-j4bjx"]
Jul 31 23:47:13.884401 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:47:13.884366    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e93b652_caf7_4e5f_9c0a_ef498fa18697.slice/crio-1cfd552eb1a29777f8647c8ed7a88344896fdcf8cf3e2102156372a7a33b8da0 WatchSource:0}: Error finding container 1cfd552eb1a29777f8647c8ed7a88344896fdcf8cf3e2102156372a7a33b8da0: Status 404 returned error can't find the container with id 1cfd552eb1a29777f8647c8ed7a88344896fdcf8cf3e2102156372a7a33b8da0
Jul 31 23:47:14.024587 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:14.024557    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-x5njs"]
Jul 31 23:47:14.027864 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:47:14.027835    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e3db559_1129_4c9d_b1b8_3861fba9a115.slice/crio-ab68dcaf729eadeb965bc8df06d47a4c8bf79dc79d9bf49676e28360d9023027 WatchSource:0}: Error finding container ab68dcaf729eadeb965bc8df06d47a4c8bf79dc79d9bf49676e28360d9023027: Status 404 returned error can't find the container with id ab68dcaf729eadeb965bc8df06d47a4c8bf79dc79d9bf49676e28360d9023027
Jul 31 23:47:14.421293 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:14.421264    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h45s5"
Jul 31 23:47:14.421293 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:14.421281    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j97wn"
Jul 31 23:47:14.421507 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:14.421265    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7n4sw"
Jul 31 23:47:14.424098 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:14.424075    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 31 23:47:14.424231 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:14.424212    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 31 23:47:14.425507 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:14.425420    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 31 23:47:14.425653 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:14.425510    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-bfr2g\""
Jul 31 23:47:14.425726 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:14.425665    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-4hvkj\""
Jul 31 23:47:14.425789 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:14.425744    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 31 23:47:14.692916 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:14.692747    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-x5njs" event={"ID":"9e3db559-1129-4c9d-b1b8-3861fba9a115","Type":"ContainerStarted","Data":"21605129684cca7afd706fddb9edaef0f48418435f2429476872af1ebeeb0341"}
Jul 31 23:47:14.692916 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:14.692833    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-x5njs" event={"ID":"9e3db559-1129-4c9d-b1b8-3861fba9a115","Type":"ContainerStarted","Data":"ab68dcaf729eadeb965bc8df06d47a4c8bf79dc79d9bf49676e28360d9023027"}
Jul 31 23:47:14.694235 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:14.694205    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-8gxhj" event={"ID":"22c17973-7031-4c9a-bbea-39d64598f867","Type":"ContainerStarted","Data":"fd373da21844eeaae974f64d288c39473a36a338e79a6c81daa617f9b5754110"}
Jul 31 23:47:14.696545 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:14.696167    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-774c7c8d4c-hxx2p" event={"ID":"219c8401-f8b7-474e-ac64-98a06039633f","Type":"ContainerStarted","Data":"388aad1710c9d1e19bf420cf388553baafbd0a15dcb5e4f1236b35abe4f9627d"}
Jul 31 23:47:14.696545 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:14.696213    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-774c7c8d4c-hxx2p" event={"ID":"219c8401-f8b7-474e-ac64-98a06039633f","Type":"ContainerStarted","Data":"84ea139fa880ceec3453ffc2345efba7106a05108fe19465e2cc827b16b71057"}
Jul 31 23:47:14.696742 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:14.696571    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-774c7c8d4c-hxx2p"
Jul 31 23:47:14.698066 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:14.698039    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-j4bjx" event={"ID":"3e93b652-caf7-4e5f-9c0a-ef498fa18697","Type":"ContainerStarted","Data":"1cfd552eb1a29777f8647c8ed7a88344896fdcf8cf3e2102156372a7a33b8da0"}
Jul 31 23:47:14.718072 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:14.717808    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-774c7c8d4c-hxx2p" podStartSLOduration=2.717788009 podStartE2EDuration="2.717788009s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:47:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:47:14.717020371 +0000 UTC m=+52.829148069" watchObservedRunningTime="2026-07-31 23:47:14.717788009 +0000 UTC m=+52.829915662"
Jul 31 23:47:16.401200 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:16.401162    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-bngc8"]
Jul 31 23:47:16.403853 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:16.403830    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-bngc8"
Jul 31 23:47:16.407931 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:16.407909    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Jul 31 23:47:16.408039 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:16.407910    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-r2qgs\""
Jul 31 23:47:16.427472 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:16.426871    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-bngc8"]
Jul 31 23:47:16.585708 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:16.585596    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/d7ef80c3-fd39-4aae-a789-db9c906ff3fe-tls-certificates\") pod \"prometheus-operator-admission-webhook-66685ff6d4-bngc8\" (UID: \"d7ef80c3-fd39-4aae-a789-db9c906ff3fe\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-bngc8"
Jul 31 23:47:16.686028 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:16.685995    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/d7ef80c3-fd39-4aae-a789-db9c906ff3fe-tls-certificates\") pod \"prometheus-operator-admission-webhook-66685ff6d4-bngc8\" (UID: \"d7ef80c3-fd39-4aae-a789-db9c906ff3fe\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-bngc8"
Jul 31 23:47:16.688437 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:16.688411    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/d7ef80c3-fd39-4aae-a789-db9c906ff3fe-tls-certificates\") pod \"prometheus-operator-admission-webhook-66685ff6d4-bngc8\" (UID: \"d7ef80c3-fd39-4aae-a789-db9c906ff3fe\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-bngc8"
Jul 31 23:47:16.706103 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:16.706069    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-x5njs" event={"ID":"9e3db559-1129-4c9d-b1b8-3861fba9a115","Type":"ContainerStarted","Data":"dd257506680d269d2d1fa9acf18d31daa3a46df12200102681ca316411fc1ea7"}
Jul 31 23:47:16.708033 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:16.708006    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-8gxhj" event={"ID":"22c17973-7031-4c9a-bbea-39d64598f867","Type":"ContainerStarted","Data":"fa8b0af9a0ed347b68182600ec36299c8f0455cdbc246c0f909cf4e75b31ca24"}
Jul 31 23:47:16.708164 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:16.708054    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-8gxhj" event={"ID":"22c17973-7031-4c9a-bbea-39d64598f867","Type":"ContainerStarted","Data":"c3b4944f83be3eb4106065d4fef734fcbaa0cbd9d567a50a66139993fe264535"}
Jul 31 23:47:16.708164 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:16.708106    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-8gxhj"
Jul 31 23:47:16.709567 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:16.709544    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-j4bjx" event={"ID":"3e93b652-caf7-4e5f-9c0a-ef498fa18697","Type":"ContainerStarted","Data":"03cc8604febb2c14ff400864fcfd688e8505336eaf82b255ef79e4b1ba10da8e"}
Jul 31 23:47:16.729599 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:16.729556    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-bngc8"
Jul 31 23:47:16.730240 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:16.730197    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-8gxhj" podStartSLOduration=1.294725901 podStartE2EDuration="3.730178624s" totalImagesPullingTime="2.435452723s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:47:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:47:16.728463676 +0000 UTC m=+54.840591373" watchObservedRunningTime="2026-07-31 23:47:16.730178624 +0000 UTC m=+54.842306295"
Jul 31 23:47:16.873250 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:16.873069    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-j4bjx" podStartSLOduration=1.487309697 podStartE2EDuration="3.873044583s" totalImagesPullingTime="2.385734886s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:47:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:47:16.745771031 +0000 UTC m=+54.857898704" watchObservedRunningTime="2026-07-31 23:47:16.873044583 +0000 UTC m=+54.985172253"
Jul 31 23:47:16.873494 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:16.873430    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-bngc8"]
Jul 31 23:47:16.877249 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:47:16.877214    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7ef80c3_fd39_4aae_a789_db9c906ff3fe.slice/crio-4f589af9c74a1a236f5eac729086bc3a6aa89dc65a3c79ec908c30444fe0b805 WatchSource:0}: Error finding container 4f589af9c74a1a236f5eac729086bc3a6aa89dc65a3c79ec908c30444fe0b805: Status 404 returned error can't find the container with id 4f589af9c74a1a236f5eac729086bc3a6aa89dc65a3c79ec908c30444fe0b805
Jul 31 23:47:17.714675 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:17.714574    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-bngc8" event={"ID":"d7ef80c3-fd39-4aae-a789-db9c906ff3fe","Type":"ContainerStarted","Data":"4f589af9c74a1a236f5eac729086bc3a6aa89dc65a3c79ec908c30444fe0b805"}
Jul 31 23:47:17.716950 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:17.716902    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-x5njs" event={"ID":"9e3db559-1129-4c9d-b1b8-3861fba9a115","Type":"ContainerStarted","Data":"3bd3fa74ee89f79e9e1fcbc4197b15520630a9c7ef7cce005fdd7ad17e039c41"}
Jul 31 23:47:17.736095 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:17.736050    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-x5njs" podStartSLOduration=1.45171738 podStartE2EDuration="4.736035339s" totalImagesPullingTime="3.284317959s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-31 23:47:13 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:47:17.73516609 +0000 UTC m=+55.847293829" watchObservedRunningTime="2026-07-31 23:47:17.736035339 +0000 UTC m=+55.848163012"
Jul 31 23:47:18.721455 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:18.721413    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-bngc8" event={"ID":"d7ef80c3-fd39-4aae-a789-db9c906ff3fe","Type":"ContainerStarted","Data":"238297d86561eaa7af154507631fa604867e32838c5509236cff07d2e6c180cf"}
Jul 31 23:47:18.721890 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:18.721749    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-bngc8"
Jul 31 23:47:18.726351 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:18.726334    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-bngc8"
Jul 31 23:47:18.738417 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:18.738377    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-bngc8" podStartSLOduration=1.5962885500000001 podStartE2EDuration="2.738364337s" totalImagesPullingTime="1.142075787s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:47:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:47:18.737952659 +0000 UTC m=+56.850080330" watchObservedRunningTime="2026-07-31 23:47:18.738364337 +0000 UTC m=+56.850492009"
Jul 31 23:47:21.106503 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:21.106464    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-695467d899-rp2fg"]
Jul 31 23:47:21.138367 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:21.138335    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-695467d899-rp2fg"
Jul 31 23:47:21.138523 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:21.138336    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-695467d899-rp2fg"]
Jul 31 23:47:21.141498 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:21.141483    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 31 23:47:21.142511 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:21.142492    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-kcdp7\""
Jul 31 23:47:21.142511 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:21.142503    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 31 23:47:21.142697 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:21.142526    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 31 23:47:21.142697 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:21.142505    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 31 23:47:21.142697 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:21.142563    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 31 23:47:21.142697 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:21.142589    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 31 23:47:21.142897 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:21.142835    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 31 23:47:21.147689 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:21.147662    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 31 23:47:21.223013 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:21.222981    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/09fe4360-7a24-4668-ad81-3df13cdd2516-trusted-ca-bundle\") pod \"console-695467d899-rp2fg\" (UID: \"09fe4360-7a24-4668-ad81-3df13cdd2516\") " pod="openshift-console/console-695467d899-rp2fg"
Jul 31 23:47:21.223167 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:21.223033    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/09fe4360-7a24-4668-ad81-3df13cdd2516-console-serving-cert\") pod \"console-695467d899-rp2fg\" (UID: \"09fe4360-7a24-4668-ad81-3df13cdd2516\") " pod="openshift-console/console-695467d899-rp2fg"
Jul 31 23:47:21.223167 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:21.223105    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/09fe4360-7a24-4668-ad81-3df13cdd2516-console-config\") pod \"console-695467d899-rp2fg\" (UID: \"09fe4360-7a24-4668-ad81-3df13cdd2516\") " pod="openshift-console/console-695467d899-rp2fg"
Jul 31 23:47:21.223167 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:21.223144    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rt8nc\" (UniqueName: \"kubernetes.io/projected/09fe4360-7a24-4668-ad81-3df13cdd2516-kube-api-access-rt8nc\") pod \"console-695467d899-rp2fg\" (UID: \"09fe4360-7a24-4668-ad81-3df13cdd2516\") " pod="openshift-console/console-695467d899-rp2fg"
Jul 31 23:47:21.223324 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:21.223199    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/09fe4360-7a24-4668-ad81-3df13cdd2516-console-oauth-config\") pod \"console-695467d899-rp2fg\" (UID: \"09fe4360-7a24-4668-ad81-3df13cdd2516\") " pod="openshift-console/console-695467d899-rp2fg"
Jul 31 23:47:21.223324 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:21.223225    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/09fe4360-7a24-4668-ad81-3df13cdd2516-oauth-serving-cert\") pod \"console-695467d899-rp2fg\" (UID: \"09fe4360-7a24-4668-ad81-3df13cdd2516\") " pod="openshift-console/console-695467d899-rp2fg"
Jul 31 23:47:21.223324 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:21.223255    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/09fe4360-7a24-4668-ad81-3df13cdd2516-service-ca\") pod \"console-695467d899-rp2fg\" (UID: \"09fe4360-7a24-4668-ad81-3df13cdd2516\") " pod="openshift-console/console-695467d899-rp2fg"
Jul 31 23:47:21.324133 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:21.324099    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/09fe4360-7a24-4668-ad81-3df13cdd2516-console-config\") pod \"console-695467d899-rp2fg\" (UID: \"09fe4360-7a24-4668-ad81-3df13cdd2516\") " pod="openshift-console/console-695467d899-rp2fg"
Jul 31 23:47:21.324133 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:21.324137    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rt8nc\" (UniqueName: \"kubernetes.io/projected/09fe4360-7a24-4668-ad81-3df13cdd2516-kube-api-access-rt8nc\") pod \"console-695467d899-rp2fg\" (UID: \"09fe4360-7a24-4668-ad81-3df13cdd2516\") " pod="openshift-console/console-695467d899-rp2fg"
Jul 31 23:47:21.324323 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:21.324172    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/09fe4360-7a24-4668-ad81-3df13cdd2516-console-oauth-config\") pod \"console-695467d899-rp2fg\" (UID: \"09fe4360-7a24-4668-ad81-3df13cdd2516\") " pod="openshift-console/console-695467d899-rp2fg"
Jul 31 23:47:21.324323 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:21.324189    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/09fe4360-7a24-4668-ad81-3df13cdd2516-oauth-serving-cert\") pod \"console-695467d899-rp2fg\" (UID: \"09fe4360-7a24-4668-ad81-3df13cdd2516\") " pod="openshift-console/console-695467d899-rp2fg"
Jul 31 23:47:21.324323 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:21.324216    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/09fe4360-7a24-4668-ad81-3df13cdd2516-service-ca\") pod \"console-695467d899-rp2fg\" (UID: \"09fe4360-7a24-4668-ad81-3df13cdd2516\") " pod="openshift-console/console-695467d899-rp2fg"
Jul 31 23:47:21.324323 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:21.324236    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/09fe4360-7a24-4668-ad81-3df13cdd2516-trusted-ca-bundle\") pod \"console-695467d899-rp2fg\" (UID: \"09fe4360-7a24-4668-ad81-3df13cdd2516\") " pod="openshift-console/console-695467d899-rp2fg"
Jul 31 23:47:21.324323 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:21.324269    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/09fe4360-7a24-4668-ad81-3df13cdd2516-console-serving-cert\") pod \"console-695467d899-rp2fg\" (UID: \"09fe4360-7a24-4668-ad81-3df13cdd2516\") " pod="openshift-console/console-695467d899-rp2fg"
Jul 31 23:47:21.324898 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:21.324876    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/09fe4360-7a24-4668-ad81-3df13cdd2516-console-config\") pod \"console-695467d899-rp2fg\" (UID: \"09fe4360-7a24-4668-ad81-3df13cdd2516\") " pod="openshift-console/console-695467d899-rp2fg"
Jul 31 23:47:21.325003 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:21.324928    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/09fe4360-7a24-4668-ad81-3df13cdd2516-service-ca\") pod \"console-695467d899-rp2fg\" (UID: \"09fe4360-7a24-4668-ad81-3df13cdd2516\") " pod="openshift-console/console-695467d899-rp2fg"
Jul 31 23:47:21.325075 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:21.325056    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/09fe4360-7a24-4668-ad81-3df13cdd2516-oauth-serving-cert\") pod \"console-695467d899-rp2fg\" (UID: \"09fe4360-7a24-4668-ad81-3df13cdd2516\") " pod="openshift-console/console-695467d899-rp2fg"
Jul 31 23:47:21.325333 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:21.325312    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/09fe4360-7a24-4668-ad81-3df13cdd2516-trusted-ca-bundle\") pod \"console-695467d899-rp2fg\" (UID: \"09fe4360-7a24-4668-ad81-3df13cdd2516\") " pod="openshift-console/console-695467d899-rp2fg"
Jul 31 23:47:21.326845 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:21.326820    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/09fe4360-7a24-4668-ad81-3df13cdd2516-console-oauth-config\") pod \"console-695467d899-rp2fg\" (UID: \"09fe4360-7a24-4668-ad81-3df13cdd2516\") " pod="openshift-console/console-695467d899-rp2fg"
Jul 31 23:47:21.326931 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:21.326895    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/09fe4360-7a24-4668-ad81-3df13cdd2516-console-serving-cert\") pod \"console-695467d899-rp2fg\" (UID: \"09fe4360-7a24-4668-ad81-3df13cdd2516\") " pod="openshift-console/console-695467d899-rp2fg"
Jul 31 23:47:21.332660 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:21.332622    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rt8nc\" (UniqueName: \"kubernetes.io/projected/09fe4360-7a24-4668-ad81-3df13cdd2516-kube-api-access-rt8nc\") pod \"console-695467d899-rp2fg\" (UID: \"09fe4360-7a24-4668-ad81-3df13cdd2516\") " pod="openshift-console/console-695467d899-rp2fg"
Jul 31 23:47:21.447519 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:21.447448    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-695467d899-rp2fg"
Jul 31 23:47:21.601964 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:21.601934    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-695467d899-rp2fg"]
Jul 31 23:47:21.605033 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:47:21.605010    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09fe4360_7a24_4668_ad81_3df13cdd2516.slice/crio-37e0d171fe1dc5bdab1ac0eff378f61e4b15357b33a78e461ed97ea12354eb28 WatchSource:0}: Error finding container 37e0d171fe1dc5bdab1ac0eff378f61e4b15357b33a78e461ed97ea12354eb28: Status 404 returned error can't find the container with id 37e0d171fe1dc5bdab1ac0eff378f61e4b15357b33a78e461ed97ea12354eb28
Jul 31 23:47:21.731065 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:21.730989    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-695467d899-rp2fg" event={"ID":"09fe4360-7a24-4668-ad81-3df13cdd2516","Type":"ContainerStarted","Data":"37e0d171fe1dc5bdab1ac0eff378f61e4b15357b33a78e461ed97ea12354eb28"}
Jul 31 23:47:22.646569 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:22.646539    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-j5545"
Jul 31 23:47:23.941906 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:23.941867    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-wz649"]
Jul 31 23:47:23.949262 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:23.949237    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-wz649"
Jul 31 23:47:23.954588 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:23.954520    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 31 23:47:23.954748 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:23.954650    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Jul 31 23:47:23.954874 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:23.954776    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 31 23:47:23.955463 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:23.955262    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-z97bh\""
Jul 31 23:47:23.955463 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:23.955298    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 31 23:47:23.961989 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:23.961967    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Jul 31 23:47:23.964742 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:23.964611    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-wz649"]
Jul 31 23:47:23.993129 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:23.993090    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-6bd9664c4-n7p8z"]
Jul 31 23:47:23.996259 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:23.996237    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-jnmzc"]
Jul 31 23:47:23.996385 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:23.996367    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-6bd9664c4-n7p8z"
Jul 31 23:47:24.001566 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:23.999767    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Jul 31 23:47:24.001566 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:24.000034    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Jul 31 23:47:24.001566 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:24.000396    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-jnmzc"
Jul 31 23:47:24.001566 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:24.000681    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-dw229\""
Jul 31 23:47:24.002717 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:24.002401    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Jul 31 23:47:24.007497 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:24.007483    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 31 23:47:24.007807 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:24.007785    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 31 23:47:24.007807 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:24.007804    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-fllzf\""
Jul 31 23:47:24.008520 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:24.008505    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 31 23:47:24.019987 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:24.019962    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-6bd9664c4-n7p8z"]
Jul 31 23:47:24.045417 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:24.045375    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/019f1fa4-9e80-41f5-b48d-82e064824829-metrics-client-ca\") pod \"node-exporter-jnmzc\" (UID: \"019f1fa4-9e80-41f5-b48d-82e064824829\") " pod="openshift-monitoring/node-exporter-jnmzc"
Jul 31 23:47:24.045417 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:24.045416    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/5b8598a5-98e5-4e88-b538-c47bbc69cf7f-volume-directive-shadow\") pod \"kube-state-metrics-6bd9664c4-n7p8z\" (UID: \"5b8598a5-98e5-4e88-b538-c47bbc69cf7f\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-n7p8z"
Jul 31 23:47:24.045647 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:24.045441    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/019f1fa4-9e80-41f5-b48d-82e064824829-node-exporter-tls\") pod \"node-exporter-jnmzc\" (UID: \"019f1fa4-9e80-41f5-b48d-82e064824829\") " pod="openshift-monitoring/node-exporter-jnmzc"
Jul 31 23:47:24.045647 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:24.045469    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c647d292-b5cd-4840-a69c-8c0e9ae41793-openshift-state-metrics-tls\") pod \"openshift-state-metrics-6bb4d7b4dd-wz649\" (UID: \"c647d292-b5cd-4840-a69c-8c0e9ae41793\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-wz649"
Jul 31 23:47:24.045647 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:24.045553    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/019f1fa4-9e80-41f5-b48d-82e064824829-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-jnmzc\" (UID: \"019f1fa4-9e80-41f5-b48d-82e064824829\") " pod="openshift-monitoring/node-exporter-jnmzc"
Jul 31 23:47:24.045647 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:24.045588    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/019f1fa4-9e80-41f5-b48d-82e064824829-root\") pod \"node-exporter-jnmzc\" (UID: \"019f1fa4-9e80-41f5-b48d-82e064824829\") " pod="openshift-monitoring/node-exporter-jnmzc"
Jul 31 23:47:24.045647 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:24.045611    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/019f1fa4-9e80-41f5-b48d-82e064824829-node-exporter-wtmp\") pod \"node-exporter-jnmzc\" (UID: \"019f1fa4-9e80-41f5-b48d-82e064824829\") " pod="openshift-monitoring/node-exporter-jnmzc"
Jul 31 23:47:24.045881 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:24.045676    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c647d292-b5cd-4840-a69c-8c0e9ae41793-metrics-client-ca\") pod \"openshift-state-metrics-6bb4d7b4dd-wz649\" (UID: \"c647d292-b5cd-4840-a69c-8c0e9ae41793\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-wz649"
Jul 31 23:47:24.045881 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:24.045702    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/5b8598a5-98e5-4e88-b538-c47bbc69cf7f-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-6bd9664c4-n7p8z\" (UID: \"5b8598a5-98e5-4e88-b538-c47bbc69cf7f\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-n7p8z"
Jul 31 23:47:24.045881 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:24.045730    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/019f1fa4-9e80-41f5-b48d-82e064824829-node-exporter-textfile\") pod \"node-exporter-jnmzc\" (UID: \"019f1fa4-9e80-41f5-b48d-82e064824829\") " pod="openshift-monitoring/node-exporter-jnmzc"
Jul 31 23:47:24.045881 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:24.045761    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5b8598a5-98e5-4e88-b538-c47bbc69cf7f-metrics-client-ca\") pod \"kube-state-metrics-6bd9664c4-n7p8z\" (UID: \"5b8598a5-98e5-4e88-b538-c47bbc69cf7f\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-n7p8z"
Jul 31 23:47:24.045881 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:24.045816    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c647d292-b5cd-4840-a69c-8c0e9ae41793-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-6bb4d7b4dd-wz649\" (UID: \"c647d292-b5cd-4840-a69c-8c0e9ae41793\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-wz649"
Jul 31 23:47:24.045881 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:24.045840    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/5b8598a5-98e5-4e88-b538-c47bbc69cf7f-kube-state-metrics-tls\") pod \"kube-state-metrics-6bd9664c4-n7p8z\" (UID: \"5b8598a5-98e5-4e88-b538-c47bbc69cf7f\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-n7p8z"
Jul 31 23:47:24.045881 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:24.045863    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/019f1fa4-9e80-41f5-b48d-82e064824829-sys\") pod \"node-exporter-jnmzc\" (UID: \"019f1fa4-9e80-41f5-b48d-82e064824829\") " pod="openshift-monitoring/node-exporter-jnmzc"
Jul 31 23:47:24.046186 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:24.046001    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l2r6z\" (UniqueName: \"kubernetes.io/projected/c647d292-b5cd-4840-a69c-8c0e9ae41793-kube-api-access-l2r6z\") pod \"openshift-state-metrics-6bb4d7b4dd-wz649\" (UID: \"c647d292-b5cd-4840-a69c-8c0e9ae41793\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-wz649"
Jul 31 23:47:24.046186 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:24.046037    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tf6hk\" (UniqueName: \"kubernetes.io/projected/5b8598a5-98e5-4e88-b538-c47bbc69cf7f-kube-api-access-tf6hk\") pod \"kube-state-metrics-6bd9664c4-n7p8z\" (UID: \"5b8598a5-98e5-4e88-b538-c47bbc69cf7f\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-n7p8z"
Jul 31 23:47:24.046186 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:24.046109    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pbcp4\" (UniqueName: \"kubernetes.io/projected/019f1fa4-9e80-41f5-b48d-82e064824829-kube-api-access-pbcp4\") pod \"node-exporter-jnmzc\" (UID: \"019f1fa4-9e80-41f5-b48d-82e064824829\") " pod="openshift-monitoring/node-exporter-jnmzc"
Jul 31 23:47:24.046186 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:24.046139    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5b8598a5-98e5-4e88-b538-c47bbc69cf7f-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-6bd9664c4-n7p8z\" (UID: \"5b8598a5-98e5-4e88-b538-c47bbc69cf7f\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-n7p8z"
Jul 31 23:47:24.046353 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:24.046184    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/019f1fa4-9e80-41f5-b48d-82e064824829-node-exporter-accelerators-collector-config\") pod \"node-exporter-jnmzc\" (UID: \"019f1fa4-9e80-41f5-b48d-82e064824829\") " pod="openshift-monitoring/node-exporter-jnmzc"
Jul 31 23:47:24.146687 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:24.146648    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c647d292-b5cd-4840-a69c-8c0e9ae41793-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-6bb4d7b4dd-wz649\" (UID: \"c647d292-b5cd-4840-a69c-8c0e9ae41793\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-wz649"
Jul 31 23:47:24.146884 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:24.146699    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/5b8598a5-98e5-4e88-b538-c47bbc69cf7f-kube-state-metrics-tls\") pod \"kube-state-metrics-6bd9664c4-n7p8z\" (UID: \"5b8598a5-98e5-4e88-b538-c47bbc69cf7f\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-n7p8z"
Jul 31 23:47:24.146884 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:24.146724    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/019f1fa4-9e80-41f5-b48d-82e064824829-sys\") pod \"node-exporter-jnmzc\" (UID: \"019f1fa4-9e80-41f5-b48d-82e064824829\") " pod="openshift-monitoring/node-exporter-jnmzc"
Jul 31 23:47:24.146884 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:24.146755    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l2r6z\" (UniqueName: \"kubernetes.io/projected/c647d292-b5cd-4840-a69c-8c0e9ae41793-kube-api-access-l2r6z\") pod \"openshift-state-metrics-6bb4d7b4dd-wz649\" (UID: \"c647d292-b5cd-4840-a69c-8c0e9ae41793\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-wz649"
Jul 31 23:47:24.146884 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:24.146781    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tf6hk\" (UniqueName: \"kubernetes.io/projected/5b8598a5-98e5-4e88-b538-c47bbc69cf7f-kube-api-access-tf6hk\") pod \"kube-state-metrics-6bd9664c4-n7p8z\" (UID: \"5b8598a5-98e5-4e88-b538-c47bbc69cf7f\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-n7p8z"
Jul 31 23:47:24.147104 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:24.146868    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/019f1fa4-9e80-41f5-b48d-82e064824829-sys\") pod \"node-exporter-jnmzc\" (UID: \"019f1fa4-9e80-41f5-b48d-82e064824829\") " pod="openshift-monitoring/node-exporter-jnmzc"
Jul 31 23:47:24.147104 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:24.146941    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pbcp4\" (UniqueName: \"kubernetes.io/projected/019f1fa4-9e80-41f5-b48d-82e064824829-kube-api-access-pbcp4\") pod \"node-exporter-jnmzc\" (UID: \"019f1fa4-9e80-41f5-b48d-82e064824829\") " pod="openshift-monitoring/node-exporter-jnmzc"
Jul 31 23:47:24.147104 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:24.146982    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5b8598a5-98e5-4e88-b538-c47bbc69cf7f-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-6bd9664c4-n7p8z\" (UID: \"5b8598a5-98e5-4e88-b538-c47bbc69cf7f\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-n7p8z"
Jul 31 23:47:24.147104 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:24.147031    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/019f1fa4-9e80-41f5-b48d-82e064824829-node-exporter-accelerators-collector-config\") pod \"node-exporter-jnmzc\" (UID: \"019f1fa4-9e80-41f5-b48d-82e064824829\") " pod="openshift-monitoring/node-exporter-jnmzc"
Jul 31 23:47:24.147104 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:24.147068    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/019f1fa4-9e80-41f5-b48d-82e064824829-metrics-client-ca\") pod \"node-exporter-jnmzc\" (UID: \"019f1fa4-9e80-41f5-b48d-82e064824829\") " pod="openshift-monitoring/node-exporter-jnmzc"
Jul 31 23:47:24.147281 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:47:24.147245    2564 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found
Jul 31 23:47:24.147281 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:24.147250    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/5b8598a5-98e5-4e88-b538-c47bbc69cf7f-volume-directive-shadow\") pod \"kube-state-metrics-6bd9664c4-n7p8z\" (UID: \"5b8598a5-98e5-4e88-b538-c47bbc69cf7f\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-n7p8z"
Jul 31 23:47:24.147338 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:24.147282    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/019f1fa4-9e80-41f5-b48d-82e064824829-node-exporter-tls\") pod \"node-exporter-jnmzc\" (UID: \"019f1fa4-9e80-41f5-b48d-82e064824829\") " pod="openshift-monitoring/node-exporter-jnmzc"
Jul 31 23:47:24.147338 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:47:24.147308    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5b8598a5-98e5-4e88-b538-c47bbc69cf7f-kube-state-metrics-tls podName:5b8598a5-98e5-4e88-b538-c47bbc69cf7f nodeName:}" failed. No retries permitted until 2026-07-31 23:47:24.647286646 +0000 UTC m=+62.759414315 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/5b8598a5-98e5-4e88-b538-c47bbc69cf7f-kube-state-metrics-tls") pod "kube-state-metrics-6bd9664c4-n7p8z" (UID: "5b8598a5-98e5-4e88-b538-c47bbc69cf7f") : secret "kube-state-metrics-tls" not found
Jul 31 23:47:24.147446 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:24.147342    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c647d292-b5cd-4840-a69c-8c0e9ae41793-openshift-state-metrics-tls\") pod \"openshift-state-metrics-6bb4d7b4dd-wz649\" (UID: \"c647d292-b5cd-4840-a69c-8c0e9ae41793\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-wz649"
Jul 31 23:47:24.147446 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:47:24.147349    2564 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 31 23:47:24.147446 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:47:24.147386    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/019f1fa4-9e80-41f5-b48d-82e064824829-node-exporter-tls podName:019f1fa4-9e80-41f5-b48d-82e064824829 nodeName:}" failed. No retries permitted until 2026-07-31 23:47:24.647374173 +0000 UTC m=+62.759501840 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/019f1fa4-9e80-41f5-b48d-82e064824829-node-exporter-tls") pod "node-exporter-jnmzc" (UID: "019f1fa4-9e80-41f5-b48d-82e064824829") : secret "node-exporter-tls" not found
Jul 31 23:47:24.147446 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:24.147402    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/019f1fa4-9e80-41f5-b48d-82e064824829-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-jnmzc\" (UID: \"019f1fa4-9e80-41f5-b48d-82e064824829\") " pod="openshift-monitoring/node-exporter-jnmzc"
Jul 31 23:47:24.147446 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:47:24.147421    2564 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found
Jul 31 23:47:24.147446 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:24.147432    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/019f1fa4-9e80-41f5-b48d-82e064824829-root\") pod \"node-exporter-jnmzc\" (UID: \"019f1fa4-9e80-41f5-b48d-82e064824829\") " pod="openshift-monitoring/node-exporter-jnmzc"
Jul 31 23:47:24.147790 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:47:24.147454    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c647d292-b5cd-4840-a69c-8c0e9ae41793-openshift-state-metrics-tls podName:c647d292-b5cd-4840-a69c-8c0e9ae41793 nodeName:}" failed. No retries permitted until 2026-07-31 23:47:24.647444414 +0000 UTC m=+62.759572066 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/c647d292-b5cd-4840-a69c-8c0e9ae41793-openshift-state-metrics-tls") pod "openshift-state-metrics-6bb4d7b4dd-wz649" (UID: "c647d292-b5cd-4840-a69c-8c0e9ae41793") : secret "openshift-state-metrics-tls" not found
Jul 31 23:47:24.147790 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:24.147471    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/019f1fa4-9e80-41f5-b48d-82e064824829-node-exporter-wtmp\") pod \"node-exporter-jnmzc\" (UID: \"019f1fa4-9e80-41f5-b48d-82e064824829\") " pod="openshift-monitoring/node-exporter-jnmzc"
Jul 31 23:47:24.147790 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:24.147487    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/019f1fa4-9e80-41f5-b48d-82e064824829-root\") pod \"node-exporter-jnmzc\" (UID: \"019f1fa4-9e80-41f5-b48d-82e064824829\") " pod="openshift-monitoring/node-exporter-jnmzc"
Jul 31 23:47:24.147790 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:24.147526    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c647d292-b5cd-4840-a69c-8c0e9ae41793-metrics-client-ca\") pod \"openshift-state-metrics-6bb4d7b4dd-wz649\" (UID: \"c647d292-b5cd-4840-a69c-8c0e9ae41793\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-wz649"
Jul 31 23:47:24.147790 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:24.147684    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/019f1fa4-9e80-41f5-b48d-82e064824829-node-exporter-wtmp\") pod \"node-exporter-jnmzc\" (UID: \"019f1fa4-9e80-41f5-b48d-82e064824829\") " pod="openshift-monitoring/node-exporter-jnmzc"
Jul 31 23:47:24.147790 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:24.147754    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/019f1fa4-9e80-41f5-b48d-82e064824829-metrics-client-ca\") pod \"node-exporter-jnmzc\" (UID: \"019f1fa4-9e80-41f5-b48d-82e064824829\") " pod="openshift-monitoring/node-exporter-jnmzc"
Jul 31 23:47:24.148078 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:24.147797    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/019f1fa4-9e80-41f5-b48d-82e064824829-node-exporter-accelerators-collector-config\") pod \"node-exporter-jnmzc\" (UID: \"019f1fa4-9e80-41f5-b48d-82e064824829\") " pod="openshift-monitoring/node-exporter-jnmzc"
Jul 31 23:47:24.148078 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:24.147800    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/5b8598a5-98e5-4e88-b538-c47bbc69cf7f-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-6bd9664c4-n7p8z\" (UID: \"5b8598a5-98e5-4e88-b538-c47bbc69cf7f\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-n7p8z"
Jul 31 23:47:24.148078 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:24.147853    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/5b8598a5-98e5-4e88-b538-c47bbc69cf7f-volume-directive-shadow\") pod \"kube-state-metrics-6bd9664c4-n7p8z\" (UID: \"5b8598a5-98e5-4e88-b538-c47bbc69cf7f\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-n7p8z"
Jul 31 23:47:24.148078 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:24.147868    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/019f1fa4-9e80-41f5-b48d-82e064824829-node-exporter-textfile\") pod \"node-exporter-jnmzc\" (UID: \"019f1fa4-9e80-41f5-b48d-82e064824829\") " pod="openshift-monitoring/node-exporter-jnmzc"
Jul 31 23:47:24.148078 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:24.147913    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5b8598a5-98e5-4e88-b538-c47bbc69cf7f-metrics-client-ca\") pod \"kube-state-metrics-6bd9664c4-n7p8z\" (UID: \"5b8598a5-98e5-4e88-b538-c47bbc69cf7f\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-n7p8z"
Jul 31 23:47:24.148319 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:24.148168    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/019f1fa4-9e80-41f5-b48d-82e064824829-node-exporter-textfile\") pod \"node-exporter-jnmzc\" (UID: \"019f1fa4-9e80-41f5-b48d-82e064824829\") " pod="openshift-monitoring/node-exporter-jnmzc"
Jul 31 23:47:24.148319 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:24.148208    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c647d292-b5cd-4840-a69c-8c0e9ae41793-metrics-client-ca\") pod \"openshift-state-metrics-6bb4d7b4dd-wz649\" (UID: \"c647d292-b5cd-4840-a69c-8c0e9ae41793\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-wz649"
Jul 31 23:47:24.148475 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:24.148454    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/5b8598a5-98e5-4e88-b538-c47bbc69cf7f-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-6bd9664c4-n7p8z\" (UID: \"5b8598a5-98e5-4e88-b538-c47bbc69cf7f\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-n7p8z"
Jul 31 23:47:24.148616 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:24.148574    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5b8598a5-98e5-4e88-b538-c47bbc69cf7f-metrics-client-ca\") pod \"kube-state-metrics-6bd9664c4-n7p8z\" (UID: \"5b8598a5-98e5-4e88-b538-c47bbc69cf7f\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-n7p8z"
Jul 31 23:47:24.149961 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:24.149939    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5b8598a5-98e5-4e88-b538-c47bbc69cf7f-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-6bd9664c4-n7p8z\" (UID: \"5b8598a5-98e5-4e88-b538-c47bbc69cf7f\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-n7p8z"
Jul 31 23:47:24.149961 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:24.149949    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c647d292-b5cd-4840-a69c-8c0e9ae41793-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-6bb4d7b4dd-wz649\" (UID: \"c647d292-b5cd-4840-a69c-8c0e9ae41793\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-wz649"
Jul 31 23:47:24.150496 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:24.150475    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/019f1fa4-9e80-41f5-b48d-82e064824829-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-jnmzc\" (UID: \"019f1fa4-9e80-41f5-b48d-82e064824829\") " pod="openshift-monitoring/node-exporter-jnmzc"
Jul 31 23:47:24.158386 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:24.158362    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l2r6z\" (UniqueName: \"kubernetes.io/projected/c647d292-b5cd-4840-a69c-8c0e9ae41793-kube-api-access-l2r6z\") pod \"openshift-state-metrics-6bb4d7b4dd-wz649\" (UID: \"c647d292-b5cd-4840-a69c-8c0e9ae41793\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-wz649"
Jul 31 23:47:24.159893 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:24.159850    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pbcp4\" (UniqueName: \"kubernetes.io/projected/019f1fa4-9e80-41f5-b48d-82e064824829-kube-api-access-pbcp4\") pod \"node-exporter-jnmzc\" (UID: \"019f1fa4-9e80-41f5-b48d-82e064824829\") " pod="openshift-monitoring/node-exporter-jnmzc"
Jul 31 23:47:24.160088 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:24.160070    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tf6hk\" (UniqueName: \"kubernetes.io/projected/5b8598a5-98e5-4e88-b538-c47bbc69cf7f-kube-api-access-tf6hk\") pod \"kube-state-metrics-6bd9664c4-n7p8z\" (UID: \"5b8598a5-98e5-4e88-b538-c47bbc69cf7f\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-n7p8z"
Jul 31 23:47:24.653226 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:24.653196    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/5b8598a5-98e5-4e88-b538-c47bbc69cf7f-kube-state-metrics-tls\") pod \"kube-state-metrics-6bd9664c4-n7p8z\" (UID: \"5b8598a5-98e5-4e88-b538-c47bbc69cf7f\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-n7p8z"
Jul 31 23:47:24.653391 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:24.653254    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/019f1fa4-9e80-41f5-b48d-82e064824829-node-exporter-tls\") pod \"node-exporter-jnmzc\" (UID: \"019f1fa4-9e80-41f5-b48d-82e064824829\") " pod="openshift-monitoring/node-exporter-jnmzc"
Jul 31 23:47:24.653391 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:24.653278    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c647d292-b5cd-4840-a69c-8c0e9ae41793-openshift-state-metrics-tls\") pod \"openshift-state-metrics-6bb4d7b4dd-wz649\" (UID: \"c647d292-b5cd-4840-a69c-8c0e9ae41793\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-wz649"
Jul 31 23:47:24.653491 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:47:24.653467    2564 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found
Jul 31 23:47:24.653571 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:47:24.653558    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c647d292-b5cd-4840-a69c-8c0e9ae41793-openshift-state-metrics-tls podName:c647d292-b5cd-4840-a69c-8c0e9ae41793 nodeName:}" failed. No retries permitted until 2026-07-31 23:47:25.653535796 +0000 UTC m=+63.765663984 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/c647d292-b5cd-4840-a69c-8c0e9ae41793-openshift-state-metrics-tls") pod "openshift-state-metrics-6bb4d7b4dd-wz649" (UID: "c647d292-b5cd-4840-a69c-8c0e9ae41793") : secret "openshift-state-metrics-tls" not found
Jul 31 23:47:24.655645 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:24.655602    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/019f1fa4-9e80-41f5-b48d-82e064824829-node-exporter-tls\") pod \"node-exporter-jnmzc\" (UID: \"019f1fa4-9e80-41f5-b48d-82e064824829\") " pod="openshift-monitoring/node-exporter-jnmzc"
Jul 31 23:47:24.655752 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:24.655719    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/5b8598a5-98e5-4e88-b538-c47bbc69cf7f-kube-state-metrics-tls\") pod \"kube-state-metrics-6bd9664c4-n7p8z\" (UID: \"5b8598a5-98e5-4e88-b538-c47bbc69cf7f\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-n7p8z"
Jul 31 23:47:24.742163 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:24.742134    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-695467d899-rp2fg" event={"ID":"09fe4360-7a24-4668-ad81-3df13cdd2516","Type":"ContainerStarted","Data":"a477d94e3b2262042596715e88866a7a8b0de3b5d297ce197a4a37841f670e1c"}
Jul 31 23:47:24.761560 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:24.761515    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-695467d899-rp2fg" podStartSLOduration=1.117625161 podStartE2EDuration="3.761499557s" totalImagesPullingTime="2.643874396s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:47:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:47:24.760660888 +0000 UTC m=+62.872788583" watchObservedRunningTime="2026-07-31 23:47:24.761499557 +0000 UTC m=+62.873627230"
Jul 31 23:47:24.908574 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:24.908490    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-6bd9664c4-n7p8z"
Jul 31 23:47:24.914637 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:24.914611    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-jnmzc"
Jul 31 23:47:24.923226 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:47:24.923201    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod019f1fa4_9e80_41f5_b48d_82e064824829.slice/crio-9145a75dc84e91d2fc2a24e503c8b124a89a9fbe278b85a0ae3ce86d458fec9b WatchSource:0}: Error finding container 9145a75dc84e91d2fc2a24e503c8b124a89a9fbe278b85a0ae3ce86d458fec9b: Status 404 returned error can't find the container with id 9145a75dc84e91d2fc2a24e503c8b124a89a9fbe278b85a0ae3ce86d458fec9b
Jul 31 23:47:25.045006 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:25.044975    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-6bd9664c4-n7p8z"]
Jul 31 23:47:25.048454 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:47:25.048426    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b8598a5_98e5_4e88_b538_c47bbc69cf7f.slice/crio-c6db9cbe5751b39b16b20b1db8c507439cfd4fc8bf7d230eab52c6cd76c6dba0 WatchSource:0}: Error finding container c6db9cbe5751b39b16b20b1db8c507439cfd4fc8bf7d230eab52c6cd76c6dba0: Status 404 returned error can't find the container with id c6db9cbe5751b39b16b20b1db8c507439cfd4fc8bf7d230eab52c6cd76c6dba0
Jul 31 23:47:25.663495 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:25.663462    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c647d292-b5cd-4840-a69c-8c0e9ae41793-openshift-state-metrics-tls\") pod \"openshift-state-metrics-6bb4d7b4dd-wz649\" (UID: \"c647d292-b5cd-4840-a69c-8c0e9ae41793\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-wz649"
Jul 31 23:47:25.666659 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:25.666614    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c647d292-b5cd-4840-a69c-8c0e9ae41793-openshift-state-metrics-tls\") pod \"openshift-state-metrics-6bb4d7b4dd-wz649\" (UID: \"c647d292-b5cd-4840-a69c-8c0e9ae41793\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-wz649"
Jul 31 23:47:25.749560 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:25.749522    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-jnmzc" event={"ID":"019f1fa4-9e80-41f5-b48d-82e064824829","Type":"ContainerStarted","Data":"9145a75dc84e91d2fc2a24e503c8b124a89a9fbe278b85a0ae3ce86d458fec9b"}
Jul 31 23:47:25.750896 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:25.750870    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-6bd9664c4-n7p8z" event={"ID":"5b8598a5-98e5-4e88-b538-c47bbc69cf7f","Type":"ContainerStarted","Data":"c6db9cbe5751b39b16b20b1db8c507439cfd4fc8bf7d230eab52c6cd76c6dba0"}
Jul 31 23:47:25.760520 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:25.760497    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-wz649"
Jul 31 23:47:25.944943 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:25.944915    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-wz649"]
Jul 31 23:47:26.245229 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:47:26.245150    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc647d292_b5cd_4840_a69c_8c0e9ae41793.slice/crio-0ac0ba3999182148517fa59399739239a4749f1ef2497454e915f91deebfa332 WatchSource:0}: Error finding container 0ac0ba3999182148517fa59399739239a4749f1ef2497454e915f91deebfa332: Status 404 returned error can't find the container with id 0ac0ba3999182148517fa59399739239a4749f1ef2497454e915f91deebfa332
Jul 31 23:47:26.719511 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:26.719482    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-8gxhj"
Jul 31 23:47:26.757015 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:26.756985    2564 generic.go:356] "Generic (PLEG): container finished" podID="019f1fa4-9e80-41f5-b48d-82e064824829" containerID="e01be1e4ec092d41a239781bd1fce1652ebf717f5edbd5bff9e8055af4edf2bd" exitCode=0
Jul 31 23:47:26.757177 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:26.757069    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-jnmzc" event={"ID":"019f1fa4-9e80-41f5-b48d-82e064824829","Type":"ContainerDied","Data":"e01be1e4ec092d41a239781bd1fce1652ebf717f5edbd5bff9e8055af4edf2bd"}
Jul 31 23:47:26.759232 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:26.759202    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-wz649" event={"ID":"c647d292-b5cd-4840-a69c-8c0e9ae41793","Type":"ContainerStarted","Data":"3f6b64bc7235121157be049a49f07a24ba3a731051255d535bcfac7478a3d4d9"}
Jul 31 23:47:26.759346 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:26.759265    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-wz649" event={"ID":"c647d292-b5cd-4840-a69c-8c0e9ae41793","Type":"ContainerStarted","Data":"ebf6843f5f618e17e883b54e11651fd7006063a63a9e424e8e607f6534c6df93"}
Jul 31 23:47:26.759346 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:26.759309    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-wz649" event={"ID":"c647d292-b5cd-4840-a69c-8c0e9ae41793","Type":"ContainerStarted","Data":"0ac0ba3999182148517fa59399739239a4749f1ef2497454e915f91deebfa332"}
Jul 31 23:47:26.762123 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:26.762101    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-6bd9664c4-n7p8z" event={"ID":"5b8598a5-98e5-4e88-b538-c47bbc69cf7f","Type":"ContainerStarted","Data":"05f188f84547bb9fcdb9f19e58821e5f1a4396386bbd07be6b1abd277e18db69"}
Jul 31 23:47:26.762209 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:26.762156    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-6bd9664c4-n7p8z" event={"ID":"5b8598a5-98e5-4e88-b538-c47bbc69cf7f","Type":"ContainerStarted","Data":"9698c389b1bbc8908637684e1194309bf1a58a1fa5413dc2752f14fe0b2e2466"}
Jul 31 23:47:26.762209 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:26.762185    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-6bd9664c4-n7p8z" event={"ID":"5b8598a5-98e5-4e88-b538-c47bbc69cf7f","Type":"ContainerStarted","Data":"50d478f1917b2fe8f797ad19bccea0fc3963389673554c86a1753ace4796ade3"}
Jul 31 23:47:26.794841 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:26.794793    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-6bd9664c4-n7p8z" podStartSLOduration=2.573154756 podStartE2EDuration="3.794773388s" totalImagesPullingTime="1.221618632s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:47:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:47:26.793931981 +0000 UTC m=+64.906059677" watchObservedRunningTime="2026-07-31 23:47:26.794773388 +0000 UTC m=+64.906901039"
Jul 31 23:47:27.178534 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:27.178496    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/69e78750-573b-4dbd-a585-cf0d88056f4f-original-pull-secret\") pod \"global-pull-secret-syncer-h45s5\" (UID: \"69e78750-573b-4dbd-a585-cf0d88056f4f\") " pod="kube-system/global-pull-secret-syncer-h45s5"
Jul 31 23:47:27.178823 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:27.178606    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/815ee183-a99c-4636-9937-da826945f531-metrics-certs\") pod \"network-metrics-daemon-j97wn\" (UID: \"815ee183-a99c-4636-9937-da826945f531\") " pod="openshift-multus/network-metrics-daemon-j97wn"
Jul 31 23:47:27.181583 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:27.181560    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 31 23:47:27.181720 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:27.181681    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 31 23:47:27.191326 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:27.191305    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/815ee183-a99c-4636-9937-da826945f531-metrics-certs\") pod \"network-metrics-daemon-j97wn\" (UID: \"815ee183-a99c-4636-9937-da826945f531\") " pod="openshift-multus/network-metrics-daemon-j97wn"
Jul 31 23:47:27.191439 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:27.191359    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/69e78750-573b-4dbd-a585-cf0d88056f4f-original-pull-secret\") pod \"global-pull-secret-syncer-h45s5\" (UID: \"69e78750-573b-4dbd-a585-cf0d88056f4f\") " pod="kube-system/global-pull-secret-syncer-h45s5"
Jul 31 23:47:27.279580 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:27.279557    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cv7k5\" (UniqueName: \"kubernetes.io/projected/2887ad3f-73c9-4fce-8359-c86e96300a84-kube-api-access-cv7k5\") pod \"network-check-target-7n4sw\" (UID: \"2887ad3f-73c9-4fce-8359-c86e96300a84\") " pod="openshift-network-diagnostics/network-check-target-7n4sw"
Jul 31 23:47:27.282552 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:27.282535    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 31 23:47:27.292080 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:27.292059    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 31 23:47:27.302251 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:27.302233    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cv7k5\" (UniqueName: \"kubernetes.io/projected/2887ad3f-73c9-4fce-8359-c86e96300a84-kube-api-access-cv7k5\") pod \"network-check-target-7n4sw\" (UID: \"2887ad3f-73c9-4fce-8359-c86e96300a84\") " pod="openshift-network-diagnostics/network-check-target-7n4sw"
Jul 31 23:47:27.334635 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:27.334606    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h45s5"
Jul 31 23:47:27.345016 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:27.344995    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-bfr2g\""
Jul 31 23:47:27.351241 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:27.351219    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-4hvkj\""
Jul 31 23:47:27.353165 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:27.353153    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7n4sw"
Jul 31 23:47:27.359941 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:27.359925    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j97wn"
Jul 31 23:47:27.498193 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:27.498162    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-h45s5"]
Jul 31 23:47:27.503021 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:47:27.502989    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69e78750_573b_4dbd_a585_cf0d88056f4f.slice/crio-65eccbc45216d07afbdb146d47fe435bf79f845f7baece192926b6eb70a16af6 WatchSource:0}: Error finding container 65eccbc45216d07afbdb146d47fe435bf79f845f7baece192926b6eb70a16af6: Status 404 returned error can't find the container with id 65eccbc45216d07afbdb146d47fe435bf79f845f7baece192926b6eb70a16af6
Jul 31 23:47:27.524281 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:27.524187    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-j97wn"]
Jul 31 23:47:27.526180 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:47:27.526156    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod815ee183_a99c_4636_9937_da826945f531.slice/crio-8d56ea9fcfb13add14733df076b44bab49c639bca09d7f6ede24e3c51d0c46f6 WatchSource:0}: Error finding container 8d56ea9fcfb13add14733df076b44bab49c639bca09d7f6ede24e3c51d0c46f6: Status 404 returned error can't find the container with id 8d56ea9fcfb13add14733df076b44bab49c639bca09d7f6ede24e3c51d0c46f6
Jul 31 23:47:27.539795 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:27.539772    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-7n4sw"]
Jul 31 23:47:27.542470 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:47:27.542445    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2887ad3f_73c9_4fce_8359_c86e96300a84.slice/crio-83a008d50203cf9ffee9e678726c9b0db9dbd788bf36c2c326e16f0b9187a3cc WatchSource:0}: Error finding container 83a008d50203cf9ffee9e678726c9b0db9dbd788bf36c2c326e16f0b9187a3cc: Status 404 returned error can't find the container with id 83a008d50203cf9ffee9e678726c9b0db9dbd788bf36c2c326e16f0b9187a3cc
Jul 31 23:47:27.770888 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:27.770791    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-7n4sw" event={"ID":"2887ad3f-73c9-4fce-8359-c86e96300a84","Type":"ContainerStarted","Data":"83a008d50203cf9ffee9e678726c9b0db9dbd788bf36c2c326e16f0b9187a3cc"}
Jul 31 23:47:27.771907 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:27.771878    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-h45s5" event={"ID":"69e78750-573b-4dbd-a585-cf0d88056f4f","Type":"ContainerStarted","Data":"65eccbc45216d07afbdb146d47fe435bf79f845f7baece192926b6eb70a16af6"}
Jul 31 23:47:27.773786 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:27.773756    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-jnmzc" event={"ID":"019f1fa4-9e80-41f5-b48d-82e064824829","Type":"ContainerStarted","Data":"1bd317a0bba61ffaf5e842281baf6896419c0844d18592c4d37aea38a74278dc"}
Jul 31 23:47:27.773906 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:27.773799    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-jnmzc" event={"ID":"019f1fa4-9e80-41f5-b48d-82e064824829","Type":"ContainerStarted","Data":"8f3330f83bcab9cf94911a9e42c329359424ff5f30b06a928864d38ea555f676"}
Jul 31 23:47:27.775723 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:27.775702    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-wz649" event={"ID":"c647d292-b5cd-4840-a69c-8c0e9ae41793","Type":"ContainerStarted","Data":"818571fd1a192773d7070455d6927f3b43837438daa78502cdcefddc7f663e20"}
Jul 31 23:47:27.776796 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:27.776777    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-j97wn" event={"ID":"815ee183-a99c-4636-9937-da826945f531","Type":"ContainerStarted","Data":"8d56ea9fcfb13add14733df076b44bab49c639bca09d7f6ede24e3c51d0c46f6"}
Jul 31 23:47:27.793682 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:27.793610    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-jnmzc" podStartSLOduration=3.855998464 podStartE2EDuration="4.793593926s" totalImagesPullingTime="861.086923ms" totalInitContainerRuntime="76.508539ms" isStatefulPod=true podCreationTimestamp="2026-07-31 23:47:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:47:27.792291337 +0000 UTC m=+65.904419012" watchObservedRunningTime="2026-07-31 23:47:27.793593926 +0000 UTC m=+65.905721577"
Jul 31 23:47:27.811122 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:27.811070    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-wz649" podStartSLOduration=3.938853761 podStartE2EDuration="4.811054985s" totalImagesPullingTime="872.201224ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:47:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:47:27.809893044 +0000 UTC m=+65.922020738" watchObservedRunningTime="2026-07-31 23:47:27.811054985 +0000 UTC m=+65.923182658"
Jul 31 23:47:29.788025 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:29.787920    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-j97wn" event={"ID":"815ee183-a99c-4636-9937-da826945f531","Type":"ContainerStarted","Data":"dcc6958d5b108e7031ff571da9add47591a70d6344956c894aa2e723139b9d56"}
Jul 31 23:47:29.788025 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:29.787983    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-j97wn" event={"ID":"815ee183-a99c-4636-9937-da826945f531","Type":"ContainerStarted","Data":"f5bd0cb24fa3001a5e841cf1db7635918ce6797bef07c212d790f8eafb577542"}
Jul 31 23:47:29.805716 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:29.805662    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-j97wn" podStartSLOduration=66.476442996 podStartE2EDuration="1m7.805620729s" totalImagesPullingTime="1.329177733s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:46:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:47:29.80537941 +0000 UTC m=+67.917507084" watchObservedRunningTime="2026-07-31 23:47:29.805620729 +0000 UTC m=+67.917748402"
Jul 31 23:47:30.253592 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:30.253557    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 31 23:47:30.256275 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:30.256255    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 31 23:47:30.259048 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:30.259028    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 31 23:47:30.259164 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:30.259065    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 31 23:47:30.259164 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:30.259145    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 31 23:47:30.261266 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:30.261243    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 31 23:47:30.261491 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:30.261476    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 31 23:47:30.261585 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:30.261495    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-746p9\""
Jul 31 23:47:30.261665 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:30.261611    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 31 23:47:30.261913 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:30.261895    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 31 23:47:30.262034 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:30.261897    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 31 23:47:30.262260 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:30.262236    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 31 23:47:30.262353 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:30.262235    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 31 23:47:30.262353 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:30.262327    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 31 23:47:30.262484 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:30.262468    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-2fth5a5eh0iid\""
Jul 31 23:47:30.262540 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:30.262483    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 31 23:47:30.262952 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:30.262935    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 31 23:47:30.268707 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:30.268684    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 31 23:47:30.272450 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:30.272427    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 31 23:47:30.285247 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:30.285227    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 31 23:47:30.306784 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:30.306735    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/136d0bc8-ebce-4191-9680-948f869e4b26-config-out\") pod \"prometheus-k8s-0\" (UID: \"136d0bc8-ebce-4191-9680-948f869e4b26\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 31 23:47:30.306923 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:30.306793    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/136d0bc8-ebce-4191-9680-948f869e4b26-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"136d0bc8-ebce-4191-9680-948f869e4b26\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 31 23:47:30.307270 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:30.306873    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/136d0bc8-ebce-4191-9680-948f869e4b26-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"136d0bc8-ebce-4191-9680-948f869e4b26\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 31 23:47:30.307368 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:30.307338    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/136d0bc8-ebce-4191-9680-948f869e4b26-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"136d0bc8-ebce-4191-9680-948f869e4b26\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 31 23:47:30.307415 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:30.307398    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/136d0bc8-ebce-4191-9680-948f869e4b26-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"136d0bc8-ebce-4191-9680-948f869e4b26\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 31 23:47:30.307467 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:30.307432    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ws8mr\" (UniqueName: \"kubernetes.io/projected/136d0bc8-ebce-4191-9680-948f869e4b26-kube-api-access-ws8mr\") pod \"prometheus-k8s-0\" (UID: \"136d0bc8-ebce-4191-9680-948f869e4b26\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 31 23:47:30.307566 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:30.307510    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/136d0bc8-ebce-4191-9680-948f869e4b26-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"136d0bc8-ebce-4191-9680-948f869e4b26\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 31 23:47:30.308428 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:30.307711    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/136d0bc8-ebce-4191-9680-948f869e4b26-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"136d0bc8-ebce-4191-9680-948f869e4b26\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 31 23:47:30.308428 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:30.307760    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/136d0bc8-ebce-4191-9680-948f869e4b26-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"136d0bc8-ebce-4191-9680-948f869e4b26\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 31 23:47:30.308428 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:30.307802    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/136d0bc8-ebce-4191-9680-948f869e4b26-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"136d0bc8-ebce-4191-9680-948f869e4b26\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 31 23:47:30.308428 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:30.307849    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/136d0bc8-ebce-4191-9680-948f869e4b26-config\") pod \"prometheus-k8s-0\" (UID: \"136d0bc8-ebce-4191-9680-948f869e4b26\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 31 23:47:30.308428 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:30.307954    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/136d0bc8-ebce-4191-9680-948f869e4b26-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"136d0bc8-ebce-4191-9680-948f869e4b26\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 31 23:47:30.308428 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:30.307995    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/136d0bc8-ebce-4191-9680-948f869e4b26-web-config\") pod \"prometheus-k8s-0\" (UID: \"136d0bc8-ebce-4191-9680-948f869e4b26\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 31 23:47:30.308428 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:30.308028    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/136d0bc8-ebce-4191-9680-948f869e4b26-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"136d0bc8-ebce-4191-9680-948f869e4b26\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 31 23:47:30.308428 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:30.308066    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/136d0bc8-ebce-4191-9680-948f869e4b26-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"136d0bc8-ebce-4191-9680-948f869e4b26\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 31 23:47:30.308428 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:30.308097    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/136d0bc8-ebce-4191-9680-948f869e4b26-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"136d0bc8-ebce-4191-9680-948f869e4b26\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 31 23:47:30.308428 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:30.308134    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/136d0bc8-ebce-4191-9680-948f869e4b26-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"136d0bc8-ebce-4191-9680-948f869e4b26\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 31 23:47:30.308428 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:30.308170    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/136d0bc8-ebce-4191-9680-948f869e4b26-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"136d0bc8-ebce-4191-9680-948f869e4b26\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 31 23:47:30.308428 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:30.308206    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/136d0bc8-ebce-4191-9680-948f869e4b26-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"136d0bc8-ebce-4191-9680-948f869e4b26\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 31 23:47:30.308428 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:30.308237    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/136d0bc8-ebce-4191-9680-948f869e4b26-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"136d0bc8-ebce-4191-9680-948f869e4b26\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 31 23:47:30.408735 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:30.408687    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/136d0bc8-ebce-4191-9680-948f869e4b26-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"136d0bc8-ebce-4191-9680-948f869e4b26\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 31 23:47:30.408918 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:30.408743    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/136d0bc8-ebce-4191-9680-948f869e4b26-web-config\") pod \"prometheus-k8s-0\" (UID: \"136d0bc8-ebce-4191-9680-948f869e4b26\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 31 23:47:30.408918 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:30.408774    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/136d0bc8-ebce-4191-9680-948f869e4b26-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"136d0bc8-ebce-4191-9680-948f869e4b26\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 31 23:47:30.408918 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:30.408805    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/136d0bc8-ebce-4191-9680-948f869e4b26-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"136d0bc8-ebce-4191-9680-948f869e4b26\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 31 23:47:30.408918 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:30.408831    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/136d0bc8-ebce-4191-9680-948f869e4b26-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"136d0bc8-ebce-4191-9680-948f869e4b26\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 31 23:47:30.408918 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:30.408879    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/136d0bc8-ebce-4191-9680-948f869e4b26-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"136d0bc8-ebce-4191-9680-948f869e4b26\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 31 23:47:30.409762 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:30.409266    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/136d0bc8-ebce-4191-9680-948f869e4b26-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"136d0bc8-ebce-4191-9680-948f869e4b26\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 31 23:47:30.409762 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:30.409325    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/136d0bc8-ebce-4191-9680-948f869e4b26-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"136d0bc8-ebce-4191-9680-948f869e4b26\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 31 23:47:30.409762 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:30.409360    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/136d0bc8-ebce-4191-9680-948f869e4b26-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"136d0bc8-ebce-4191-9680-948f869e4b26\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 31 23:47:30.409762 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:30.409407    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/136d0bc8-ebce-4191-9680-948f869e4b26-config-out\") pod \"prometheus-k8s-0\" (UID: \"136d0bc8-ebce-4191-9680-948f869e4b26\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 31 23:47:30.409762 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:30.409426    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/136d0bc8-ebce-4191-9680-948f869e4b26-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"136d0bc8-ebce-4191-9680-948f869e4b26\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 31 23:47:30.409762 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:30.409437    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/136d0bc8-ebce-4191-9680-948f869e4b26-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"136d0bc8-ebce-4191-9680-948f869e4b26\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 31 23:47:30.409762 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:30.409470    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/136d0bc8-ebce-4191-9680-948f869e4b26-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"136d0bc8-ebce-4191-9680-948f869e4b26\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 31 23:47:30.409762 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:30.409495    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/136d0bc8-ebce-4191-9680-948f869e4b26-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"136d0bc8-ebce-4191-9680-948f869e4b26\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 31 23:47:30.409762 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:30.409546    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/136d0bc8-ebce-4191-9680-948f869e4b26-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"136d0bc8-ebce-4191-9680-948f869e4b26\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 31 23:47:30.409762 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:30.409564    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/136d0bc8-ebce-4191-9680-948f869e4b26-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"136d0bc8-ebce-4191-9680-948f869e4b26\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 31 23:47:30.409762 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:30.409588    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/136d0bc8-ebce-4191-9680-948f869e4b26-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"136d0bc8-ebce-4191-9680-948f869e4b26\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 31 23:47:30.409762 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:30.409616    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ws8mr\" (UniqueName: \"kubernetes.io/projected/136d0bc8-ebce-4191-9680-948f869e4b26-kube-api-access-ws8mr\") pod \"prometheus-k8s-0\" (UID: \"136d0bc8-ebce-4191-9680-948f869e4b26\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 31 23:47:30.409762 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:30.409679    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/136d0bc8-ebce-4191-9680-948f869e4b26-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"136d0bc8-ebce-4191-9680-948f869e4b26\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 31 23:47:30.409762 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:30.409709    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/136d0bc8-ebce-4191-9680-948f869e4b26-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"136d0bc8-ebce-4191-9680-948f869e4b26\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 31 23:47:30.409762 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:30.409731    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/136d0bc8-ebce-4191-9680-948f869e4b26-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"136d0bc8-ebce-4191-9680-948f869e4b26\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 31 23:47:30.409762 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:30.409754    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/136d0bc8-ebce-4191-9680-948f869e4b26-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"136d0bc8-ebce-4191-9680-948f869e4b26\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 31 23:47:30.410584 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:30.409780    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/136d0bc8-ebce-4191-9680-948f869e4b26-config\") pod \"prometheus-k8s-0\" (UID: \"136d0bc8-ebce-4191-9680-948f869e4b26\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 31 23:47:30.410584 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:30.410440    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/136d0bc8-ebce-4191-9680-948f869e4b26-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"136d0bc8-ebce-4191-9680-948f869e4b26\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 31 23:47:30.410715 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:30.410668    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/136d0bc8-ebce-4191-9680-948f869e4b26-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"136d0bc8-ebce-4191-9680-948f869e4b26\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 31 23:47:30.411075 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:30.410757    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/136d0bc8-ebce-4191-9680-948f869e4b26-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"136d0bc8-ebce-4191-9680-948f869e4b26\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 31 23:47:30.411803 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:30.411618    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/136d0bc8-ebce-4191-9680-948f869e4b26-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"136d0bc8-ebce-4191-9680-948f869e4b26\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 31 23:47:30.412101 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:30.412077    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/136d0bc8-ebce-4191-9680-948f869e4b26-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"136d0bc8-ebce-4191-9680-948f869e4b26\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 31 23:47:30.412878 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:30.412842    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/136d0bc8-ebce-4191-9680-948f869e4b26-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"136d0bc8-ebce-4191-9680-948f869e4b26\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 31 23:47:30.413263 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:30.413242    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/136d0bc8-ebce-4191-9680-948f869e4b26-config\") pod \"prometheus-k8s-0\" (UID: \"136d0bc8-ebce-4191-9680-948f869e4b26\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 31 23:47:30.413772 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:30.413744    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/136d0bc8-ebce-4191-9680-948f869e4b26-web-config\") pod \"prometheus-k8s-0\" (UID: \"136d0bc8-ebce-4191-9680-948f869e4b26\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 31 23:47:30.414431 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:30.414403    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/136d0bc8-ebce-4191-9680-948f869e4b26-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"136d0bc8-ebce-4191-9680-948f869e4b26\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 31 23:47:30.414924 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:30.414901    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/136d0bc8-ebce-4191-9680-948f869e4b26-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"136d0bc8-ebce-4191-9680-948f869e4b26\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 31 23:47:30.415130 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:30.415059    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/136d0bc8-ebce-4191-9680-948f869e4b26-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"136d0bc8-ebce-4191-9680-948f869e4b26\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 31 23:47:30.415207 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:30.415135    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/136d0bc8-ebce-4191-9680-948f869e4b26-config-out\") pod \"prometheus-k8s-0\" (UID: \"136d0bc8-ebce-4191-9680-948f869e4b26\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 31 23:47:30.415456 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:30.415430    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/136d0bc8-ebce-4191-9680-948f869e4b26-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"136d0bc8-ebce-4191-9680-948f869e4b26\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 31 23:47:30.415547 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:30.415528    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/136d0bc8-ebce-4191-9680-948f869e4b26-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"136d0bc8-ebce-4191-9680-948f869e4b26\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 31 23:47:30.416362 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:30.416336    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/136d0bc8-ebce-4191-9680-948f869e4b26-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"136d0bc8-ebce-4191-9680-948f869e4b26\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 31 23:47:30.416649 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:30.416602    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/136d0bc8-ebce-4191-9680-948f869e4b26-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"136d0bc8-ebce-4191-9680-948f869e4b26\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 31 23:47:30.420718 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:30.420700    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ws8mr\" (UniqueName: \"kubernetes.io/projected/136d0bc8-ebce-4191-9680-948f869e4b26-kube-api-access-ws8mr\") pod \"prometheus-k8s-0\" (UID: \"136d0bc8-ebce-4191-9680-948f869e4b26\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 31 23:47:30.567575 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:30.567485    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 31 23:47:31.447588 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:31.447550    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-695467d899-rp2fg"
Jul 31 23:47:31.448120 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:31.447641    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-695467d899-rp2fg"
Jul 31 23:47:31.453146 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:31.453120    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-695467d899-rp2fg"
Jul 31 23:47:31.802528 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:31.802455    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-695467d899-rp2fg"
Jul 31 23:47:32.123332 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:32.123306    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 31 23:47:32.125938 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:47:32.125913    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod136d0bc8_ebce_4191_9680_948f869e4b26.slice/crio-04e99cc31d71e0136c441978f883c018a44a34d71a8cf60b9bf6ed32d5178093 WatchSource:0}: Error finding container 04e99cc31d71e0136c441978f883c018a44a34d71a8cf60b9bf6ed32d5178093: Status 404 returned error can't find the container with id 04e99cc31d71e0136c441978f883c018a44a34d71a8cf60b9bf6ed32d5178093
Jul 31 23:47:32.802288 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:32.802244    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-h45s5" event={"ID":"69e78750-573b-4dbd-a585-cf0d88056f4f","Type":"ContainerStarted","Data":"1daa96ec80cdf5f532e627460d58e4b8ed02d628b7039e793ab7745ebc9ebaa0"}
Jul 31 23:47:32.803829 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:32.803799    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"136d0bc8-ebce-4191-9680-948f869e4b26","Type":"ContainerStarted","Data":"04e99cc31d71e0136c441978f883c018a44a34d71a8cf60b9bf6ed32d5178093"}
Jul 31 23:47:32.805405 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:32.805378    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-7n4sw" event={"ID":"2887ad3f-73c9-4fce-8359-c86e96300a84","Type":"ContainerStarted","Data":"d2700b7aa33262adba479297629beed8fdb01b9258e25fdf6a5509e2beaf7f38"}
Jul 31 23:47:32.805594 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:32.805575    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-7n4sw"
Jul 31 23:47:32.821475 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:32.821423    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-h45s5" podStartSLOduration=66.343826127 podStartE2EDuration="1m10.821404266s" totalImagesPullingTime="4.477578139s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-31 23:46:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:47:32.819353394 +0000 UTC m=+70.931481094" watchObservedRunningTime="2026-07-31 23:47:32.821404266 +0000 UTC m=+70.933531941"
Jul 31 23:47:32.838775 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:32.838727    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-7n4sw" podStartSLOduration=66.405924132 podStartE2EDuration="1m10.838712503s" totalImagesPullingTime="4.432788371s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:46:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:47:32.837699491 +0000 UTC m=+70.949827203" watchObservedRunningTime="2026-07-31 23:47:32.838712503 +0000 UTC m=+70.950840175"
Jul 31 23:47:33.589712 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:33.589616    2564 patch_prober.go:28] interesting pod/image-registry-774c7c8d4c-hxx2p container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 31 23:47:33.589712 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:33.589701    2564 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-774c7c8d4c-hxx2p" podUID="219c8401-f8b7-474e-ac64-98a06039633f" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 31 23:47:33.812062 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:33.812027    2564 generic.go:356] "Generic (PLEG): container finished" podID="136d0bc8-ebce-4191-9680-948f869e4b26" containerID="21028cdfd2d860f3c4732d29ed3bf17e81ee2d29e684d4aceb732c981a100cb7" exitCode=0
Jul 31 23:47:33.812535 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:33.812112    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"136d0bc8-ebce-4191-9680-948f869e4b26","Type":"ContainerDied","Data":"21028cdfd2d860f3c4732d29ed3bf17e81ee2d29e684d4aceb732c981a100cb7"}
Jul 31 23:47:35.707275 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:35.707247    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-774c7c8d4c-hxx2p"
Jul 31 23:47:36.833833 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:36.833686    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"136d0bc8-ebce-4191-9680-948f869e4b26","Type":"ContainerStarted","Data":"f3abbe6325b9f83136e4e8b0122539602f1d95ca72a8b4f7df1e5d6b6cd50316"}
Jul 31 23:47:36.833833 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:36.833769    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"136d0bc8-ebce-4191-9680-948f869e4b26","Type":"ContainerStarted","Data":"6084762f8c53e7c4ecb1f093ed956686c55a95ad6549c4abb00e987a97d4f622"}
Jul 31 23:47:38.843826 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:38.843794    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"136d0bc8-ebce-4191-9680-948f869e4b26","Type":"ContainerStarted","Data":"d060dea73a8f05fdf8306dcdc7194c58b5786eb312ea6425f4c08da5b1d737d0"}
Jul 31 23:47:38.844183 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:38.843856    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"136d0bc8-ebce-4191-9680-948f869e4b26","Type":"ContainerStarted","Data":"4003b250e7a69bc18ce674ebbbbc4510aa66ca22667f2cbbc1f350a401c2e64a"}
Jul 31 23:47:38.844183 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:38.843883    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"136d0bc8-ebce-4191-9680-948f869e4b26","Type":"ContainerStarted","Data":"7e8f55cd72cc6343251f106f58f8f605441877c6eed78c9f7fc88ac988b7a6e9"}
Jul 31 23:47:38.844183 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:38.843909    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"136d0bc8-ebce-4191-9680-948f869e4b26","Type":"ContainerStarted","Data":"864705041d63baad5e10a434566300b23a9cdf73dc257f2f9a5d7823f0901762"}
Jul 31 23:47:38.875922 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:38.875873    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.361128363 podStartE2EDuration="8.875857048s" totalImagesPullingTime="5.498527757s" totalInitContainerRuntime="16.200928ms" isStatefulPod=false podCreationTimestamp="2026-07-31 23:47:30 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:47:38.873656708 +0000 UTC m=+76.985784401" watchObservedRunningTime="2026-07-31 23:47:38.875857048 +0000 UTC m=+76.987984721"
Jul 31 23:47:40.056265 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:40.056233    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-695467d899-rp2fg"]
Jul 31 23:47:40.567847 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:47:40.567817    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 31 23:48:03.814279 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:48:03.814248    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-7n4sw"
Jul 31 23:48:05.075080 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:48:05.075032    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-695467d899-rp2fg" podUID="09fe4360-7a24-4668-ad81-3df13cdd2516" containerName="console" containerID="cri-o://a477d94e3b2262042596715e88866a7a8b0de3b5d297ce197a4a37841f670e1c" gracePeriod=15
Jul 31 23:48:05.312165 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:48:05.312130    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-695467d899-rp2fg"
Jul 31 23:48:05.400295 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:48:05.400264    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/09fe4360-7a24-4668-ad81-3df13cdd2516-service-ca\") pod \"09fe4360-7a24-4668-ad81-3df13cdd2516\" (UID: \"09fe4360-7a24-4668-ad81-3df13cdd2516\") "
Jul 31 23:48:05.400295 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:48:05.400301    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/09fe4360-7a24-4668-ad81-3df13cdd2516-trusted-ca-bundle\") pod \"09fe4360-7a24-4668-ad81-3df13cdd2516\" (UID: \"09fe4360-7a24-4668-ad81-3df13cdd2516\") "
Jul 31 23:48:05.400607 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:48:05.400330    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/09fe4360-7a24-4668-ad81-3df13cdd2516-console-oauth-config\") pod \"09fe4360-7a24-4668-ad81-3df13cdd2516\" (UID: \"09fe4360-7a24-4668-ad81-3df13cdd2516\") "
Jul 31 23:48:05.400607 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:48:05.400353    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rt8nc\" (UniqueName: \"kubernetes.io/projected/09fe4360-7a24-4668-ad81-3df13cdd2516-kube-api-access-rt8nc\") pod \"09fe4360-7a24-4668-ad81-3df13cdd2516\" (UID: \"09fe4360-7a24-4668-ad81-3df13cdd2516\") "
Jul 31 23:48:05.400607 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:48:05.400372    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/09fe4360-7a24-4668-ad81-3df13cdd2516-console-serving-cert\") pod \"09fe4360-7a24-4668-ad81-3df13cdd2516\" (UID: \"09fe4360-7a24-4668-ad81-3df13cdd2516\") "
Jul 31 23:48:05.400607 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:48:05.400392    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/09fe4360-7a24-4668-ad81-3df13cdd2516-oauth-serving-cert\") pod \"09fe4360-7a24-4668-ad81-3df13cdd2516\" (UID: \"09fe4360-7a24-4668-ad81-3df13cdd2516\") "
Jul 31 23:48:05.400607 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:48:05.400426    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/09fe4360-7a24-4668-ad81-3df13cdd2516-console-config\") pod \"09fe4360-7a24-4668-ad81-3df13cdd2516\" (UID: \"09fe4360-7a24-4668-ad81-3df13cdd2516\") "
Jul 31 23:48:05.400885 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:48:05.400861    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/09fe4360-7a24-4668-ad81-3df13cdd2516-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "09fe4360-7a24-4668-ad81-3df13cdd2516" (UID: "09fe4360-7a24-4668-ad81-3df13cdd2516"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 31 23:48:05.400952 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:48:05.400890    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/09fe4360-7a24-4668-ad81-3df13cdd2516-service-ca" (OuterVolumeSpecName: "service-ca") pod "09fe4360-7a24-4668-ad81-3df13cdd2516" (UID: "09fe4360-7a24-4668-ad81-3df13cdd2516"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 31 23:48:05.401007 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:48:05.400942    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/09fe4360-7a24-4668-ad81-3df13cdd2516-console-config" (OuterVolumeSpecName: "console-config") pod "09fe4360-7a24-4668-ad81-3df13cdd2516" (UID: "09fe4360-7a24-4668-ad81-3df13cdd2516"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 31 23:48:05.401007 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:48:05.400964    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/09fe4360-7a24-4668-ad81-3df13cdd2516-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "09fe4360-7a24-4668-ad81-3df13cdd2516" (UID: "09fe4360-7a24-4668-ad81-3df13cdd2516"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 31 23:48:05.402896 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:48:05.402855    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/09fe4360-7a24-4668-ad81-3df13cdd2516-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "09fe4360-7a24-4668-ad81-3df13cdd2516" (UID: "09fe4360-7a24-4668-ad81-3df13cdd2516"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 31 23:48:05.402979 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:48:05.402895    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/09fe4360-7a24-4668-ad81-3df13cdd2516-kube-api-access-rt8nc" (OuterVolumeSpecName: "kube-api-access-rt8nc") pod "09fe4360-7a24-4668-ad81-3df13cdd2516" (UID: "09fe4360-7a24-4668-ad81-3df13cdd2516"). InnerVolumeSpecName "kube-api-access-rt8nc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 31 23:48:05.402979 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:48:05.402937    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/09fe4360-7a24-4668-ad81-3df13cdd2516-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "09fe4360-7a24-4668-ad81-3df13cdd2516" (UID: "09fe4360-7a24-4668-ad81-3df13cdd2516"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 31 23:48:05.501378 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:48:05.501330    2564 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/09fe4360-7a24-4668-ad81-3df13cdd2516-service-ca\") on node \"ip-10-0-128-75.ec2.internal\" DevicePath \"\""
Jul 31 23:48:05.501378 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:48:05.501372    2564 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/09fe4360-7a24-4668-ad81-3df13cdd2516-trusted-ca-bundle\") on node \"ip-10-0-128-75.ec2.internal\" DevicePath \"\""
Jul 31 23:48:05.501378 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:48:05.501384    2564 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/09fe4360-7a24-4668-ad81-3df13cdd2516-console-oauth-config\") on node \"ip-10-0-128-75.ec2.internal\" DevicePath \"\""
Jul 31 23:48:05.501378 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:48:05.501393    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rt8nc\" (UniqueName: \"kubernetes.io/projected/09fe4360-7a24-4668-ad81-3df13cdd2516-kube-api-access-rt8nc\") on node \"ip-10-0-128-75.ec2.internal\" DevicePath \"\""
Jul 31 23:48:05.501649 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:48:05.501401    2564 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/09fe4360-7a24-4668-ad81-3df13cdd2516-console-serving-cert\") on node \"ip-10-0-128-75.ec2.internal\" DevicePath \"\""
Jul 31 23:48:05.501649 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:48:05.501409    2564 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/09fe4360-7a24-4668-ad81-3df13cdd2516-oauth-serving-cert\") on node \"ip-10-0-128-75.ec2.internal\" DevicePath \"\""
Jul 31 23:48:05.501649 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:48:05.501419    2564 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/09fe4360-7a24-4668-ad81-3df13cdd2516-console-config\") on node \"ip-10-0-128-75.ec2.internal\" DevicePath \"\""
Jul 31 23:48:05.942251 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:48:05.942219    2564 generic.go:356] "Generic (PLEG): container finished" podID="09fe4360-7a24-4668-ad81-3df13cdd2516" containerID="a477d94e3b2262042596715e88866a7a8b0de3b5d297ce197a4a37841f670e1c" exitCode=0
Jul 31 23:48:05.942440 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:48:05.942267    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-695467d899-rp2fg"
Jul 31 23:48:05.942440 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:48:05.942306    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-695467d899-rp2fg" event={"ID":"09fe4360-7a24-4668-ad81-3df13cdd2516","Type":"ContainerDied","Data":"a477d94e3b2262042596715e88866a7a8b0de3b5d297ce197a4a37841f670e1c"}
Jul 31 23:48:05.942440 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:48:05.942363    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-695467d899-rp2fg" event={"ID":"09fe4360-7a24-4668-ad81-3df13cdd2516","Type":"ContainerDied","Data":"37e0d171fe1dc5bdab1ac0eff378f61e4b15357b33a78e461ed97ea12354eb28"}
Jul 31 23:48:05.942440 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:48:05.942404    2564 scope.go:117] "RemoveContainer" containerID="a477d94e3b2262042596715e88866a7a8b0de3b5d297ce197a4a37841f670e1c"
Jul 31 23:48:05.950714 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:48:05.950696    2564 scope.go:117] "RemoveContainer" containerID="a477d94e3b2262042596715e88866a7a8b0de3b5d297ce197a4a37841f670e1c"
Jul 31 23:48:05.950978 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:48:05.950957    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a477d94e3b2262042596715e88866a7a8b0de3b5d297ce197a4a37841f670e1c\": container with ID starting with a477d94e3b2262042596715e88866a7a8b0de3b5d297ce197a4a37841f670e1c not found: ID does not exist" containerID="a477d94e3b2262042596715e88866a7a8b0de3b5d297ce197a4a37841f670e1c"
Jul 31 23:48:05.951021 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:48:05.950986    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a477d94e3b2262042596715e88866a7a8b0de3b5d297ce197a4a37841f670e1c"} err="failed to get container status \"a477d94e3b2262042596715e88866a7a8b0de3b5d297ce197a4a37841f670e1c\": rpc error: code = NotFound desc = could not find container \"a477d94e3b2262042596715e88866a7a8b0de3b5d297ce197a4a37841f670e1c\": container with ID starting with a477d94e3b2262042596715e88866a7a8b0de3b5d297ce197a4a37841f670e1c not found: ID does not exist"
Jul 31 23:48:05.963264 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:48:05.963240    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-695467d899-rp2fg"]
Jul 31 23:48:05.965838 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:48:05.965818    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-695467d899-rp2fg"]
Jul 31 23:48:06.420907 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:48:06.420871    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="09fe4360-7a24-4668-ad81-3df13cdd2516" path="/var/lib/kubelet/pods/09fe4360-7a24-4668-ad81-3df13cdd2516/volumes"
Jul 31 23:48:30.568005 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:48:30.567960    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 31 23:48:30.587556 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:48:30.587530    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 31 23:48:31.059699 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:48:31.059655    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 31 23:51:22.296118 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:51:22.296084    2564 kubelet.go:1648] "Image garbage collection succeeded"
Jul 31 23:52:13.160240 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:52:13.160206    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/seaweedfs-64568bcd49-ssq4m"]
Jul 31 23:52:13.160863 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:52:13.160612    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="09fe4360-7a24-4668-ad81-3df13cdd2516" containerName="console"
Jul 31 23:52:13.160863 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:52:13.160654    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="09fe4360-7a24-4668-ad81-3df13cdd2516" containerName="console"
Jul 31 23:52:13.160863 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:52:13.160766    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="09fe4360-7a24-4668-ad81-3df13cdd2516" containerName="console"
Jul 31 23:52:13.163733 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:52:13.163710    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/seaweedfs-64568bcd49-ssq4m"
Jul 31 23:52:13.169581 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:52:13.169562    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"mlpipeline-s3-artifact\""
Jul 31 23:52:13.170290 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:52:13.170260    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"openshift-service-ca.crt\""
Jul 31 23:52:13.170663 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:52:13.170642    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"default-dockercfg-grmxd\""
Jul 31 23:52:13.170776 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:52:13.170669    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"kube-root-ca.crt\""
Jul 31 23:52:13.178947 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:52:13.178926    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/seaweedfs-64568bcd49-ssq4m"]
Jul 31 23:52:13.272944 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:52:13.272915    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/10fb8451-20c5-45c7-8367-2902bddab100-data\") pod \"seaweedfs-64568bcd49-ssq4m\" (UID: \"10fb8451-20c5-45c7-8367-2902bddab100\") " pod="kserve/seaweedfs-64568bcd49-ssq4m"
Jul 31 23:52:13.273134 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:52:13.272952    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bwkfd\" (UniqueName: \"kubernetes.io/projected/10fb8451-20c5-45c7-8367-2902bddab100-kube-api-access-bwkfd\") pod \"seaweedfs-64568bcd49-ssq4m\" (UID: \"10fb8451-20c5-45c7-8367-2902bddab100\") " pod="kserve/seaweedfs-64568bcd49-ssq4m"
Jul 31 23:52:13.373540 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:52:13.373505    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/10fb8451-20c5-45c7-8367-2902bddab100-data\") pod \"seaweedfs-64568bcd49-ssq4m\" (UID: \"10fb8451-20c5-45c7-8367-2902bddab100\") " pod="kserve/seaweedfs-64568bcd49-ssq4m"
Jul 31 23:52:13.373540 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:52:13.373548    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bwkfd\" (UniqueName: \"kubernetes.io/projected/10fb8451-20c5-45c7-8367-2902bddab100-kube-api-access-bwkfd\") pod \"seaweedfs-64568bcd49-ssq4m\" (UID: \"10fb8451-20c5-45c7-8367-2902bddab100\") " pod="kserve/seaweedfs-64568bcd49-ssq4m"
Jul 31 23:52:13.374008 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:52:13.373982    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/10fb8451-20c5-45c7-8367-2902bddab100-data\") pod \"seaweedfs-64568bcd49-ssq4m\" (UID: \"10fb8451-20c5-45c7-8367-2902bddab100\") " pod="kserve/seaweedfs-64568bcd49-ssq4m"
Jul 31 23:52:13.385311 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:52:13.385275    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bwkfd\" (UniqueName: \"kubernetes.io/projected/10fb8451-20c5-45c7-8367-2902bddab100-kube-api-access-bwkfd\") pod \"seaweedfs-64568bcd49-ssq4m\" (UID: \"10fb8451-20c5-45c7-8367-2902bddab100\") " pod="kserve/seaweedfs-64568bcd49-ssq4m"
Jul 31 23:52:13.473296 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:52:13.473211    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/seaweedfs-64568bcd49-ssq4m"
Jul 31 23:52:13.622078 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:52:13.622050    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/seaweedfs-64568bcd49-ssq4m"]
Jul 31 23:52:13.625031 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:52:13.625005    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10fb8451_20c5_45c7_8367_2902bddab100.slice/crio-a91292f5ed402c012ad93285b06a27288b320a494e5a12fd9d87264e484a3aaf WatchSource:0}: Error finding container a91292f5ed402c012ad93285b06a27288b320a494e5a12fd9d87264e484a3aaf: Status 404 returned error can't find the container with id a91292f5ed402c012ad93285b06a27288b320a494e5a12fd9d87264e484a3aaf
Jul 31 23:52:13.626367 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:52:13.626344    2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 31 23:52:13.807691 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:52:13.807592    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/seaweedfs-64568bcd49-ssq4m" event={"ID":"10fb8451-20c5-45c7-8367-2902bddab100","Type":"ContainerStarted","Data":"a91292f5ed402c012ad93285b06a27288b320a494e5a12fd9d87264e484a3aaf"}
Jul 31 23:52:16.823160 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:52:16.823122    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/seaweedfs-64568bcd49-ssq4m" event={"ID":"10fb8451-20c5-45c7-8367-2902bddab100","Type":"ContainerStarted","Data":"06d92c384d1bb49d5537dd76faa0855f6c9717c3fdb85dcb5d72d284b9ef3746"}
Jul 31 23:52:16.823670 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:52:16.823257    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/seaweedfs-64568bcd49-ssq4m"
Jul 31 23:52:16.839348 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:52:16.839301    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/seaweedfs-64568bcd49-ssq4m" podStartSLOduration=1.417948875 podStartE2EDuration="3.839285891s" totalImagesPullingTime="2.421337016s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:52:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:52:16.838523512 +0000 UTC m=+354.950651187" watchObservedRunningTime="2026-07-31 23:52:16.839285891 +0000 UTC m=+354.951413611"
Jul 31 23:52:22.829086 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:52:22.829057    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/seaweedfs-64568bcd49-ssq4m"
Jul 31 23:53:46.395478 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:53:46.395400    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/s3-init-phkrf"]
Jul 31 23:53:46.398435 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:53:46.398419    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-phkrf"
Jul 31 23:53:46.405836 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:53:46.405813    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/s3-init-phkrf"]
Jul 31 23:53:46.508427 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:53:46.508385    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/08ae5adf-ffd8-4f08-a500-4cbf4dadce15-model-cache\") pod \"s3-init-phkrf\" (UID: \"08ae5adf-ffd8-4f08-a500-4cbf4dadce15\") " pod="kserve/s3-init-phkrf"
Jul 31 23:53:46.508427 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:53:46.508426    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fmzxg\" (UniqueName: \"kubernetes.io/projected/08ae5adf-ffd8-4f08-a500-4cbf4dadce15-kube-api-access-fmzxg\") pod \"s3-init-phkrf\" (UID: \"08ae5adf-ffd8-4f08-a500-4cbf4dadce15\") " pod="kserve/s3-init-phkrf"
Jul 31 23:53:46.609741 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:53:46.609691    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/08ae5adf-ffd8-4f08-a500-4cbf4dadce15-model-cache\") pod \"s3-init-phkrf\" (UID: \"08ae5adf-ffd8-4f08-a500-4cbf4dadce15\") " pod="kserve/s3-init-phkrf"
Jul 31 23:53:46.609741 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:53:46.609740    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fmzxg\" (UniqueName: \"kubernetes.io/projected/08ae5adf-ffd8-4f08-a500-4cbf4dadce15-kube-api-access-fmzxg\") pod \"s3-init-phkrf\" (UID: \"08ae5adf-ffd8-4f08-a500-4cbf4dadce15\") " pod="kserve/s3-init-phkrf"
Jul 31 23:53:46.610126 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:53:46.610106    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/08ae5adf-ffd8-4f08-a500-4cbf4dadce15-model-cache\") pod \"s3-init-phkrf\" (UID: \"08ae5adf-ffd8-4f08-a500-4cbf4dadce15\") " pod="kserve/s3-init-phkrf"
Jul 31 23:53:46.618804 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:53:46.618778    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fmzxg\" (UniqueName: \"kubernetes.io/projected/08ae5adf-ffd8-4f08-a500-4cbf4dadce15-kube-api-access-fmzxg\") pod \"s3-init-phkrf\" (UID: \"08ae5adf-ffd8-4f08-a500-4cbf4dadce15\") " pod="kserve/s3-init-phkrf"
Jul 31 23:53:46.719744 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:53:46.719656    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-phkrf"
Jul 31 23:53:46.862552 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:53:46.862518    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/s3-init-phkrf"]
Jul 31 23:53:46.865734 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:53:46.865707    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod08ae5adf_ffd8_4f08_a500_4cbf4dadce15.slice/crio-c5eff2496044f0d4575f9dfd15a1da2e0f1db0c8cbbb8ff96390fbb71c210f3b WatchSource:0}: Error finding container c5eff2496044f0d4575f9dfd15a1da2e0f1db0c8cbbb8ff96390fbb71c210f3b: Status 404 returned error can't find the container with id c5eff2496044f0d4575f9dfd15a1da2e0f1db0c8cbbb8ff96390fbb71c210f3b
Jul 31 23:53:47.135263 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:53:47.135220    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-phkrf" event={"ID":"08ae5adf-ffd8-4f08-a500-4cbf4dadce15","Type":"ContainerStarted","Data":"c5eff2496044f0d4575f9dfd15a1da2e0f1db0c8cbbb8ff96390fbb71c210f3b"}
Jul 31 23:53:50.148484 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:53:50.148444    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-phkrf" event={"ID":"08ae5adf-ffd8-4f08-a500-4cbf4dadce15","Type":"ContainerStarted","Data":"a8761206fc07f4c1eed5e2a31f0ddb5bae50107218b621fdfc7f4305283bdd7f"}
Jul 31 23:53:52.156677 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:53:52.156616    2564 generic.go:356] "Generic (PLEG): container finished" podID="08ae5adf-ffd8-4f08-a500-4cbf4dadce15" containerID="a8761206fc07f4c1eed5e2a31f0ddb5bae50107218b621fdfc7f4305283bdd7f" exitCode=0
Jul 31 23:53:52.157238 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:53:52.156690    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-phkrf" event={"ID":"08ae5adf-ffd8-4f08-a500-4cbf4dadce15","Type":"ContainerDied","Data":"a8761206fc07f4c1eed5e2a31f0ddb5bae50107218b621fdfc7f4305283bdd7f"}
Jul 31 23:53:57.179169 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:53:57.179133    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-phkrf" event={"ID":"08ae5adf-ffd8-4f08-a500-4cbf4dadce15","Type":"ContainerStarted","Data":"2a6390e341f87423e45e6288d9b26d54c684f6db9fd80d4185f5815c0e78cf85"}
Jul 31 23:53:57.199120 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:53:57.199070    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/s3-init-phkrf" podStartSLOduration=2.265561228 podStartE2EDuration="11.199051465s" totalImagesPullingTime="7.466569464s" totalInitContainerRuntime="1.466920773s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:53:46 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:53:57.197576821 +0000 UTC m=+455.309704496" watchObservedRunningTime="2026-07-31 23:53:57.199051465 +0000 UTC m=+455.311179139"
Jul 31 23:55:02.413004 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:02.412971    2564 generic.go:356] "Generic (PLEG): container finished" podID="08ae5adf-ffd8-4f08-a500-4cbf4dadce15" containerID="2a6390e341f87423e45e6288d9b26d54c684f6db9fd80d4185f5815c0e78cf85" exitCode=0
Jul 31 23:55:02.413004 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:02.413007    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-phkrf" event={"ID":"08ae5adf-ffd8-4f08-a500-4cbf4dadce15","Type":"ContainerDied","Data":"2a6390e341f87423e45e6288d9b26d54c684f6db9fd80d4185f5815c0e78cf85"}
Jul 31 23:55:03.539478 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:03.539457    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-phkrf"
Jul 31 23:55:03.603746 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:03.603720    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fmzxg\" (UniqueName: \"kubernetes.io/projected/08ae5adf-ffd8-4f08-a500-4cbf4dadce15-kube-api-access-fmzxg\") pod \"08ae5adf-ffd8-4f08-a500-4cbf4dadce15\" (UID: \"08ae5adf-ffd8-4f08-a500-4cbf4dadce15\") "
Jul 31 23:55:03.603919 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:03.603768    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/08ae5adf-ffd8-4f08-a500-4cbf4dadce15-model-cache\") pod \"08ae5adf-ffd8-4f08-a500-4cbf4dadce15\" (UID: \"08ae5adf-ffd8-4f08-a500-4cbf4dadce15\") "
Jul 31 23:55:03.605948 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:03.605915    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/08ae5adf-ffd8-4f08-a500-4cbf4dadce15-kube-api-access-fmzxg" (OuterVolumeSpecName: "kube-api-access-fmzxg") pod "08ae5adf-ffd8-4f08-a500-4cbf4dadce15" (UID: "08ae5adf-ffd8-4f08-a500-4cbf4dadce15"). InnerVolumeSpecName "kube-api-access-fmzxg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 31 23:55:03.626206 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:03.626172    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/08ae5adf-ffd8-4f08-a500-4cbf4dadce15-model-cache" (OuterVolumeSpecName: "model-cache") pod "08ae5adf-ffd8-4f08-a500-4cbf4dadce15" (UID: "08ae5adf-ffd8-4f08-a500-4cbf4dadce15"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 31 23:55:03.704605 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:03.704517    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fmzxg\" (UniqueName: \"kubernetes.io/projected/08ae5adf-ffd8-4f08-a500-4cbf4dadce15-kube-api-access-fmzxg\") on node \"ip-10-0-128-75.ec2.internal\" DevicePath \"\""
Jul 31 23:55:03.704605 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:03.704546    2564 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/08ae5adf-ffd8-4f08-a500-4cbf4dadce15-model-cache\") on node \"ip-10-0-128-75.ec2.internal\" DevicePath \"\""
Jul 31 23:55:04.422176 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:04.422146    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-phkrf"
Jul 31 23:55:04.422176 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:04.422163    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-phkrf" event={"ID":"08ae5adf-ffd8-4f08-a500-4cbf4dadce15","Type":"ContainerDied","Data":"c5eff2496044f0d4575f9dfd15a1da2e0f1db0c8cbbb8ff96390fbb71c210f3b"}
Jul 31 23:55:04.422369 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:04.422200    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c5eff2496044f0d4575f9dfd15a1da2e0f1db0c8cbbb8ff96390fbb71c210f3b"
Jul 31 23:55:16.961919 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:16.961842    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-75774fc8c9-5mn9p"]
Jul 31 23:55:16.962287 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:16.962124    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="08ae5adf-ffd8-4f08-a500-4cbf4dadce15" containerName="download-hf-model"
Jul 31 23:55:16.962287 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:16.962135    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="08ae5adf-ffd8-4f08-a500-4cbf4dadce15" containerName="download-hf-model"
Jul 31 23:55:16.962287 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:16.962146    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="08ae5adf-ffd8-4f08-a500-4cbf4dadce15" containerName="s3-init"
Jul 31 23:55:16.962287 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:16.962151    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="08ae5adf-ffd8-4f08-a500-4cbf4dadce15" containerName="s3-init"
Jul 31 23:55:16.962287 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:16.962210    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="08ae5adf-ffd8-4f08-a500-4cbf4dadce15" containerName="s3-init"
Jul 31 23:55:16.965317 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:16.965302    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-75774fc8c9-5mn9p"
Jul 31 23:55:16.967890 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:16.967869    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-j55jp\""
Jul 31 23:55:16.969038 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:16.969020    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 31 23:55:16.969135 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:16.969058    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\""
Jul 31 23:55:16.969135 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:16.969042    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"odh-kserve-custom-ca-bundle\""
Jul 31 23:55:16.969252 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:16.969053    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 31 23:55:16.969252 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:16.969062    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-graph-1-kube-rbac-proxy-sar-config\""
Jul 31 23:55:16.969252 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:16.969036    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-graph-1-predictor-serving-cert\""
Jul 31 23:55:16.974356 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:16.974332    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-75774fc8c9-5mn9p"]
Jul 31 23:55:17.114439 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:17.114402    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bz2gd\" (UniqueName: \"kubernetes.io/projected/76301075-df42-4218-b34e-47f97615bb7c-kube-api-access-bz2gd\") pod \"isvc-sklearn-graph-1-predictor-75774fc8c9-5mn9p\" (UID: \"76301075-df42-4218-b34e-47f97615bb7c\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-75774fc8c9-5mn9p"
Jul 31 23:55:17.114439 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:17.114441    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/76301075-df42-4218-b34e-47f97615bb7c-proxy-tls\") pod \"isvc-sklearn-graph-1-predictor-75774fc8c9-5mn9p\" (UID: \"76301075-df42-4218-b34e-47f97615bb7c\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-75774fc8c9-5mn9p"
Jul 31 23:55:17.114651 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:17.114550    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/76301075-df42-4218-b34e-47f97615bb7c-cabundle-cert\") pod \"isvc-sklearn-graph-1-predictor-75774fc8c9-5mn9p\" (UID: \"76301075-df42-4218-b34e-47f97615bb7c\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-75774fc8c9-5mn9p"
Jul 31 23:55:17.114651 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:17.114585    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/76301075-df42-4218-b34e-47f97615bb7c-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-1-predictor-75774fc8c9-5mn9p\" (UID: \"76301075-df42-4218-b34e-47f97615bb7c\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-75774fc8c9-5mn9p"
Jul 31 23:55:17.114651 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:17.114611    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/76301075-df42-4218-b34e-47f97615bb7c-kserve-provision-location\") pod \"isvc-sklearn-graph-1-predictor-75774fc8c9-5mn9p\" (UID: \"76301075-df42-4218-b34e-47f97615bb7c\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-75774fc8c9-5mn9p"
Jul 31 23:55:17.215594 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:17.215515    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/76301075-df42-4218-b34e-47f97615bb7c-cabundle-cert\") pod \"isvc-sklearn-graph-1-predictor-75774fc8c9-5mn9p\" (UID: \"76301075-df42-4218-b34e-47f97615bb7c\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-75774fc8c9-5mn9p"
Jul 31 23:55:17.215594 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:17.215550    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/76301075-df42-4218-b34e-47f97615bb7c-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-1-predictor-75774fc8c9-5mn9p\" (UID: \"76301075-df42-4218-b34e-47f97615bb7c\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-75774fc8c9-5mn9p"
Jul 31 23:55:17.215594 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:17.215575    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/76301075-df42-4218-b34e-47f97615bb7c-kserve-provision-location\") pod \"isvc-sklearn-graph-1-predictor-75774fc8c9-5mn9p\" (UID: \"76301075-df42-4218-b34e-47f97615bb7c\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-75774fc8c9-5mn9p"
Jul 31 23:55:17.215883 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:17.215620    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bz2gd\" (UniqueName: \"kubernetes.io/projected/76301075-df42-4218-b34e-47f97615bb7c-kube-api-access-bz2gd\") pod \"isvc-sklearn-graph-1-predictor-75774fc8c9-5mn9p\" (UID: \"76301075-df42-4218-b34e-47f97615bb7c\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-75774fc8c9-5mn9p"
Jul 31 23:55:17.215883 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:17.215753    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/76301075-df42-4218-b34e-47f97615bb7c-proxy-tls\") pod \"isvc-sklearn-graph-1-predictor-75774fc8c9-5mn9p\" (UID: \"76301075-df42-4218-b34e-47f97615bb7c\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-75774fc8c9-5mn9p"
Jul 31 23:55:17.216057 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:17.216035    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/76301075-df42-4218-b34e-47f97615bb7c-kserve-provision-location\") pod \"isvc-sklearn-graph-1-predictor-75774fc8c9-5mn9p\" (UID: \"76301075-df42-4218-b34e-47f97615bb7c\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-75774fc8c9-5mn9p"
Jul 31 23:55:17.216259 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:17.216242    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/76301075-df42-4218-b34e-47f97615bb7c-cabundle-cert\") pod \"isvc-sklearn-graph-1-predictor-75774fc8c9-5mn9p\" (UID: \"76301075-df42-4218-b34e-47f97615bb7c\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-75774fc8c9-5mn9p"
Jul 31 23:55:17.216312 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:17.216272    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/76301075-df42-4218-b34e-47f97615bb7c-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-1-predictor-75774fc8c9-5mn9p\" (UID: \"76301075-df42-4218-b34e-47f97615bb7c\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-75774fc8c9-5mn9p"
Jul 31 23:55:17.218151 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:17.218131    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/76301075-df42-4218-b34e-47f97615bb7c-proxy-tls\") pod \"isvc-sklearn-graph-1-predictor-75774fc8c9-5mn9p\" (UID: \"76301075-df42-4218-b34e-47f97615bb7c\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-75774fc8c9-5mn9p"
Jul 31 23:55:17.224612 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:17.224586    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bz2gd\" (UniqueName: \"kubernetes.io/projected/76301075-df42-4218-b34e-47f97615bb7c-kube-api-access-bz2gd\") pod \"isvc-sklearn-graph-1-predictor-75774fc8c9-5mn9p\" (UID: \"76301075-df42-4218-b34e-47f97615bb7c\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-75774fc8c9-5mn9p"
Jul 31 23:55:17.262063 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:17.262036    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-skw2q"]
Jul 31 23:55:17.265645 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:17.265614    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-skw2q"
Jul 31 23:55:17.267980 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:17.267956    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-xgboost-graph-predictor-serving-cert\""
Jul 31 23:55:17.268124 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:17.268107    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-xgboost-graph-kube-rbac-proxy-sar-config\""
Jul 31 23:55:17.275133 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:17.275108    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-skw2q"]
Jul 31 23:55:17.275621 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:17.275586    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-75774fc8c9-5mn9p"
Jul 31 23:55:17.349952 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:17.349919    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-716dc-predictor-58d8b8d4f4-9xkds"]
Jul 31 23:55:17.353879 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:17.353857    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-716dc-predictor-58d8b8d4f4-9xkds"
Jul 31 23:55:17.356527 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:17.356505    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-716dc-kube-rbac-proxy-sar-config\""
Jul 31 23:55:17.356741 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:17.356723    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-716dc-predictor-serving-cert\""
Jul 31 23:55:17.364537 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:17.364512    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-716dc-predictor-58d8b8d4f4-9xkds"]
Jul 31 23:55:17.417317 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:17.417145    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b7cd0c34-4fac-4008-b972-54541f103f61-kube-rbac-proxy-sar-config\") pod \"isvc-xgboost-graph-predictor-555fc69d56-skw2q\" (UID: \"b7cd0c34-4fac-4008-b972-54541f103f61\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-skw2q"
Jul 31 23:55:17.417317 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:17.417192    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b7cd0c34-4fac-4008-b972-54541f103f61-kserve-provision-location\") pod \"isvc-xgboost-graph-predictor-555fc69d56-skw2q\" (UID: \"b7cd0c34-4fac-4008-b972-54541f103f61\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-skw2q"
Jul 31 23:55:17.417317 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:17.417257    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/b7cd0c34-4fac-4008-b972-54541f103f61-cabundle-cert\") pod \"isvc-xgboost-graph-predictor-555fc69d56-skw2q\" (UID: \"b7cd0c34-4fac-4008-b972-54541f103f61\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-skw2q"
Jul 31 23:55:17.417601 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:17.417405    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b7cd0c34-4fac-4008-b972-54541f103f61-proxy-tls\") pod \"isvc-xgboost-graph-predictor-555fc69d56-skw2q\" (UID: \"b7cd0c34-4fac-4008-b972-54541f103f61\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-skw2q"
Jul 31 23:55:17.417601 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:17.417457    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nhlg2\" (UniqueName: \"kubernetes.io/projected/b7cd0c34-4fac-4008-b972-54541f103f61-kube-api-access-nhlg2\") pod \"isvc-xgboost-graph-predictor-555fc69d56-skw2q\" (UID: \"b7cd0c34-4fac-4008-b972-54541f103f61\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-skw2q"
Jul 31 23:55:17.418945 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:17.418920    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-75774fc8c9-5mn9p"]
Jul 31 23:55:17.422481 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:55:17.422453    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76301075_df42_4218_b34e_47f97615bb7c.slice/crio-74437c3eb53cebf29caedaaa9ce0d8603ad59470f32de0b7b74c6a9f202e917c WatchSource:0}: Error finding container 74437c3eb53cebf29caedaaa9ce0d8603ad59470f32de0b7b74c6a9f202e917c: Status 404 returned error can't find the container with id 74437c3eb53cebf29caedaaa9ce0d8603ad59470f32de0b7b74c6a9f202e917c
Jul 31 23:55:17.457713 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:17.457690    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76b49f6c9c-7c92t"]
Jul 31 23:55:17.462261 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:17.462241    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76b49f6c9c-7c92t"
Jul 31 23:55:17.464756 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:17.464722    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-graph-2-predictor-serving-cert\""
Jul 31 23:55:17.464864 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:17.464833    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-graph-2-kube-rbac-proxy-sar-config\""
Jul 31 23:55:17.467397 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:17.467336    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-75774fc8c9-5mn9p" event={"ID":"76301075-df42-4218-b34e-47f97615bb7c","Type":"ContainerStarted","Data":"74437c3eb53cebf29caedaaa9ce0d8603ad59470f32de0b7b74c6a9f202e917c"}
Jul 31 23:55:17.470714 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:17.470691    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76b49f6c9c-7c92t"]
Jul 31 23:55:17.517968 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:17.517928    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/b7cd0c34-4fac-4008-b972-54541f103f61-cabundle-cert\") pod \"isvc-xgboost-graph-predictor-555fc69d56-skw2q\" (UID: \"b7cd0c34-4fac-4008-b972-54541f103f61\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-skw2q"
Jul 31 23:55:17.518125 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:17.518016    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/4988fb9c-fe37-4d48-bd7b-6bea640dba66-proxy-tls\") pod \"error-404-isvc-716dc-predictor-58d8b8d4f4-9xkds\" (UID: \"4988fb9c-fe37-4d48-bd7b-6bea640dba66\") " pod="kserve-ci-e2e-test/error-404-isvc-716dc-predictor-58d8b8d4f4-9xkds"
Jul 31 23:55:17.518125 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:17.518055    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/4988fb9c-fe37-4d48-bd7b-6bea640dba66-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-716dc-predictor-58d8b8d4f4-9xkds\" (UID: \"4988fb9c-fe37-4d48-bd7b-6bea640dba66\") " pod="kserve-ci-e2e-test/error-404-isvc-716dc-predictor-58d8b8d4f4-9xkds"
Jul 31 23:55:17.518125 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:17.518085    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b7cd0c34-4fac-4008-b972-54541f103f61-proxy-tls\") pod \"isvc-xgboost-graph-predictor-555fc69d56-skw2q\" (UID: \"b7cd0c34-4fac-4008-b972-54541f103f61\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-skw2q"
Jul 31 23:55:17.518259 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:17.518147    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nhlg2\" (UniqueName: \"kubernetes.io/projected/b7cd0c34-4fac-4008-b972-54541f103f61-kube-api-access-nhlg2\") pod \"isvc-xgboost-graph-predictor-555fc69d56-skw2q\" (UID: \"b7cd0c34-4fac-4008-b972-54541f103f61\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-skw2q"
Jul 31 23:55:17.518259 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:17.518190    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4z8gp\" (UniqueName: \"kubernetes.io/projected/4988fb9c-fe37-4d48-bd7b-6bea640dba66-kube-api-access-4z8gp\") pod \"error-404-isvc-716dc-predictor-58d8b8d4f4-9xkds\" (UID: \"4988fb9c-fe37-4d48-bd7b-6bea640dba66\") " pod="kserve-ci-e2e-test/error-404-isvc-716dc-predictor-58d8b8d4f4-9xkds"
Jul 31 23:55:17.518259 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:17.518234    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b7cd0c34-4fac-4008-b972-54541f103f61-kube-rbac-proxy-sar-config\") pod \"isvc-xgboost-graph-predictor-555fc69d56-skw2q\" (UID: \"b7cd0c34-4fac-4008-b972-54541f103f61\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-skw2q"
Jul 31 23:55:17.518464 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:17.518265    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b7cd0c34-4fac-4008-b972-54541f103f61-kserve-provision-location\") pod \"isvc-xgboost-graph-predictor-555fc69d56-skw2q\" (UID: \"b7cd0c34-4fac-4008-b972-54541f103f61\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-skw2q"
Jul 31 23:55:17.518714 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:17.518690    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b7cd0c34-4fac-4008-b972-54541f103f61-kserve-provision-location\") pod \"isvc-xgboost-graph-predictor-555fc69d56-skw2q\" (UID: \"b7cd0c34-4fac-4008-b972-54541f103f61\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-skw2q"
Jul 31 23:55:17.518804 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:17.518712    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/b7cd0c34-4fac-4008-b972-54541f103f61-cabundle-cert\") pod \"isvc-xgboost-graph-predictor-555fc69d56-skw2q\" (UID: \"b7cd0c34-4fac-4008-b972-54541f103f61\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-skw2q"
Jul 31 23:55:17.518924 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:17.518905    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b7cd0c34-4fac-4008-b972-54541f103f61-kube-rbac-proxy-sar-config\") pod \"isvc-xgboost-graph-predictor-555fc69d56-skw2q\" (UID: \"b7cd0c34-4fac-4008-b972-54541f103f61\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-skw2q"
Jul 31 23:55:17.520535 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:17.520514    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b7cd0c34-4fac-4008-b972-54541f103f61-proxy-tls\") pod \"isvc-xgboost-graph-predictor-555fc69d56-skw2q\" (UID: \"b7cd0c34-4fac-4008-b972-54541f103f61\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-skw2q"
Jul 31 23:55:17.527194 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:17.527174    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nhlg2\" (UniqueName: \"kubernetes.io/projected/b7cd0c34-4fac-4008-b972-54541f103f61-kube-api-access-nhlg2\") pod \"isvc-xgboost-graph-predictor-555fc69d56-skw2q\" (UID: \"b7cd0c34-4fac-4008-b972-54541f103f61\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-skw2q"
Jul 31 23:55:17.576127 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:17.576093    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-skw2q"
Jul 31 23:55:17.619709 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:17.619664    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/f0cf1575-e02f-4cdb-9810-c55d478f4d54-cabundle-cert\") pod \"isvc-sklearn-graph-2-predictor-76b49f6c9c-7c92t\" (UID: \"f0cf1575-e02f-4cdb-9810-c55d478f4d54\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76b49f6c9c-7c92t"
Jul 31 23:55:17.619871 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:17.619758    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f0cf1575-e02f-4cdb-9810-c55d478f4d54-proxy-tls\") pod \"isvc-sklearn-graph-2-predictor-76b49f6c9c-7c92t\" (UID: \"f0cf1575-e02f-4cdb-9810-c55d478f4d54\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76b49f6c9c-7c92t"
Jul 31 23:55:17.619871 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:17.619791    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/f0cf1575-e02f-4cdb-9810-c55d478f4d54-kserve-provision-location\") pod \"isvc-sklearn-graph-2-predictor-76b49f6c9c-7c92t\" (UID: \"f0cf1575-e02f-4cdb-9810-c55d478f4d54\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76b49f6c9c-7c92t"
Jul 31 23:55:17.619871 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:17.619823    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/4988fb9c-fe37-4d48-bd7b-6bea640dba66-proxy-tls\") pod \"error-404-isvc-716dc-predictor-58d8b8d4f4-9xkds\" (UID: \"4988fb9c-fe37-4d48-bd7b-6bea640dba66\") " pod="kserve-ci-e2e-test/error-404-isvc-716dc-predictor-58d8b8d4f4-9xkds"
Jul 31 23:55:17.619991 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:17.619872    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/4988fb9c-fe37-4d48-bd7b-6bea640dba66-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-716dc-predictor-58d8b8d4f4-9xkds\" (UID: \"4988fb9c-fe37-4d48-bd7b-6bea640dba66\") " pod="kserve-ci-e2e-test/error-404-isvc-716dc-predictor-58d8b8d4f4-9xkds"
Jul 31 23:55:17.619991 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:17.619911    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4z8gp\" (UniqueName: \"kubernetes.io/projected/4988fb9c-fe37-4d48-bd7b-6bea640dba66-kube-api-access-4z8gp\") pod \"error-404-isvc-716dc-predictor-58d8b8d4f4-9xkds\" (UID: \"4988fb9c-fe37-4d48-bd7b-6bea640dba66\") " pod="kserve-ci-e2e-test/error-404-isvc-716dc-predictor-58d8b8d4f4-9xkds"
Jul 31 23:55:17.619991 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:17.619935    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/f0cf1575-e02f-4cdb-9810-c55d478f4d54-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-2-predictor-76b49f6c9c-7c92t\" (UID: \"f0cf1575-e02f-4cdb-9810-c55d478f4d54\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76b49f6c9c-7c92t"
Jul 31 23:55:17.619991 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:17.619981    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q66fn\" (UniqueName: \"kubernetes.io/projected/f0cf1575-e02f-4cdb-9810-c55d478f4d54-kube-api-access-q66fn\") pod \"isvc-sklearn-graph-2-predictor-76b49f6c9c-7c92t\" (UID: \"f0cf1575-e02f-4cdb-9810-c55d478f4d54\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76b49f6c9c-7c92t"
Jul 31 23:55:17.620609 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:17.620583    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/4988fb9c-fe37-4d48-bd7b-6bea640dba66-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-716dc-predictor-58d8b8d4f4-9xkds\" (UID: \"4988fb9c-fe37-4d48-bd7b-6bea640dba66\") " pod="kserve-ci-e2e-test/error-404-isvc-716dc-predictor-58d8b8d4f4-9xkds"
Jul 31 23:55:17.622418 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:17.622390    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/4988fb9c-fe37-4d48-bd7b-6bea640dba66-proxy-tls\") pod \"error-404-isvc-716dc-predictor-58d8b8d4f4-9xkds\" (UID: \"4988fb9c-fe37-4d48-bd7b-6bea640dba66\") " pod="kserve-ci-e2e-test/error-404-isvc-716dc-predictor-58d8b8d4f4-9xkds"
Jul 31 23:55:17.628477 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:17.628362    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4z8gp\" (UniqueName: \"kubernetes.io/projected/4988fb9c-fe37-4d48-bd7b-6bea640dba66-kube-api-access-4z8gp\") pod \"error-404-isvc-716dc-predictor-58d8b8d4f4-9xkds\" (UID: \"4988fb9c-fe37-4d48-bd7b-6bea640dba66\") " pod="kserve-ci-e2e-test/error-404-isvc-716dc-predictor-58d8b8d4f4-9xkds"
Jul 31 23:55:17.666021 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:17.665991    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-716dc-predictor-58d8b8d4f4-9xkds"
Jul 31 23:55:17.706727 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:17.706696    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-skw2q"]
Jul 31 23:55:17.707577 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:55:17.707550    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7cd0c34_4fac_4008_b972_54541f103f61.slice/crio-4a2b44ea8b00f5af7bead2572258e046387ec15ad7d85e606d0dc4358379f291 WatchSource:0}: Error finding container 4a2b44ea8b00f5af7bead2572258e046387ec15ad7d85e606d0dc4358379f291: Status 404 returned error can't find the container with id 4a2b44ea8b00f5af7bead2572258e046387ec15ad7d85e606d0dc4358379f291
Jul 31 23:55:17.720829 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:17.720774    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/f0cf1575-e02f-4cdb-9810-c55d478f4d54-cabundle-cert\") pod \"isvc-sklearn-graph-2-predictor-76b49f6c9c-7c92t\" (UID: \"f0cf1575-e02f-4cdb-9810-c55d478f4d54\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76b49f6c9c-7c92t"
Jul 31 23:55:17.720971 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:17.720953    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f0cf1575-e02f-4cdb-9810-c55d478f4d54-proxy-tls\") pod \"isvc-sklearn-graph-2-predictor-76b49f6c9c-7c92t\" (UID: \"f0cf1575-e02f-4cdb-9810-c55d478f4d54\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76b49f6c9c-7c92t"
Jul 31 23:55:17.721139 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:17.721121    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/f0cf1575-e02f-4cdb-9810-c55d478f4d54-kserve-provision-location\") pod \"isvc-sklearn-graph-2-predictor-76b49f6c9c-7c92t\" (UID: \"f0cf1575-e02f-4cdb-9810-c55d478f4d54\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76b49f6c9c-7c92t"
Jul 31 23:55:17.721788 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:17.721510    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/f0cf1575-e02f-4cdb-9810-c55d478f4d54-kserve-provision-location\") pod \"isvc-sklearn-graph-2-predictor-76b49f6c9c-7c92t\" (UID: \"f0cf1575-e02f-4cdb-9810-c55d478f4d54\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76b49f6c9c-7c92t"
Jul 31 23:55:17.721788 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:17.721517    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/f0cf1575-e02f-4cdb-9810-c55d478f4d54-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-2-predictor-76b49f6c9c-7c92t\" (UID: \"f0cf1575-e02f-4cdb-9810-c55d478f4d54\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76b49f6c9c-7c92t"
Jul 31 23:55:17.721788 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:17.721575    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q66fn\" (UniqueName: \"kubernetes.io/projected/f0cf1575-e02f-4cdb-9810-c55d478f4d54-kube-api-access-q66fn\") pod \"isvc-sklearn-graph-2-predictor-76b49f6c9c-7c92t\" (UID: \"f0cf1575-e02f-4cdb-9810-c55d478f4d54\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76b49f6c9c-7c92t"
Jul 31 23:55:17.721788 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:17.721591    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/f0cf1575-e02f-4cdb-9810-c55d478f4d54-cabundle-cert\") pod \"isvc-sklearn-graph-2-predictor-76b49f6c9c-7c92t\" (UID: \"f0cf1575-e02f-4cdb-9810-c55d478f4d54\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76b49f6c9c-7c92t"
Jul 31 23:55:17.722162 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:17.722141    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/f0cf1575-e02f-4cdb-9810-c55d478f4d54-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-2-predictor-76b49f6c9c-7c92t\" (UID: \"f0cf1575-e02f-4cdb-9810-c55d478f4d54\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76b49f6c9c-7c92t"
Jul 31 23:55:17.724972 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:17.724939    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f0cf1575-e02f-4cdb-9810-c55d478f4d54-proxy-tls\") pod \"isvc-sklearn-graph-2-predictor-76b49f6c9c-7c92t\" (UID: \"f0cf1575-e02f-4cdb-9810-c55d478f4d54\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76b49f6c9c-7c92t"
Jul 31 23:55:17.731327 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:17.731290    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q66fn\" (UniqueName: \"kubernetes.io/projected/f0cf1575-e02f-4cdb-9810-c55d478f4d54-kube-api-access-q66fn\") pod \"isvc-sklearn-graph-2-predictor-76b49f6c9c-7c92t\" (UID: \"f0cf1575-e02f-4cdb-9810-c55d478f4d54\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76b49f6c9c-7c92t"
Jul 31 23:55:17.773054 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:17.772988    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76b49f6c9c-7c92t"
Jul 31 23:55:17.819594 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:17.819099    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-716dc-predictor-58d8b8d4f4-9xkds"]
Jul 31 23:55:17.919011 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:17.918981    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76b49f6c9c-7c92t"]
Jul 31 23:55:17.921324 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:55:17.921298    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0cf1575_e02f_4cdb_9810_c55d478f4d54.slice/crio-5c8c762286e52092bc4e99a9bf56611461f2210a11a8624ae1ca7725724960db WatchSource:0}: Error finding container 5c8c762286e52092bc4e99a9bf56611461f2210a11a8624ae1ca7725724960db: Status 404 returned error can't find the container with id 5c8c762286e52092bc4e99a9bf56611461f2210a11a8624ae1ca7725724960db
Jul 31 23:55:18.480039 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:18.479997    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-skw2q" event={"ID":"b7cd0c34-4fac-4008-b972-54541f103f61","Type":"ContainerStarted","Data":"81e619df97615bc9a54a4f4dc56400f9893da5a636c4a86cb2cd7b00ae0488d2"}
Jul 31 23:55:18.480498 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:18.480067    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-skw2q" event={"ID":"b7cd0c34-4fac-4008-b972-54541f103f61","Type":"ContainerStarted","Data":"4a2b44ea8b00f5af7bead2572258e046387ec15ad7d85e606d0dc4358379f291"}
Jul 31 23:55:18.486219 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:18.486189    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-75774fc8c9-5mn9p" event={"ID":"76301075-df42-4218-b34e-47f97615bb7c","Type":"ContainerStarted","Data":"9f08aae96f35be8dc65d383a056987bf1f7100f6b3377b4cb690ebe5cda01866"}
Jul 31 23:55:18.488574 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:18.488539    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-716dc-predictor-58d8b8d4f4-9xkds" event={"ID":"4988fb9c-fe37-4d48-bd7b-6bea640dba66","Type":"ContainerStarted","Data":"b11e28cfc26424f3c2e2ab6003c8119a3404311e8ead69e347aad47da7a4a679"}
Jul 31 23:55:18.492114 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:18.491680    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76b49f6c9c-7c92t" event={"ID":"f0cf1575-e02f-4cdb-9810-c55d478f4d54","Type":"ContainerStarted","Data":"09569673e23d868b81f48980895bedcea7ae58d6926fbe473466330e8f3e0945"}
Jul 31 23:55:18.492114 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:18.491735    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76b49f6c9c-7c92t" event={"ID":"f0cf1575-e02f-4cdb-9810-c55d478f4d54","Type":"ContainerStarted","Data":"5c8c762286e52092bc4e99a9bf56611461f2210a11a8624ae1ca7725724960db"}
Jul 31 23:55:22.515443 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:22.515361    2564 generic.go:356] "Generic (PLEG): container finished" podID="b7cd0c34-4fac-4008-b972-54541f103f61" containerID="81e619df97615bc9a54a4f4dc56400f9893da5a636c4a86cb2cd7b00ae0488d2" exitCode=0
Jul 31 23:55:22.515443 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:22.515431    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-skw2q" event={"ID":"b7cd0c34-4fac-4008-b972-54541f103f61","Type":"ContainerDied","Data":"81e619df97615bc9a54a4f4dc56400f9893da5a636c4a86cb2cd7b00ae0488d2"}
Jul 31 23:55:23.522345 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:23.522233    2564 generic.go:356] "Generic (PLEG): container finished" podID="76301075-df42-4218-b34e-47f97615bb7c" containerID="9f08aae96f35be8dc65d383a056987bf1f7100f6b3377b4cb690ebe5cda01866" exitCode=0
Jul 31 23:55:23.522345 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:23.522275    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-75774fc8c9-5mn9p" event={"ID":"76301075-df42-4218-b34e-47f97615bb7c","Type":"ContainerDied","Data":"9f08aae96f35be8dc65d383a056987bf1f7100f6b3377b4cb690ebe5cda01866"}
Jul 31 23:55:23.524829 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:23.524799    2564 generic.go:356] "Generic (PLEG): container finished" podID="f0cf1575-e02f-4cdb-9810-c55d478f4d54" containerID="09569673e23d868b81f48980895bedcea7ae58d6926fbe473466330e8f3e0945" exitCode=0
Jul 31 23:55:23.524895 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:23.524868    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76b49f6c9c-7c92t" event={"ID":"f0cf1575-e02f-4cdb-9810-c55d478f4d54","Type":"ContainerDied","Data":"09569673e23d868b81f48980895bedcea7ae58d6926fbe473466330e8f3e0945"}
Jul 31 23:55:45.640911 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:45.640869    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-75774fc8c9-5mn9p" event={"ID":"76301075-df42-4218-b34e-47f97615bb7c","Type":"ContainerStarted","Data":"65524873155aeedbdbb96082b086c8d14cdc4856a7eb41f99978b38b15b3f70f"}
Jul 31 23:55:45.643781 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:45.643752    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-716dc-predictor-58d8b8d4f4-9xkds" event={"ID":"4988fb9c-fe37-4d48-bd7b-6bea640dba66","Type":"ContainerStarted","Data":"560fae720b76b5226c193fcc998e6fbb463056a76c77a5171ca7cea47c9d1352"}
Jul 31 23:55:45.647309 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:45.647279    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76b49f6c9c-7c92t" event={"ID":"f0cf1575-e02f-4cdb-9810-c55d478f4d54","Type":"ContainerStarted","Data":"222a010e745c69265ba28e9ebdd9ef1b24e435c87e18dee328170e385b8f7693"}
Jul 31 23:55:45.651359 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:45.651238    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-skw2q" event={"ID":"b7cd0c34-4fac-4008-b972-54541f103f61","Type":"ContainerStarted","Data":"5bdc5832c0174987676e09df579f30e0998e0993a69e2ca22614595b45b6f9f1"}
Jul 31 23:55:48.672944 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:48.672911    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-75774fc8c9-5mn9p" event={"ID":"76301075-df42-4218-b34e-47f97615bb7c","Type":"ContainerStarted","Data":"2b17cf90c82e012b50598cf59e3e237d2b681cd7dce3a06a6a56e7b026e70754"}
Jul 31 23:55:48.673716 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:48.673701    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-75774fc8c9-5mn9p"
Jul 31 23:55:48.673770 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:48.673740    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-75774fc8c9-5mn9p"
Jul 31 23:55:48.674975 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:48.674950    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-75774fc8c9-5mn9p" podUID="76301075-df42-4218-b34e-47f97615bb7c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.17:8080: connect: connection refused"
Jul 31 23:55:48.675927 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:48.675909    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-716dc-predictor-58d8b8d4f4-9xkds" event={"ID":"4988fb9c-fe37-4d48-bd7b-6bea640dba66","Type":"ContainerStarted","Data":"af69c85929cc5a582abe617f467b97412e3afb85afbc42c04a2bee80fd5a5b13"}
Jul 31 23:55:48.676331 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:48.676316    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-716dc-predictor-58d8b8d4f4-9xkds"
Jul 31 23:55:48.676503 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:48.676493    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-716dc-predictor-58d8b8d4f4-9xkds"
Jul 31 23:55:48.677425 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:48.677393    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-716dc-predictor-58d8b8d4f4-9xkds" podUID="4988fb9c-fe37-4d48-bd7b-6bea640dba66" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.19:8080: connect: connection refused"
Jul 31 23:55:48.679035 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:48.679020    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76b49f6c9c-7c92t" event={"ID":"f0cf1575-e02f-4cdb-9810-c55d478f4d54","Type":"ContainerStarted","Data":"cf625482934d9ebd49d01ada684e0b26a79809c44a8ac1baac8a7294ad11bb1d"}
Jul 31 23:55:48.679118 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:48.679060    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76b49f6c9c-7c92t"
Jul 31 23:55:48.679156 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:48.679128    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76b49f6c9c-7c92t"
Jul 31 23:55:48.680057 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:48.680038    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76b49f6c9c-7c92t" podUID="f0cf1575-e02f-4cdb-9810-c55d478f4d54" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.20:8080: connect: connection refused"
Jul 31 23:55:48.680977 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:48.680960    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-skw2q" event={"ID":"b7cd0c34-4fac-4008-b972-54541f103f61","Type":"ContainerStarted","Data":"b6bcd35e048e7e63b9c8d1615ef051b2f9469db8fce6f03000ec8178bfe85196"}
Jul 31 23:55:48.681166 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:48.681152    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-skw2q"
Jul 31 23:55:48.681212 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:48.681195    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-skw2q"
Jul 31 23:55:48.681975 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:48.681958    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-skw2q" podUID="b7cd0c34-4fac-4008-b972-54541f103f61" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.18:8080: connect: connection refused"
Jul 31 23:55:48.691942 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:48.691904    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-75774fc8c9-5mn9p" podStartSLOduration=3.967443449 podStartE2EDuration="32.691891472s" totalImagesPullingTime="24.728340188s" totalInitContainerRuntime="3.996107835s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:55:16 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:55:48.690816964 +0000 UTC m=+566.802944677" watchObservedRunningTime="2026-07-31 23:55:48.691891472 +0000 UTC m=+566.804019145"
Jul 31 23:55:48.707019 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:48.706975    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-716dc-predictor-58d8b8d4f4-9xkds" podStartSLOduration=9.895088363 podStartE2EDuration="31.706960003s" totalImagesPullingTime="21.81187164s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:55:17 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:55:48.705956261 +0000 UTC m=+566.818083961" watchObservedRunningTime="2026-07-31 23:55:48.706960003 +0000 UTC m=+566.819087677"
Jul 31 23:55:48.726600 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:48.726530    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-skw2q" podStartSLOduration=1.806757119 podStartE2EDuration="31.726511345s" totalImagesPullingTime="25.843413016s" totalInitContainerRuntime="4.07634121s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:55:17 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:55:48.724286503 +0000 UTC m=+566.836414177" watchObservedRunningTime="2026-07-31 23:55:48.726511345 +0000 UTC m=+566.838639020"
Jul 31 23:55:48.743250 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:48.743156    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76b49f6c9c-7c92t" podStartSLOduration=11.881832219 podStartE2EDuration="31.743140852s" totalImagesPullingTime="16.211849155s" totalInitContainerRuntime="3.649459478s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:55:17 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:55:48.742138597 +0000 UTC m=+566.854266296" watchObservedRunningTime="2026-07-31 23:55:48.743140852 +0000 UTC m=+566.855268525"
Jul 31 23:55:49.684758 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:49.684713    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76b49f6c9c-7c92t" podUID="f0cf1575-e02f-4cdb-9810-c55d478f4d54" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.20:8080: connect: connection refused"
Jul 31 23:55:49.685187 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:49.684861    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-skw2q" podUID="b7cd0c34-4fac-4008-b972-54541f103f61" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.18:8080: connect: connection refused"
Jul 31 23:55:49.685187 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:49.684886    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-716dc-predictor-58d8b8d4f4-9xkds" podUID="4988fb9c-fe37-4d48-bd7b-6bea640dba66" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.19:8080: connect: connection refused"
Jul 31 23:55:49.685187 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:49.684911    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-75774fc8c9-5mn9p" podUID="76301075-df42-4218-b34e-47f97615bb7c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.17:8080: connect: connection refused"
Jul 31 23:55:50.689192 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:50.689150    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-75774fc8c9-5mn9p" podUID="76301075-df42-4218-b34e-47f97615bb7c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.17:8080: connect: connection refused"
Jul 31 23:55:50.689652 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:50.689273    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-716dc-predictor-58d8b8d4f4-9xkds" podUID="4988fb9c-fe37-4d48-bd7b-6bea640dba66" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.19:8080: connect: connection refused"
Jul 31 23:55:54.689380 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:54.689350    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76b49f6c9c-7c92t"
Jul 31 23:55:54.689878 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:54.689710    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-skw2q"
Jul 31 23:55:54.689878 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:54.689847    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76b49f6c9c-7c92t" podUID="f0cf1575-e02f-4cdb-9810-c55d478f4d54" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.20:8080: connect: connection refused"
Jul 31 23:55:54.690210 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:54.690178    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-skw2q" podUID="b7cd0c34-4fac-4008-b972-54541f103f61" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.18:8080: connect: connection refused"
Jul 31 23:55:55.694574 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:55.694546    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-75774fc8c9-5mn9p"
Jul 31 23:55:55.694971 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:55.694653    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-716dc-predictor-58d8b8d4f4-9xkds"
Jul 31 23:55:55.695191 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:55.695162    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-716dc-predictor-58d8b8d4f4-9xkds" podUID="4988fb9c-fe37-4d48-bd7b-6bea640dba66" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.19:8080: connect: connection refused"
Jul 31 23:55:55.695273 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:55:55.695191    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-75774fc8c9-5mn9p" podUID="76301075-df42-4218-b34e-47f97615bb7c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.17:8080: connect: connection refused"
Jul 31 23:56:04.690271 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:56:04.690231    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76b49f6c9c-7c92t" podUID="f0cf1575-e02f-4cdb-9810-c55d478f4d54" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.20:8080: connect: connection refused"
Jul 31 23:56:04.690753 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:56:04.690231    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-skw2q" podUID="b7cd0c34-4fac-4008-b972-54541f103f61" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.18:8080: connect: connection refused"
Jul 31 23:56:05.695971 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:56:05.695931    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-75774fc8c9-5mn9p" podUID="76301075-df42-4218-b34e-47f97615bb7c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.17:8080: connect: connection refused"
Jul 31 23:56:05.696376 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:56:05.695931    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-716dc-predictor-58d8b8d4f4-9xkds" podUID="4988fb9c-fe37-4d48-bd7b-6bea640dba66" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.19:8080: connect: connection refused"
Jul 31 23:56:14.689911 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:56:14.689871    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76b49f6c9c-7c92t" podUID="f0cf1575-e02f-4cdb-9810-c55d478f4d54" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.20:8080: connect: connection refused"
Jul 31 23:56:14.690379 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:56:14.690129    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-skw2q" podUID="b7cd0c34-4fac-4008-b972-54541f103f61" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.18:8080: connect: connection refused"
Jul 31 23:56:15.696038 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:56:15.696003    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-716dc-predictor-58d8b8d4f4-9xkds" podUID="4988fb9c-fe37-4d48-bd7b-6bea640dba66" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.19:8080: connect: connection refused"
Jul 31 23:56:15.696430 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:56:15.696004    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-75774fc8c9-5mn9p" podUID="76301075-df42-4218-b34e-47f97615bb7c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.17:8080: connect: connection refused"
Jul 31 23:56:24.690432 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:56:24.690391    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76b49f6c9c-7c92t" podUID="f0cf1575-e02f-4cdb-9810-c55d478f4d54" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.20:8080: connect: connection refused"
Jul 31 23:56:24.690852 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:56:24.690391    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-skw2q" podUID="b7cd0c34-4fac-4008-b972-54541f103f61" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.18:8080: connect: connection refused"
Jul 31 23:56:25.695729 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:56:25.695688    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-716dc-predictor-58d8b8d4f4-9xkds" podUID="4988fb9c-fe37-4d48-bd7b-6bea640dba66" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.19:8080: connect: connection refused"
Jul 31 23:56:25.696188 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:56:25.695695    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-75774fc8c9-5mn9p" podUID="76301075-df42-4218-b34e-47f97615bb7c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.17:8080: connect: connection refused"
Jul 31 23:56:34.689905 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:56:34.689869    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76b49f6c9c-7c92t" podUID="f0cf1575-e02f-4cdb-9810-c55d478f4d54" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.20:8080: connect: connection refused"
Jul 31 23:56:34.690371 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:56:34.690132    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-skw2q" podUID="b7cd0c34-4fac-4008-b972-54541f103f61" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.18:8080: connect: connection refused"
Jul 31 23:56:35.695586 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:56:35.695538    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-75774fc8c9-5mn9p" podUID="76301075-df42-4218-b34e-47f97615bb7c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.17:8080: connect: connection refused"
Jul 31 23:56:35.696087 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:56:35.695782    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-716dc-predictor-58d8b8d4f4-9xkds"
Jul 31 23:56:44.690756 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:56:44.690670    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-skw2q" podUID="b7cd0c34-4fac-4008-b972-54541f103f61" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.18:8080: connect: connection refused"
Jul 31 23:56:44.691140 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:56:44.690670    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76b49f6c9c-7c92t" podUID="f0cf1575-e02f-4cdb-9810-c55d478f4d54" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.20:8080: connect: connection refused"
Jul 31 23:56:45.696067 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:56:45.696025    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-75774fc8c9-5mn9p" podUID="76301075-df42-4218-b34e-47f97615bb7c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.17:8080: connect: connection refused"
Jul 31 23:56:47.221938 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:56:47.221895    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-716dc-predictor-58d8b8d4f4-9xkds"]
Jul 31 23:56:47.222422 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:56:47.222153    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-716dc-predictor-58d8b8d4f4-9xkds" podUID="4988fb9c-fe37-4d48-bd7b-6bea640dba66" containerName="kserve-container" containerID="cri-o://560fae720b76b5226c193fcc998e6fbb463056a76c77a5171ca7cea47c9d1352" gracePeriod=30
Jul 31 23:56:47.222422 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:56:47.222372    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-716dc-predictor-58d8b8d4f4-9xkds" podUID="4988fb9c-fe37-4d48-bd7b-6bea640dba66" containerName="kube-rbac-proxy" containerID="cri-o://af69c85929cc5a582abe617f467b97412e3afb85afbc42c04a2bee80fd5a5b13" gracePeriod=30
Jul 31 23:56:47.330644 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:56:47.330590    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-32541-predictor-8d6ffd9d-mp9l6"]
Jul 31 23:56:47.334028 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:56:47.334004    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-32541-predictor-8d6ffd9d-mp9l6"
Jul 31 23:56:47.336803 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:56:47.336775    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-32541-predictor-serving-cert\""
Jul 31 23:56:47.336930 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:56:47.336776    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-32541-kube-rbac-proxy-sar-config\""
Jul 31 23:56:47.344060 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:56:47.344037    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-32541-predictor-8d6ffd9d-mp9l6"]
Jul 31 23:56:47.427355 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:56:47.427320    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4wgvd\" (UniqueName: \"kubernetes.io/projected/097e1321-7c6d-4446-98b2-21cc0442df39-kube-api-access-4wgvd\") pod \"error-404-isvc-32541-predictor-8d6ffd9d-mp9l6\" (UID: \"097e1321-7c6d-4446-98b2-21cc0442df39\") " pod="kserve-ci-e2e-test/error-404-isvc-32541-predictor-8d6ffd9d-mp9l6"
Jul 31 23:56:47.427562 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:56:47.427380    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/097e1321-7c6d-4446-98b2-21cc0442df39-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-32541-predictor-8d6ffd9d-mp9l6\" (UID: \"097e1321-7c6d-4446-98b2-21cc0442df39\") " pod="kserve-ci-e2e-test/error-404-isvc-32541-predictor-8d6ffd9d-mp9l6"
Jul 31 23:56:47.427562 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:56:47.427479    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/097e1321-7c6d-4446-98b2-21cc0442df39-proxy-tls\") pod \"error-404-isvc-32541-predictor-8d6ffd9d-mp9l6\" (UID: \"097e1321-7c6d-4446-98b2-21cc0442df39\") " pod="kserve-ci-e2e-test/error-404-isvc-32541-predictor-8d6ffd9d-mp9l6"
Jul 31 23:56:47.528063 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:56:47.527980    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4wgvd\" (UniqueName: \"kubernetes.io/projected/097e1321-7c6d-4446-98b2-21cc0442df39-kube-api-access-4wgvd\") pod \"error-404-isvc-32541-predictor-8d6ffd9d-mp9l6\" (UID: \"097e1321-7c6d-4446-98b2-21cc0442df39\") " pod="kserve-ci-e2e-test/error-404-isvc-32541-predictor-8d6ffd9d-mp9l6"
Jul 31 23:56:47.528063 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:56:47.528025    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/097e1321-7c6d-4446-98b2-21cc0442df39-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-32541-predictor-8d6ffd9d-mp9l6\" (UID: \"097e1321-7c6d-4446-98b2-21cc0442df39\") " pod="kserve-ci-e2e-test/error-404-isvc-32541-predictor-8d6ffd9d-mp9l6"
Jul 31 23:56:47.528256 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:56:47.528072    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/097e1321-7c6d-4446-98b2-21cc0442df39-proxy-tls\") pod \"error-404-isvc-32541-predictor-8d6ffd9d-mp9l6\" (UID: \"097e1321-7c6d-4446-98b2-21cc0442df39\") " pod="kserve-ci-e2e-test/error-404-isvc-32541-predictor-8d6ffd9d-mp9l6"
Jul 31 23:56:47.528256 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:56:47.528180    2564 secret.go:189] Couldn't get secret kserve-ci-e2e-test/error-404-isvc-32541-predictor-serving-cert: secret "error-404-isvc-32541-predictor-serving-cert" not found
Jul 31 23:56:47.528256 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:56:47.528242    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/097e1321-7c6d-4446-98b2-21cc0442df39-proxy-tls podName:097e1321-7c6d-4446-98b2-21cc0442df39 nodeName:}" failed. No retries permitted until 2026-07-31 23:56:48.028223393 +0000 UTC m=+626.140351046 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/097e1321-7c6d-4446-98b2-21cc0442df39-proxy-tls") pod "error-404-isvc-32541-predictor-8d6ffd9d-mp9l6" (UID: "097e1321-7c6d-4446-98b2-21cc0442df39") : secret "error-404-isvc-32541-predictor-serving-cert" not found
Jul 31 23:56:47.528733 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:56:47.528713    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/097e1321-7c6d-4446-98b2-21cc0442df39-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-32541-predictor-8d6ffd9d-mp9l6\" (UID: \"097e1321-7c6d-4446-98b2-21cc0442df39\") " pod="kserve-ci-e2e-test/error-404-isvc-32541-predictor-8d6ffd9d-mp9l6"
Jul 31 23:56:47.536480 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:56:47.536456    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4wgvd\" (UniqueName: \"kubernetes.io/projected/097e1321-7c6d-4446-98b2-21cc0442df39-kube-api-access-4wgvd\") pod \"error-404-isvc-32541-predictor-8d6ffd9d-mp9l6\" (UID: \"097e1321-7c6d-4446-98b2-21cc0442df39\") " pod="kserve-ci-e2e-test/error-404-isvc-32541-predictor-8d6ffd9d-mp9l6"
Jul 31 23:56:47.926120 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:56:47.926079    2564 generic.go:356] "Generic (PLEG): container finished" podID="4988fb9c-fe37-4d48-bd7b-6bea640dba66" containerID="af69c85929cc5a582abe617f467b97412e3afb85afbc42c04a2bee80fd5a5b13" exitCode=2
Jul 31 23:56:47.926284 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:56:47.926137    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-716dc-predictor-58d8b8d4f4-9xkds" event={"ID":"4988fb9c-fe37-4d48-bd7b-6bea640dba66","Type":"ContainerDied","Data":"af69c85929cc5a582abe617f467b97412e3afb85afbc42c04a2bee80fd5a5b13"}
Jul 31 23:56:48.032393 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:56:48.032356    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/097e1321-7c6d-4446-98b2-21cc0442df39-proxy-tls\") pod \"error-404-isvc-32541-predictor-8d6ffd9d-mp9l6\" (UID: \"097e1321-7c6d-4446-98b2-21cc0442df39\") " pod="kserve-ci-e2e-test/error-404-isvc-32541-predictor-8d6ffd9d-mp9l6"
Jul 31 23:56:48.034710 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:56:48.034691    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/097e1321-7c6d-4446-98b2-21cc0442df39-proxy-tls\") pod \"error-404-isvc-32541-predictor-8d6ffd9d-mp9l6\" (UID: \"097e1321-7c6d-4446-98b2-21cc0442df39\") " pod="kserve-ci-e2e-test/error-404-isvc-32541-predictor-8d6ffd9d-mp9l6"
Jul 31 23:56:48.246191 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:56:48.246100    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-32541-predictor-8d6ffd9d-mp9l6"
Jul 31 23:56:48.377792 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:56:48.377729    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-32541-predictor-8d6ffd9d-mp9l6"]
Jul 31 23:56:48.379975 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:56:48.379949    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod097e1321_7c6d_4446_98b2_21cc0442df39.slice/crio-35aa17629a1b49515cc93e1f9e1bdbc463cbc3de69c3ffd406cbc9e1b17cc93e WatchSource:0}: Error finding container 35aa17629a1b49515cc93e1f9e1bdbc463cbc3de69c3ffd406cbc9e1b17cc93e: Status 404 returned error can't find the container with id 35aa17629a1b49515cc93e1f9e1bdbc463cbc3de69c3ffd406cbc9e1b17cc93e
Jul 31 23:56:48.932180 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:56:48.932150    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-32541-predictor-8d6ffd9d-mp9l6" event={"ID":"097e1321-7c6d-4446-98b2-21cc0442df39","Type":"ContainerStarted","Data":"6ecacd97b1cbb6a66a6511a926831aec83e57753ca610375005d233180559cf4"}
Jul 31 23:56:48.932344 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:56:48.932205    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-32541-predictor-8d6ffd9d-mp9l6" event={"ID":"097e1321-7c6d-4446-98b2-21cc0442df39","Type":"ContainerStarted","Data":"95fcfc8694ee8f077fd61dfa912605bc5b7989824fdbc0a2aa83d130798737f0"}
Jul 31 23:56:48.932344 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:56:48.932225    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-32541-predictor-8d6ffd9d-mp9l6" event={"ID":"097e1321-7c6d-4446-98b2-21cc0442df39","Type":"ContainerStarted","Data":"35aa17629a1b49515cc93e1f9e1bdbc463cbc3de69c3ffd406cbc9e1b17cc93e"}
Jul 31 23:56:48.932344 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:56:48.932250    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-32541-predictor-8d6ffd9d-mp9l6"
Jul 31 23:56:48.950582 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:56:48.950537    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-32541-predictor-8d6ffd9d-mp9l6" podStartSLOduration=1.9505202320000001 podStartE2EDuration="1.950520232s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:56:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:56:48.950121874 +0000 UTC m=+627.062249566" watchObservedRunningTime="2026-07-31 23:56:48.950520232 +0000 UTC m=+627.062647906"
Jul 31 23:56:49.936165 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:56:49.936133    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-32541-predictor-8d6ffd9d-mp9l6"
Jul 31 23:56:49.937363 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:56:49.937339    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-32541-predictor-8d6ffd9d-mp9l6" podUID="097e1321-7c6d-4446-98b2-21cc0442df39" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.21:8080: connect: connection refused"
Jul 31 23:56:50.689722 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:56:50.689681    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-716dc-predictor-58d8b8d4f4-9xkds" podUID="4988fb9c-fe37-4d48-bd7b-6bea640dba66" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.19:8643/healthz\": dial tcp 10.134.0.19:8643: connect: connection refused"
Jul 31 23:56:50.862233 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:56:50.862210    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-716dc-predictor-58d8b8d4f4-9xkds"
Jul 31 23:56:50.941677 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:56:50.941567    2564 generic.go:356] "Generic (PLEG): container finished" podID="4988fb9c-fe37-4d48-bd7b-6bea640dba66" containerID="560fae720b76b5226c193fcc998e6fbb463056a76c77a5171ca7cea47c9d1352" exitCode=0
Jul 31 23:56:50.941677 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:56:50.941619    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-716dc-predictor-58d8b8d4f4-9xkds"
Jul 31 23:56:50.941677 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:56:50.941649    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-716dc-predictor-58d8b8d4f4-9xkds" event={"ID":"4988fb9c-fe37-4d48-bd7b-6bea640dba66","Type":"ContainerDied","Data":"560fae720b76b5226c193fcc998e6fbb463056a76c77a5171ca7cea47c9d1352"}
Jul 31 23:56:50.942201 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:56:50.941738    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-716dc-predictor-58d8b8d4f4-9xkds" event={"ID":"4988fb9c-fe37-4d48-bd7b-6bea640dba66","Type":"ContainerDied","Data":"b11e28cfc26424f3c2e2ab6003c8119a3404311e8ead69e347aad47da7a4a679"}
Jul 31 23:56:50.942201 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:56:50.941768    2564 scope.go:117] "RemoveContainer" containerID="af69c85929cc5a582abe617f467b97412e3afb85afbc42c04a2bee80fd5a5b13"
Jul 31 23:56:50.942306 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:56:50.942230    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-32541-predictor-8d6ffd9d-mp9l6" podUID="097e1321-7c6d-4446-98b2-21cc0442df39" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.21:8080: connect: connection refused"
Jul 31 23:56:50.949859 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:56:50.949837    2564 scope.go:117] "RemoveContainer" containerID="560fae720b76b5226c193fcc998e6fbb463056a76c77a5171ca7cea47c9d1352"
Jul 31 23:56:50.954911 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:56:50.954890    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/4988fb9c-fe37-4d48-bd7b-6bea640dba66-kube-rbac-proxy-sar-config\") pod \"4988fb9c-fe37-4d48-bd7b-6bea640dba66\" (UID: \"4988fb9c-fe37-4d48-bd7b-6bea640dba66\") "
Jul 31 23:56:50.955005 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:56:50.954923    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4z8gp\" (UniqueName: \"kubernetes.io/projected/4988fb9c-fe37-4d48-bd7b-6bea640dba66-kube-api-access-4z8gp\") pod \"4988fb9c-fe37-4d48-bd7b-6bea640dba66\" (UID: \"4988fb9c-fe37-4d48-bd7b-6bea640dba66\") "
Jul 31 23:56:50.955005 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:56:50.954971    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/4988fb9c-fe37-4d48-bd7b-6bea640dba66-proxy-tls\") pod \"4988fb9c-fe37-4d48-bd7b-6bea640dba66\" (UID: \"4988fb9c-fe37-4d48-bd7b-6bea640dba66\") "
Jul 31 23:56:50.955285 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:56:50.955264    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4988fb9c-fe37-4d48-bd7b-6bea640dba66-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "4988fb9c-fe37-4d48-bd7b-6bea640dba66" (UID: "4988fb9c-fe37-4d48-bd7b-6bea640dba66"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 31 23:56:50.957067 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:56:50.956998    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4988fb9c-fe37-4d48-bd7b-6bea640dba66-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "4988fb9c-fe37-4d48-bd7b-6bea640dba66" (UID: "4988fb9c-fe37-4d48-bd7b-6bea640dba66"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 31 23:56:50.957116 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:56:50.957088    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4988fb9c-fe37-4d48-bd7b-6bea640dba66-kube-api-access-4z8gp" (OuterVolumeSpecName: "kube-api-access-4z8gp") pod "4988fb9c-fe37-4d48-bd7b-6bea640dba66" (UID: "4988fb9c-fe37-4d48-bd7b-6bea640dba66"). InnerVolumeSpecName "kube-api-access-4z8gp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 31 23:56:50.957220 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:56:50.957200    2564 scope.go:117] "RemoveContainer" containerID="af69c85929cc5a582abe617f467b97412e3afb85afbc42c04a2bee80fd5a5b13"
Jul 31 23:56:50.957498 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:56:50.957475    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"af69c85929cc5a582abe617f467b97412e3afb85afbc42c04a2bee80fd5a5b13\": container with ID starting with af69c85929cc5a582abe617f467b97412e3afb85afbc42c04a2bee80fd5a5b13 not found: ID does not exist" containerID="af69c85929cc5a582abe617f467b97412e3afb85afbc42c04a2bee80fd5a5b13"
Jul 31 23:56:50.957591 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:56:50.957508    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"af69c85929cc5a582abe617f467b97412e3afb85afbc42c04a2bee80fd5a5b13"} err="failed to get container status \"af69c85929cc5a582abe617f467b97412e3afb85afbc42c04a2bee80fd5a5b13\": rpc error: code = NotFound desc = could not find container \"af69c85929cc5a582abe617f467b97412e3afb85afbc42c04a2bee80fd5a5b13\": container with ID starting with af69c85929cc5a582abe617f467b97412e3afb85afbc42c04a2bee80fd5a5b13 not found: ID does not exist"
Jul 31 23:56:50.957591 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:56:50.957534    2564 scope.go:117] "RemoveContainer" containerID="560fae720b76b5226c193fcc998e6fbb463056a76c77a5171ca7cea47c9d1352"
Jul 31 23:56:50.957835 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:56:50.957817    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"560fae720b76b5226c193fcc998e6fbb463056a76c77a5171ca7cea47c9d1352\": container with ID starting with 560fae720b76b5226c193fcc998e6fbb463056a76c77a5171ca7cea47c9d1352 not found: ID does not exist" containerID="560fae720b76b5226c193fcc998e6fbb463056a76c77a5171ca7cea47c9d1352"
Jul 31 23:56:50.957882 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:56:50.957840    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"560fae720b76b5226c193fcc998e6fbb463056a76c77a5171ca7cea47c9d1352"} err="failed to get container status \"560fae720b76b5226c193fcc998e6fbb463056a76c77a5171ca7cea47c9d1352\": rpc error: code = NotFound desc = could not find container \"560fae720b76b5226c193fcc998e6fbb463056a76c77a5171ca7cea47c9d1352\": container with ID starting with 560fae720b76b5226c193fcc998e6fbb463056a76c77a5171ca7cea47c9d1352 not found: ID does not exist"
Jul 31 23:56:51.055951 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:56:51.055903    2564 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/4988fb9c-fe37-4d48-bd7b-6bea640dba66-kube-rbac-proxy-sar-config\") on node \"ip-10-0-128-75.ec2.internal\" DevicePath \"\""
Jul 31 23:56:51.055951 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:56:51.055939    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4z8gp\" (UniqueName: \"kubernetes.io/projected/4988fb9c-fe37-4d48-bd7b-6bea640dba66-kube-api-access-4z8gp\") on node \"ip-10-0-128-75.ec2.internal\" DevicePath \"\""
Jul 31 23:56:51.055951 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:56:51.055954    2564 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/4988fb9c-fe37-4d48-bd7b-6bea640dba66-proxy-tls\") on node \"ip-10-0-128-75.ec2.internal\" DevicePath \"\""
Jul 31 23:56:51.265055 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:56:51.265022    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-716dc-predictor-58d8b8d4f4-9xkds"]
Jul 31 23:56:51.268101 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:56:51.268078    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-716dc-predictor-58d8b8d4f4-9xkds"]
Jul 31 23:56:52.422730 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:56:52.422693    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4988fb9c-fe37-4d48-bd7b-6bea640dba66" path="/var/lib/kubelet/pods/4988fb9c-fe37-4d48-bd7b-6bea640dba66/volumes"
Jul 31 23:56:54.689894 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:56:54.689850    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76b49f6c9c-7c92t" podUID="f0cf1575-e02f-4cdb-9810-c55d478f4d54" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.20:8080: connect: connection refused"
Jul 31 23:56:54.690765 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:56:54.690744    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-skw2q"
Jul 31 23:56:55.695809 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:56:55.695765    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-75774fc8c9-5mn9p" podUID="76301075-df42-4218-b34e-47f97615bb7c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.17:8080: connect: connection refused"
Jul 31 23:56:55.947604 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:56:55.947515    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-32541-predictor-8d6ffd9d-mp9l6"
Jul 31 23:56:55.948087 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:56:55.948062    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-32541-predictor-8d6ffd9d-mp9l6" podUID="097e1321-7c6d-4446-98b2-21cc0442df39" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.21:8080: connect: connection refused"
Jul 31 23:57:04.690824 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:04.690791    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76b49f6c9c-7c92t"
Jul 31 23:57:05.696484 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:05.696458    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-75774fc8c9-5mn9p"
Jul 31 23:57:05.949000 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:05.948913    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-32541-predictor-8d6ffd9d-mp9l6" podUID="097e1321-7c6d-4446-98b2-21cc0442df39" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.21:8080: connect: connection refused"
Jul 31 23:57:15.948832 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:15.948784    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-32541-predictor-8d6ffd9d-mp9l6" podUID="097e1321-7c6d-4446-98b2-21cc0442df39" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.21:8080: connect: connection refused"
Jul 31 23:57:25.948856 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:25.948816    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-32541-predictor-8d6ffd9d-mp9l6" podUID="097e1321-7c6d-4446-98b2-21cc0442df39" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.21:8080: connect: connection refused"
Jul 31 23:57:35.949385 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:35.949354    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-32541-predictor-8d6ffd9d-mp9l6"
Jul 31 23:57:37.115786 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:37.115748    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-skw2q"]
Jul 31 23:57:37.116295 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:37.116045    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-skw2q" podUID="b7cd0c34-4fac-4008-b972-54541f103f61" containerName="kserve-container" containerID="cri-o://5bdc5832c0174987676e09df579f30e0998e0993a69e2ca22614595b45b6f9f1" gracePeriod=30
Jul 31 23:57:37.116295 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:37.116105    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-skw2q" podUID="b7cd0c34-4fac-4008-b972-54541f103f61" containerName="kube-rbac-proxy" containerID="cri-o://b6bcd35e048e7e63b9c8d1615ef051b2f9469db8fce6f03000ec8178bfe85196" gracePeriod=30
Jul 31 23:57:37.162577 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:37.162543    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76b49f6c9c-7c92t"]
Jul 31 23:57:37.162851 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:37.162822    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76b49f6c9c-7c92t" podUID="f0cf1575-e02f-4cdb-9810-c55d478f4d54" containerName="kserve-container" containerID="cri-o://222a010e745c69265ba28e9ebdd9ef1b24e435c87e18dee328170e385b8f7693" gracePeriod=30
Jul 31 23:57:37.162939 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:37.162851    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76b49f6c9c-7c92t" podUID="f0cf1575-e02f-4cdb-9810-c55d478f4d54" containerName="kube-rbac-proxy" containerID="cri-o://cf625482934d9ebd49d01ada684e0b26a79809c44a8ac1baac8a7294ad11bb1d" gracePeriod=30
Jul 31 23:57:37.223120 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:37.223090    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-75774fc8c9-5mn9p"]
Jul 31 23:57:37.223354 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:37.223323    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-75774fc8c9-5mn9p" podUID="76301075-df42-4218-b34e-47f97615bb7c" containerName="kserve-container" containerID="cri-o://65524873155aeedbdbb96082b086c8d14cdc4856a7eb41f99978b38b15b3f70f" gracePeriod=30
Jul 31 23:57:37.223499 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:37.223361    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-75774fc8c9-5mn9p" podUID="76301075-df42-4218-b34e-47f97615bb7c" containerName="kube-rbac-proxy" containerID="cri-o://2b17cf90c82e012b50598cf59e3e237d2b681cd7dce3a06a6a56e7b026e70754" gracePeriod=30
Jul 31 23:57:37.328926 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:37.328893    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-a3ab7-predictor-746489cd7b-qfpmr"]
Jul 31 23:57:37.329231 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:37.329220    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4988fb9c-fe37-4d48-bd7b-6bea640dba66" containerName="kserve-container"
Jul 31 23:57:37.329273 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:37.329232    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="4988fb9c-fe37-4d48-bd7b-6bea640dba66" containerName="kserve-container"
Jul 31 23:57:37.329273 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:37.329254    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4988fb9c-fe37-4d48-bd7b-6bea640dba66" containerName="kube-rbac-proxy"
Jul 31 23:57:37.329273 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:37.329260    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="4988fb9c-fe37-4d48-bd7b-6bea640dba66" containerName="kube-rbac-proxy"
Jul 31 23:57:37.329371 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:37.329351    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="4988fb9c-fe37-4d48-bd7b-6bea640dba66" containerName="kube-rbac-proxy"
Jul 31 23:57:37.329371 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:37.329362    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="4988fb9c-fe37-4d48-bd7b-6bea640dba66" containerName="kserve-container"
Jul 31 23:57:37.332405 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:37.332385    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-a3ab7-predictor-746489cd7b-qfpmr"
Jul 31 23:57:37.334902 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:37.334882    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-a3ab7-kube-rbac-proxy-sar-config\""
Jul 31 23:57:37.334902 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:37.334892    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-a3ab7-predictor-serving-cert\""
Jul 31 23:57:37.341032 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:37.341011    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-a3ab7-predictor-746489cd7b-qfpmr"]
Jul 31 23:57:37.439198 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:37.439116    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lw8xs\" (UniqueName: \"kubernetes.io/projected/ff102fa8-a979-4023-8e8f-fd67bd0e15b9-kube-api-access-lw8xs\") pod \"error-404-isvc-a3ab7-predictor-746489cd7b-qfpmr\" (UID: \"ff102fa8-a979-4023-8e8f-fd67bd0e15b9\") " pod="kserve-ci-e2e-test/error-404-isvc-a3ab7-predictor-746489cd7b-qfpmr"
Jul 31 23:57:37.439198 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:37.439168    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ff102fa8-a979-4023-8e8f-fd67bd0e15b9-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-a3ab7-predictor-746489cd7b-qfpmr\" (UID: \"ff102fa8-a979-4023-8e8f-fd67bd0e15b9\") " pod="kserve-ci-e2e-test/error-404-isvc-a3ab7-predictor-746489cd7b-qfpmr"
Jul 31 23:57:37.439372 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:37.439278    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ff102fa8-a979-4023-8e8f-fd67bd0e15b9-proxy-tls\") pod \"error-404-isvc-a3ab7-predictor-746489cd7b-qfpmr\" (UID: \"ff102fa8-a979-4023-8e8f-fd67bd0e15b9\") " pod="kserve-ci-e2e-test/error-404-isvc-a3ab7-predictor-746489cd7b-qfpmr"
Jul 31 23:57:37.539978 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:37.539939    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ff102fa8-a979-4023-8e8f-fd67bd0e15b9-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-a3ab7-predictor-746489cd7b-qfpmr\" (UID: \"ff102fa8-a979-4023-8e8f-fd67bd0e15b9\") " pod="kserve-ci-e2e-test/error-404-isvc-a3ab7-predictor-746489cd7b-qfpmr"
Jul 31 23:57:37.540170 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:37.539994    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ff102fa8-a979-4023-8e8f-fd67bd0e15b9-proxy-tls\") pod \"error-404-isvc-a3ab7-predictor-746489cd7b-qfpmr\" (UID: \"ff102fa8-a979-4023-8e8f-fd67bd0e15b9\") " pod="kserve-ci-e2e-test/error-404-isvc-a3ab7-predictor-746489cd7b-qfpmr"
Jul 31 23:57:37.540170 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:57:37.540102    2564 secret.go:189] Couldn't get secret kserve-ci-e2e-test/error-404-isvc-a3ab7-predictor-serving-cert: secret "error-404-isvc-a3ab7-predictor-serving-cert" not found
Jul 31 23:57:37.540170 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:57:37.540153    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff102fa8-a979-4023-8e8f-fd67bd0e15b9-proxy-tls podName:ff102fa8-a979-4023-8e8f-fd67bd0e15b9 nodeName:}" failed. No retries permitted until 2026-07-31 23:57:38.040137121 +0000 UTC m=+676.152264774 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/ff102fa8-a979-4023-8e8f-fd67bd0e15b9-proxy-tls") pod "error-404-isvc-a3ab7-predictor-746489cd7b-qfpmr" (UID: "ff102fa8-a979-4023-8e8f-fd67bd0e15b9") : secret "error-404-isvc-a3ab7-predictor-serving-cert" not found
Jul 31 23:57:37.540170 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:37.540166    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lw8xs\" (UniqueName: \"kubernetes.io/projected/ff102fa8-a979-4023-8e8f-fd67bd0e15b9-kube-api-access-lw8xs\") pod \"error-404-isvc-a3ab7-predictor-746489cd7b-qfpmr\" (UID: \"ff102fa8-a979-4023-8e8f-fd67bd0e15b9\") " pod="kserve-ci-e2e-test/error-404-isvc-a3ab7-predictor-746489cd7b-qfpmr"
Jul 31 23:57:37.540706 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:37.540685    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ff102fa8-a979-4023-8e8f-fd67bd0e15b9-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-a3ab7-predictor-746489cd7b-qfpmr\" (UID: \"ff102fa8-a979-4023-8e8f-fd67bd0e15b9\") " pod="kserve-ci-e2e-test/error-404-isvc-a3ab7-predictor-746489cd7b-qfpmr"
Jul 31 23:57:37.548687 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:37.548665    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lw8xs\" (UniqueName: \"kubernetes.io/projected/ff102fa8-a979-4023-8e8f-fd67bd0e15b9-kube-api-access-lw8xs\") pod \"error-404-isvc-a3ab7-predictor-746489cd7b-qfpmr\" (UID: \"ff102fa8-a979-4023-8e8f-fd67bd0e15b9\") " pod="kserve-ci-e2e-test/error-404-isvc-a3ab7-predictor-746489cd7b-qfpmr"
Jul 31 23:57:38.044148 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:38.044111    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ff102fa8-a979-4023-8e8f-fd67bd0e15b9-proxy-tls\") pod \"error-404-isvc-a3ab7-predictor-746489cd7b-qfpmr\" (UID: \"ff102fa8-a979-4023-8e8f-fd67bd0e15b9\") " pod="kserve-ci-e2e-test/error-404-isvc-a3ab7-predictor-746489cd7b-qfpmr"
Jul 31 23:57:38.046423 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:38.046406    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ff102fa8-a979-4023-8e8f-fd67bd0e15b9-proxy-tls\") pod \"error-404-isvc-a3ab7-predictor-746489cd7b-qfpmr\" (UID: \"ff102fa8-a979-4023-8e8f-fd67bd0e15b9\") " pod="kserve-ci-e2e-test/error-404-isvc-a3ab7-predictor-746489cd7b-qfpmr"
Jul 31 23:57:38.127560 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:38.127524    2564 generic.go:356] "Generic (PLEG): container finished" podID="76301075-df42-4218-b34e-47f97615bb7c" containerID="2b17cf90c82e012b50598cf59e3e237d2b681cd7dce3a06a6a56e7b026e70754" exitCode=2
Jul 31 23:57:38.128019 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:38.127600    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-75774fc8c9-5mn9p" event={"ID":"76301075-df42-4218-b34e-47f97615bb7c","Type":"ContainerDied","Data":"2b17cf90c82e012b50598cf59e3e237d2b681cd7dce3a06a6a56e7b026e70754"}
Jul 31 23:57:38.129307 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:38.129289    2564 generic.go:356] "Generic (PLEG): container finished" podID="f0cf1575-e02f-4cdb-9810-c55d478f4d54" containerID="cf625482934d9ebd49d01ada684e0b26a79809c44a8ac1baac8a7294ad11bb1d" exitCode=2
Jul 31 23:57:38.129378 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:38.129358    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76b49f6c9c-7c92t" event={"ID":"f0cf1575-e02f-4cdb-9810-c55d478f4d54","Type":"ContainerDied","Data":"cf625482934d9ebd49d01ada684e0b26a79809c44a8ac1baac8a7294ad11bb1d"}
Jul 31 23:57:38.131165 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:38.131145    2564 generic.go:356] "Generic (PLEG): container finished" podID="b7cd0c34-4fac-4008-b972-54541f103f61" containerID="b6bcd35e048e7e63b9c8d1615ef051b2f9469db8fce6f03000ec8178bfe85196" exitCode=2
Jul 31 23:57:38.131264 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:38.131183    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-skw2q" event={"ID":"b7cd0c34-4fac-4008-b972-54541f103f61","Type":"ContainerDied","Data":"b6bcd35e048e7e63b9c8d1615ef051b2f9469db8fce6f03000ec8178bfe85196"}
Jul 31 23:57:38.243279 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:38.243234    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-a3ab7-predictor-746489cd7b-qfpmr"
Jul 31 23:57:38.375593 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:38.375468    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-a3ab7-predictor-746489cd7b-qfpmr"]
Jul 31 23:57:38.378059 ip-10-0-128-75 kubenswrapper[2564]: W0731 23:57:38.378035    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff102fa8_a979_4023_8e8f_fd67bd0e15b9.slice/crio-e2be7e1be49a857fe7081ac530ac3a57a71e9f9bfdaa3cb13c5ec763805faa4d WatchSource:0}: Error finding container e2be7e1be49a857fe7081ac530ac3a57a71e9f9bfdaa3cb13c5ec763805faa4d: Status 404 returned error can't find the container with id e2be7e1be49a857fe7081ac530ac3a57a71e9f9bfdaa3cb13c5ec763805faa4d
Jul 31 23:57:39.137049 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:39.137014    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-a3ab7-predictor-746489cd7b-qfpmr" event={"ID":"ff102fa8-a979-4023-8e8f-fd67bd0e15b9","Type":"ContainerStarted","Data":"27d481128c5767d2b1d9be8e07e40f6412e6957359858b045dd49b6e4289da98"}
Jul 31 23:57:39.137438 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:39.137072    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-a3ab7-predictor-746489cd7b-qfpmr" event={"ID":"ff102fa8-a979-4023-8e8f-fd67bd0e15b9","Type":"ContainerStarted","Data":"687a791d1f61ae03fff02083fc77776ada71b511f33b8cf300293f0142ac8969"}
Jul 31 23:57:39.137438 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:39.137097    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-a3ab7-predictor-746489cd7b-qfpmr" event={"ID":"ff102fa8-a979-4023-8e8f-fd67bd0e15b9","Type":"ContainerStarted","Data":"e2be7e1be49a857fe7081ac530ac3a57a71e9f9bfdaa3cb13c5ec763805faa4d"}
Jul 31 23:57:39.137438 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:39.137147    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-a3ab7-predictor-746489cd7b-qfpmr"
Jul 31 23:57:39.154421 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:39.154371    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-a3ab7-predictor-746489cd7b-qfpmr" podStartSLOduration=2.154354283 podStartE2EDuration="2.154354283s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-31 23:57:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-31 23:57:39.153088958 +0000 UTC m=+677.265216674" watchObservedRunningTime="2026-07-31 23:57:39.154354283 +0000 UTC m=+677.266482023"
Jul 31 23:57:39.689655 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:39.685532    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-skw2q" podUID="b7cd0c34-4fac-4008-b972-54541f103f61" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.18:8643/healthz\": dial tcp 10.134.0.18:8643: connect: connection refused"
Jul 31 23:57:39.689655 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:39.685788    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76b49f6c9c-7c92t" podUID="f0cf1575-e02f-4cdb-9810-c55d478f4d54" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.20:8643/healthz\": dial tcp 10.134.0.20:8643: connect: connection refused"
Jul 31 23:57:40.140987 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:40.140954    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-a3ab7-predictor-746489cd7b-qfpmr"
Jul 31 23:57:40.142333 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:40.142303    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-a3ab7-predictor-746489cd7b-qfpmr" podUID="ff102fa8-a979-4023-8e8f-fd67bd0e15b9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.22:8080: connect: connection refused"
Jul 31 23:57:40.690302 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:40.690265    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-75774fc8c9-5mn9p" podUID="76301075-df42-4218-b34e-47f97615bb7c" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.17:8643/healthz\": dial tcp 10.134.0.17:8643: connect: connection refused"
Jul 31 23:57:41.144603 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:41.144567    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-a3ab7-predictor-746489cd7b-qfpmr" podUID="ff102fa8-a979-4023-8e8f-fd67bd0e15b9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.22:8080: connect: connection refused"
Jul 31 23:57:41.951922 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:41.951900    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-skw2q"
Jul 31 23:57:42.079818 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:42.079794    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b7cd0c34-4fac-4008-b972-54541f103f61-proxy-tls\") pod \"b7cd0c34-4fac-4008-b972-54541f103f61\" (UID: \"b7cd0c34-4fac-4008-b972-54541f103f61\") "
Jul 31 23:57:42.079936 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:42.079835    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nhlg2\" (UniqueName: \"kubernetes.io/projected/b7cd0c34-4fac-4008-b972-54541f103f61-kube-api-access-nhlg2\") pod \"b7cd0c34-4fac-4008-b972-54541f103f61\" (UID: \"b7cd0c34-4fac-4008-b972-54541f103f61\") "
Jul 31 23:57:42.079936 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:42.079859    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b7cd0c34-4fac-4008-b972-54541f103f61-kserve-provision-location\") pod \"b7cd0c34-4fac-4008-b972-54541f103f61\" (UID: \"b7cd0c34-4fac-4008-b972-54541f103f61\") "
Jul 31 23:57:42.079936 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:42.079919    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b7cd0c34-4fac-4008-b972-54541f103f61-kube-rbac-proxy-sar-config\") pod \"b7cd0c34-4fac-4008-b972-54541f103f61\" (UID: \"b7cd0c34-4fac-4008-b972-54541f103f61\") "
Jul 31 23:57:42.080098 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:42.079952    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/b7cd0c34-4fac-4008-b972-54541f103f61-cabundle-cert\") pod \"b7cd0c34-4fac-4008-b972-54541f103f61\" (UID: \"b7cd0c34-4fac-4008-b972-54541f103f61\") "
Jul 31 23:57:42.080256 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:42.080232    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7cd0c34-4fac-4008-b972-54541f103f61-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "b7cd0c34-4fac-4008-b972-54541f103f61" (UID: "b7cd0c34-4fac-4008-b972-54541f103f61"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 31 23:57:42.080497 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:42.080469    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b7cd0c34-4fac-4008-b972-54541f103f61-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "b7cd0c34-4fac-4008-b972-54541f103f61" (UID: "b7cd0c34-4fac-4008-b972-54541f103f61"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 31 23:57:42.080572 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:42.080477    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b7cd0c34-4fac-4008-b972-54541f103f61-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "b7cd0c34-4fac-4008-b972-54541f103f61" (UID: "b7cd0c34-4fac-4008-b972-54541f103f61"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 31 23:57:42.082037 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:42.082008    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b7cd0c34-4fac-4008-b972-54541f103f61-kube-api-access-nhlg2" (OuterVolumeSpecName: "kube-api-access-nhlg2") pod "b7cd0c34-4fac-4008-b972-54541f103f61" (UID: "b7cd0c34-4fac-4008-b972-54541f103f61"). InnerVolumeSpecName "kube-api-access-nhlg2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 31 23:57:42.082305 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:42.082270    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b7cd0c34-4fac-4008-b972-54541f103f61-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "b7cd0c34-4fac-4008-b972-54541f103f61" (UID: "b7cd0c34-4fac-4008-b972-54541f103f61"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 31 23:57:42.151243 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:42.151215    2564 generic.go:356] "Generic (PLEG): container finished" podID="f0cf1575-e02f-4cdb-9810-c55d478f4d54" containerID="222a010e745c69265ba28e9ebdd9ef1b24e435c87e18dee328170e385b8f7693" exitCode=0
Jul 31 23:57:42.151598 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:42.151288    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76b49f6c9c-7c92t" event={"ID":"f0cf1575-e02f-4cdb-9810-c55d478f4d54","Type":"ContainerDied","Data":"222a010e745c69265ba28e9ebdd9ef1b24e435c87e18dee328170e385b8f7693"}
Jul 31 23:57:42.153423 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:42.153398    2564 generic.go:356] "Generic (PLEG): container finished" podID="b7cd0c34-4fac-4008-b972-54541f103f61" containerID="5bdc5832c0174987676e09df579f30e0998e0993a69e2ca22614595b45b6f9f1" exitCode=0
Jul 31 23:57:42.153555 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:42.153466    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-skw2q" event={"ID":"b7cd0c34-4fac-4008-b972-54541f103f61","Type":"ContainerDied","Data":"5bdc5832c0174987676e09df579f30e0998e0993a69e2ca22614595b45b6f9f1"}
Jul 31 23:57:42.153555 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:42.153475    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-skw2q"
Jul 31 23:57:42.153555 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:42.153510    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-skw2q" event={"ID":"b7cd0c34-4fac-4008-b972-54541f103f61","Type":"ContainerDied","Data":"4a2b44ea8b00f5af7bead2572258e046387ec15ad7d85e606d0dc4358379f291"}
Jul 31 23:57:42.153555 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:42.153535    2564 scope.go:117] "RemoveContainer" containerID="b6bcd35e048e7e63b9c8d1615ef051b2f9469db8fce6f03000ec8178bfe85196"
Jul 31 23:57:42.162068 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:42.161943    2564 scope.go:117] "RemoveContainer" containerID="5bdc5832c0174987676e09df579f30e0998e0993a69e2ca22614595b45b6f9f1"
Jul 31 23:57:42.169778 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:42.169763    2564 scope.go:117] "RemoveContainer" containerID="81e619df97615bc9a54a4f4dc56400f9893da5a636c4a86cb2cd7b00ae0488d2"
Jul 31 23:57:42.177287 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:42.177268    2564 scope.go:117] "RemoveContainer" containerID="b6bcd35e048e7e63b9c8d1615ef051b2f9469db8fce6f03000ec8178bfe85196"
Jul 31 23:57:42.177469 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:42.177425    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-skw2q"]
Jul 31 23:57:42.177761 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:57:42.177735    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b6bcd35e048e7e63b9c8d1615ef051b2f9469db8fce6f03000ec8178bfe85196\": container with ID starting with b6bcd35e048e7e63b9c8d1615ef051b2f9469db8fce6f03000ec8178bfe85196 not found: ID does not exist" containerID="b6bcd35e048e7e63b9c8d1615ef051b2f9469db8fce6f03000ec8178bfe85196"
Jul 31 23:57:42.177835 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:42.177776    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b6bcd35e048e7e63b9c8d1615ef051b2f9469db8fce6f03000ec8178bfe85196"} err="failed to get container status \"b6bcd35e048e7e63b9c8d1615ef051b2f9469db8fce6f03000ec8178bfe85196\": rpc error: code = NotFound desc = could not find container \"b6bcd35e048e7e63b9c8d1615ef051b2f9469db8fce6f03000ec8178bfe85196\": container with ID starting with b6bcd35e048e7e63b9c8d1615ef051b2f9469db8fce6f03000ec8178bfe85196 not found: ID does not exist"
Jul 31 23:57:42.177835 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:42.177800    2564 scope.go:117] "RemoveContainer" containerID="5bdc5832c0174987676e09df579f30e0998e0993a69e2ca22614595b45b6f9f1"
Jul 31 23:57:42.178377 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:57:42.178354    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5bdc5832c0174987676e09df579f30e0998e0993a69e2ca22614595b45b6f9f1\": container with ID starting with 5bdc5832c0174987676e09df579f30e0998e0993a69e2ca22614595b45b6f9f1 not found: ID does not exist" containerID="5bdc5832c0174987676e09df579f30e0998e0993a69e2ca22614595b45b6f9f1"
Jul 31 23:57:42.178460 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:42.178384    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5bdc5832c0174987676e09df579f30e0998e0993a69e2ca22614595b45b6f9f1"} err="failed to get container status \"5bdc5832c0174987676e09df579f30e0998e0993a69e2ca22614595b45b6f9f1\": rpc error: code = NotFound desc = could not find container \"5bdc5832c0174987676e09df579f30e0998e0993a69e2ca22614595b45b6f9f1\": container with ID starting with 5bdc5832c0174987676e09df579f30e0998e0993a69e2ca22614595b45b6f9f1 not found: ID does not exist"
Jul 31 23:57:42.178460 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:42.178407    2564 scope.go:117] "RemoveContainer" containerID="81e619df97615bc9a54a4f4dc56400f9893da5a636c4a86cb2cd7b00ae0488d2"
Jul 31 23:57:42.178727 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:57:42.178700    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"81e619df97615bc9a54a4f4dc56400f9893da5a636c4a86cb2cd7b00ae0488d2\": container with ID starting with 81e619df97615bc9a54a4f4dc56400f9893da5a636c4a86cb2cd7b00ae0488d2 not found: ID does not exist" containerID="81e619df97615bc9a54a4f4dc56400f9893da5a636c4a86cb2cd7b00ae0488d2"
Jul 31 23:57:42.178792 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:42.178733    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"81e619df97615bc9a54a4f4dc56400f9893da5a636c4a86cb2cd7b00ae0488d2"} err="failed to get container status \"81e619df97615bc9a54a4f4dc56400f9893da5a636c4a86cb2cd7b00ae0488d2\": rpc error: code = NotFound desc = could not find container \"81e619df97615bc9a54a4f4dc56400f9893da5a636c4a86cb2cd7b00ae0488d2\": container with ID starting with 81e619df97615bc9a54a4f4dc56400f9893da5a636c4a86cb2cd7b00ae0488d2 not found: ID does not exist"
Jul 31 23:57:42.180115 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:42.180097    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-skw2q"]
Jul 31 23:57:42.181217 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:42.181201    2564 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b7cd0c34-4fac-4008-b972-54541f103f61-kube-rbac-proxy-sar-config\") on node \"ip-10-0-128-75.ec2.internal\" DevicePath \"\""
Jul 31 23:57:42.181270 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:42.181220    2564 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/b7cd0c34-4fac-4008-b972-54541f103f61-cabundle-cert\") on node \"ip-10-0-128-75.ec2.internal\" DevicePath \"\""
Jul 31 23:57:42.181270 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:42.181230    2564 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b7cd0c34-4fac-4008-b972-54541f103f61-proxy-tls\") on node \"ip-10-0-128-75.ec2.internal\" DevicePath \"\""
Jul 31 23:57:42.181270 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:42.181239    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nhlg2\" (UniqueName: \"kubernetes.io/projected/b7cd0c34-4fac-4008-b972-54541f103f61-kube-api-access-nhlg2\") on node \"ip-10-0-128-75.ec2.internal\" DevicePath \"\""
Jul 31 23:57:42.181270 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:42.181248    2564 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b7cd0c34-4fac-4008-b972-54541f103f61-kserve-provision-location\") on node \"ip-10-0-128-75.ec2.internal\" DevicePath \"\""
Jul 31 23:57:42.213307 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:42.213289    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76b49f6c9c-7c92t"
Jul 31 23:57:42.282351 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:42.282264    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/f0cf1575-e02f-4cdb-9810-c55d478f4d54-kserve-provision-location\") pod \"f0cf1575-e02f-4cdb-9810-c55d478f4d54\" (UID: \"f0cf1575-e02f-4cdb-9810-c55d478f4d54\") "
Jul 31 23:57:42.282351 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:42.282303    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/f0cf1575-e02f-4cdb-9810-c55d478f4d54-kube-rbac-proxy-sar-config\") pod \"f0cf1575-e02f-4cdb-9810-c55d478f4d54\" (UID: \"f0cf1575-e02f-4cdb-9810-c55d478f4d54\") "
Jul 31 23:57:42.282576 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:42.282367    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f0cf1575-e02f-4cdb-9810-c55d478f4d54-proxy-tls\") pod \"f0cf1575-e02f-4cdb-9810-c55d478f4d54\" (UID: \"f0cf1575-e02f-4cdb-9810-c55d478f4d54\") "
Jul 31 23:57:42.282576 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:42.282397    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/f0cf1575-e02f-4cdb-9810-c55d478f4d54-cabundle-cert\") pod \"f0cf1575-e02f-4cdb-9810-c55d478f4d54\" (UID: \"f0cf1575-e02f-4cdb-9810-c55d478f4d54\") "
Jul 31 23:57:42.282576 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:42.282414    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q66fn\" (UniqueName: \"kubernetes.io/projected/f0cf1575-e02f-4cdb-9810-c55d478f4d54-kube-api-access-q66fn\") pod \"f0cf1575-e02f-4cdb-9810-c55d478f4d54\" (UID: \"f0cf1575-e02f-4cdb-9810-c55d478f4d54\") "
Jul 31 23:57:42.282758 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:42.282730    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0cf1575-e02f-4cdb-9810-c55d478f4d54-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "f0cf1575-e02f-4cdb-9810-c55d478f4d54" (UID: "f0cf1575-e02f-4cdb-9810-c55d478f4d54"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 31 23:57:42.282807 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:42.282748    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f0cf1575-e02f-4cdb-9810-c55d478f4d54-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "f0cf1575-e02f-4cdb-9810-c55d478f4d54" (UID: "f0cf1575-e02f-4cdb-9810-c55d478f4d54"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 31 23:57:42.282807 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:42.282785    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f0cf1575-e02f-4cdb-9810-c55d478f4d54-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "f0cf1575-e02f-4cdb-9810-c55d478f4d54" (UID: "f0cf1575-e02f-4cdb-9810-c55d478f4d54"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 31 23:57:42.284503 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:42.284483    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f0cf1575-e02f-4cdb-9810-c55d478f4d54-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "f0cf1575-e02f-4cdb-9810-c55d478f4d54" (UID: "f0cf1575-e02f-4cdb-9810-c55d478f4d54"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 31 23:57:42.284580 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:42.284528    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f0cf1575-e02f-4cdb-9810-c55d478f4d54-kube-api-access-q66fn" (OuterVolumeSpecName: "kube-api-access-q66fn") pod "f0cf1575-e02f-4cdb-9810-c55d478f4d54" (UID: "f0cf1575-e02f-4cdb-9810-c55d478f4d54"). InnerVolumeSpecName "kube-api-access-q66fn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 31 23:57:42.383722 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:42.383592    2564 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/f0cf1575-e02f-4cdb-9810-c55d478f4d54-kserve-provision-location\") on node \"ip-10-0-128-75.ec2.internal\" DevicePath \"\""
Jul 31 23:57:42.383722 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:42.383652    2564 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/f0cf1575-e02f-4cdb-9810-c55d478f4d54-kube-rbac-proxy-sar-config\") on node \"ip-10-0-128-75.ec2.internal\" DevicePath \"\""
Jul 31 23:57:42.383722 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:42.383666    2564 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f0cf1575-e02f-4cdb-9810-c55d478f4d54-proxy-tls\") on node \"ip-10-0-128-75.ec2.internal\" DevicePath \"\""
Jul 31 23:57:42.383722 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:42.383682    2564 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/f0cf1575-e02f-4cdb-9810-c55d478f4d54-cabundle-cert\") on node \"ip-10-0-128-75.ec2.internal\" DevicePath \"\""
Jul 31 23:57:42.383722 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:42.383701    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q66fn\" (UniqueName: \"kubernetes.io/projected/f0cf1575-e02f-4cdb-9810-c55d478f4d54-kube-api-access-q66fn\") on node \"ip-10-0-128-75.ec2.internal\" DevicePath \"\""
Jul 31 23:57:42.423150 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:42.423106    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b7cd0c34-4fac-4008-b972-54541f103f61" path="/var/lib/kubelet/pods/b7cd0c34-4fac-4008-b972-54541f103f61/volumes"
Jul 31 23:57:43.070968 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:43.070941    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-75774fc8c9-5mn9p"
Jul 31 23:57:43.160384 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:43.160349    2564 generic.go:356] "Generic (PLEG): container finished" podID="76301075-df42-4218-b34e-47f97615bb7c" containerID="65524873155aeedbdbb96082b086c8d14cdc4856a7eb41f99978b38b15b3f70f" exitCode=0
Jul 31 23:57:43.160801 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:43.160418    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-75774fc8c9-5mn9p"
Jul 31 23:57:43.160801 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:43.160437    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-75774fc8c9-5mn9p" event={"ID":"76301075-df42-4218-b34e-47f97615bb7c","Type":"ContainerDied","Data":"65524873155aeedbdbb96082b086c8d14cdc4856a7eb41f99978b38b15b3f70f"}
Jul 31 23:57:43.160801 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:43.160520    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-75774fc8c9-5mn9p" event={"ID":"76301075-df42-4218-b34e-47f97615bb7c","Type":"ContainerDied","Data":"74437c3eb53cebf29caedaaa9ce0d8603ad59470f32de0b7b74c6a9f202e917c"}
Jul 31 23:57:43.160801 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:43.160568    2564 scope.go:117] "RemoveContainer" containerID="2b17cf90c82e012b50598cf59e3e237d2b681cd7dce3a06a6a56e7b026e70754"
Jul 31 23:57:43.162298 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:43.162279    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76b49f6c9c-7c92t" event={"ID":"f0cf1575-e02f-4cdb-9810-c55d478f4d54","Type":"ContainerDied","Data":"5c8c762286e52092bc4e99a9bf56611461f2210a11a8624ae1ca7725724960db"}
Jul 31 23:57:43.162392 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:43.162310    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76b49f6c9c-7c92t"
Jul 31 23:57:43.171095 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:43.171073    2564 scope.go:117] "RemoveContainer" containerID="65524873155aeedbdbb96082b086c8d14cdc4856a7eb41f99978b38b15b3f70f"
Jul 31 23:57:43.179811 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:43.179746    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76b49f6c9c-7c92t"]
Jul 31 23:57:43.179906 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:43.179878    2564 scope.go:117] "RemoveContainer" containerID="9f08aae96f35be8dc65d383a056987bf1f7100f6b3377b4cb690ebe5cda01866"
Jul 31 23:57:43.184055 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:43.184034    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76b49f6c9c-7c92t"]
Jul 31 23:57:43.187649 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:43.187610    2564 scope.go:117] "RemoveContainer" containerID="2b17cf90c82e012b50598cf59e3e237d2b681cd7dce3a06a6a56e7b026e70754"
Jul 31 23:57:43.187997 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:57:43.187978    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2b17cf90c82e012b50598cf59e3e237d2b681cd7dce3a06a6a56e7b026e70754\": container with ID starting with 2b17cf90c82e012b50598cf59e3e237d2b681cd7dce3a06a6a56e7b026e70754 not found: ID does not exist" containerID="2b17cf90c82e012b50598cf59e3e237d2b681cd7dce3a06a6a56e7b026e70754"
Jul 31 23:57:43.188055 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:43.188006    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2b17cf90c82e012b50598cf59e3e237d2b681cd7dce3a06a6a56e7b026e70754"} err="failed to get container status \"2b17cf90c82e012b50598cf59e3e237d2b681cd7dce3a06a6a56e7b026e70754\": rpc error: code = NotFound desc = could not find container \"2b17cf90c82e012b50598cf59e3e237d2b681cd7dce3a06a6a56e7b026e70754\": container with ID starting with 2b17cf90c82e012b50598cf59e3e237d2b681cd7dce3a06a6a56e7b026e70754 not found: ID does not exist"
Jul 31 23:57:43.188055 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:43.188026    2564 scope.go:117] "RemoveContainer" containerID="65524873155aeedbdbb96082b086c8d14cdc4856a7eb41f99978b38b15b3f70f"
Jul 31 23:57:43.188304 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:57:43.188285    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"65524873155aeedbdbb96082b086c8d14cdc4856a7eb41f99978b38b15b3f70f\": container with ID starting with 65524873155aeedbdbb96082b086c8d14cdc4856a7eb41f99978b38b15b3f70f not found: ID does not exist" containerID="65524873155aeedbdbb96082b086c8d14cdc4856a7eb41f99978b38b15b3f70f"
Jul 31 23:57:43.188351 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:43.188310    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"65524873155aeedbdbb96082b086c8d14cdc4856a7eb41f99978b38b15b3f70f"} err="failed to get container status \"65524873155aeedbdbb96082b086c8d14cdc4856a7eb41f99978b38b15b3f70f\": rpc error: code = NotFound desc = could not find container \"65524873155aeedbdbb96082b086c8d14cdc4856a7eb41f99978b38b15b3f70f\": container with ID starting with 65524873155aeedbdbb96082b086c8d14cdc4856a7eb41f99978b38b15b3f70f not found: ID does not exist"
Jul 31 23:57:43.188351 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:43.188326    2564 scope.go:117] "RemoveContainer" containerID="9f08aae96f35be8dc65d383a056987bf1f7100f6b3377b4cb690ebe5cda01866"
Jul 31 23:57:43.188574 ip-10-0-128-75 kubenswrapper[2564]: E0731 23:57:43.188550    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9f08aae96f35be8dc65d383a056987bf1f7100f6b3377b4cb690ebe5cda01866\": container with ID starting with 9f08aae96f35be8dc65d383a056987bf1f7100f6b3377b4cb690ebe5cda01866 not found: ID does not exist" containerID="9f08aae96f35be8dc65d383a056987bf1f7100f6b3377b4cb690ebe5cda01866"
Jul 31 23:57:43.188616 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:43.188579    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f08aae96f35be8dc65d383a056987bf1f7100f6b3377b4cb690ebe5cda01866"} err="failed to get container status \"9f08aae96f35be8dc65d383a056987bf1f7100f6b3377b4cb690ebe5cda01866\": rpc error: code = NotFound desc = could not find container \"9f08aae96f35be8dc65d383a056987bf1f7100f6b3377b4cb690ebe5cda01866\": container with ID starting with 9f08aae96f35be8dc65d383a056987bf1f7100f6b3377b4cb690ebe5cda01866 not found: ID does not exist"
Jul 31 23:57:43.188616 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:43.188594    2564 scope.go:117] "RemoveContainer" containerID="cf625482934d9ebd49d01ada684e0b26a79809c44a8ac1baac8a7294ad11bb1d"
Jul 31 23:57:43.189057 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:43.189039    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/76301075-df42-4218-b34e-47f97615bb7c-proxy-tls\") pod \"76301075-df42-4218-b34e-47f97615bb7c\" (UID: \"76301075-df42-4218-b34e-47f97615bb7c\") "
Jul 31 23:57:43.189112 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:43.189074    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/76301075-df42-4218-b34e-47f97615bb7c-cabundle-cert\") pod \"76301075-df42-4218-b34e-47f97615bb7c\" (UID: \"76301075-df42-4218-b34e-47f97615bb7c\") "
Jul 31 23:57:43.189154 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:43.189133    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/76301075-df42-4218-b34e-47f97615bb7c-kserve-provision-location\") pod \"76301075-df42-4218-b34e-47f97615bb7c\" (UID: \"76301075-df42-4218-b34e-47f97615bb7c\") "
Jul 31 23:57:43.189187 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:43.189175    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bz2gd\" (UniqueName: \"kubernetes.io/projected/76301075-df42-4218-b34e-47f97615bb7c-kube-api-access-bz2gd\") pod \"76301075-df42-4218-b34e-47f97615bb7c\" (UID: \"76301075-df42-4218-b34e-47f97615bb7c\") "
Jul 31 23:57:43.189235 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:43.189213    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/76301075-df42-4218-b34e-47f97615bb7c-kube-rbac-proxy-sar-config\") pod \"76301075-df42-4218-b34e-47f97615bb7c\" (UID: \"76301075-df42-4218-b34e-47f97615bb7c\") "
Jul 31 23:57:43.189679 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:43.189551    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/76301075-df42-4218-b34e-47f97615bb7c-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "76301075-df42-4218-b34e-47f97615bb7c" (UID: "76301075-df42-4218-b34e-47f97615bb7c"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 31 23:57:43.189679 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:43.189584    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/76301075-df42-4218-b34e-47f97615bb7c-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "76301075-df42-4218-b34e-47f97615bb7c" (UID: "76301075-df42-4218-b34e-47f97615bb7c"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 31 23:57:43.189933 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:43.189907    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76301075-df42-4218-b34e-47f97615bb7c-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "76301075-df42-4218-b34e-47f97615bb7c" (UID: "76301075-df42-4218-b34e-47f97615bb7c"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 31 23:57:43.191113 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:43.191090    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/76301075-df42-4218-b34e-47f97615bb7c-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "76301075-df42-4218-b34e-47f97615bb7c" (UID: "76301075-df42-4218-b34e-47f97615bb7c"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 31 23:57:43.191232 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:43.191214    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/76301075-df42-4218-b34e-47f97615bb7c-kube-api-access-bz2gd" (OuterVolumeSpecName: "kube-api-access-bz2gd") pod "76301075-df42-4218-b34e-47f97615bb7c" (UID: "76301075-df42-4218-b34e-47f97615bb7c"). InnerVolumeSpecName "kube-api-access-bz2gd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 31 23:57:43.200262 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:43.200240    2564 scope.go:117] "RemoveContainer" containerID="222a010e745c69265ba28e9ebdd9ef1b24e435c87e18dee328170e385b8f7693"
Jul 31 23:57:43.207720 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:43.207700    2564 scope.go:117] "RemoveContainer" containerID="09569673e23d868b81f48980895bedcea7ae58d6926fbe473466330e8f3e0945"
Jul 31 23:57:43.290479 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:43.290392    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bz2gd\" (UniqueName: \"kubernetes.io/projected/76301075-df42-4218-b34e-47f97615bb7c-kube-api-access-bz2gd\") on node \"ip-10-0-128-75.ec2.internal\" DevicePath \"\""
Jul 31 23:57:43.290479 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:43.290420    2564 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/76301075-df42-4218-b34e-47f97615bb7c-kube-rbac-proxy-sar-config\") on node \"ip-10-0-128-75.ec2.internal\" DevicePath \"\""
Jul 31 23:57:43.290479 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:43.290429    2564 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/76301075-df42-4218-b34e-47f97615bb7c-proxy-tls\") on node \"ip-10-0-128-75.ec2.internal\" DevicePath \"\""
Jul 31 23:57:43.290479 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:43.290441    2564 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/76301075-df42-4218-b34e-47f97615bb7c-cabundle-cert\") on node \"ip-10-0-128-75.ec2.internal\" DevicePath \"\""
Jul 31 23:57:43.290479 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:43.290449    2564 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/76301075-df42-4218-b34e-47f97615bb7c-kserve-provision-location\") on node \"ip-10-0-128-75.ec2.internal\" DevicePath \"\""
Jul 31 23:57:43.482237 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:43.482196    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-75774fc8c9-5mn9p"]
Jul 31 23:57:43.487973 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:43.487946    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-75774fc8c9-5mn9p"]
Jul 31 23:57:44.422590 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:44.422551    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="76301075-df42-4218-b34e-47f97615bb7c" path="/var/lib/kubelet/pods/76301075-df42-4218-b34e-47f97615bb7c/volumes"
Jul 31 23:57:44.423104 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:44.423089    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f0cf1575-e02f-4cdb-9810-c55d478f4d54" path="/var/lib/kubelet/pods/f0cf1575-e02f-4cdb-9810-c55d478f4d54/volumes"
Jul 31 23:57:46.149819 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:46.149715    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-a3ab7-predictor-746489cd7b-qfpmr"
Jul 31 23:57:46.150376 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:46.150347    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-a3ab7-predictor-746489cd7b-qfpmr" podUID="ff102fa8-a979-4023-8e8f-fd67bd0e15b9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.22:8080: connect: connection refused"
Jul 31 23:57:56.151148 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:57:56.151108    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-a3ab7-predictor-746489cd7b-qfpmr" podUID="ff102fa8-a979-4023-8e8f-fd67bd0e15b9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.22:8080: connect: connection refused"
Jul 31 23:58:06.151199 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:58:06.151156    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-a3ab7-predictor-746489cd7b-qfpmr" podUID="ff102fa8-a979-4023-8e8f-fd67bd0e15b9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.22:8080: connect: connection refused"
Jul 31 23:58:16.150749 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:58:16.150664    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-a3ab7-predictor-746489cd7b-qfpmr" podUID="ff102fa8-a979-4023-8e8f-fd67bd0e15b9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.22:8080: connect: connection refused"
Jul 31 23:58:26.151127 ip-10-0-128-75 kubenswrapper[2564]: I0731 23:58:26.151093    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-a3ab7-predictor-746489cd7b-qfpmr"
Aug 01 00:00:00.145475 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:00:00.145395    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-pruner-29759040-k5csx"]
Aug 01 00:00:00.145934 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:00:00.145694    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="76301075-df42-4218-b34e-47f97615bb7c" containerName="kube-rbac-proxy"
Aug 01 00:00:00.145934 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:00:00.145706    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="76301075-df42-4218-b34e-47f97615bb7c" containerName="kube-rbac-proxy"
Aug 01 00:00:00.145934 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:00:00.145718    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f0cf1575-e02f-4cdb-9810-c55d478f4d54" containerName="kube-rbac-proxy"
Aug 01 00:00:00.145934 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:00:00.145723    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="f0cf1575-e02f-4cdb-9810-c55d478f4d54" containerName="kube-rbac-proxy"
Aug 01 00:00:00.145934 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:00:00.145731    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f0cf1575-e02f-4cdb-9810-c55d478f4d54" containerName="kserve-container"
Aug 01 00:00:00.145934 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:00:00.145738    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="f0cf1575-e02f-4cdb-9810-c55d478f4d54" containerName="kserve-container"
Aug 01 00:00:00.145934 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:00:00.145750    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="76301075-df42-4218-b34e-47f97615bb7c" containerName="storage-initializer"
Aug 01 00:00:00.145934 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:00:00.145757    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="76301075-df42-4218-b34e-47f97615bb7c" containerName="storage-initializer"
Aug 01 00:00:00.145934 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:00:00.145770    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="76301075-df42-4218-b34e-47f97615bb7c" containerName="kserve-container"
Aug 01 00:00:00.145934 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:00:00.145778    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="76301075-df42-4218-b34e-47f97615bb7c" containerName="kserve-container"
Aug 01 00:00:00.145934 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:00:00.145790    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b7cd0c34-4fac-4008-b972-54541f103f61" containerName="storage-initializer"
Aug 01 00:00:00.145934 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:00:00.145798    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="b7cd0c34-4fac-4008-b972-54541f103f61" containerName="storage-initializer"
Aug 01 00:00:00.145934 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:00:00.145810    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b7cd0c34-4fac-4008-b972-54541f103f61" containerName="kube-rbac-proxy"
Aug 01 00:00:00.145934 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:00:00.145817    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="b7cd0c34-4fac-4008-b972-54541f103f61" containerName="kube-rbac-proxy"
Aug 01 00:00:00.145934 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:00:00.145828    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b7cd0c34-4fac-4008-b972-54541f103f61" containerName="kserve-container"
Aug 01 00:00:00.145934 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:00:00.145835    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="b7cd0c34-4fac-4008-b972-54541f103f61" containerName="kserve-container"
Aug 01 00:00:00.145934 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:00:00.145848    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f0cf1575-e02f-4cdb-9810-c55d478f4d54" containerName="storage-initializer"
Aug 01 00:00:00.145934 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:00:00.145857    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="f0cf1575-e02f-4cdb-9810-c55d478f4d54" containerName="storage-initializer"
Aug 01 00:00:00.145934 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:00:00.145927    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="f0cf1575-e02f-4cdb-9810-c55d478f4d54" containerName="kube-rbac-proxy"
Aug 01 00:00:00.145934 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:00:00.145941    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="f0cf1575-e02f-4cdb-9810-c55d478f4d54" containerName="kserve-container"
Aug 01 00:00:00.146826 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:00:00.145949    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="76301075-df42-4218-b34e-47f97615bb7c" containerName="kube-rbac-proxy"
Aug 01 00:00:00.146826 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:00:00.145957    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="b7cd0c34-4fac-4008-b972-54541f103f61" containerName="kube-rbac-proxy"
Aug 01 00:00:00.146826 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:00:00.145965    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="b7cd0c34-4fac-4008-b972-54541f103f61" containerName="kserve-container"
Aug 01 00:00:00.146826 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:00:00.145972    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="76301075-df42-4218-b34e-47f97615bb7c" containerName="kserve-container"
Aug 01 00:00:00.148978 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:00:00.148959    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-pruner-29759040-k5csx"
Aug 01 00:00:00.152951 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:00:00.152920    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"serviceca\""
Aug 01 00:00:00.153038 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:00:00.152922    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"pruner-dockercfg-8r844\""
Aug 01 00:00:00.158300 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:00:00.158278    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-pruner-29759040-k5csx"]
Aug 01 00:00:00.238440 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:00:00.238410    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h247x\" (UniqueName: \"kubernetes.io/projected/3cabb612-38bd-4e6e-b904-284efbe28a18-kube-api-access-h247x\") pod \"image-pruner-29759040-k5csx\" (UID: \"3cabb612-38bd-4e6e-b904-284efbe28a18\") " pod="openshift-image-registry/image-pruner-29759040-k5csx"
Aug 01 00:00:00.238440 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:00:00.238445    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/3cabb612-38bd-4e6e-b904-284efbe28a18-serviceca\") pod \"image-pruner-29759040-k5csx\" (UID: \"3cabb612-38bd-4e6e-b904-284efbe28a18\") " pod="openshift-image-registry/image-pruner-29759040-k5csx"
Aug 01 00:00:00.339269 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:00:00.339238    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h247x\" (UniqueName: \"kubernetes.io/projected/3cabb612-38bd-4e6e-b904-284efbe28a18-kube-api-access-h247x\") pod \"image-pruner-29759040-k5csx\" (UID: \"3cabb612-38bd-4e6e-b904-284efbe28a18\") " pod="openshift-image-registry/image-pruner-29759040-k5csx"
Aug 01 00:00:00.339452 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:00:00.339279    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/3cabb612-38bd-4e6e-b904-284efbe28a18-serviceca\") pod \"image-pruner-29759040-k5csx\" (UID: \"3cabb612-38bd-4e6e-b904-284efbe28a18\") " pod="openshift-image-registry/image-pruner-29759040-k5csx"
Aug 01 00:00:00.339974 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:00:00.339951    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/3cabb612-38bd-4e6e-b904-284efbe28a18-serviceca\") pod \"image-pruner-29759040-k5csx\" (UID: \"3cabb612-38bd-4e6e-b904-284efbe28a18\") " pod="openshift-image-registry/image-pruner-29759040-k5csx"
Aug 01 00:00:00.348195 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:00:00.348174    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h247x\" (UniqueName: \"kubernetes.io/projected/3cabb612-38bd-4e6e-b904-284efbe28a18-kube-api-access-h247x\") pod \"image-pruner-29759040-k5csx\" (UID: \"3cabb612-38bd-4e6e-b904-284efbe28a18\") " pod="openshift-image-registry/image-pruner-29759040-k5csx"
Aug 01 00:00:00.483324 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:00:00.483279    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-pruner-29759040-k5csx"
Aug 01 00:00:00.612733 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:00:00.612668    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-pruner-29759040-k5csx"]
Aug 01 00:00:00.615395 ip-10-0-128-75 kubenswrapper[2564]: W0801 00:00:00.615365    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3cabb612_38bd_4e6e_b904_284efbe28a18.slice/crio-702673248ff1ad6492ceec50067fe6345b9ba4178c04a7ed53e62c37bc731fb5 WatchSource:0}: Error finding container 702673248ff1ad6492ceec50067fe6345b9ba4178c04a7ed53e62c37bc731fb5: Status 404 returned error can't find the container with id 702673248ff1ad6492ceec50067fe6345b9ba4178c04a7ed53e62c37bc731fb5
Aug 01 00:00:00.673513 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:00:00.673488    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-pruner-29759040-k5csx" event={"ID":"3cabb612-38bd-4e6e-b904-284efbe28a18","Type":"ContainerStarted","Data":"702673248ff1ad6492ceec50067fe6345b9ba4178c04a7ed53e62c37bc731fb5"}
Aug 01 00:00:01.678775 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:00:01.678693    2564 generic.go:356] "Generic (PLEG): container finished" podID="3cabb612-38bd-4e6e-b904-284efbe28a18" containerID="2a7f3808b846853dd4a1fdb0e50e7351a3b96ecdf9346ff2f79c2cf34160cdf9" exitCode=0
Aug 01 00:00:01.679122 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:00:01.678776    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-pruner-29759040-k5csx" event={"ID":"3cabb612-38bd-4e6e-b904-284efbe28a18","Type":"ContainerDied","Data":"2a7f3808b846853dd4a1fdb0e50e7351a3b96ecdf9346ff2f79c2cf34160cdf9"}
Aug 01 00:00:02.821585 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:00:02.821558    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-pruner-29759040-k5csx"
Aug 01 00:00:02.963213 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:00:02.963123    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h247x\" (UniqueName: \"kubernetes.io/projected/3cabb612-38bd-4e6e-b904-284efbe28a18-kube-api-access-h247x\") pod \"3cabb612-38bd-4e6e-b904-284efbe28a18\" (UID: \"3cabb612-38bd-4e6e-b904-284efbe28a18\") "
Aug 01 00:00:02.963213 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:00:02.963205    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/3cabb612-38bd-4e6e-b904-284efbe28a18-serviceca\") pod \"3cabb612-38bd-4e6e-b904-284efbe28a18\" (UID: \"3cabb612-38bd-4e6e-b904-284efbe28a18\") "
Aug 01 00:00:02.963598 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:00:02.963569    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3cabb612-38bd-4e6e-b904-284efbe28a18-serviceca" (OuterVolumeSpecName: "serviceca") pod "3cabb612-38bd-4e6e-b904-284efbe28a18" (UID: "3cabb612-38bd-4e6e-b904-284efbe28a18"). InnerVolumeSpecName "serviceca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 01 00:00:02.965308 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:00:02.965284    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3cabb612-38bd-4e6e-b904-284efbe28a18-kube-api-access-h247x" (OuterVolumeSpecName: "kube-api-access-h247x") pod "3cabb612-38bd-4e6e-b904-284efbe28a18" (UID: "3cabb612-38bd-4e6e-b904-284efbe28a18"). InnerVolumeSpecName "kube-api-access-h247x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 01 00:00:03.064504 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:00:03.064469    2564 reconciler_common.go:299] "Volume detached for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/3cabb612-38bd-4e6e-b904-284efbe28a18-serviceca\") on node \"ip-10-0-128-75.ec2.internal\" DevicePath \"\""
Aug 01 00:00:03.064504 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:00:03.064497    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h247x\" (UniqueName: \"kubernetes.io/projected/3cabb612-38bd-4e6e-b904-284efbe28a18-kube-api-access-h247x\") on node \"ip-10-0-128-75.ec2.internal\" DevicePath \"\""
Aug 01 00:00:03.686717 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:00:03.686680    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-pruner-29759040-k5csx" event={"ID":"3cabb612-38bd-4e6e-b904-284efbe28a18","Type":"ContainerDied","Data":"702673248ff1ad6492ceec50067fe6345b9ba4178c04a7ed53e62c37bc731fb5"}
Aug 01 00:00:03.686895 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:00:03.686732    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="702673248ff1ad6492ceec50067fe6345b9ba4178c04a7ed53e62c37bc731fb5"
Aug 01 00:00:03.686895 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:00:03.686751    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-pruner-29759040-k5csx"
Aug 01 00:06:02.365328 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:06:02.365240    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-32541-predictor-8d6ffd9d-mp9l6"]
Aug 01 00:06:02.367981 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:06:02.365531    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-32541-predictor-8d6ffd9d-mp9l6" podUID="097e1321-7c6d-4446-98b2-21cc0442df39" containerName="kserve-container" containerID="cri-o://95fcfc8694ee8f077fd61dfa912605bc5b7989824fdbc0a2aa83d130798737f0" gracePeriod=30
Aug 01 00:06:02.367981 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:06:02.365583    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-32541-predictor-8d6ffd9d-mp9l6" podUID="097e1321-7c6d-4446-98b2-21cc0442df39" containerName="kube-rbac-proxy" containerID="cri-o://6ecacd97b1cbb6a66a6511a926831aec83e57753ca610375005d233180559cf4" gracePeriod=30
Aug 01 00:06:02.463496 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:06:02.463458    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-fe2ca-predictor-bb856dc98-9pjb9"]
Aug 01 00:06:02.463843 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:06:02.463830    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3cabb612-38bd-4e6e-b904-284efbe28a18" containerName="image-pruner"
Aug 01 00:06:02.463898 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:06:02.463844    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="3cabb612-38bd-4e6e-b904-284efbe28a18" containerName="image-pruner"
Aug 01 00:06:02.463898 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:06:02.463896    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="3cabb612-38bd-4e6e-b904-284efbe28a18" containerName="image-pruner"
Aug 01 00:06:02.466829 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:06:02.466809    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-fe2ca-predictor-bb856dc98-9pjb9"
Aug 01 00:06:02.469523 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:06:02.469492    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-fe2ca-kube-rbac-proxy-sar-config\""
Aug 01 00:06:02.469523 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:06:02.469508    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-fe2ca-predictor-serving-cert\""
Aug 01 00:06:02.476764 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:06:02.476735    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-fe2ca-predictor-bb856dc98-9pjb9"]
Aug 01 00:06:02.524540 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:06:02.524497    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/576ba641-ea65-4b9e-8b4a-a08086f095cb-proxy-tls\") pod \"error-404-isvc-fe2ca-predictor-bb856dc98-9pjb9\" (UID: \"576ba641-ea65-4b9e-8b4a-a08086f095cb\") " pod="kserve-ci-e2e-test/error-404-isvc-fe2ca-predictor-bb856dc98-9pjb9"
Aug 01 00:06:02.524783 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:06:02.524553    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/576ba641-ea65-4b9e-8b4a-a08086f095cb-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-fe2ca-predictor-bb856dc98-9pjb9\" (UID: \"576ba641-ea65-4b9e-8b4a-a08086f095cb\") " pod="kserve-ci-e2e-test/error-404-isvc-fe2ca-predictor-bb856dc98-9pjb9"
Aug 01 00:06:02.524783 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:06:02.524696    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nrcgw\" (UniqueName: \"kubernetes.io/projected/576ba641-ea65-4b9e-8b4a-a08086f095cb-kube-api-access-nrcgw\") pod \"error-404-isvc-fe2ca-predictor-bb856dc98-9pjb9\" (UID: \"576ba641-ea65-4b9e-8b4a-a08086f095cb\") " pod="kserve-ci-e2e-test/error-404-isvc-fe2ca-predictor-bb856dc98-9pjb9"
Aug 01 00:06:02.625758 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:06:02.625622    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/576ba641-ea65-4b9e-8b4a-a08086f095cb-proxy-tls\") pod \"error-404-isvc-fe2ca-predictor-bb856dc98-9pjb9\" (UID: \"576ba641-ea65-4b9e-8b4a-a08086f095cb\") " pod="kserve-ci-e2e-test/error-404-isvc-fe2ca-predictor-bb856dc98-9pjb9"
Aug 01 00:06:02.625758 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:06:02.625716    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/576ba641-ea65-4b9e-8b4a-a08086f095cb-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-fe2ca-predictor-bb856dc98-9pjb9\" (UID: \"576ba641-ea65-4b9e-8b4a-a08086f095cb\") " pod="kserve-ci-e2e-test/error-404-isvc-fe2ca-predictor-bb856dc98-9pjb9"
Aug 01 00:06:02.626000 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:06:02.625763    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nrcgw\" (UniqueName: \"kubernetes.io/projected/576ba641-ea65-4b9e-8b4a-a08086f095cb-kube-api-access-nrcgw\") pod \"error-404-isvc-fe2ca-predictor-bb856dc98-9pjb9\" (UID: \"576ba641-ea65-4b9e-8b4a-a08086f095cb\") " pod="kserve-ci-e2e-test/error-404-isvc-fe2ca-predictor-bb856dc98-9pjb9"
Aug 01 00:06:02.626452 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:06:02.626426    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/576ba641-ea65-4b9e-8b4a-a08086f095cb-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-fe2ca-predictor-bb856dc98-9pjb9\" (UID: \"576ba641-ea65-4b9e-8b4a-a08086f095cb\") " pod="kserve-ci-e2e-test/error-404-isvc-fe2ca-predictor-bb856dc98-9pjb9"
Aug 01 00:06:02.628173 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:06:02.628149    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/576ba641-ea65-4b9e-8b4a-a08086f095cb-proxy-tls\") pod \"error-404-isvc-fe2ca-predictor-bb856dc98-9pjb9\" (UID: \"576ba641-ea65-4b9e-8b4a-a08086f095cb\") " pod="kserve-ci-e2e-test/error-404-isvc-fe2ca-predictor-bb856dc98-9pjb9"
Aug 01 00:06:02.634926 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:06:02.634895    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nrcgw\" (UniqueName: \"kubernetes.io/projected/576ba641-ea65-4b9e-8b4a-a08086f095cb-kube-api-access-nrcgw\") pod \"error-404-isvc-fe2ca-predictor-bb856dc98-9pjb9\" (UID: \"576ba641-ea65-4b9e-8b4a-a08086f095cb\") " pod="kserve-ci-e2e-test/error-404-isvc-fe2ca-predictor-bb856dc98-9pjb9"
Aug 01 00:06:02.778879 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:06:02.778837    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-fe2ca-predictor-bb856dc98-9pjb9"
Aug 01 00:06:02.921186 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:06:02.921098    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-fe2ca-predictor-bb856dc98-9pjb9"]
Aug 01 00:06:02.924912 ip-10-0-128-75 kubenswrapper[2564]: W0801 00:06:02.924884    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod576ba641_ea65_4b9e_8b4a_a08086f095cb.slice/crio-698296c47c39cfe63efb7d21e28efa6985a2a134834e44a16dfe610ad345b0a5 WatchSource:0}: Error finding container 698296c47c39cfe63efb7d21e28efa6985a2a134834e44a16dfe610ad345b0a5: Status 404 returned error can't find the container with id 698296c47c39cfe63efb7d21e28efa6985a2a134834e44a16dfe610ad345b0a5
Aug 01 00:06:03.024540 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:06:03.024503    2564 generic.go:356] "Generic (PLEG): container finished" podID="097e1321-7c6d-4446-98b2-21cc0442df39" containerID="6ecacd97b1cbb6a66a6511a926831aec83e57753ca610375005d233180559cf4" exitCode=2
Aug 01 00:06:03.024730 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:06:03.024581    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-32541-predictor-8d6ffd9d-mp9l6" event={"ID":"097e1321-7c6d-4446-98b2-21cc0442df39","Type":"ContainerDied","Data":"6ecacd97b1cbb6a66a6511a926831aec83e57753ca610375005d233180559cf4"}
Aug 01 00:06:03.026109 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:06:03.026082    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-fe2ca-predictor-bb856dc98-9pjb9" event={"ID":"576ba641-ea65-4b9e-8b4a-a08086f095cb","Type":"ContainerStarted","Data":"5af8e569dc0ce37c8c9677d31c1ddf67123c02965804b0b0eb21dc7c6aa23447"}
Aug 01 00:06:03.026234 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:06:03.026135    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-fe2ca-predictor-bb856dc98-9pjb9" event={"ID":"576ba641-ea65-4b9e-8b4a-a08086f095cb","Type":"ContainerStarted","Data":"698296c47c39cfe63efb7d21e28efa6985a2a134834e44a16dfe610ad345b0a5"}
Aug 01 00:06:04.031444 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:06:04.031410    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-fe2ca-predictor-bb856dc98-9pjb9" event={"ID":"576ba641-ea65-4b9e-8b4a-a08086f095cb","Type":"ContainerStarted","Data":"0da0554fc06c826a94fba1b5be559b4aeee7fcee2f19b452738e264ddd120a5d"}
Aug 01 00:06:04.031941 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:06:04.031471    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-fe2ca-predictor-bb856dc98-9pjb9"
Aug 01 00:06:04.031941 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:06:04.031517    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-fe2ca-predictor-bb856dc98-9pjb9"
Aug 01 00:06:04.033113 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:06:04.033085    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-fe2ca-predictor-bb856dc98-9pjb9" podUID="576ba641-ea65-4b9e-8b4a-a08086f095cb" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.24:8080: connect: connection refused"
Aug 01 00:06:04.049334 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:06:04.049280    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-fe2ca-predictor-bb856dc98-9pjb9" podStartSLOduration=2.049265165 podStartE2EDuration="2.049265165s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-01 00:06:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-01 00:06:04.048094462 +0000 UTC m=+1182.160222181" watchObservedRunningTime="2026-08-01 00:06:04.049265165 +0000 UTC m=+1182.161392838"
Aug 01 00:06:05.035599 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:06:05.035552    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-fe2ca-predictor-bb856dc98-9pjb9" podUID="576ba641-ea65-4b9e-8b4a-a08086f095cb" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.24:8080: connect: connection refused"
Aug 01 00:06:06.020568 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:06:06.020540    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-32541-predictor-8d6ffd9d-mp9l6"
Aug 01 00:06:06.041637 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:06:06.041587    2564 generic.go:356] "Generic (PLEG): container finished" podID="097e1321-7c6d-4446-98b2-21cc0442df39" containerID="95fcfc8694ee8f077fd61dfa912605bc5b7989824fdbc0a2aa83d130798737f0" exitCode=0
Aug 01 00:06:06.042093 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:06:06.041680    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-32541-predictor-8d6ffd9d-mp9l6" event={"ID":"097e1321-7c6d-4446-98b2-21cc0442df39","Type":"ContainerDied","Data":"95fcfc8694ee8f077fd61dfa912605bc5b7989824fdbc0a2aa83d130798737f0"}
Aug 01 00:06:06.042093 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:06:06.041695    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-32541-predictor-8d6ffd9d-mp9l6"
Aug 01 00:06:06.042093 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:06:06.041747    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-32541-predictor-8d6ffd9d-mp9l6" event={"ID":"097e1321-7c6d-4446-98b2-21cc0442df39","Type":"ContainerDied","Data":"35aa17629a1b49515cc93e1f9e1bdbc463cbc3de69c3ffd406cbc9e1b17cc93e"}
Aug 01 00:06:06.042093 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:06:06.041785    2564 scope.go:117] "RemoveContainer" containerID="6ecacd97b1cbb6a66a6511a926831aec83e57753ca610375005d233180559cf4"
Aug 01 00:06:06.050977 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:06:06.050955    2564 scope.go:117] "RemoveContainer" containerID="95fcfc8694ee8f077fd61dfa912605bc5b7989824fdbc0a2aa83d130798737f0"
Aug 01 00:06:06.058873 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:06:06.058849    2564 scope.go:117] "RemoveContainer" containerID="6ecacd97b1cbb6a66a6511a926831aec83e57753ca610375005d233180559cf4"
Aug 01 00:06:06.059269 ip-10-0-128-75 kubenswrapper[2564]: E0801 00:06:06.059245    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6ecacd97b1cbb6a66a6511a926831aec83e57753ca610375005d233180559cf4\": container with ID starting with 6ecacd97b1cbb6a66a6511a926831aec83e57753ca610375005d233180559cf4 not found: ID does not exist" containerID="6ecacd97b1cbb6a66a6511a926831aec83e57753ca610375005d233180559cf4"
Aug 01 00:06:06.059371 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:06:06.059276    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ecacd97b1cbb6a66a6511a926831aec83e57753ca610375005d233180559cf4"} err="failed to get container status \"6ecacd97b1cbb6a66a6511a926831aec83e57753ca610375005d233180559cf4\": rpc error: code = NotFound desc = could not find container \"6ecacd97b1cbb6a66a6511a926831aec83e57753ca610375005d233180559cf4\": container with ID starting with 6ecacd97b1cbb6a66a6511a926831aec83e57753ca610375005d233180559cf4 not found: ID does not exist"
Aug 01 00:06:06.059371 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:06:06.059296    2564 scope.go:117] "RemoveContainer" containerID="95fcfc8694ee8f077fd61dfa912605bc5b7989824fdbc0a2aa83d130798737f0"
Aug 01 00:06:06.059642 ip-10-0-128-75 kubenswrapper[2564]: E0801 00:06:06.059606    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"95fcfc8694ee8f077fd61dfa912605bc5b7989824fdbc0a2aa83d130798737f0\": container with ID starting with 95fcfc8694ee8f077fd61dfa912605bc5b7989824fdbc0a2aa83d130798737f0 not found: ID does not exist" containerID="95fcfc8694ee8f077fd61dfa912605bc5b7989824fdbc0a2aa83d130798737f0"
Aug 01 00:06:06.059720 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:06:06.059649    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"95fcfc8694ee8f077fd61dfa912605bc5b7989824fdbc0a2aa83d130798737f0"} err="failed to get container status \"95fcfc8694ee8f077fd61dfa912605bc5b7989824fdbc0a2aa83d130798737f0\": rpc error: code = NotFound desc = could not find container \"95fcfc8694ee8f077fd61dfa912605bc5b7989824fdbc0a2aa83d130798737f0\": container with ID starting with 95fcfc8694ee8f077fd61dfa912605bc5b7989824fdbc0a2aa83d130798737f0 not found: ID does not exist"
Aug 01 00:06:06.157868 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:06:06.157833    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/097e1321-7c6d-4446-98b2-21cc0442df39-kube-rbac-proxy-sar-config\") pod \"097e1321-7c6d-4446-98b2-21cc0442df39\" (UID: \"097e1321-7c6d-4446-98b2-21cc0442df39\") "
Aug 01 00:06:06.158047 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:06:06.157887    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4wgvd\" (UniqueName: \"kubernetes.io/projected/097e1321-7c6d-4446-98b2-21cc0442df39-kube-api-access-4wgvd\") pod \"097e1321-7c6d-4446-98b2-21cc0442df39\" (UID: \"097e1321-7c6d-4446-98b2-21cc0442df39\") "
Aug 01 00:06:06.158047 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:06:06.157946    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/097e1321-7c6d-4446-98b2-21cc0442df39-proxy-tls\") pod \"097e1321-7c6d-4446-98b2-21cc0442df39\" (UID: \"097e1321-7c6d-4446-98b2-21cc0442df39\") "
Aug 01 00:06:06.158260 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:06:06.158228    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/097e1321-7c6d-4446-98b2-21cc0442df39-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "097e1321-7c6d-4446-98b2-21cc0442df39" (UID: "097e1321-7c6d-4446-98b2-21cc0442df39"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 01 00:06:06.160076 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:06:06.160052    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/097e1321-7c6d-4446-98b2-21cc0442df39-kube-api-access-4wgvd" (OuterVolumeSpecName: "kube-api-access-4wgvd") pod "097e1321-7c6d-4446-98b2-21cc0442df39" (UID: "097e1321-7c6d-4446-98b2-21cc0442df39"). InnerVolumeSpecName "kube-api-access-4wgvd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 01 00:06:06.160186 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:06:06.160084    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/097e1321-7c6d-4446-98b2-21cc0442df39-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "097e1321-7c6d-4446-98b2-21cc0442df39" (UID: "097e1321-7c6d-4446-98b2-21cc0442df39"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 01 00:06:06.259039 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:06:06.258986    2564 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/097e1321-7c6d-4446-98b2-21cc0442df39-kube-rbac-proxy-sar-config\") on node \"ip-10-0-128-75.ec2.internal\" DevicePath \"\""
Aug 01 00:06:06.259039 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:06:06.259029    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4wgvd\" (UniqueName: \"kubernetes.io/projected/097e1321-7c6d-4446-98b2-21cc0442df39-kube-api-access-4wgvd\") on node \"ip-10-0-128-75.ec2.internal\" DevicePath \"\""
Aug 01 00:06:06.259039 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:06:06.259040    2564 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/097e1321-7c6d-4446-98b2-21cc0442df39-proxy-tls\") on node \"ip-10-0-128-75.ec2.internal\" DevicePath \"\""
Aug 01 00:06:06.362781 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:06:06.362746    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-32541-predictor-8d6ffd9d-mp9l6"]
Aug 01 00:06:06.366996 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:06:06.366966    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-32541-predictor-8d6ffd9d-mp9l6"]
Aug 01 00:06:06.422164 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:06:06.422075    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="097e1321-7c6d-4446-98b2-21cc0442df39" path="/var/lib/kubelet/pods/097e1321-7c6d-4446-98b2-21cc0442df39/volumes"
Aug 01 00:06:06.943442 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:06:06.943391    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-32541-predictor-8d6ffd9d-mp9l6" podUID="097e1321-7c6d-4446-98b2-21cc0442df39" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.21:8643/healthz\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Aug 01 00:06:06.948608 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:06:06.948570    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-32541-predictor-8d6ffd9d-mp9l6" podUID="097e1321-7c6d-4446-98b2-21cc0442df39" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.21:8080: i/o timeout"
Aug 01 00:06:10.040449 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:06:10.040419    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-fe2ca-predictor-bb856dc98-9pjb9"
Aug 01 00:06:10.040993 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:06:10.040948    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-fe2ca-predictor-bb856dc98-9pjb9" podUID="576ba641-ea65-4b9e-8b4a-a08086f095cb" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.24:8080: connect: connection refused"
Aug 01 00:06:20.041607 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:06:20.041565    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-fe2ca-predictor-bb856dc98-9pjb9" podUID="576ba641-ea65-4b9e-8b4a-a08086f095cb" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.24:8080: connect: connection refused"
Aug 01 00:06:30.041337 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:06:30.041294    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-fe2ca-predictor-bb856dc98-9pjb9" podUID="576ba641-ea65-4b9e-8b4a-a08086f095cb" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.24:8080: connect: connection refused"
Aug 01 00:06:40.041007 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:06:40.040970    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-fe2ca-predictor-bb856dc98-9pjb9" podUID="576ba641-ea65-4b9e-8b4a-a08086f095cb" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.24:8080: connect: connection refused"
Aug 01 00:06:50.041861 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:06:50.041835    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-fe2ca-predictor-bb856dc98-9pjb9"
Aug 01 00:06:52.160421 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:06:52.160378    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-a3ab7-predictor-746489cd7b-qfpmr"]
Aug 01 00:06:52.160917 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:06:52.160708    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-a3ab7-predictor-746489cd7b-qfpmr" podUID="ff102fa8-a979-4023-8e8f-fd67bd0e15b9" containerName="kserve-container" containerID="cri-o://687a791d1f61ae03fff02083fc77776ada71b511f33b8cf300293f0142ac8969" gracePeriod=30
Aug 01 00:06:52.160917 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:06:52.160783    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-a3ab7-predictor-746489cd7b-qfpmr" podUID="ff102fa8-a979-4023-8e8f-fd67bd0e15b9" containerName="kube-rbac-proxy" containerID="cri-o://27d481128c5767d2b1d9be8e07e40f6412e6957359858b045dd49b6e4289da98" gracePeriod=30
Aug 01 00:06:52.298420 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:06:52.298381    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-df362-predictor-6769b69b98-bnk4q"]
Aug 01 00:06:52.298704 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:06:52.298693    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="097e1321-7c6d-4446-98b2-21cc0442df39" containerName="kube-rbac-proxy"
Aug 01 00:06:52.298753 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:06:52.298706    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="097e1321-7c6d-4446-98b2-21cc0442df39" containerName="kube-rbac-proxy"
Aug 01 00:06:52.298753 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:06:52.298727    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="097e1321-7c6d-4446-98b2-21cc0442df39" containerName="kserve-container"
Aug 01 00:06:52.298753 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:06:52.298733    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="097e1321-7c6d-4446-98b2-21cc0442df39" containerName="kserve-container"
Aug 01 00:06:52.298847 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:06:52.298779    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="097e1321-7c6d-4446-98b2-21cc0442df39" containerName="kube-rbac-proxy"
Aug 01 00:06:52.298847 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:06:52.298788    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="097e1321-7c6d-4446-98b2-21cc0442df39" containerName="kserve-container"
Aug 01 00:06:52.301538 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:06:52.301516    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-df362-predictor-6769b69b98-bnk4q"
Aug 01 00:06:52.305021 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:06:52.304990    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-df362-predictor-serving-cert\""
Aug 01 00:06:52.305168 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:06:52.305130    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-df362-kube-rbac-proxy-sar-config\""
Aug 01 00:06:52.310984 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:06:52.310957    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-df362-predictor-6769b69b98-bnk4q"]
Aug 01 00:06:52.315413 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:06:52.315383    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/bcbd1dbf-5ca2-475a-a217-076476f448c3-proxy-tls\") pod \"error-404-isvc-df362-predictor-6769b69b98-bnk4q\" (UID: \"bcbd1dbf-5ca2-475a-a217-076476f448c3\") " pod="kserve-ci-e2e-test/error-404-isvc-df362-predictor-6769b69b98-bnk4q"
Aug 01 00:06:52.315547 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:06:52.315435    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9fjl4\" (UniqueName: \"kubernetes.io/projected/bcbd1dbf-5ca2-475a-a217-076476f448c3-kube-api-access-9fjl4\") pod \"error-404-isvc-df362-predictor-6769b69b98-bnk4q\" (UID: \"bcbd1dbf-5ca2-475a-a217-076476f448c3\") " pod="kserve-ci-e2e-test/error-404-isvc-df362-predictor-6769b69b98-bnk4q"
Aug 01 00:06:52.315547 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:06:52.315510    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/bcbd1dbf-5ca2-475a-a217-076476f448c3-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-df362-predictor-6769b69b98-bnk4q\" (UID: \"bcbd1dbf-5ca2-475a-a217-076476f448c3\") " pod="kserve-ci-e2e-test/error-404-isvc-df362-predictor-6769b69b98-bnk4q"
Aug 01 00:06:52.416798 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:06:52.416693    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9fjl4\" (UniqueName: \"kubernetes.io/projected/bcbd1dbf-5ca2-475a-a217-076476f448c3-kube-api-access-9fjl4\") pod \"error-404-isvc-df362-predictor-6769b69b98-bnk4q\" (UID: \"bcbd1dbf-5ca2-475a-a217-076476f448c3\") " pod="kserve-ci-e2e-test/error-404-isvc-df362-predictor-6769b69b98-bnk4q"
Aug 01 00:06:52.416798 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:06:52.416768    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/bcbd1dbf-5ca2-475a-a217-076476f448c3-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-df362-predictor-6769b69b98-bnk4q\" (UID: \"bcbd1dbf-5ca2-475a-a217-076476f448c3\") " pod="kserve-ci-e2e-test/error-404-isvc-df362-predictor-6769b69b98-bnk4q"
Aug 01 00:06:52.417042 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:06:52.416828    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/bcbd1dbf-5ca2-475a-a217-076476f448c3-proxy-tls\") pod \"error-404-isvc-df362-predictor-6769b69b98-bnk4q\" (UID: \"bcbd1dbf-5ca2-475a-a217-076476f448c3\") " pod="kserve-ci-e2e-test/error-404-isvc-df362-predictor-6769b69b98-bnk4q"
Aug 01 00:06:52.417042 ip-10-0-128-75 kubenswrapper[2564]: E0801 00:06:52.416948    2564 secret.go:189] Couldn't get secret kserve-ci-e2e-test/error-404-isvc-df362-predictor-serving-cert: secret "error-404-isvc-df362-predictor-serving-cert" not found
Aug 01 00:06:52.417042 ip-10-0-128-75 kubenswrapper[2564]: E0801 00:06:52.417012    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bcbd1dbf-5ca2-475a-a217-076476f448c3-proxy-tls podName:bcbd1dbf-5ca2-475a-a217-076476f448c3 nodeName:}" failed. No retries permitted until 2026-08-01 00:06:52.916990254 +0000 UTC m=+1231.029117906 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/bcbd1dbf-5ca2-475a-a217-076476f448c3-proxy-tls") pod "error-404-isvc-df362-predictor-6769b69b98-bnk4q" (UID: "bcbd1dbf-5ca2-475a-a217-076476f448c3") : secret "error-404-isvc-df362-predictor-serving-cert" not found
Aug 01 00:06:52.417657 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:06:52.417608    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/bcbd1dbf-5ca2-475a-a217-076476f448c3-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-df362-predictor-6769b69b98-bnk4q\" (UID: \"bcbd1dbf-5ca2-475a-a217-076476f448c3\") " pod="kserve-ci-e2e-test/error-404-isvc-df362-predictor-6769b69b98-bnk4q"
Aug 01 00:06:52.426056 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:06:52.426024    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9fjl4\" (UniqueName: \"kubernetes.io/projected/bcbd1dbf-5ca2-475a-a217-076476f448c3-kube-api-access-9fjl4\") pod \"error-404-isvc-df362-predictor-6769b69b98-bnk4q\" (UID: \"bcbd1dbf-5ca2-475a-a217-076476f448c3\") " pod="kserve-ci-e2e-test/error-404-isvc-df362-predictor-6769b69b98-bnk4q"
Aug 01 00:06:52.919573 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:06:52.919521    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/bcbd1dbf-5ca2-475a-a217-076476f448c3-proxy-tls\") pod \"error-404-isvc-df362-predictor-6769b69b98-bnk4q\" (UID: \"bcbd1dbf-5ca2-475a-a217-076476f448c3\") " pod="kserve-ci-e2e-test/error-404-isvc-df362-predictor-6769b69b98-bnk4q"
Aug 01 00:06:52.922234 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:06:52.922192    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/bcbd1dbf-5ca2-475a-a217-076476f448c3-proxy-tls\") pod \"error-404-isvc-df362-predictor-6769b69b98-bnk4q\" (UID: \"bcbd1dbf-5ca2-475a-a217-076476f448c3\") " pod="kserve-ci-e2e-test/error-404-isvc-df362-predictor-6769b69b98-bnk4q"
Aug 01 00:06:53.213085 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:06:53.212994    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-df362-predictor-6769b69b98-bnk4q"
Aug 01 00:06:53.234125 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:06:53.234087    2564 generic.go:356] "Generic (PLEG): container finished" podID="ff102fa8-a979-4023-8e8f-fd67bd0e15b9" containerID="27d481128c5767d2b1d9be8e07e40f6412e6957359858b045dd49b6e4289da98" exitCode=2
Aug 01 00:06:53.234272 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:06:53.234135    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-a3ab7-predictor-746489cd7b-qfpmr" event={"ID":"ff102fa8-a979-4023-8e8f-fd67bd0e15b9","Type":"ContainerDied","Data":"27d481128c5767d2b1d9be8e07e40f6412e6957359858b045dd49b6e4289da98"}
Aug 01 00:06:53.354533 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:06:53.354198    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-df362-predictor-6769b69b98-bnk4q"]
Aug 01 00:06:53.357092 ip-10-0-128-75 kubenswrapper[2564]: W0801 00:06:53.357058    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbcbd1dbf_5ca2_475a_a217_076476f448c3.slice/crio-91686c26ac8e28cede066700fcd20aa768e3fef393ea263fefcd86da95f6f7c0 WatchSource:0}: Error finding container 91686c26ac8e28cede066700fcd20aa768e3fef393ea263fefcd86da95f6f7c0: Status 404 returned error can't find the container with id 91686c26ac8e28cede066700fcd20aa768e3fef393ea263fefcd86da95f6f7c0
Aug 01 00:06:54.239544 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:06:54.239506    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-df362-predictor-6769b69b98-bnk4q" event={"ID":"bcbd1dbf-5ca2-475a-a217-076476f448c3","Type":"ContainerStarted","Data":"23a3e4bd26225ebb2147eb18ad00f83fc442bcd9c01fd0d1d509f0d792fb98cf"}
Aug 01 00:06:54.240003 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:06:54.239570    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-df362-predictor-6769b69b98-bnk4q" event={"ID":"bcbd1dbf-5ca2-475a-a217-076476f448c3","Type":"ContainerStarted","Data":"473e36ecf3a1f3a923c6aa009d2e4685926e4c69fcaa5232a3f1a272e45dc619"}
Aug 01 00:06:54.240003 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:06:54.239601    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-df362-predictor-6769b69b98-bnk4q" event={"ID":"bcbd1dbf-5ca2-475a-a217-076476f448c3","Type":"ContainerStarted","Data":"91686c26ac8e28cede066700fcd20aa768e3fef393ea263fefcd86da95f6f7c0"}
Aug 01 00:06:54.240003 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:06:54.239750    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-df362-predictor-6769b69b98-bnk4q"
Aug 01 00:06:54.258167 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:06:54.258116    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-df362-predictor-6769b69b98-bnk4q" podStartSLOduration=2.258097125 podStartE2EDuration="2.258097125s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-01 00:06:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-01 00:06:54.257168597 +0000 UTC m=+1232.369296325" watchObservedRunningTime="2026-08-01 00:06:54.258097125 +0000 UTC m=+1232.370224800"
Aug 01 00:06:55.243518 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:06:55.243478    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-df362-predictor-6769b69b98-bnk4q"
Aug 01 00:06:55.244854 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:06:55.244825    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-df362-predictor-6769b69b98-bnk4q" podUID="bcbd1dbf-5ca2-475a-a217-076476f448c3" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.25:8080: connect: connection refused"
Aug 01 00:06:55.909658 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:06:55.909612    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-a3ab7-predictor-746489cd7b-qfpmr"
Aug 01 00:06:55.938445 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:06:55.938409    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lw8xs\" (UniqueName: \"kubernetes.io/projected/ff102fa8-a979-4023-8e8f-fd67bd0e15b9-kube-api-access-lw8xs\") pod \"ff102fa8-a979-4023-8e8f-fd67bd0e15b9\" (UID: \"ff102fa8-a979-4023-8e8f-fd67bd0e15b9\") "
Aug 01 00:06:55.938673 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:06:55.938470    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ff102fa8-a979-4023-8e8f-fd67bd0e15b9-kube-rbac-proxy-sar-config\") pod \"ff102fa8-a979-4023-8e8f-fd67bd0e15b9\" (UID: \"ff102fa8-a979-4023-8e8f-fd67bd0e15b9\") "
Aug 01 00:06:55.938673 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:06:55.938492    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ff102fa8-a979-4023-8e8f-fd67bd0e15b9-proxy-tls\") pod \"ff102fa8-a979-4023-8e8f-fd67bd0e15b9\" (UID: \"ff102fa8-a979-4023-8e8f-fd67bd0e15b9\") "
Aug 01 00:06:55.938867 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:06:55.938838    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ff102fa8-a979-4023-8e8f-fd67bd0e15b9-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "ff102fa8-a979-4023-8e8f-fd67bd0e15b9" (UID: "ff102fa8-a979-4023-8e8f-fd67bd0e15b9"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 01 00:06:55.940802 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:06:55.940755    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ff102fa8-a979-4023-8e8f-fd67bd0e15b9-kube-api-access-lw8xs" (OuterVolumeSpecName: "kube-api-access-lw8xs") pod "ff102fa8-a979-4023-8e8f-fd67bd0e15b9" (UID: "ff102fa8-a979-4023-8e8f-fd67bd0e15b9"). InnerVolumeSpecName "kube-api-access-lw8xs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 01 00:06:55.940932 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:06:55.940857    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ff102fa8-a979-4023-8e8f-fd67bd0e15b9-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "ff102fa8-a979-4023-8e8f-fd67bd0e15b9" (UID: "ff102fa8-a979-4023-8e8f-fd67bd0e15b9"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 01 00:06:56.039925 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:06:56.039830    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lw8xs\" (UniqueName: \"kubernetes.io/projected/ff102fa8-a979-4023-8e8f-fd67bd0e15b9-kube-api-access-lw8xs\") on node \"ip-10-0-128-75.ec2.internal\" DevicePath \"\""
Aug 01 00:06:56.039925 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:06:56.039863    2564 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ff102fa8-a979-4023-8e8f-fd67bd0e15b9-kube-rbac-proxy-sar-config\") on node \"ip-10-0-128-75.ec2.internal\" DevicePath \"\""
Aug 01 00:06:56.039925 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:06:56.039873    2564 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ff102fa8-a979-4023-8e8f-fd67bd0e15b9-proxy-tls\") on node \"ip-10-0-128-75.ec2.internal\" DevicePath \"\""
Aug 01 00:06:56.249071 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:06:56.249033    2564 generic.go:356] "Generic (PLEG): container finished" podID="ff102fa8-a979-4023-8e8f-fd67bd0e15b9" containerID="687a791d1f61ae03fff02083fc77776ada71b511f33b8cf300293f0142ac8969" exitCode=0
Aug 01 00:06:56.249564 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:06:56.249100    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-a3ab7-predictor-746489cd7b-qfpmr"
Aug 01 00:06:56.249564 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:06:56.249116    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-a3ab7-predictor-746489cd7b-qfpmr" event={"ID":"ff102fa8-a979-4023-8e8f-fd67bd0e15b9","Type":"ContainerDied","Data":"687a791d1f61ae03fff02083fc77776ada71b511f33b8cf300293f0142ac8969"}
Aug 01 00:06:56.249564 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:06:56.249170    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-a3ab7-predictor-746489cd7b-qfpmr" event={"ID":"ff102fa8-a979-4023-8e8f-fd67bd0e15b9","Type":"ContainerDied","Data":"e2be7e1be49a857fe7081ac530ac3a57a71e9f9bfdaa3cb13c5ec763805faa4d"}
Aug 01 00:06:56.249564 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:06:56.249198    2564 scope.go:117] "RemoveContainer" containerID="27d481128c5767d2b1d9be8e07e40f6412e6957359858b045dd49b6e4289da98"
Aug 01 00:06:56.249841 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:06:56.249818    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-df362-predictor-6769b69b98-bnk4q" podUID="bcbd1dbf-5ca2-475a-a217-076476f448c3" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.25:8080: connect: connection refused"
Aug 01 00:06:56.257506 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:06:56.257487    2564 scope.go:117] "RemoveContainer" containerID="687a791d1f61ae03fff02083fc77776ada71b511f33b8cf300293f0142ac8969"
Aug 01 00:06:56.265390 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:06:56.265367    2564 scope.go:117] "RemoveContainer" containerID="27d481128c5767d2b1d9be8e07e40f6412e6957359858b045dd49b6e4289da98"
Aug 01 00:06:56.265718 ip-10-0-128-75 kubenswrapper[2564]: E0801 00:06:56.265691    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"27d481128c5767d2b1d9be8e07e40f6412e6957359858b045dd49b6e4289da98\": container with ID starting with 27d481128c5767d2b1d9be8e07e40f6412e6957359858b045dd49b6e4289da98 not found: ID does not exist" containerID="27d481128c5767d2b1d9be8e07e40f6412e6957359858b045dd49b6e4289da98"
Aug 01 00:06:56.265829 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:06:56.265728    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"27d481128c5767d2b1d9be8e07e40f6412e6957359858b045dd49b6e4289da98"} err="failed to get container status \"27d481128c5767d2b1d9be8e07e40f6412e6957359858b045dd49b6e4289da98\": rpc error: code = NotFound desc = could not find container \"27d481128c5767d2b1d9be8e07e40f6412e6957359858b045dd49b6e4289da98\": container with ID starting with 27d481128c5767d2b1d9be8e07e40f6412e6957359858b045dd49b6e4289da98 not found: ID does not exist"
Aug 01 00:06:56.265829 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:06:56.265752    2564 scope.go:117] "RemoveContainer" containerID="687a791d1f61ae03fff02083fc77776ada71b511f33b8cf300293f0142ac8969"
Aug 01 00:06:56.266088 ip-10-0-128-75 kubenswrapper[2564]: E0801 00:06:56.266067    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"687a791d1f61ae03fff02083fc77776ada71b511f33b8cf300293f0142ac8969\": container with ID starting with 687a791d1f61ae03fff02083fc77776ada71b511f33b8cf300293f0142ac8969 not found: ID does not exist" containerID="687a791d1f61ae03fff02083fc77776ada71b511f33b8cf300293f0142ac8969"
Aug 01 00:06:56.266152 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:06:56.266099    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"687a791d1f61ae03fff02083fc77776ada71b511f33b8cf300293f0142ac8969"} err="failed to get container status \"687a791d1f61ae03fff02083fc77776ada71b511f33b8cf300293f0142ac8969\": rpc error: code = NotFound desc = could not find container \"687a791d1f61ae03fff02083fc77776ada71b511f33b8cf300293f0142ac8969\": container with ID starting with 687a791d1f61ae03fff02083fc77776ada71b511f33b8cf300293f0142ac8969 not found: ID does not exist"
Aug 01 00:06:56.286855 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:06:56.286819    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-a3ab7-predictor-746489cd7b-qfpmr"]
Aug 01 00:06:56.293350 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:06:56.293264    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-a3ab7-predictor-746489cd7b-qfpmr"]
Aug 01 00:06:56.421774 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:06:56.421733    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ff102fa8-a979-4023-8e8f-fd67bd0e15b9" path="/var/lib/kubelet/pods/ff102fa8-a979-4023-8e8f-fd67bd0e15b9/volumes"
Aug 01 00:07:01.254077 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:07:01.254037    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-df362-predictor-6769b69b98-bnk4q"
Aug 01 00:07:01.254741 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:07:01.254709    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-df362-predictor-6769b69b98-bnk4q" podUID="bcbd1dbf-5ca2-475a-a217-076476f448c3" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.25:8080: connect: connection refused"
Aug 01 00:07:11.254773 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:07:11.254732    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-df362-predictor-6769b69b98-bnk4q" podUID="bcbd1dbf-5ca2-475a-a217-076476f448c3" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.25:8080: connect: connection refused"
Aug 01 00:07:12.618319 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:07:12.618281    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-fe2ca-predictor-bb856dc98-9pjb9"]
Aug 01 00:07:12.618797 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:07:12.618592    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-fe2ca-predictor-bb856dc98-9pjb9" podUID="576ba641-ea65-4b9e-8b4a-a08086f095cb" containerName="kserve-container" containerID="cri-o://5af8e569dc0ce37c8c9677d31c1ddf67123c02965804b0b0eb21dc7c6aa23447" gracePeriod=30
Aug 01 00:07:12.618797 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:07:12.618656    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-fe2ca-predictor-bb856dc98-9pjb9" podUID="576ba641-ea65-4b9e-8b4a-a08086f095cb" containerName="kube-rbac-proxy" containerID="cri-o://0da0554fc06c826a94fba1b5be559b4aeee7fcee2f19b452738e264ddd120a5d" gracePeriod=30
Aug 01 00:07:12.771884 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:07:12.771842    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-01c7a-predictor-86b4c76644-xbdpz"]
Aug 01 00:07:12.772162 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:07:12.772150    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ff102fa8-a979-4023-8e8f-fd67bd0e15b9" containerName="kube-rbac-proxy"
Aug 01 00:07:12.772205 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:07:12.772164    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="ff102fa8-a979-4023-8e8f-fd67bd0e15b9" containerName="kube-rbac-proxy"
Aug 01 00:07:12.772205 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:07:12.772180    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ff102fa8-a979-4023-8e8f-fd67bd0e15b9" containerName="kserve-container"
Aug 01 00:07:12.772205 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:07:12.772185    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="ff102fa8-a979-4023-8e8f-fd67bd0e15b9" containerName="kserve-container"
Aug 01 00:07:12.772295 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:07:12.772232    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="ff102fa8-a979-4023-8e8f-fd67bd0e15b9" containerName="kube-rbac-proxy"
Aug 01 00:07:12.772295 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:07:12.772241    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="ff102fa8-a979-4023-8e8f-fd67bd0e15b9" containerName="kserve-container"
Aug 01 00:07:12.775299 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:07:12.775271    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-01c7a-predictor-86b4c76644-xbdpz"
Aug 01 00:07:12.778134 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:07:12.778111    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-01c7a-kube-rbac-proxy-sar-config\""
Aug 01 00:07:12.778271 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:07:12.778146    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-01c7a-predictor-serving-cert\""
Aug 01 00:07:12.784446 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:07:12.784421    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-01c7a-predictor-86b4c76644-xbdpz"]
Aug 01 00:07:12.884585 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:07:12.884498    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/50116272-1adb-4354-9026-e97947524575-proxy-tls\") pod \"error-404-isvc-01c7a-predictor-86b4c76644-xbdpz\" (UID: \"50116272-1adb-4354-9026-e97947524575\") " pod="kserve-ci-e2e-test/error-404-isvc-01c7a-predictor-86b4c76644-xbdpz"
Aug 01 00:07:12.884585 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:07:12.884537    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z9m75\" (UniqueName: \"kubernetes.io/projected/50116272-1adb-4354-9026-e97947524575-kube-api-access-z9m75\") pod \"error-404-isvc-01c7a-predictor-86b4c76644-xbdpz\" (UID: \"50116272-1adb-4354-9026-e97947524575\") " pod="kserve-ci-e2e-test/error-404-isvc-01c7a-predictor-86b4c76644-xbdpz"
Aug 01 00:07:12.884585 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:07:12.884570    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/50116272-1adb-4354-9026-e97947524575-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-01c7a-predictor-86b4c76644-xbdpz\" (UID: \"50116272-1adb-4354-9026-e97947524575\") " pod="kserve-ci-e2e-test/error-404-isvc-01c7a-predictor-86b4c76644-xbdpz"
Aug 01 00:07:12.985956 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:07:12.985922    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/50116272-1adb-4354-9026-e97947524575-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-01c7a-predictor-86b4c76644-xbdpz\" (UID: \"50116272-1adb-4354-9026-e97947524575\") " pod="kserve-ci-e2e-test/error-404-isvc-01c7a-predictor-86b4c76644-xbdpz"
Aug 01 00:07:12.986141 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:07:12.985994    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/50116272-1adb-4354-9026-e97947524575-proxy-tls\") pod \"error-404-isvc-01c7a-predictor-86b4c76644-xbdpz\" (UID: \"50116272-1adb-4354-9026-e97947524575\") " pod="kserve-ci-e2e-test/error-404-isvc-01c7a-predictor-86b4c76644-xbdpz"
Aug 01 00:07:12.986141 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:07:12.986016    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z9m75\" (UniqueName: \"kubernetes.io/projected/50116272-1adb-4354-9026-e97947524575-kube-api-access-z9m75\") pod \"error-404-isvc-01c7a-predictor-86b4c76644-xbdpz\" (UID: \"50116272-1adb-4354-9026-e97947524575\") " pod="kserve-ci-e2e-test/error-404-isvc-01c7a-predictor-86b4c76644-xbdpz"
Aug 01 00:07:12.986141 ip-10-0-128-75 kubenswrapper[2564]: E0801 00:07:12.986114    2564 secret.go:189] Couldn't get secret kserve-ci-e2e-test/error-404-isvc-01c7a-predictor-serving-cert: secret "error-404-isvc-01c7a-predictor-serving-cert" not found
Aug 01 00:07:12.986278 ip-10-0-128-75 kubenswrapper[2564]: E0801 00:07:12.986186    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/50116272-1adb-4354-9026-e97947524575-proxy-tls podName:50116272-1adb-4354-9026-e97947524575 nodeName:}" failed. No retries permitted until 2026-08-01 00:07:13.48616848 +0000 UTC m=+1251.598296136 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/50116272-1adb-4354-9026-e97947524575-proxy-tls") pod "error-404-isvc-01c7a-predictor-86b4c76644-xbdpz" (UID: "50116272-1adb-4354-9026-e97947524575") : secret "error-404-isvc-01c7a-predictor-serving-cert" not found
Aug 01 00:07:12.986620 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:07:12.986601    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/50116272-1adb-4354-9026-e97947524575-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-01c7a-predictor-86b4c76644-xbdpz\" (UID: \"50116272-1adb-4354-9026-e97947524575\") " pod="kserve-ci-e2e-test/error-404-isvc-01c7a-predictor-86b4c76644-xbdpz"
Aug 01 00:07:12.995366 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:07:12.995335    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z9m75\" (UniqueName: \"kubernetes.io/projected/50116272-1adb-4354-9026-e97947524575-kube-api-access-z9m75\") pod \"error-404-isvc-01c7a-predictor-86b4c76644-xbdpz\" (UID: \"50116272-1adb-4354-9026-e97947524575\") " pod="kserve-ci-e2e-test/error-404-isvc-01c7a-predictor-86b4c76644-xbdpz"
Aug 01 00:07:13.319482 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:07:13.319392    2564 generic.go:356] "Generic (PLEG): container finished" podID="576ba641-ea65-4b9e-8b4a-a08086f095cb" containerID="0da0554fc06c826a94fba1b5be559b4aeee7fcee2f19b452738e264ddd120a5d" exitCode=2
Aug 01 00:07:13.319482 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:07:13.319438    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-fe2ca-predictor-bb856dc98-9pjb9" event={"ID":"576ba641-ea65-4b9e-8b4a-a08086f095cb","Type":"ContainerDied","Data":"0da0554fc06c826a94fba1b5be559b4aeee7fcee2f19b452738e264ddd120a5d"}
Aug 01 00:07:13.489765 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:07:13.489725    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/50116272-1adb-4354-9026-e97947524575-proxy-tls\") pod \"error-404-isvc-01c7a-predictor-86b4c76644-xbdpz\" (UID: \"50116272-1adb-4354-9026-e97947524575\") " pod="kserve-ci-e2e-test/error-404-isvc-01c7a-predictor-86b4c76644-xbdpz"
Aug 01 00:07:13.492406 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:07:13.492374    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/50116272-1adb-4354-9026-e97947524575-proxy-tls\") pod \"error-404-isvc-01c7a-predictor-86b4c76644-xbdpz\" (UID: \"50116272-1adb-4354-9026-e97947524575\") " pod="kserve-ci-e2e-test/error-404-isvc-01c7a-predictor-86b4c76644-xbdpz"
Aug 01 00:07:13.686193 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:07:13.686151    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-01c7a-predictor-86b4c76644-xbdpz"
Aug 01 00:07:13.823465 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:07:13.823438    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-01c7a-predictor-86b4c76644-xbdpz"]
Aug 01 00:07:13.826077 ip-10-0-128-75 kubenswrapper[2564]: W0801 00:07:13.826047    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50116272_1adb_4354_9026_e97947524575.slice/crio-16b5c40f241f4f51426ff9be56a74664785618eb88edbd12f0527dc094418a60 WatchSource:0}: Error finding container 16b5c40f241f4f51426ff9be56a74664785618eb88edbd12f0527dc094418a60: Status 404 returned error can't find the container with id 16b5c40f241f4f51426ff9be56a74664785618eb88edbd12f0527dc094418a60
Aug 01 00:07:14.329300 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:07:14.329170    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-01c7a-predictor-86b4c76644-xbdpz" event={"ID":"50116272-1adb-4354-9026-e97947524575","Type":"ContainerStarted","Data":"b01b31a430ef297fec3106be1879e33e01e4b5c83e5a17e37d92f6e68eb2527d"}
Aug 01 00:07:14.329300 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:07:14.329247    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-01c7a-predictor-86b4c76644-xbdpz" event={"ID":"50116272-1adb-4354-9026-e97947524575","Type":"ContainerStarted","Data":"f5f8a0facf6074a5065c54515662e3a505fd3d9b862076fa7ecc1bf861026e77"}
Aug 01 00:07:14.329300 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:07:14.329275    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-01c7a-predictor-86b4c76644-xbdpz" event={"ID":"50116272-1adb-4354-9026-e97947524575","Type":"ContainerStarted","Data":"16b5c40f241f4f51426ff9be56a74664785618eb88edbd12f0527dc094418a60"}
Aug 01 00:07:14.329646 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:07:14.329307    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-01c7a-predictor-86b4c76644-xbdpz"
Aug 01 00:07:14.348035 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:07:14.347980    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-01c7a-predictor-86b4c76644-xbdpz" podStartSLOduration=2.347960954 podStartE2EDuration="2.347960954s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-01 00:07:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-01 00:07:14.346052861 +0000 UTC m=+1252.458180568" watchObservedRunningTime="2026-08-01 00:07:14.347960954 +0000 UTC m=+1252.460088622"
Aug 01 00:07:15.036553 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:07:15.036502    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-fe2ca-predictor-bb856dc98-9pjb9" podUID="576ba641-ea65-4b9e-8b4a-a08086f095cb" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.24:8643/healthz\": dial tcp 10.134.0.24:8643: connect: connection refused"
Aug 01 00:07:15.333412 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:07:15.333319    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-01c7a-predictor-86b4c76644-xbdpz"
Aug 01 00:07:15.334583 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:07:15.334558    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-01c7a-predictor-86b4c76644-xbdpz" podUID="50116272-1adb-4354-9026-e97947524575" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.26:8080: connect: connection refused"
Aug 01 00:07:16.338518 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:07:16.338489    2564 generic.go:356] "Generic (PLEG): container finished" podID="576ba641-ea65-4b9e-8b4a-a08086f095cb" containerID="5af8e569dc0ce37c8c9677d31c1ddf67123c02965804b0b0eb21dc7c6aa23447" exitCode=0
Aug 01 00:07:16.338962 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:07:16.338566    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-fe2ca-predictor-bb856dc98-9pjb9" event={"ID":"576ba641-ea65-4b9e-8b4a-a08086f095cb","Type":"ContainerDied","Data":"5af8e569dc0ce37c8c9677d31c1ddf67123c02965804b0b0eb21dc7c6aa23447"}
Aug 01 00:07:16.338962 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:07:16.338908    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-01c7a-predictor-86b4c76644-xbdpz" podUID="50116272-1adb-4354-9026-e97947524575" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.26:8080: connect: connection refused"
Aug 01 00:07:16.479807 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:07:16.479778    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-fe2ca-predictor-bb856dc98-9pjb9"
Aug 01 00:07:16.616567 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:07:16.616517    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nrcgw\" (UniqueName: \"kubernetes.io/projected/576ba641-ea65-4b9e-8b4a-a08086f095cb-kube-api-access-nrcgw\") pod \"576ba641-ea65-4b9e-8b4a-a08086f095cb\" (UID: \"576ba641-ea65-4b9e-8b4a-a08086f095cb\") "
Aug 01 00:07:16.616762 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:07:16.616682    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/576ba641-ea65-4b9e-8b4a-a08086f095cb-kube-rbac-proxy-sar-config\") pod \"576ba641-ea65-4b9e-8b4a-a08086f095cb\" (UID: \"576ba641-ea65-4b9e-8b4a-a08086f095cb\") "
Aug 01 00:07:16.616762 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:07:16.616726    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/576ba641-ea65-4b9e-8b4a-a08086f095cb-proxy-tls\") pod \"576ba641-ea65-4b9e-8b4a-a08086f095cb\" (UID: \"576ba641-ea65-4b9e-8b4a-a08086f095cb\") "
Aug 01 00:07:16.617072 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:07:16.617043    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/576ba641-ea65-4b9e-8b4a-a08086f095cb-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "576ba641-ea65-4b9e-8b4a-a08086f095cb" (UID: "576ba641-ea65-4b9e-8b4a-a08086f095cb"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 01 00:07:16.618874 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:07:16.618845    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/576ba641-ea65-4b9e-8b4a-a08086f095cb-kube-api-access-nrcgw" (OuterVolumeSpecName: "kube-api-access-nrcgw") pod "576ba641-ea65-4b9e-8b4a-a08086f095cb" (UID: "576ba641-ea65-4b9e-8b4a-a08086f095cb"). InnerVolumeSpecName "kube-api-access-nrcgw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 01 00:07:16.619002 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:07:16.618845    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/576ba641-ea65-4b9e-8b4a-a08086f095cb-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "576ba641-ea65-4b9e-8b4a-a08086f095cb" (UID: "576ba641-ea65-4b9e-8b4a-a08086f095cb"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 01 00:07:16.717703 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:07:16.717660    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nrcgw\" (UniqueName: \"kubernetes.io/projected/576ba641-ea65-4b9e-8b4a-a08086f095cb-kube-api-access-nrcgw\") on node \"ip-10-0-128-75.ec2.internal\" DevicePath \"\""
Aug 01 00:07:16.717703 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:07:16.717688    2564 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/576ba641-ea65-4b9e-8b4a-a08086f095cb-kube-rbac-proxy-sar-config\") on node \"ip-10-0-128-75.ec2.internal\" DevicePath \"\""
Aug 01 00:07:16.717703 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:07:16.717698    2564 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/576ba641-ea65-4b9e-8b4a-a08086f095cb-proxy-tls\") on node \"ip-10-0-128-75.ec2.internal\" DevicePath \"\""
Aug 01 00:07:17.344119 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:07:17.344085    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-fe2ca-predictor-bb856dc98-9pjb9" event={"ID":"576ba641-ea65-4b9e-8b4a-a08086f095cb","Type":"ContainerDied","Data":"698296c47c39cfe63efb7d21e28efa6985a2a134834e44a16dfe610ad345b0a5"}
Aug 01 00:07:17.344119 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:07:17.344105    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-fe2ca-predictor-bb856dc98-9pjb9"
Aug 01 00:07:17.344614 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:07:17.344157    2564 scope.go:117] "RemoveContainer" containerID="0da0554fc06c826a94fba1b5be559b4aeee7fcee2f19b452738e264ddd120a5d"
Aug 01 00:07:17.353234 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:07:17.353207    2564 scope.go:117] "RemoveContainer" containerID="5af8e569dc0ce37c8c9677d31c1ddf67123c02965804b0b0eb21dc7c6aa23447"
Aug 01 00:07:17.365123 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:07:17.365096    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-fe2ca-predictor-bb856dc98-9pjb9"]
Aug 01 00:07:17.368067 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:07:17.368040    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-fe2ca-predictor-bb856dc98-9pjb9"]
Aug 01 00:07:18.420839 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:07:18.420804    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="576ba641-ea65-4b9e-8b4a-a08086f095cb" path="/var/lib/kubelet/pods/576ba641-ea65-4b9e-8b4a-a08086f095cb/volumes"
Aug 01 00:07:21.254706 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:07:21.254667    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-df362-predictor-6769b69b98-bnk4q" podUID="bcbd1dbf-5ca2-475a-a217-076476f448c3" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.25:8080: connect: connection refused"
Aug 01 00:07:21.343093 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:07:21.343063    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-01c7a-predictor-86b4c76644-xbdpz"
Aug 01 00:07:21.343597 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:07:21.343563    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-01c7a-predictor-86b4c76644-xbdpz" podUID="50116272-1adb-4354-9026-e97947524575" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.26:8080: connect: connection refused"
Aug 01 00:07:31.255377 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:07:31.255337    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-df362-predictor-6769b69b98-bnk4q" podUID="bcbd1dbf-5ca2-475a-a217-076476f448c3" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.25:8080: connect: connection refused"
Aug 01 00:07:31.343885 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:07:31.343845    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-01c7a-predictor-86b4c76644-xbdpz" podUID="50116272-1adb-4354-9026-e97947524575" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.26:8080: connect: connection refused"
Aug 01 00:07:41.254849 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:07:41.254804    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-df362-predictor-6769b69b98-bnk4q" podUID="bcbd1dbf-5ca2-475a-a217-076476f448c3" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.25:8080: connect: connection refused"
Aug 01 00:07:41.343736 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:07:41.343693    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-01c7a-predictor-86b4c76644-xbdpz" podUID="50116272-1adb-4354-9026-e97947524575" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.26:8080: connect: connection refused"
Aug 01 00:07:51.255667 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:07:51.255614    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-df362-predictor-6769b69b98-bnk4q"
Aug 01 00:07:51.343972 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:07:51.343931    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-01c7a-predictor-86b4c76644-xbdpz" podUID="50116272-1adb-4354-9026-e97947524575" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.26:8080: connect: connection refused"
Aug 01 00:08:01.344329 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:08:01.344301    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-01c7a-predictor-86b4c76644-xbdpz"
Aug 01 00:08:02.492021 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:08:02.491989    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-df362-predictor-6769b69b98-bnk4q"]
Aug 01 00:08:02.492579 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:08:02.492244    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-df362-predictor-6769b69b98-bnk4q" podUID="bcbd1dbf-5ca2-475a-a217-076476f448c3" containerName="kserve-container" containerID="cri-o://473e36ecf3a1f3a923c6aa009d2e4685926e4c69fcaa5232a3f1a272e45dc619" gracePeriod=30
Aug 01 00:08:02.492579 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:08:02.492274    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-df362-predictor-6769b69b98-bnk4q" podUID="bcbd1dbf-5ca2-475a-a217-076476f448c3" containerName="kube-rbac-proxy" containerID="cri-o://23a3e4bd26225ebb2147eb18ad00f83fc442bcd9c01fd0d1d509f0d792fb98cf" gracePeriod=30
Aug 01 00:08:02.606518 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:08:02.606487    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-d99eb-predictor-597b578c4-hvl99"]
Aug 01 00:08:02.606804 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:08:02.606792    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="576ba641-ea65-4b9e-8b4a-a08086f095cb" containerName="kube-rbac-proxy"
Aug 01 00:08:02.606855 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:08:02.606806    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="576ba641-ea65-4b9e-8b4a-a08086f095cb" containerName="kube-rbac-proxy"
Aug 01 00:08:02.606855 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:08:02.606820    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="576ba641-ea65-4b9e-8b4a-a08086f095cb" containerName="kserve-container"
Aug 01 00:08:02.606855 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:08:02.606825    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="576ba641-ea65-4b9e-8b4a-a08086f095cb" containerName="kserve-container"
Aug 01 00:08:02.607239 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:08:02.606881    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="576ba641-ea65-4b9e-8b4a-a08086f095cb" containerName="kube-rbac-proxy"
Aug 01 00:08:02.607239 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:08:02.606891    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="576ba641-ea65-4b9e-8b4a-a08086f095cb" containerName="kserve-container"
Aug 01 00:08:02.610868 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:08:02.610817    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-d99eb-predictor-597b578c4-hvl99"
Aug 01 00:08:02.613423 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:08:02.613398    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-d99eb-kube-rbac-proxy-sar-config\""
Aug 01 00:08:02.613423 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:08:02.613399    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-d99eb-predictor-serving-cert\""
Aug 01 00:08:02.620688 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:08:02.620659    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-d99eb-predictor-597b578c4-hvl99"]
Aug 01 00:08:02.695593 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:08:02.695558    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2498b\" (UniqueName: \"kubernetes.io/projected/63f601af-2cf4-4dd4-bf90-b7d2e98b97c1-kube-api-access-2498b\") pod \"error-404-isvc-d99eb-predictor-597b578c4-hvl99\" (UID: \"63f601af-2cf4-4dd4-bf90-b7d2e98b97c1\") " pod="kserve-ci-e2e-test/error-404-isvc-d99eb-predictor-597b578c4-hvl99"
Aug 01 00:08:02.695593 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:08:02.695605    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/63f601af-2cf4-4dd4-bf90-b7d2e98b97c1-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-d99eb-predictor-597b578c4-hvl99\" (UID: \"63f601af-2cf4-4dd4-bf90-b7d2e98b97c1\") " pod="kserve-ci-e2e-test/error-404-isvc-d99eb-predictor-597b578c4-hvl99"
Aug 01 00:08:02.695849 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:08:02.695688    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/63f601af-2cf4-4dd4-bf90-b7d2e98b97c1-proxy-tls\") pod \"error-404-isvc-d99eb-predictor-597b578c4-hvl99\" (UID: \"63f601af-2cf4-4dd4-bf90-b7d2e98b97c1\") " pod="kserve-ci-e2e-test/error-404-isvc-d99eb-predictor-597b578c4-hvl99"
Aug 01 00:08:02.796170 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:08:02.796081    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/63f601af-2cf4-4dd4-bf90-b7d2e98b97c1-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-d99eb-predictor-597b578c4-hvl99\" (UID: \"63f601af-2cf4-4dd4-bf90-b7d2e98b97c1\") " pod="kserve-ci-e2e-test/error-404-isvc-d99eb-predictor-597b578c4-hvl99"
Aug 01 00:08:02.796170 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:08:02.796139    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/63f601af-2cf4-4dd4-bf90-b7d2e98b97c1-proxy-tls\") pod \"error-404-isvc-d99eb-predictor-597b578c4-hvl99\" (UID: \"63f601af-2cf4-4dd4-bf90-b7d2e98b97c1\") " pod="kserve-ci-e2e-test/error-404-isvc-d99eb-predictor-597b578c4-hvl99"
Aug 01 00:08:02.796392 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:08:02.796211    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2498b\" (UniqueName: \"kubernetes.io/projected/63f601af-2cf4-4dd4-bf90-b7d2e98b97c1-kube-api-access-2498b\") pod \"error-404-isvc-d99eb-predictor-597b578c4-hvl99\" (UID: \"63f601af-2cf4-4dd4-bf90-b7d2e98b97c1\") " pod="kserve-ci-e2e-test/error-404-isvc-d99eb-predictor-597b578c4-hvl99"
Aug 01 00:08:02.796808 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:08:02.796790    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/63f601af-2cf4-4dd4-bf90-b7d2e98b97c1-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-d99eb-predictor-597b578c4-hvl99\" (UID: \"63f601af-2cf4-4dd4-bf90-b7d2e98b97c1\") " pod="kserve-ci-e2e-test/error-404-isvc-d99eb-predictor-597b578c4-hvl99"
Aug 01 00:08:02.798805 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:08:02.798776    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/63f601af-2cf4-4dd4-bf90-b7d2e98b97c1-proxy-tls\") pod \"error-404-isvc-d99eb-predictor-597b578c4-hvl99\" (UID: \"63f601af-2cf4-4dd4-bf90-b7d2e98b97c1\") " pod="kserve-ci-e2e-test/error-404-isvc-d99eb-predictor-597b578c4-hvl99"
Aug 01 00:08:02.805041 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:08:02.805010    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2498b\" (UniqueName: \"kubernetes.io/projected/63f601af-2cf4-4dd4-bf90-b7d2e98b97c1-kube-api-access-2498b\") pod \"error-404-isvc-d99eb-predictor-597b578c4-hvl99\" (UID: \"63f601af-2cf4-4dd4-bf90-b7d2e98b97c1\") " pod="kserve-ci-e2e-test/error-404-isvc-d99eb-predictor-597b578c4-hvl99"
Aug 01 00:08:02.922395 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:08:02.922360    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-d99eb-predictor-597b578c4-hvl99"
Aug 01 00:08:03.057151 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:08:03.057125    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-d99eb-predictor-597b578c4-hvl99"]
Aug 01 00:08:03.059666 ip-10-0-128-75 kubenswrapper[2564]: W0801 00:08:03.059620    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63f601af_2cf4_4dd4_bf90_b7d2e98b97c1.slice/crio-eb331105dcba363e41b1aaffe00d1c6b2d20c7eca403385c7a055d54ac8ce022 WatchSource:0}: Error finding container eb331105dcba363e41b1aaffe00d1c6b2d20c7eca403385c7a055d54ac8ce022: Status 404 returned error can't find the container with id eb331105dcba363e41b1aaffe00d1c6b2d20c7eca403385c7a055d54ac8ce022
Aug 01 00:08:03.527079 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:08:03.527044    2564 generic.go:356] "Generic (PLEG): container finished" podID="bcbd1dbf-5ca2-475a-a217-076476f448c3" containerID="23a3e4bd26225ebb2147eb18ad00f83fc442bcd9c01fd0d1d509f0d792fb98cf" exitCode=2
Aug 01 00:08:03.527556 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:08:03.527115    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-df362-predictor-6769b69b98-bnk4q" event={"ID":"bcbd1dbf-5ca2-475a-a217-076476f448c3","Type":"ContainerDied","Data":"23a3e4bd26225ebb2147eb18ad00f83fc442bcd9c01fd0d1d509f0d792fb98cf"}
Aug 01 00:08:03.528735 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:08:03.528707    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-d99eb-predictor-597b578c4-hvl99" event={"ID":"63f601af-2cf4-4dd4-bf90-b7d2e98b97c1","Type":"ContainerStarted","Data":"b47c025516d35a9832c61d240a47fed6e7de3843b8f5d9cc73e0f66f2d97c925"}
Aug 01 00:08:03.528856 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:08:03.528761    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-d99eb-predictor-597b578c4-hvl99" event={"ID":"63f601af-2cf4-4dd4-bf90-b7d2e98b97c1","Type":"ContainerStarted","Data":"a6f9e2ac4aaeece12ad0761911af08a95aa51b03f499905eaa4ad35fdeda5adf"}
Aug 01 00:08:03.528856 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:08:03.528792    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-d99eb-predictor-597b578c4-hvl99" event={"ID":"63f601af-2cf4-4dd4-bf90-b7d2e98b97c1","Type":"ContainerStarted","Data":"eb331105dcba363e41b1aaffe00d1c6b2d20c7eca403385c7a055d54ac8ce022"}
Aug 01 00:08:03.528926 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:08:03.528869    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-d99eb-predictor-597b578c4-hvl99"
Aug 01 00:08:03.547529 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:08:03.547475    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-d99eb-predictor-597b578c4-hvl99" podStartSLOduration=1.547453744 podStartE2EDuration="1.547453744s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-01 00:08:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-01 00:08:03.546020149 +0000 UTC m=+1301.658147844" watchObservedRunningTime="2026-08-01 00:08:03.547453744 +0000 UTC m=+1301.659581477"
Aug 01 00:08:04.533092 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:08:04.533058    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-d99eb-predictor-597b578c4-hvl99"
Aug 01 00:08:04.534224 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:08:04.534199    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-d99eb-predictor-597b578c4-hvl99" podUID="63f601af-2cf4-4dd4-bf90-b7d2e98b97c1" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.27:8080: connect: connection refused"
Aug 01 00:08:05.536620 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:08:05.536579    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-d99eb-predictor-597b578c4-hvl99" podUID="63f601af-2cf4-4dd4-bf90-b7d2e98b97c1" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.27:8080: connect: connection refused"
Aug 01 00:08:05.836196 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:08:05.836171    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-df362-predictor-6769b69b98-bnk4q"
Aug 01 00:08:05.923834 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:08:05.923792    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9fjl4\" (UniqueName: \"kubernetes.io/projected/bcbd1dbf-5ca2-475a-a217-076476f448c3-kube-api-access-9fjl4\") pod \"bcbd1dbf-5ca2-475a-a217-076476f448c3\" (UID: \"bcbd1dbf-5ca2-475a-a217-076476f448c3\") "
Aug 01 00:08:05.924029 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:08:05.923898    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/bcbd1dbf-5ca2-475a-a217-076476f448c3-proxy-tls\") pod \"bcbd1dbf-5ca2-475a-a217-076476f448c3\" (UID: \"bcbd1dbf-5ca2-475a-a217-076476f448c3\") "
Aug 01 00:08:05.924029 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:08:05.923932    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/bcbd1dbf-5ca2-475a-a217-076476f448c3-kube-rbac-proxy-sar-config\") pod \"bcbd1dbf-5ca2-475a-a217-076476f448c3\" (UID: \"bcbd1dbf-5ca2-475a-a217-076476f448c3\") "
Aug 01 00:08:05.924349 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:08:05.924323    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bcbd1dbf-5ca2-475a-a217-076476f448c3-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "bcbd1dbf-5ca2-475a-a217-076476f448c3" (UID: "bcbd1dbf-5ca2-475a-a217-076476f448c3"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 01 00:08:05.925960 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:08:05.925934    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bcbd1dbf-5ca2-475a-a217-076476f448c3-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "bcbd1dbf-5ca2-475a-a217-076476f448c3" (UID: "bcbd1dbf-5ca2-475a-a217-076476f448c3"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 01 00:08:05.926047 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:08:05.925935    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bcbd1dbf-5ca2-475a-a217-076476f448c3-kube-api-access-9fjl4" (OuterVolumeSpecName: "kube-api-access-9fjl4") pod "bcbd1dbf-5ca2-475a-a217-076476f448c3" (UID: "bcbd1dbf-5ca2-475a-a217-076476f448c3"). InnerVolumeSpecName "kube-api-access-9fjl4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 01 00:08:06.024535 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:08:06.024498    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9fjl4\" (UniqueName: \"kubernetes.io/projected/bcbd1dbf-5ca2-475a-a217-076476f448c3-kube-api-access-9fjl4\") on node \"ip-10-0-128-75.ec2.internal\" DevicePath \"\""
Aug 01 00:08:06.024535 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:08:06.024527    2564 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/bcbd1dbf-5ca2-475a-a217-076476f448c3-proxy-tls\") on node \"ip-10-0-128-75.ec2.internal\" DevicePath \"\""
Aug 01 00:08:06.024535 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:08:06.024540    2564 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/bcbd1dbf-5ca2-475a-a217-076476f448c3-kube-rbac-proxy-sar-config\") on node \"ip-10-0-128-75.ec2.internal\" DevicePath \"\""
Aug 01 00:08:06.542023 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:08:06.541975    2564 generic.go:356] "Generic (PLEG): container finished" podID="bcbd1dbf-5ca2-475a-a217-076476f448c3" containerID="473e36ecf3a1f3a923c6aa009d2e4685926e4c69fcaa5232a3f1a272e45dc619" exitCode=0
Aug 01 00:08:06.542023 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:08:06.542021    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-df362-predictor-6769b69b98-bnk4q"
Aug 01 00:08:06.542505 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:08:06.542047    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-df362-predictor-6769b69b98-bnk4q" event={"ID":"bcbd1dbf-5ca2-475a-a217-076476f448c3","Type":"ContainerDied","Data":"473e36ecf3a1f3a923c6aa009d2e4685926e4c69fcaa5232a3f1a272e45dc619"}
Aug 01 00:08:06.542505 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:08:06.542108    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-df362-predictor-6769b69b98-bnk4q" event={"ID":"bcbd1dbf-5ca2-475a-a217-076476f448c3","Type":"ContainerDied","Data":"91686c26ac8e28cede066700fcd20aa768e3fef393ea263fefcd86da95f6f7c0"}
Aug 01 00:08:06.542505 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:08:06.542134    2564 scope.go:117] "RemoveContainer" containerID="23a3e4bd26225ebb2147eb18ad00f83fc442bcd9c01fd0d1d509f0d792fb98cf"
Aug 01 00:08:06.550025 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:08:06.550011    2564 scope.go:117] "RemoveContainer" containerID="473e36ecf3a1f3a923c6aa009d2e4685926e4c69fcaa5232a3f1a272e45dc619"
Aug 01 00:08:06.557513 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:08:06.557495    2564 scope.go:117] "RemoveContainer" containerID="23a3e4bd26225ebb2147eb18ad00f83fc442bcd9c01fd0d1d509f0d792fb98cf"
Aug 01 00:08:06.557813 ip-10-0-128-75 kubenswrapper[2564]: E0801 00:08:06.557795    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"23a3e4bd26225ebb2147eb18ad00f83fc442bcd9c01fd0d1d509f0d792fb98cf\": container with ID starting with 23a3e4bd26225ebb2147eb18ad00f83fc442bcd9c01fd0d1d509f0d792fb98cf not found: ID does not exist" containerID="23a3e4bd26225ebb2147eb18ad00f83fc442bcd9c01fd0d1d509f0d792fb98cf"
Aug 01 00:08:06.557892 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:08:06.557829    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"23a3e4bd26225ebb2147eb18ad00f83fc442bcd9c01fd0d1d509f0d792fb98cf"} err="failed to get container status \"23a3e4bd26225ebb2147eb18ad00f83fc442bcd9c01fd0d1d509f0d792fb98cf\": rpc error: code = NotFound desc = could not find container \"23a3e4bd26225ebb2147eb18ad00f83fc442bcd9c01fd0d1d509f0d792fb98cf\": container with ID starting with 23a3e4bd26225ebb2147eb18ad00f83fc442bcd9c01fd0d1d509f0d792fb98cf not found: ID does not exist"
Aug 01 00:08:06.557892 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:08:06.557854    2564 scope.go:117] "RemoveContainer" containerID="473e36ecf3a1f3a923c6aa009d2e4685926e4c69fcaa5232a3f1a272e45dc619"
Aug 01 00:08:06.558125 ip-10-0-128-75 kubenswrapper[2564]: E0801 00:08:06.558106    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"473e36ecf3a1f3a923c6aa009d2e4685926e4c69fcaa5232a3f1a272e45dc619\": container with ID starting with 473e36ecf3a1f3a923c6aa009d2e4685926e4c69fcaa5232a3f1a272e45dc619 not found: ID does not exist" containerID="473e36ecf3a1f3a923c6aa009d2e4685926e4c69fcaa5232a3f1a272e45dc619"
Aug 01 00:08:06.558168 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:08:06.558133    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"473e36ecf3a1f3a923c6aa009d2e4685926e4c69fcaa5232a3f1a272e45dc619"} err="failed to get container status \"473e36ecf3a1f3a923c6aa009d2e4685926e4c69fcaa5232a3f1a272e45dc619\": rpc error: code = NotFound desc = could not find container \"473e36ecf3a1f3a923c6aa009d2e4685926e4c69fcaa5232a3f1a272e45dc619\": container with ID starting with 473e36ecf3a1f3a923c6aa009d2e4685926e4c69fcaa5232a3f1a272e45dc619 not found: ID does not exist"
Aug 01 00:08:06.560845 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:08:06.560820    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-df362-predictor-6769b69b98-bnk4q"]
Aug 01 00:08:06.564221 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:08:06.564198    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-df362-predictor-6769b69b98-bnk4q"]
Aug 01 00:08:08.422008 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:08:08.421973    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bcbd1dbf-5ca2-475a-a217-076476f448c3" path="/var/lib/kubelet/pods/bcbd1dbf-5ca2-475a-a217-076476f448c3/volumes"
Aug 01 00:08:10.540789 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:08:10.540762    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-d99eb-predictor-597b578c4-hvl99"
Aug 01 00:08:10.541385 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:08:10.541360    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-d99eb-predictor-597b578c4-hvl99" podUID="63f601af-2cf4-4dd4-bf90-b7d2e98b97c1" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.27:8080: connect: connection refused"
Aug 01 00:08:20.542283 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:08:20.542249    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-d99eb-predictor-597b578c4-hvl99" podUID="63f601af-2cf4-4dd4-bf90-b7d2e98b97c1" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.27:8080: connect: connection refused"
Aug 01 00:08:30.542244 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:08:30.542203    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-d99eb-predictor-597b578c4-hvl99" podUID="63f601af-2cf4-4dd4-bf90-b7d2e98b97c1" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.27:8080: connect: connection refused"
Aug 01 00:08:40.541745 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:08:40.541699    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-d99eb-predictor-597b578c4-hvl99" podUID="63f601af-2cf4-4dd4-bf90-b7d2e98b97c1" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.27:8080: connect: connection refused"
Aug 01 00:08:50.541874 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:08:50.541783    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-d99eb-predictor-597b578c4-hvl99"
Aug 01 00:16:27.740283 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:16:27.740247    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-01c7a-predictor-86b4c76644-xbdpz"]
Aug 01 00:16:27.740881 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:16:27.740482    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-01c7a-predictor-86b4c76644-xbdpz" podUID="50116272-1adb-4354-9026-e97947524575" containerName="kserve-container" containerID="cri-o://f5f8a0facf6074a5065c54515662e3a505fd3d9b862076fa7ecc1bf861026e77" gracePeriod=30
Aug 01 00:16:27.740881 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:16:27.740511    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-01c7a-predictor-86b4c76644-xbdpz" podUID="50116272-1adb-4354-9026-e97947524575" containerName="kube-rbac-proxy" containerID="cri-o://b01b31a430ef297fec3106be1879e33e01e4b5c83e5a17e37d92f6e68eb2527d" gracePeriod=30
Aug 01 00:16:27.767689 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:16:27.767621    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-d0074-predictor-64b97bfbdb-84kpw"]
Aug 01 00:16:27.767967 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:16:27.767956    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bcbd1dbf-5ca2-475a-a217-076476f448c3" containerName="kube-rbac-proxy"
Aug 01 00:16:27.768010 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:16:27.767969    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="bcbd1dbf-5ca2-475a-a217-076476f448c3" containerName="kube-rbac-proxy"
Aug 01 00:16:27.768010 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:16:27.767979    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bcbd1dbf-5ca2-475a-a217-076476f448c3" containerName="kserve-container"
Aug 01 00:16:27.768010 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:16:27.767985    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="bcbd1dbf-5ca2-475a-a217-076476f448c3" containerName="kserve-container"
Aug 01 00:16:27.768102 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:16:27.768037    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="bcbd1dbf-5ca2-475a-a217-076476f448c3" containerName="kserve-container"
Aug 01 00:16:27.768102 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:16:27.768045    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="bcbd1dbf-5ca2-475a-a217-076476f448c3" containerName="kube-rbac-proxy"
Aug 01 00:16:27.770971 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:16:27.770953    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-d0074-predictor-64b97bfbdb-84kpw"
Aug 01 00:16:27.773322 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:16:27.773300    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-d0074-predictor-serving-cert\""
Aug 01 00:16:27.773408 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:16:27.773299    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-d0074-kube-rbac-proxy-sar-config\""
Aug 01 00:16:27.779119 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:16:27.779092    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-d0074-predictor-64b97bfbdb-84kpw"]
Aug 01 00:16:27.885331 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:16:27.885288    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/54cbebd2-2258-4358-a3f9-0cb5fc27f46c-proxy-tls\") pod \"error-404-isvc-d0074-predictor-64b97bfbdb-84kpw\" (UID: \"54cbebd2-2258-4358-a3f9-0cb5fc27f46c\") " pod="kserve-ci-e2e-test/error-404-isvc-d0074-predictor-64b97bfbdb-84kpw"
Aug 01 00:16:27.885331 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:16:27.885334    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cskpc\" (UniqueName: \"kubernetes.io/projected/54cbebd2-2258-4358-a3f9-0cb5fc27f46c-kube-api-access-cskpc\") pod \"error-404-isvc-d0074-predictor-64b97bfbdb-84kpw\" (UID: \"54cbebd2-2258-4358-a3f9-0cb5fc27f46c\") " pod="kserve-ci-e2e-test/error-404-isvc-d0074-predictor-64b97bfbdb-84kpw"
Aug 01 00:16:27.885537 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:16:27.885374    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/54cbebd2-2258-4358-a3f9-0cb5fc27f46c-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-d0074-predictor-64b97bfbdb-84kpw\" (UID: \"54cbebd2-2258-4358-a3f9-0cb5fc27f46c\") " pod="kserve-ci-e2e-test/error-404-isvc-d0074-predictor-64b97bfbdb-84kpw"
Aug 01 00:16:27.986772 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:16:27.986734    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/54cbebd2-2258-4358-a3f9-0cb5fc27f46c-proxy-tls\") pod \"error-404-isvc-d0074-predictor-64b97bfbdb-84kpw\" (UID: \"54cbebd2-2258-4358-a3f9-0cb5fc27f46c\") " pod="kserve-ci-e2e-test/error-404-isvc-d0074-predictor-64b97bfbdb-84kpw"
Aug 01 00:16:27.986978 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:16:27.986779    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cskpc\" (UniqueName: \"kubernetes.io/projected/54cbebd2-2258-4358-a3f9-0cb5fc27f46c-kube-api-access-cskpc\") pod \"error-404-isvc-d0074-predictor-64b97bfbdb-84kpw\" (UID: \"54cbebd2-2258-4358-a3f9-0cb5fc27f46c\") " pod="kserve-ci-e2e-test/error-404-isvc-d0074-predictor-64b97bfbdb-84kpw"
Aug 01 00:16:27.986978 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:16:27.986828    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/54cbebd2-2258-4358-a3f9-0cb5fc27f46c-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-d0074-predictor-64b97bfbdb-84kpw\" (UID: \"54cbebd2-2258-4358-a3f9-0cb5fc27f46c\") " pod="kserve-ci-e2e-test/error-404-isvc-d0074-predictor-64b97bfbdb-84kpw"
Aug 01 00:16:27.987519 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:16:27.987495    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/54cbebd2-2258-4358-a3f9-0cb5fc27f46c-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-d0074-predictor-64b97bfbdb-84kpw\" (UID: \"54cbebd2-2258-4358-a3f9-0cb5fc27f46c\") " pod="kserve-ci-e2e-test/error-404-isvc-d0074-predictor-64b97bfbdb-84kpw"
Aug 01 00:16:27.989270 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:16:27.989246    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/54cbebd2-2258-4358-a3f9-0cb5fc27f46c-proxy-tls\") pod \"error-404-isvc-d0074-predictor-64b97bfbdb-84kpw\" (UID: \"54cbebd2-2258-4358-a3f9-0cb5fc27f46c\") " pod="kserve-ci-e2e-test/error-404-isvc-d0074-predictor-64b97bfbdb-84kpw"
Aug 01 00:16:27.995890 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:16:27.995819    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cskpc\" (UniqueName: \"kubernetes.io/projected/54cbebd2-2258-4358-a3f9-0cb5fc27f46c-kube-api-access-cskpc\") pod \"error-404-isvc-d0074-predictor-64b97bfbdb-84kpw\" (UID: \"54cbebd2-2258-4358-a3f9-0cb5fc27f46c\") " pod="kserve-ci-e2e-test/error-404-isvc-d0074-predictor-64b97bfbdb-84kpw"
Aug 01 00:16:28.081914 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:16:28.081877    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-d0074-predictor-64b97bfbdb-84kpw"
Aug 01 00:16:28.224653 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:16:28.221391    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-d0074-predictor-64b97bfbdb-84kpw"]
Aug 01 00:16:28.453908 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:16:28.453871    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-d0074-predictor-64b97bfbdb-84kpw" event={"ID":"54cbebd2-2258-4358-a3f9-0cb5fc27f46c","Type":"ContainerStarted","Data":"6c43bc16433b164718994da13d2b16dfb7f9c58ee957979e860475a19fe55ca3"}
Aug 01 00:16:28.454113 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:16:28.453932    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-d0074-predictor-64b97bfbdb-84kpw" event={"ID":"54cbebd2-2258-4358-a3f9-0cb5fc27f46c","Type":"ContainerStarted","Data":"8555659eabadbc61c6fb3c6dc1f03a4d20213f65f187a118fdac6e49c5353896"}
Aug 01 00:16:28.454113 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:16:28.454035    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-d0074-predictor-64b97bfbdb-84kpw"
Aug 01 00:16:28.454113 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:16:28.454077    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-d0074-predictor-64b97bfbdb-84kpw" event={"ID":"54cbebd2-2258-4358-a3f9-0cb5fc27f46c","Type":"ContainerStarted","Data":"b557d939363621b5159a993d5e667eb01dc38c21663526af350c88b28a4a03fc"}
Aug 01 00:16:28.454113 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:16:28.454118    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-d0074-predictor-64b97bfbdb-84kpw"
Aug 01 00:16:28.455872 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:16:28.455840    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-d0074-predictor-64b97bfbdb-84kpw" podUID="54cbebd2-2258-4358-a3f9-0cb5fc27f46c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.28:8080: connect: connection refused"
Aug 01 00:16:28.456001 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:16:28.455883    2564 generic.go:356] "Generic (PLEG): container finished" podID="50116272-1adb-4354-9026-e97947524575" containerID="b01b31a430ef297fec3106be1879e33e01e4b5c83e5a17e37d92f6e68eb2527d" exitCode=2
Aug 01 00:16:28.456001 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:16:28.455920    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-01c7a-predictor-86b4c76644-xbdpz" event={"ID":"50116272-1adb-4354-9026-e97947524575","Type":"ContainerDied","Data":"b01b31a430ef297fec3106be1879e33e01e4b5c83e5a17e37d92f6e68eb2527d"}
Aug 01 00:16:28.470903 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:16:28.470846    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-d0074-predictor-64b97bfbdb-84kpw" podStartSLOduration=1.4708292090000001 podStartE2EDuration="1.470829209s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-01 00:16:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-01 00:16:28.469362684 +0000 UTC m=+1806.581490385" watchObservedRunningTime="2026-08-01 00:16:28.470829209 +0000 UTC m=+1806.582956882"
Aug 01 00:16:29.460239 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:16:29.460199    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-d0074-predictor-64b97bfbdb-84kpw" podUID="54cbebd2-2258-4358-a3f9-0cb5fc27f46c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.28:8080: connect: connection refused"
Aug 01 00:16:31.091970 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:16:31.091946    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-01c7a-predictor-86b4c76644-xbdpz"
Aug 01 00:16:31.215254 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:16:31.215167    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z9m75\" (UniqueName: \"kubernetes.io/projected/50116272-1adb-4354-9026-e97947524575-kube-api-access-z9m75\") pod \"50116272-1adb-4354-9026-e97947524575\" (UID: \"50116272-1adb-4354-9026-e97947524575\") "
Aug 01 00:16:31.215254 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:16:31.215231    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/50116272-1adb-4354-9026-e97947524575-proxy-tls\") pod \"50116272-1adb-4354-9026-e97947524575\" (UID: \"50116272-1adb-4354-9026-e97947524575\") "
Aug 01 00:16:31.215479 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:16:31.215286    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/50116272-1adb-4354-9026-e97947524575-kube-rbac-proxy-sar-config\") pod \"50116272-1adb-4354-9026-e97947524575\" (UID: \"50116272-1adb-4354-9026-e97947524575\") "
Aug 01 00:16:31.215741 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:16:31.215709    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/50116272-1adb-4354-9026-e97947524575-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "50116272-1adb-4354-9026-e97947524575" (UID: "50116272-1adb-4354-9026-e97947524575"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 01 00:16:31.217407 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:16:31.217377    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/50116272-1adb-4354-9026-e97947524575-kube-api-access-z9m75" (OuterVolumeSpecName: "kube-api-access-z9m75") pod "50116272-1adb-4354-9026-e97947524575" (UID: "50116272-1adb-4354-9026-e97947524575"). InnerVolumeSpecName "kube-api-access-z9m75". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 01 00:16:31.217464 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:16:31.217440    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/50116272-1adb-4354-9026-e97947524575-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "50116272-1adb-4354-9026-e97947524575" (UID: "50116272-1adb-4354-9026-e97947524575"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 01 00:16:31.316498 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:16:31.316464    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z9m75\" (UniqueName: \"kubernetes.io/projected/50116272-1adb-4354-9026-e97947524575-kube-api-access-z9m75\") on node \"ip-10-0-128-75.ec2.internal\" DevicePath \"\""
Aug 01 00:16:31.316498 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:16:31.316493    2564 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/50116272-1adb-4354-9026-e97947524575-proxy-tls\") on node \"ip-10-0-128-75.ec2.internal\" DevicePath \"\""
Aug 01 00:16:31.316498 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:16:31.316504    2564 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/50116272-1adb-4354-9026-e97947524575-kube-rbac-proxy-sar-config\") on node \"ip-10-0-128-75.ec2.internal\" DevicePath \"\""
Aug 01 00:16:31.468548 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:16:31.468464    2564 generic.go:356] "Generic (PLEG): container finished" podID="50116272-1adb-4354-9026-e97947524575" containerID="f5f8a0facf6074a5065c54515662e3a505fd3d9b862076fa7ecc1bf861026e77" exitCode=0
Aug 01 00:16:31.468548 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:16:31.468509    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-01c7a-predictor-86b4c76644-xbdpz"
Aug 01 00:16:31.468766 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:16:31.468545    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-01c7a-predictor-86b4c76644-xbdpz" event={"ID":"50116272-1adb-4354-9026-e97947524575","Type":"ContainerDied","Data":"f5f8a0facf6074a5065c54515662e3a505fd3d9b862076fa7ecc1bf861026e77"}
Aug 01 00:16:31.468766 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:16:31.468608    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-01c7a-predictor-86b4c76644-xbdpz" event={"ID":"50116272-1adb-4354-9026-e97947524575","Type":"ContainerDied","Data":"16b5c40f241f4f51426ff9be56a74664785618eb88edbd12f0527dc094418a60"}
Aug 01 00:16:31.468766 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:16:31.468662    2564 scope.go:117] "RemoveContainer" containerID="b01b31a430ef297fec3106be1879e33e01e4b5c83e5a17e37d92f6e68eb2527d"
Aug 01 00:16:31.478254 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:16:31.478233    2564 scope.go:117] "RemoveContainer" containerID="f5f8a0facf6074a5065c54515662e3a505fd3d9b862076fa7ecc1bf861026e77"
Aug 01 00:16:31.485709 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:16:31.485690    2564 scope.go:117] "RemoveContainer" containerID="b01b31a430ef297fec3106be1879e33e01e4b5c83e5a17e37d92f6e68eb2527d"
Aug 01 00:16:31.485995 ip-10-0-128-75 kubenswrapper[2564]: E0801 00:16:31.485968    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b01b31a430ef297fec3106be1879e33e01e4b5c83e5a17e37d92f6e68eb2527d\": container with ID starting with b01b31a430ef297fec3106be1879e33e01e4b5c83e5a17e37d92f6e68eb2527d not found: ID does not exist" containerID="b01b31a430ef297fec3106be1879e33e01e4b5c83e5a17e37d92f6e68eb2527d"
Aug 01 00:16:31.486046 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:16:31.486004    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b01b31a430ef297fec3106be1879e33e01e4b5c83e5a17e37d92f6e68eb2527d"} err="failed to get container status \"b01b31a430ef297fec3106be1879e33e01e4b5c83e5a17e37d92f6e68eb2527d\": rpc error: code = NotFound desc = could not find container \"b01b31a430ef297fec3106be1879e33e01e4b5c83e5a17e37d92f6e68eb2527d\": container with ID starting with b01b31a430ef297fec3106be1879e33e01e4b5c83e5a17e37d92f6e68eb2527d not found: ID does not exist"
Aug 01 00:16:31.486046 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:16:31.486024    2564 scope.go:117] "RemoveContainer" containerID="f5f8a0facf6074a5065c54515662e3a505fd3d9b862076fa7ecc1bf861026e77"
Aug 01 00:16:31.486270 ip-10-0-128-75 kubenswrapper[2564]: E0801 00:16:31.486249    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f5f8a0facf6074a5065c54515662e3a505fd3d9b862076fa7ecc1bf861026e77\": container with ID starting with f5f8a0facf6074a5065c54515662e3a505fd3d9b862076fa7ecc1bf861026e77 not found: ID does not exist" containerID="f5f8a0facf6074a5065c54515662e3a505fd3d9b862076fa7ecc1bf861026e77"
Aug 01 00:16:31.486307 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:16:31.486280    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f5f8a0facf6074a5065c54515662e3a505fd3d9b862076fa7ecc1bf861026e77"} err="failed to get container status \"f5f8a0facf6074a5065c54515662e3a505fd3d9b862076fa7ecc1bf861026e77\": rpc error: code = NotFound desc = could not find container \"f5f8a0facf6074a5065c54515662e3a505fd3d9b862076fa7ecc1bf861026e77\": container with ID starting with f5f8a0facf6074a5065c54515662e3a505fd3d9b862076fa7ecc1bf861026e77 not found: ID does not exist"
Aug 01 00:16:31.493688 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:16:31.493649    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-01c7a-predictor-86b4c76644-xbdpz"]
Aug 01 00:16:31.498405 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:16:31.498378    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-01c7a-predictor-86b4c76644-xbdpz"]
Aug 01 00:16:32.420874 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:16:32.420836    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="50116272-1adb-4354-9026-e97947524575" path="/var/lib/kubelet/pods/50116272-1adb-4354-9026-e97947524575/volumes"
Aug 01 00:16:34.465517 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:16:34.465488    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-d0074-predictor-64b97bfbdb-84kpw"
Aug 01 00:16:34.466144 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:16:34.466112    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-d0074-predictor-64b97bfbdb-84kpw" podUID="54cbebd2-2258-4358-a3f9-0cb5fc27f46c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.28:8080: connect: connection refused"
Aug 01 00:16:44.466325 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:16:44.466289    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-d0074-predictor-64b97bfbdb-84kpw" podUID="54cbebd2-2258-4358-a3f9-0cb5fc27f46c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.28:8080: connect: connection refused"
Aug 01 00:16:54.466304 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:16:54.466263    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-d0074-predictor-64b97bfbdb-84kpw" podUID="54cbebd2-2258-4358-a3f9-0cb5fc27f46c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.28:8080: connect: connection refused"
Aug 01 00:17:04.466161 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:17:04.466114    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-d0074-predictor-64b97bfbdb-84kpw" podUID="54cbebd2-2258-4358-a3f9-0cb5fc27f46c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.28:8080: connect: connection refused"
Aug 01 00:17:14.466052 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:17:14.466009    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-d0074-predictor-64b97bfbdb-84kpw" podUID="54cbebd2-2258-4358-a3f9-0cb5fc27f46c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.28:8080: connect: connection refused"
Aug 01 00:17:17.431263 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:17:17.431223    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-d99eb-predictor-597b578c4-hvl99"]
Aug 01 00:17:17.431767 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:17:17.431502    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-d99eb-predictor-597b578c4-hvl99" podUID="63f601af-2cf4-4dd4-bf90-b7d2e98b97c1" containerName="kserve-container" containerID="cri-o://a6f9e2ac4aaeece12ad0761911af08a95aa51b03f499905eaa4ad35fdeda5adf" gracePeriod=30
Aug 01 00:17:17.431767 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:17:17.431582    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-d99eb-predictor-597b578c4-hvl99" podUID="63f601af-2cf4-4dd4-bf90-b7d2e98b97c1" containerName="kube-rbac-proxy" containerID="cri-o://b47c025516d35a9832c61d240a47fed6e7de3843b8f5d9cc73e0f66f2d97c925" gracePeriod=30
Aug 01 00:17:17.549436 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:17:17.549411    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-bb639-predictor-7bbf8d789b-6tfmv"]
Aug 01 00:17:17.549727 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:17:17.549714    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="50116272-1adb-4354-9026-e97947524575" containerName="kube-rbac-proxy"
Aug 01 00:17:17.549780 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:17:17.549728    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="50116272-1adb-4354-9026-e97947524575" containerName="kube-rbac-proxy"
Aug 01 00:17:17.549780 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:17:17.549744    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="50116272-1adb-4354-9026-e97947524575" containerName="kserve-container"
Aug 01 00:17:17.549780 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:17:17.549750    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="50116272-1adb-4354-9026-e97947524575" containerName="kserve-container"
Aug 01 00:17:17.549931 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:17:17.549806    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="50116272-1adb-4354-9026-e97947524575" containerName="kube-rbac-proxy"
Aug 01 00:17:17.549931 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:17:17.549814    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="50116272-1adb-4354-9026-e97947524575" containerName="kserve-container"
Aug 01 00:17:17.552622 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:17:17.552605    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-bb639-predictor-7bbf8d789b-6tfmv"
Aug 01 00:17:17.555054 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:17:17.555029    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-bb639-predictor-serving-cert\""
Aug 01 00:17:17.555174 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:17:17.555029    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-bb639-kube-rbac-proxy-sar-config\""
Aug 01 00:17:17.561953 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:17:17.561918    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-bb639-predictor-7bbf8d789b-6tfmv"]
Aug 01 00:17:17.603915 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:17:17.603874    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tvgkc\" (UniqueName: \"kubernetes.io/projected/12afa0f2-0926-4e52-a74d-bba84664f6c5-kube-api-access-tvgkc\") pod \"error-404-isvc-bb639-predictor-7bbf8d789b-6tfmv\" (UID: \"12afa0f2-0926-4e52-a74d-bba84664f6c5\") " pod="kserve-ci-e2e-test/error-404-isvc-bb639-predictor-7bbf8d789b-6tfmv"
Aug 01 00:17:17.604111 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:17:17.603930    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/12afa0f2-0926-4e52-a74d-bba84664f6c5-proxy-tls\") pod \"error-404-isvc-bb639-predictor-7bbf8d789b-6tfmv\" (UID: \"12afa0f2-0926-4e52-a74d-bba84664f6c5\") " pod="kserve-ci-e2e-test/error-404-isvc-bb639-predictor-7bbf8d789b-6tfmv"
Aug 01 00:17:17.604111 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:17:17.604022    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/12afa0f2-0926-4e52-a74d-bba84664f6c5-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-bb639-predictor-7bbf8d789b-6tfmv\" (UID: \"12afa0f2-0926-4e52-a74d-bba84664f6c5\") " pod="kserve-ci-e2e-test/error-404-isvc-bb639-predictor-7bbf8d789b-6tfmv"
Aug 01 00:17:17.651844 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:17:17.651811    2564 generic.go:356] "Generic (PLEG): container finished" podID="63f601af-2cf4-4dd4-bf90-b7d2e98b97c1" containerID="b47c025516d35a9832c61d240a47fed6e7de3843b8f5d9cc73e0f66f2d97c925" exitCode=2
Aug 01 00:17:17.652004 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:17:17.651849    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-d99eb-predictor-597b578c4-hvl99" event={"ID":"63f601af-2cf4-4dd4-bf90-b7d2e98b97c1","Type":"ContainerDied","Data":"b47c025516d35a9832c61d240a47fed6e7de3843b8f5d9cc73e0f66f2d97c925"}
Aug 01 00:17:17.705303 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:17:17.705218    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tvgkc\" (UniqueName: \"kubernetes.io/projected/12afa0f2-0926-4e52-a74d-bba84664f6c5-kube-api-access-tvgkc\") pod \"error-404-isvc-bb639-predictor-7bbf8d789b-6tfmv\" (UID: \"12afa0f2-0926-4e52-a74d-bba84664f6c5\") " pod="kserve-ci-e2e-test/error-404-isvc-bb639-predictor-7bbf8d789b-6tfmv"
Aug 01 00:17:17.705303 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:17:17.705264    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/12afa0f2-0926-4e52-a74d-bba84664f6c5-proxy-tls\") pod \"error-404-isvc-bb639-predictor-7bbf8d789b-6tfmv\" (UID: \"12afa0f2-0926-4e52-a74d-bba84664f6c5\") " pod="kserve-ci-e2e-test/error-404-isvc-bb639-predictor-7bbf8d789b-6tfmv"
Aug 01 00:17:17.705303 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:17:17.705294    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/12afa0f2-0926-4e52-a74d-bba84664f6c5-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-bb639-predictor-7bbf8d789b-6tfmv\" (UID: \"12afa0f2-0926-4e52-a74d-bba84664f6c5\") " pod="kserve-ci-e2e-test/error-404-isvc-bb639-predictor-7bbf8d789b-6tfmv"
Aug 01 00:17:17.705527 ip-10-0-128-75 kubenswrapper[2564]: E0801 00:17:17.705416    2564 secret.go:189] Couldn't get secret kserve-ci-e2e-test/error-404-isvc-bb639-predictor-serving-cert: secret "error-404-isvc-bb639-predictor-serving-cert" not found
Aug 01 00:17:17.705527 ip-10-0-128-75 kubenswrapper[2564]: E0801 00:17:17.705482    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/12afa0f2-0926-4e52-a74d-bba84664f6c5-proxy-tls podName:12afa0f2-0926-4e52-a74d-bba84664f6c5 nodeName:}" failed. No retries permitted until 2026-08-01 00:17:18.205464994 +0000 UTC m=+1856.317592646 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/12afa0f2-0926-4e52-a74d-bba84664f6c5-proxy-tls") pod "error-404-isvc-bb639-predictor-7bbf8d789b-6tfmv" (UID: "12afa0f2-0926-4e52-a74d-bba84664f6c5") : secret "error-404-isvc-bb639-predictor-serving-cert" not found
Aug 01 00:17:17.705976 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:17:17.705959    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/12afa0f2-0926-4e52-a74d-bba84664f6c5-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-bb639-predictor-7bbf8d789b-6tfmv\" (UID: \"12afa0f2-0926-4e52-a74d-bba84664f6c5\") " pod="kserve-ci-e2e-test/error-404-isvc-bb639-predictor-7bbf8d789b-6tfmv"
Aug 01 00:17:17.713320 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:17:17.713284    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tvgkc\" (UniqueName: \"kubernetes.io/projected/12afa0f2-0926-4e52-a74d-bba84664f6c5-kube-api-access-tvgkc\") pod \"error-404-isvc-bb639-predictor-7bbf8d789b-6tfmv\" (UID: \"12afa0f2-0926-4e52-a74d-bba84664f6c5\") " pod="kserve-ci-e2e-test/error-404-isvc-bb639-predictor-7bbf8d789b-6tfmv"
Aug 01 00:17:18.209823 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:17:18.209777    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/12afa0f2-0926-4e52-a74d-bba84664f6c5-proxy-tls\") pod \"error-404-isvc-bb639-predictor-7bbf8d789b-6tfmv\" (UID: \"12afa0f2-0926-4e52-a74d-bba84664f6c5\") " pod="kserve-ci-e2e-test/error-404-isvc-bb639-predictor-7bbf8d789b-6tfmv"
Aug 01 00:17:18.212234 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:17:18.212201    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/12afa0f2-0926-4e52-a74d-bba84664f6c5-proxy-tls\") pod \"error-404-isvc-bb639-predictor-7bbf8d789b-6tfmv\" (UID: \"12afa0f2-0926-4e52-a74d-bba84664f6c5\") " pod="kserve-ci-e2e-test/error-404-isvc-bb639-predictor-7bbf8d789b-6tfmv"
Aug 01 00:17:18.463197 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:17:18.463099    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-bb639-predictor-7bbf8d789b-6tfmv"
Aug 01 00:17:18.597399 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:17:18.597368    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-bb639-predictor-7bbf8d789b-6tfmv"]
Aug 01 00:17:18.599933 ip-10-0-128-75 kubenswrapper[2564]: W0801 00:17:18.599890    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12afa0f2_0926_4e52_a74d_bba84664f6c5.slice/crio-4a246787baaa4460223234c4f6c8986586ab307ffd334011df4e684583199355 WatchSource:0}: Error finding container 4a246787baaa4460223234c4f6c8986586ab307ffd334011df4e684583199355: Status 404 returned error can't find the container with id 4a246787baaa4460223234c4f6c8986586ab307ffd334011df4e684583199355
Aug 01 00:17:18.657693 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:17:18.657666    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-bb639-predictor-7bbf8d789b-6tfmv" event={"ID":"12afa0f2-0926-4e52-a74d-bba84664f6c5","Type":"ContainerStarted","Data":"4a246787baaa4460223234c4f6c8986586ab307ffd334011df4e684583199355"}
Aug 01 00:17:19.663701 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:17:19.663659    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-bb639-predictor-7bbf8d789b-6tfmv" event={"ID":"12afa0f2-0926-4e52-a74d-bba84664f6c5","Type":"ContainerStarted","Data":"dac9c5a3de64a7d5753a8bf7b74440ca09b46c6440ba1967094d1d54c1e5fa2b"}
Aug 01 00:17:19.664176 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:17:19.663733    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-bb639-predictor-7bbf8d789b-6tfmv" event={"ID":"12afa0f2-0926-4e52-a74d-bba84664f6c5","Type":"ContainerStarted","Data":"572c16a74ccbefc29ad0475b05c04ca8a7031e749eb13d8cbc297f9ef1a46090"}
Aug 01 00:17:19.664176 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:17:19.663790    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-bb639-predictor-7bbf8d789b-6tfmv"
Aug 01 00:17:19.682499 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:17:19.682448    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-bb639-predictor-7bbf8d789b-6tfmv" podStartSLOduration=2.682429099 podStartE2EDuration="2.682429099s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-01 00:17:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-01 00:17:19.680847654 +0000 UTC m=+1857.792975331" watchObservedRunningTime="2026-08-01 00:17:19.682429099 +0000 UTC m=+1857.794556772"
Aug 01 00:17:20.537100 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:17:20.537057    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-d99eb-predictor-597b578c4-hvl99" podUID="63f601af-2cf4-4dd4-bf90-b7d2e98b97c1" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.27:8643/healthz\": dial tcp 10.134.0.27:8643: connect: connection refused"
Aug 01 00:17:20.541457 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:17:20.541425    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-d99eb-predictor-597b578c4-hvl99" podUID="63f601af-2cf4-4dd4-bf90-b7d2e98b97c1" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.27:8080: connect: connection refused"
Aug 01 00:17:20.668160 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:17:20.668126    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-bb639-predictor-7bbf8d789b-6tfmv"
Aug 01 00:17:20.669512 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:17:20.669481    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-bb639-predictor-7bbf8d789b-6tfmv" podUID="12afa0f2-0926-4e52-a74d-bba84664f6c5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.29:8080: connect: connection refused"
Aug 01 00:17:20.882532 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:17:20.882503    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-d99eb-predictor-597b578c4-hvl99"
Aug 01 00:17:20.934572 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:17:20.934534    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/63f601af-2cf4-4dd4-bf90-b7d2e98b97c1-kube-rbac-proxy-sar-config\") pod \"63f601af-2cf4-4dd4-bf90-b7d2e98b97c1\" (UID: \"63f601af-2cf4-4dd4-bf90-b7d2e98b97c1\") "
Aug 01 00:17:20.934771 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:17:20.934658    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2498b\" (UniqueName: \"kubernetes.io/projected/63f601af-2cf4-4dd4-bf90-b7d2e98b97c1-kube-api-access-2498b\") pod \"63f601af-2cf4-4dd4-bf90-b7d2e98b97c1\" (UID: \"63f601af-2cf4-4dd4-bf90-b7d2e98b97c1\") "
Aug 01 00:17:20.934771 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:17:20.934686    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/63f601af-2cf4-4dd4-bf90-b7d2e98b97c1-proxy-tls\") pod \"63f601af-2cf4-4dd4-bf90-b7d2e98b97c1\" (UID: \"63f601af-2cf4-4dd4-bf90-b7d2e98b97c1\") "
Aug 01 00:17:20.934999 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:17:20.934980    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/63f601af-2cf4-4dd4-bf90-b7d2e98b97c1-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "63f601af-2cf4-4dd4-bf90-b7d2e98b97c1" (UID: "63f601af-2cf4-4dd4-bf90-b7d2e98b97c1"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 01 00:17:20.936845 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:17:20.936808    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/63f601af-2cf4-4dd4-bf90-b7d2e98b97c1-kube-api-access-2498b" (OuterVolumeSpecName: "kube-api-access-2498b") pod "63f601af-2cf4-4dd4-bf90-b7d2e98b97c1" (UID: "63f601af-2cf4-4dd4-bf90-b7d2e98b97c1"). InnerVolumeSpecName "kube-api-access-2498b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 01 00:17:20.936967 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:17:20.936891    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/63f601af-2cf4-4dd4-bf90-b7d2e98b97c1-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "63f601af-2cf4-4dd4-bf90-b7d2e98b97c1" (UID: "63f601af-2cf4-4dd4-bf90-b7d2e98b97c1"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 01 00:17:21.035459 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:17:21.035418    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2498b\" (UniqueName: \"kubernetes.io/projected/63f601af-2cf4-4dd4-bf90-b7d2e98b97c1-kube-api-access-2498b\") on node \"ip-10-0-128-75.ec2.internal\" DevicePath \"\""
Aug 01 00:17:21.035459 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:17:21.035451    2564 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/63f601af-2cf4-4dd4-bf90-b7d2e98b97c1-proxy-tls\") on node \"ip-10-0-128-75.ec2.internal\" DevicePath \"\""
Aug 01 00:17:21.035459 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:17:21.035463    2564 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/63f601af-2cf4-4dd4-bf90-b7d2e98b97c1-kube-rbac-proxy-sar-config\") on node \"ip-10-0-128-75.ec2.internal\" DevicePath \"\""
Aug 01 00:17:21.672522 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:17:21.672486    2564 generic.go:356] "Generic (PLEG): container finished" podID="63f601af-2cf4-4dd4-bf90-b7d2e98b97c1" containerID="a6f9e2ac4aaeece12ad0761911af08a95aa51b03f499905eaa4ad35fdeda5adf" exitCode=0
Aug 01 00:17:21.672987 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:17:21.672535    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-d99eb-predictor-597b578c4-hvl99"
Aug 01 00:17:21.672987 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:17:21.672569    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-d99eb-predictor-597b578c4-hvl99" event={"ID":"63f601af-2cf4-4dd4-bf90-b7d2e98b97c1","Type":"ContainerDied","Data":"a6f9e2ac4aaeece12ad0761911af08a95aa51b03f499905eaa4ad35fdeda5adf"}
Aug 01 00:17:21.672987 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:17:21.672622    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-d99eb-predictor-597b578c4-hvl99" event={"ID":"63f601af-2cf4-4dd4-bf90-b7d2e98b97c1","Type":"ContainerDied","Data":"eb331105dcba363e41b1aaffe00d1c6b2d20c7eca403385c7a055d54ac8ce022"}
Aug 01 00:17:21.672987 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:17:21.672657    2564 scope.go:117] "RemoveContainer" containerID="b47c025516d35a9832c61d240a47fed6e7de3843b8f5d9cc73e0f66f2d97c925"
Aug 01 00:17:21.673217 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:17:21.673190    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-bb639-predictor-7bbf8d789b-6tfmv" podUID="12afa0f2-0926-4e52-a74d-bba84664f6c5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.29:8080: connect: connection refused"
Aug 01 00:17:21.680924 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:17:21.680823    2564 scope.go:117] "RemoveContainer" containerID="a6f9e2ac4aaeece12ad0761911af08a95aa51b03f499905eaa4ad35fdeda5adf"
Aug 01 00:17:21.688774 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:17:21.688702    2564 scope.go:117] "RemoveContainer" containerID="b47c025516d35a9832c61d240a47fed6e7de3843b8f5d9cc73e0f66f2d97c925"
Aug 01 00:17:21.689042 ip-10-0-128-75 kubenswrapper[2564]: E0801 00:17:21.689023    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b47c025516d35a9832c61d240a47fed6e7de3843b8f5d9cc73e0f66f2d97c925\": container with ID starting with b47c025516d35a9832c61d240a47fed6e7de3843b8f5d9cc73e0f66f2d97c925 not found: ID does not exist" containerID="b47c025516d35a9832c61d240a47fed6e7de3843b8f5d9cc73e0f66f2d97c925"
Aug 01 00:17:21.689085 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:17:21.689051    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b47c025516d35a9832c61d240a47fed6e7de3843b8f5d9cc73e0f66f2d97c925"} err="failed to get container status \"b47c025516d35a9832c61d240a47fed6e7de3843b8f5d9cc73e0f66f2d97c925\": rpc error: code = NotFound desc = could not find container \"b47c025516d35a9832c61d240a47fed6e7de3843b8f5d9cc73e0f66f2d97c925\": container with ID starting with b47c025516d35a9832c61d240a47fed6e7de3843b8f5d9cc73e0f66f2d97c925 not found: ID does not exist"
Aug 01 00:17:21.689085 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:17:21.689070    2564 scope.go:117] "RemoveContainer" containerID="a6f9e2ac4aaeece12ad0761911af08a95aa51b03f499905eaa4ad35fdeda5adf"
Aug 01 00:17:21.689314 ip-10-0-128-75 kubenswrapper[2564]: E0801 00:17:21.689296    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a6f9e2ac4aaeece12ad0761911af08a95aa51b03f499905eaa4ad35fdeda5adf\": container with ID starting with a6f9e2ac4aaeece12ad0761911af08a95aa51b03f499905eaa4ad35fdeda5adf not found: ID does not exist" containerID="a6f9e2ac4aaeece12ad0761911af08a95aa51b03f499905eaa4ad35fdeda5adf"
Aug 01 00:17:21.689360 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:17:21.689320    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a6f9e2ac4aaeece12ad0761911af08a95aa51b03f499905eaa4ad35fdeda5adf"} err="failed to get container status \"a6f9e2ac4aaeece12ad0761911af08a95aa51b03f499905eaa4ad35fdeda5adf\": rpc error: code = NotFound desc = could not find container \"a6f9e2ac4aaeece12ad0761911af08a95aa51b03f499905eaa4ad35fdeda5adf\": container with ID starting with a6f9e2ac4aaeece12ad0761911af08a95aa51b03f499905eaa4ad35fdeda5adf not found: ID does not exist"
Aug 01 00:17:21.693998 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:17:21.693973    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-d99eb-predictor-597b578c4-hvl99"]
Aug 01 00:17:21.697414 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:17:21.697391    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-d99eb-predictor-597b578c4-hvl99"]
Aug 01 00:17:22.422098 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:17:22.422062    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="63f601af-2cf4-4dd4-bf90-b7d2e98b97c1" path="/var/lib/kubelet/pods/63f601af-2cf4-4dd4-bf90-b7d2e98b97c1/volumes"
Aug 01 00:17:24.466769 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:17:24.466743    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-d0074-predictor-64b97bfbdb-84kpw"
Aug 01 00:17:26.677145 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:17:26.677117    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-bb639-predictor-7bbf8d789b-6tfmv"
Aug 01 00:17:26.677783 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:17:26.677756    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-bb639-predictor-7bbf8d789b-6tfmv" podUID="12afa0f2-0926-4e52-a74d-bba84664f6c5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.29:8080: connect: connection refused"
Aug 01 00:17:36.678183 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:17:36.678141    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-bb639-predictor-7bbf8d789b-6tfmv" podUID="12afa0f2-0926-4e52-a74d-bba84664f6c5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.29:8080: connect: connection refused"
Aug 01 00:17:38.021015 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:17:38.020975    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-d0074-predictor-64b97bfbdb-84kpw"]
Aug 01 00:17:38.021500 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:17:38.021260    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-d0074-predictor-64b97bfbdb-84kpw" podUID="54cbebd2-2258-4358-a3f9-0cb5fc27f46c" containerName="kserve-container" containerID="cri-o://8555659eabadbc61c6fb3c6dc1f03a4d20213f65f187a118fdac6e49c5353896" gracePeriod=30
Aug 01 00:17:38.021500 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:17:38.021317    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-d0074-predictor-64b97bfbdb-84kpw" podUID="54cbebd2-2258-4358-a3f9-0cb5fc27f46c" containerName="kube-rbac-proxy" containerID="cri-o://6c43bc16433b164718994da13d2b16dfb7f9c58ee957979e860475a19fe55ca3" gracePeriod=30
Aug 01 00:17:38.138445 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:17:38.138413    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-2f013-predictor-6456f4cb4-hjp52"]
Aug 01 00:17:38.138802 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:17:38.138785    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="63f601af-2cf4-4dd4-bf90-b7d2e98b97c1" containerName="kserve-container"
Aug 01 00:17:38.138893 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:17:38.138804    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="63f601af-2cf4-4dd4-bf90-b7d2e98b97c1" containerName="kserve-container"
Aug 01 00:17:38.138893 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:17:38.138837    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="63f601af-2cf4-4dd4-bf90-b7d2e98b97c1" containerName="kube-rbac-proxy"
Aug 01 00:17:38.138893 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:17:38.138847    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="63f601af-2cf4-4dd4-bf90-b7d2e98b97c1" containerName="kube-rbac-proxy"
Aug 01 00:17:38.139054 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:17:38.138911    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="63f601af-2cf4-4dd4-bf90-b7d2e98b97c1" containerName="kube-rbac-proxy"
Aug 01 00:17:38.139054 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:17:38.138925    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="63f601af-2cf4-4dd4-bf90-b7d2e98b97c1" containerName="kserve-container"
Aug 01 00:17:38.142032 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:17:38.142010    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-2f013-predictor-6456f4cb4-hjp52"
Aug 01 00:17:38.144803 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:17:38.144783    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-2f013-predictor-serving-cert\""
Aug 01 00:17:38.144920 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:17:38.144841    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-2f013-kube-rbac-proxy-sar-config\""
Aug 01 00:17:38.151242 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:17:38.151215    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-2f013-predictor-6456f4cb4-hjp52"]
Aug 01 00:17:38.287903 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:17:38.287814    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/bbd728e2-bb32-4672-961a-930b5d3c4a6d-proxy-tls\") pod \"error-404-isvc-2f013-predictor-6456f4cb4-hjp52\" (UID: \"bbd728e2-bb32-4672-961a-930b5d3c4a6d\") " pod="kserve-ci-e2e-test/error-404-isvc-2f013-predictor-6456f4cb4-hjp52"
Aug 01 00:17:38.287903 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:17:38.287861    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m57sr\" (UniqueName: \"kubernetes.io/projected/bbd728e2-bb32-4672-961a-930b5d3c4a6d-kube-api-access-m57sr\") pod \"error-404-isvc-2f013-predictor-6456f4cb4-hjp52\" (UID: \"bbd728e2-bb32-4672-961a-930b5d3c4a6d\") " pod="kserve-ci-e2e-test/error-404-isvc-2f013-predictor-6456f4cb4-hjp52"
Aug 01 00:17:38.287903 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:17:38.287879    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/bbd728e2-bb32-4672-961a-930b5d3c4a6d-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-2f013-predictor-6456f4cb4-hjp52\" (UID: \"bbd728e2-bb32-4672-961a-930b5d3c4a6d\") " pod="kserve-ci-e2e-test/error-404-isvc-2f013-predictor-6456f4cb4-hjp52"
Aug 01 00:17:38.388805 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:17:38.388766    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m57sr\" (UniqueName: \"kubernetes.io/projected/bbd728e2-bb32-4672-961a-930b5d3c4a6d-kube-api-access-m57sr\") pod \"error-404-isvc-2f013-predictor-6456f4cb4-hjp52\" (UID: \"bbd728e2-bb32-4672-961a-930b5d3c4a6d\") " pod="kserve-ci-e2e-test/error-404-isvc-2f013-predictor-6456f4cb4-hjp52"
Aug 01 00:17:38.388805 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:17:38.388803    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/bbd728e2-bb32-4672-961a-930b5d3c4a6d-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-2f013-predictor-6456f4cb4-hjp52\" (UID: \"bbd728e2-bb32-4672-961a-930b5d3c4a6d\") " pod="kserve-ci-e2e-test/error-404-isvc-2f013-predictor-6456f4cb4-hjp52"
Aug 01 00:17:38.389056 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:17:38.388868    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/bbd728e2-bb32-4672-961a-930b5d3c4a6d-proxy-tls\") pod \"error-404-isvc-2f013-predictor-6456f4cb4-hjp52\" (UID: \"bbd728e2-bb32-4672-961a-930b5d3c4a6d\") " pod="kserve-ci-e2e-test/error-404-isvc-2f013-predictor-6456f4cb4-hjp52"
Aug 01 00:17:38.389546 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:17:38.389525    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/bbd728e2-bb32-4672-961a-930b5d3c4a6d-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-2f013-predictor-6456f4cb4-hjp52\" (UID: \"bbd728e2-bb32-4672-961a-930b5d3c4a6d\") " pod="kserve-ci-e2e-test/error-404-isvc-2f013-predictor-6456f4cb4-hjp52"
Aug 01 00:17:38.391238 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:17:38.391221    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/bbd728e2-bb32-4672-961a-930b5d3c4a6d-proxy-tls\") pod \"error-404-isvc-2f013-predictor-6456f4cb4-hjp52\" (UID: \"bbd728e2-bb32-4672-961a-930b5d3c4a6d\") " pod="kserve-ci-e2e-test/error-404-isvc-2f013-predictor-6456f4cb4-hjp52"
Aug 01 00:17:38.397491 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:17:38.397469    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m57sr\" (UniqueName: \"kubernetes.io/projected/bbd728e2-bb32-4672-961a-930b5d3c4a6d-kube-api-access-m57sr\") pod \"error-404-isvc-2f013-predictor-6456f4cb4-hjp52\" (UID: \"bbd728e2-bb32-4672-961a-930b5d3c4a6d\") " pod="kserve-ci-e2e-test/error-404-isvc-2f013-predictor-6456f4cb4-hjp52"
Aug 01 00:17:38.453418 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:17:38.453373    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-2f013-predictor-6456f4cb4-hjp52"
Aug 01 00:17:38.590905 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:17:38.590787    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-2f013-predictor-6456f4cb4-hjp52"]
Aug 01 00:17:38.593476 ip-10-0-128-75 kubenswrapper[2564]: W0801 00:17:38.593446    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbbd728e2_bb32_4672_961a_930b5d3c4a6d.slice/crio-200c842e82ab4edab6d06d1ac1c9e11919cc8b68ba924ee6b0c69327e2841646 WatchSource:0}: Error finding container 200c842e82ab4edab6d06d1ac1c9e11919cc8b68ba924ee6b0c69327e2841646: Status 404 returned error can't find the container with id 200c842e82ab4edab6d06d1ac1c9e11919cc8b68ba924ee6b0c69327e2841646
Aug 01 00:17:38.742057 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:17:38.742009    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-2f013-predictor-6456f4cb4-hjp52" event={"ID":"bbd728e2-bb32-4672-961a-930b5d3c4a6d","Type":"ContainerStarted","Data":"9d3590726654e81345b7e7e882a0bdf35e929076e99df1b097ea0b678169df64"}
Aug 01 00:17:38.742253 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:17:38.742084    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-2f013-predictor-6456f4cb4-hjp52"
Aug 01 00:17:38.742253 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:17:38.742113    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-2f013-predictor-6456f4cb4-hjp52" event={"ID":"bbd728e2-bb32-4672-961a-930b5d3c4a6d","Type":"ContainerStarted","Data":"277750b13c0c5aa0c01716a456607809e6c49a9b1856314f5e09e6551eb1f0bf"}
Aug 01 00:17:38.742253 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:17:38.742137    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-2f013-predictor-6456f4cb4-hjp52" event={"ID":"bbd728e2-bb32-4672-961a-930b5d3c4a6d","Type":"ContainerStarted","Data":"200c842e82ab4edab6d06d1ac1c9e11919cc8b68ba924ee6b0c69327e2841646"}
Aug 01 00:17:38.743783 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:17:38.743755    2564 generic.go:356] "Generic (PLEG): container finished" podID="54cbebd2-2258-4358-a3f9-0cb5fc27f46c" containerID="6c43bc16433b164718994da13d2b16dfb7f9c58ee957979e860475a19fe55ca3" exitCode=2
Aug 01 00:17:38.743895 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:17:38.743817    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-d0074-predictor-64b97bfbdb-84kpw" event={"ID":"54cbebd2-2258-4358-a3f9-0cb5fc27f46c","Type":"ContainerDied","Data":"6c43bc16433b164718994da13d2b16dfb7f9c58ee957979e860475a19fe55ca3"}
Aug 01 00:17:38.758541 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:17:38.758487    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-2f013-predictor-6456f4cb4-hjp52" podStartSLOduration=0.758467816 podStartE2EDuration="758.467816ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-01 00:17:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-01 00:17:38.757642589 +0000 UTC m=+1876.869770286" watchObservedRunningTime="2026-08-01 00:17:38.758467816 +0000 UTC m=+1876.870595490"
Aug 01 00:17:39.460892 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:17:39.460843    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-d0074-predictor-64b97bfbdb-84kpw" podUID="54cbebd2-2258-4358-a3f9-0cb5fc27f46c" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.28:8643/healthz\": dial tcp 10.134.0.28:8643: connect: connection refused"
Aug 01 00:17:39.748064 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:17:39.747968    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-2f013-predictor-6456f4cb4-hjp52"
Aug 01 00:17:39.749290 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:17:39.749263    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-2f013-predictor-6456f4cb4-hjp52" podUID="bbd728e2-bb32-4672-961a-930b5d3c4a6d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.30:8080: connect: connection refused"
Aug 01 00:17:40.751821 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:17:40.751773    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-2f013-predictor-6456f4cb4-hjp52" podUID="bbd728e2-bb32-4672-961a-930b5d3c4a6d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.30:8080: connect: connection refused"
Aug 01 00:17:41.569146 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:17:41.569118    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-d0074-predictor-64b97bfbdb-84kpw"
Aug 01 00:17:41.717121 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:17:41.717027    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/54cbebd2-2258-4358-a3f9-0cb5fc27f46c-proxy-tls\") pod \"54cbebd2-2258-4358-a3f9-0cb5fc27f46c\" (UID: \"54cbebd2-2258-4358-a3f9-0cb5fc27f46c\") "
Aug 01 00:17:41.717121 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:17:41.717086    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cskpc\" (UniqueName: \"kubernetes.io/projected/54cbebd2-2258-4358-a3f9-0cb5fc27f46c-kube-api-access-cskpc\") pod \"54cbebd2-2258-4358-a3f9-0cb5fc27f46c\" (UID: \"54cbebd2-2258-4358-a3f9-0cb5fc27f46c\") "
Aug 01 00:17:41.717367 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:17:41.717123    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/54cbebd2-2258-4358-a3f9-0cb5fc27f46c-kube-rbac-proxy-sar-config\") pod \"54cbebd2-2258-4358-a3f9-0cb5fc27f46c\" (UID: \"54cbebd2-2258-4358-a3f9-0cb5fc27f46c\") "
Aug 01 00:17:41.717607 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:17:41.717545    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/54cbebd2-2258-4358-a3f9-0cb5fc27f46c-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "54cbebd2-2258-4358-a3f9-0cb5fc27f46c" (UID: "54cbebd2-2258-4358-a3f9-0cb5fc27f46c"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 01 00:17:41.719356 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:17:41.719329    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/54cbebd2-2258-4358-a3f9-0cb5fc27f46c-kube-api-access-cskpc" (OuterVolumeSpecName: "kube-api-access-cskpc") pod "54cbebd2-2258-4358-a3f9-0cb5fc27f46c" (UID: "54cbebd2-2258-4358-a3f9-0cb5fc27f46c"). InnerVolumeSpecName "kube-api-access-cskpc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 01 00:17:41.719591 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:17:41.719573    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/54cbebd2-2258-4358-a3f9-0cb5fc27f46c-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "54cbebd2-2258-4358-a3f9-0cb5fc27f46c" (UID: "54cbebd2-2258-4358-a3f9-0cb5fc27f46c"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 01 00:17:41.756737 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:17:41.756701    2564 generic.go:356] "Generic (PLEG): container finished" podID="54cbebd2-2258-4358-a3f9-0cb5fc27f46c" containerID="8555659eabadbc61c6fb3c6dc1f03a4d20213f65f187a118fdac6e49c5353896" exitCode=0
Aug 01 00:17:41.756737 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:17:41.756743    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-d0074-predictor-64b97bfbdb-84kpw" event={"ID":"54cbebd2-2258-4358-a3f9-0cb5fc27f46c","Type":"ContainerDied","Data":"8555659eabadbc61c6fb3c6dc1f03a4d20213f65f187a118fdac6e49c5353896"}
Aug 01 00:17:41.757195 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:17:41.756753    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-d0074-predictor-64b97bfbdb-84kpw"
Aug 01 00:17:41.757195 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:17:41.756790    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-d0074-predictor-64b97bfbdb-84kpw" event={"ID":"54cbebd2-2258-4358-a3f9-0cb5fc27f46c","Type":"ContainerDied","Data":"b557d939363621b5159a993d5e667eb01dc38c21663526af350c88b28a4a03fc"}
Aug 01 00:17:41.757195 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:17:41.756818    2564 scope.go:117] "RemoveContainer" containerID="6c43bc16433b164718994da13d2b16dfb7f9c58ee957979e860475a19fe55ca3"
Aug 01 00:17:41.765583 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:17:41.765562    2564 scope.go:117] "RemoveContainer" containerID="8555659eabadbc61c6fb3c6dc1f03a4d20213f65f187a118fdac6e49c5353896"
Aug 01 00:17:41.773372 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:17:41.773352    2564 scope.go:117] "RemoveContainer" containerID="6c43bc16433b164718994da13d2b16dfb7f9c58ee957979e860475a19fe55ca3"
Aug 01 00:17:41.773692 ip-10-0-128-75 kubenswrapper[2564]: E0801 00:17:41.773672    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6c43bc16433b164718994da13d2b16dfb7f9c58ee957979e860475a19fe55ca3\": container with ID starting with 6c43bc16433b164718994da13d2b16dfb7f9c58ee957979e860475a19fe55ca3 not found: ID does not exist" containerID="6c43bc16433b164718994da13d2b16dfb7f9c58ee957979e860475a19fe55ca3"
Aug 01 00:17:41.773754 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:17:41.773703    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6c43bc16433b164718994da13d2b16dfb7f9c58ee957979e860475a19fe55ca3"} err="failed to get container status \"6c43bc16433b164718994da13d2b16dfb7f9c58ee957979e860475a19fe55ca3\": rpc error: code = NotFound desc = could not find container \"6c43bc16433b164718994da13d2b16dfb7f9c58ee957979e860475a19fe55ca3\": container with ID starting with 6c43bc16433b164718994da13d2b16dfb7f9c58ee957979e860475a19fe55ca3 not found: ID does not exist"
Aug 01 00:17:41.773754 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:17:41.773720    2564 scope.go:117] "RemoveContainer" containerID="8555659eabadbc61c6fb3c6dc1f03a4d20213f65f187a118fdac6e49c5353896"
Aug 01 00:17:41.773992 ip-10-0-128-75 kubenswrapper[2564]: E0801 00:17:41.773970    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8555659eabadbc61c6fb3c6dc1f03a4d20213f65f187a118fdac6e49c5353896\": container with ID starting with 8555659eabadbc61c6fb3c6dc1f03a4d20213f65f187a118fdac6e49c5353896 not found: ID does not exist" containerID="8555659eabadbc61c6fb3c6dc1f03a4d20213f65f187a118fdac6e49c5353896"
Aug 01 00:17:41.774037 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:17:41.774001    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8555659eabadbc61c6fb3c6dc1f03a4d20213f65f187a118fdac6e49c5353896"} err="failed to get container status \"8555659eabadbc61c6fb3c6dc1f03a4d20213f65f187a118fdac6e49c5353896\": rpc error: code = NotFound desc = could not find container \"8555659eabadbc61c6fb3c6dc1f03a4d20213f65f187a118fdac6e49c5353896\": container with ID starting with 8555659eabadbc61c6fb3c6dc1f03a4d20213f65f187a118fdac6e49c5353896 not found: ID does not exist"
Aug 01 00:17:41.779076 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:17:41.779052    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-d0074-predictor-64b97bfbdb-84kpw"]
Aug 01 00:17:41.782761 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:17:41.782740    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-d0074-predictor-64b97bfbdb-84kpw"]
Aug 01 00:17:41.818488 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:17:41.818450    2564 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/54cbebd2-2258-4358-a3f9-0cb5fc27f46c-kube-rbac-proxy-sar-config\") on node \"ip-10-0-128-75.ec2.internal\" DevicePath \"\""
Aug 01 00:17:41.818488 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:17:41.818474    2564 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/54cbebd2-2258-4358-a3f9-0cb5fc27f46c-proxy-tls\") on node \"ip-10-0-128-75.ec2.internal\" DevicePath \"\""
Aug 01 00:17:41.818488 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:17:41.818485    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cskpc\" (UniqueName: \"kubernetes.io/projected/54cbebd2-2258-4358-a3f9-0cb5fc27f46c-kube-api-access-cskpc\") on node \"ip-10-0-128-75.ec2.internal\" DevicePath \"\""
Aug 01 00:17:42.426036 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:17:42.426003    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="54cbebd2-2258-4358-a3f9-0cb5fc27f46c" path="/var/lib/kubelet/pods/54cbebd2-2258-4358-a3f9-0cb5fc27f46c/volumes"
Aug 01 00:17:45.756880 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:17:45.756809    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-2f013-predictor-6456f4cb4-hjp52"
Aug 01 00:17:45.757308 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:17:45.757283    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-2f013-predictor-6456f4cb4-hjp52" podUID="bbd728e2-bb32-4672-961a-930b5d3c4a6d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.30:8080: connect: connection refused"
Aug 01 00:17:46.678264 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:17:46.678220    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-bb639-predictor-7bbf8d789b-6tfmv" podUID="12afa0f2-0926-4e52-a74d-bba84664f6c5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.29:8080: connect: connection refused"
Aug 01 00:17:55.757373 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:17:55.757332    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-2f013-predictor-6456f4cb4-hjp52" podUID="bbd728e2-bb32-4672-961a-930b5d3c4a6d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.30:8080: connect: connection refused"
Aug 01 00:17:56.678142 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:17:56.678101    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-bb639-predictor-7bbf8d789b-6tfmv" podUID="12afa0f2-0926-4e52-a74d-bba84664f6c5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.29:8080: connect: connection refused"
Aug 01 00:18:05.758179 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:18:05.758136    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-2f013-predictor-6456f4cb4-hjp52" podUID="bbd728e2-bb32-4672-961a-930b5d3c4a6d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.30:8080: connect: connection refused"
Aug 01 00:18:06.678793 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:18:06.678756    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-bb639-predictor-7bbf8d789b-6tfmv"
Aug 01 00:18:15.757859 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:18:15.757815    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-2f013-predictor-6456f4cb4-hjp52" podUID="bbd728e2-bb32-4672-961a-930b5d3c4a6d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.30:8080: connect: connection refused"
Aug 01 00:18:25.757836 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:18:25.757798    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-2f013-predictor-6456f4cb4-hjp52"
Aug 01 00:26:52.953401 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:26:52.953312    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-2f013-predictor-6456f4cb4-hjp52"]
Aug 01 00:26:52.953929 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:26:52.953611    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-2f013-predictor-6456f4cb4-hjp52" podUID="bbd728e2-bb32-4672-961a-930b5d3c4a6d" containerName="kserve-container" containerID="cri-o://277750b13c0c5aa0c01716a456607809e6c49a9b1856314f5e09e6551eb1f0bf" gracePeriod=30
Aug 01 00:26:52.953929 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:26:52.953688    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-2f013-predictor-6456f4cb4-hjp52" podUID="bbd728e2-bb32-4672-961a-930b5d3c4a6d" containerName="kube-rbac-proxy" containerID="cri-o://9d3590726654e81345b7e7e882a0bdf35e929076e99df1b097ea0b678169df64" gracePeriod=30
Aug 01 00:26:53.837682 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:26:53.837621    2564 generic.go:356] "Generic (PLEG): container finished" podID="bbd728e2-bb32-4672-961a-930b5d3c4a6d" containerID="9d3590726654e81345b7e7e882a0bdf35e929076e99df1b097ea0b678169df64" exitCode=2
Aug 01 00:26:53.837867 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:26:53.837698    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-2f013-predictor-6456f4cb4-hjp52" event={"ID":"bbd728e2-bb32-4672-961a-930b5d3c4a6d","Type":"ContainerDied","Data":"9d3590726654e81345b7e7e882a0bdf35e929076e99df1b097ea0b678169df64"}
Aug 01 00:26:55.752406 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:26:55.752355    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-2f013-predictor-6456f4cb4-hjp52" podUID="bbd728e2-bb32-4672-961a-930b5d3c4a6d" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.30:8643/healthz\": dial tcp 10.134.0.30:8643: connect: connection refused"
Aug 01 00:26:55.757604 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:26:55.757577    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-2f013-predictor-6456f4cb4-hjp52" podUID="bbd728e2-bb32-4672-961a-930b5d3c4a6d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.30:8080: connect: connection refused"
Aug 01 00:26:56.295893 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:26:56.295864    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-2f013-predictor-6456f4cb4-hjp52"
Aug 01 00:26:56.334237 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:26:56.334201    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m57sr\" (UniqueName: \"kubernetes.io/projected/bbd728e2-bb32-4672-961a-930b5d3c4a6d-kube-api-access-m57sr\") pod \"bbd728e2-bb32-4672-961a-930b5d3c4a6d\" (UID: \"bbd728e2-bb32-4672-961a-930b5d3c4a6d\") "
Aug 01 00:26:56.334237 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:26:56.334241    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/bbd728e2-bb32-4672-961a-930b5d3c4a6d-kube-rbac-proxy-sar-config\") pod \"bbd728e2-bb32-4672-961a-930b5d3c4a6d\" (UID: \"bbd728e2-bb32-4672-961a-930b5d3c4a6d\") "
Aug 01 00:26:56.334501 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:26:56.334273    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/bbd728e2-bb32-4672-961a-930b5d3c4a6d-proxy-tls\") pod \"bbd728e2-bb32-4672-961a-930b5d3c4a6d\" (UID: \"bbd728e2-bb32-4672-961a-930b5d3c4a6d\") "
Aug 01 00:26:56.334738 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:26:56.334708    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bbd728e2-bb32-4672-961a-930b5d3c4a6d-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "bbd728e2-bb32-4672-961a-930b5d3c4a6d" (UID: "bbd728e2-bb32-4672-961a-930b5d3c4a6d"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 01 00:26:56.336449 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:26:56.336424    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bbd728e2-bb32-4672-961a-930b5d3c4a6d-kube-api-access-m57sr" (OuterVolumeSpecName: "kube-api-access-m57sr") pod "bbd728e2-bb32-4672-961a-930b5d3c4a6d" (UID: "bbd728e2-bb32-4672-961a-930b5d3c4a6d"). InnerVolumeSpecName "kube-api-access-m57sr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 01 00:26:56.336536 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:26:56.336480    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bbd728e2-bb32-4672-961a-930b5d3c4a6d-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "bbd728e2-bb32-4672-961a-930b5d3c4a6d" (UID: "bbd728e2-bb32-4672-961a-930b5d3c4a6d"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 01 00:26:56.435285 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:26:56.435258    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m57sr\" (UniqueName: \"kubernetes.io/projected/bbd728e2-bb32-4672-961a-930b5d3c4a6d-kube-api-access-m57sr\") on node \"ip-10-0-128-75.ec2.internal\" DevicePath \"\""
Aug 01 00:26:56.435285 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:26:56.435282    2564 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/bbd728e2-bb32-4672-961a-930b5d3c4a6d-kube-rbac-proxy-sar-config\") on node \"ip-10-0-128-75.ec2.internal\" DevicePath \"\""
Aug 01 00:26:56.435451 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:26:56.435293    2564 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/bbd728e2-bb32-4672-961a-930b5d3c4a6d-proxy-tls\") on node \"ip-10-0-128-75.ec2.internal\" DevicePath \"\""
Aug 01 00:26:56.851061 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:26:56.851029    2564 generic.go:356] "Generic (PLEG): container finished" podID="bbd728e2-bb32-4672-961a-930b5d3c4a6d" containerID="277750b13c0c5aa0c01716a456607809e6c49a9b1856314f5e09e6551eb1f0bf" exitCode=0
Aug 01 00:26:56.851522 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:26:56.851078    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-2f013-predictor-6456f4cb4-hjp52"
Aug 01 00:26:56.851522 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:26:56.851113    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-2f013-predictor-6456f4cb4-hjp52" event={"ID":"bbd728e2-bb32-4672-961a-930b5d3c4a6d","Type":"ContainerDied","Data":"277750b13c0c5aa0c01716a456607809e6c49a9b1856314f5e09e6551eb1f0bf"}
Aug 01 00:26:56.851522 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:26:56.851191    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-2f013-predictor-6456f4cb4-hjp52" event={"ID":"bbd728e2-bb32-4672-961a-930b5d3c4a6d","Type":"ContainerDied","Data":"200c842e82ab4edab6d06d1ac1c9e11919cc8b68ba924ee6b0c69327e2841646"}
Aug 01 00:26:56.851522 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:26:56.851233    2564 scope.go:117] "RemoveContainer" containerID="9d3590726654e81345b7e7e882a0bdf35e929076e99df1b097ea0b678169df64"
Aug 01 00:26:56.859332 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:26:56.859310    2564 scope.go:117] "RemoveContainer" containerID="277750b13c0c5aa0c01716a456607809e6c49a9b1856314f5e09e6551eb1f0bf"
Aug 01 00:26:56.867033 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:26:56.866996    2564 scope.go:117] "RemoveContainer" containerID="9d3590726654e81345b7e7e882a0bdf35e929076e99df1b097ea0b678169df64"
Aug 01 00:26:56.867328 ip-10-0-128-75 kubenswrapper[2564]: E0801 00:26:56.867308    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9d3590726654e81345b7e7e882a0bdf35e929076e99df1b097ea0b678169df64\": container with ID starting with 9d3590726654e81345b7e7e882a0bdf35e929076e99df1b097ea0b678169df64 not found: ID does not exist" containerID="9d3590726654e81345b7e7e882a0bdf35e929076e99df1b097ea0b678169df64"
Aug 01 00:26:56.867395 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:26:56.867339    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d3590726654e81345b7e7e882a0bdf35e929076e99df1b097ea0b678169df64"} err="failed to get container status \"9d3590726654e81345b7e7e882a0bdf35e929076e99df1b097ea0b678169df64\": rpc error: code = NotFound desc = could not find container \"9d3590726654e81345b7e7e882a0bdf35e929076e99df1b097ea0b678169df64\": container with ID starting with 9d3590726654e81345b7e7e882a0bdf35e929076e99df1b097ea0b678169df64 not found: ID does not exist"
Aug 01 00:26:56.867395 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:26:56.867359    2564 scope.go:117] "RemoveContainer" containerID="277750b13c0c5aa0c01716a456607809e6c49a9b1856314f5e09e6551eb1f0bf"
Aug 01 00:26:56.867659 ip-10-0-128-75 kubenswrapper[2564]: E0801 00:26:56.867617    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"277750b13c0c5aa0c01716a456607809e6c49a9b1856314f5e09e6551eb1f0bf\": container with ID starting with 277750b13c0c5aa0c01716a456607809e6c49a9b1856314f5e09e6551eb1f0bf not found: ID does not exist" containerID="277750b13c0c5aa0c01716a456607809e6c49a9b1856314f5e09e6551eb1f0bf"
Aug 01 00:26:56.867742 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:26:56.867669    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"277750b13c0c5aa0c01716a456607809e6c49a9b1856314f5e09e6551eb1f0bf"} err="failed to get container status \"277750b13c0c5aa0c01716a456607809e6c49a9b1856314f5e09e6551eb1f0bf\": rpc error: code = NotFound desc = could not find container \"277750b13c0c5aa0c01716a456607809e6c49a9b1856314f5e09e6551eb1f0bf\": container with ID starting with 277750b13c0c5aa0c01716a456607809e6c49a9b1856314f5e09e6551eb1f0bf not found: ID does not exist"
Aug 01 00:26:56.868006 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:26:56.867990    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-2f013-predictor-6456f4cb4-hjp52"]
Aug 01 00:26:56.871654 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:26:56.871610    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-2f013-predictor-6456f4cb4-hjp52"]
Aug 01 00:26:58.421479 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:26:58.421440    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bbd728e2-bb32-4672-961a-930b5d3c4a6d" path="/var/lib/kubelet/pods/bbd728e2-bb32-4672-961a-930b5d3c4a6d/volumes"
Aug 01 00:34:37.074724 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:34:37.074643    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-bb639-predictor-7bbf8d789b-6tfmv"]
Aug 01 00:34:37.077255 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:34:37.074909    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-bb639-predictor-7bbf8d789b-6tfmv" podUID="12afa0f2-0926-4e52-a74d-bba84664f6c5" containerName="kserve-container" containerID="cri-o://572c16a74ccbefc29ad0475b05c04ca8a7031e749eb13d8cbc297f9ef1a46090" gracePeriod=30
Aug 01 00:34:37.077255 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:34:37.074950    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-bb639-predictor-7bbf8d789b-6tfmv" podUID="12afa0f2-0926-4e52-a74d-bba84664f6c5" containerName="kube-rbac-proxy" containerID="cri-o://dac9c5a3de64a7d5753a8bf7b74440ca09b46c6440ba1967094d1d54c1e5fa2b" gracePeriod=30
Aug 01 00:34:37.581208 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:34:37.581173    2564 generic.go:356] "Generic (PLEG): container finished" podID="12afa0f2-0926-4e52-a74d-bba84664f6c5" containerID="dac9c5a3de64a7d5753a8bf7b74440ca09b46c6440ba1967094d1d54c1e5fa2b" exitCode=2
Aug 01 00:34:37.581378 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:34:37.581247    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-bb639-predictor-7bbf8d789b-6tfmv" event={"ID":"12afa0f2-0926-4e52-a74d-bba84664f6c5","Type":"ContainerDied","Data":"dac9c5a3de64a7d5753a8bf7b74440ca09b46c6440ba1967094d1d54c1e5fa2b"}
Aug 01 00:34:38.590981 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:34:38.590950    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-lvfql/must-gather-zpmqg"]
Aug 01 00:34:38.591385 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:34:38.591249    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="54cbebd2-2258-4358-a3f9-0cb5fc27f46c" containerName="kserve-container"
Aug 01 00:34:38.591385 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:34:38.591260    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="54cbebd2-2258-4358-a3f9-0cb5fc27f46c" containerName="kserve-container"
Aug 01 00:34:38.591385 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:34:38.591274    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bbd728e2-bb32-4672-961a-930b5d3c4a6d" containerName="kube-rbac-proxy"
Aug 01 00:34:38.591385 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:34:38.591280    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="bbd728e2-bb32-4672-961a-930b5d3c4a6d" containerName="kube-rbac-proxy"
Aug 01 00:34:38.591385 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:34:38.591287    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="54cbebd2-2258-4358-a3f9-0cb5fc27f46c" containerName="kube-rbac-proxy"
Aug 01 00:34:38.591385 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:34:38.591292    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="54cbebd2-2258-4358-a3f9-0cb5fc27f46c" containerName="kube-rbac-proxy"
Aug 01 00:34:38.591385 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:34:38.591299    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bbd728e2-bb32-4672-961a-930b5d3c4a6d" containerName="kserve-container"
Aug 01 00:34:38.591385 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:34:38.591304    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="bbd728e2-bb32-4672-961a-930b5d3c4a6d" containerName="kserve-container"
Aug 01 00:34:38.591385 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:34:38.591357    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="54cbebd2-2258-4358-a3f9-0cb5fc27f46c" containerName="kube-rbac-proxy"
Aug 01 00:34:38.591385 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:34:38.591363    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="54cbebd2-2258-4358-a3f9-0cb5fc27f46c" containerName="kserve-container"
Aug 01 00:34:38.591385 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:34:38.591368    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="bbd728e2-bb32-4672-961a-930b5d3c4a6d" containerName="kserve-container"
Aug 01 00:34:38.591385 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:34:38.591376    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="bbd728e2-bb32-4672-961a-930b5d3c4a6d" containerName="kube-rbac-proxy"
Aug 01 00:34:38.594183 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:34:38.594166    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-lvfql/must-gather-zpmqg"
Aug 01 00:34:38.597115 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:34:38.597090    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-lvfql\"/\"default-dockercfg-djgxw\""
Aug 01 00:34:38.597273 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:34:38.597090    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-lvfql\"/\"kube-root-ca.crt\""
Aug 01 00:34:38.597273 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:34:38.597151    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-lvfql\"/\"openshift-service-ca.crt\""
Aug 01 00:34:38.602539 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:34:38.602513    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-lvfql/must-gather-zpmqg"]
Aug 01 00:34:38.721903 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:34:38.721871    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/d348aa25-45c0-49fa-ad49-d0a429ba68ef-must-gather-output\") pod \"must-gather-zpmqg\" (UID: \"d348aa25-45c0-49fa-ad49-d0a429ba68ef\") " pod="openshift-must-gather-lvfql/must-gather-zpmqg"
Aug 01 00:34:38.721903 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:34:38.721911    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p6vs2\" (UniqueName: \"kubernetes.io/projected/d348aa25-45c0-49fa-ad49-d0a429ba68ef-kube-api-access-p6vs2\") pod \"must-gather-zpmqg\" (UID: \"d348aa25-45c0-49fa-ad49-d0a429ba68ef\") " pod="openshift-must-gather-lvfql/must-gather-zpmqg"
Aug 01 00:34:38.823275 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:34:38.823233    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/d348aa25-45c0-49fa-ad49-d0a429ba68ef-must-gather-output\") pod \"must-gather-zpmqg\" (UID: \"d348aa25-45c0-49fa-ad49-d0a429ba68ef\") " pod="openshift-must-gather-lvfql/must-gather-zpmqg"
Aug 01 00:34:38.823454 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:34:38.823283    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p6vs2\" (UniqueName: \"kubernetes.io/projected/d348aa25-45c0-49fa-ad49-d0a429ba68ef-kube-api-access-p6vs2\") pod \"must-gather-zpmqg\" (UID: \"d348aa25-45c0-49fa-ad49-d0a429ba68ef\") " pod="openshift-must-gather-lvfql/must-gather-zpmqg"
Aug 01 00:34:38.823648 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:34:38.823604    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/d348aa25-45c0-49fa-ad49-d0a429ba68ef-must-gather-output\") pod \"must-gather-zpmqg\" (UID: \"d348aa25-45c0-49fa-ad49-d0a429ba68ef\") " pod="openshift-must-gather-lvfql/must-gather-zpmqg"
Aug 01 00:34:38.831704 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:34:38.831677    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p6vs2\" (UniqueName: \"kubernetes.io/projected/d348aa25-45c0-49fa-ad49-d0a429ba68ef-kube-api-access-p6vs2\") pod \"must-gather-zpmqg\" (UID: \"d348aa25-45c0-49fa-ad49-d0a429ba68ef\") " pod="openshift-must-gather-lvfql/must-gather-zpmqg"
Aug 01 00:34:38.921022 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:34:38.920977    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-lvfql/must-gather-zpmqg"
Aug 01 00:34:39.056522 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:34:39.056497    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-lvfql/must-gather-zpmqg"]
Aug 01 00:34:39.058920 ip-10-0-128-75 kubenswrapper[2564]: W0801 00:34:39.058891    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd348aa25_45c0_49fa_ad49_d0a429ba68ef.slice/crio-6cac774c48cc2bacded94cb83ac49c43ee94266b844c7fae23d06157a4040708 WatchSource:0}: Error finding container 6cac774c48cc2bacded94cb83ac49c43ee94266b844c7fae23d06157a4040708: Status 404 returned error can't find the container with id 6cac774c48cc2bacded94cb83ac49c43ee94266b844c7fae23d06157a4040708
Aug 01 00:34:39.060745 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:34:39.060728    2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Aug 01 00:34:39.589813 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:34:39.589780    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-lvfql/must-gather-zpmqg" event={"ID":"d348aa25-45c0-49fa-ad49-d0a429ba68ef","Type":"ContainerStarted","Data":"6cac774c48cc2bacded94cb83ac49c43ee94266b844c7fae23d06157a4040708"}
Aug 01 00:34:40.599103 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:34:40.599066    2564 generic.go:356] "Generic (PLEG): container finished" podID="12afa0f2-0926-4e52-a74d-bba84664f6c5" containerID="572c16a74ccbefc29ad0475b05c04ca8a7031e749eb13d8cbc297f9ef1a46090" exitCode=0
Aug 01 00:34:40.599501 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:34:40.599166    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-bb639-predictor-7bbf8d789b-6tfmv" event={"ID":"12afa0f2-0926-4e52-a74d-bba84664f6c5","Type":"ContainerDied","Data":"572c16a74ccbefc29ad0475b05c04ca8a7031e749eb13d8cbc297f9ef1a46090"}
Aug 01 00:34:40.664365 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:34:40.664333    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-bb639-predictor-7bbf8d789b-6tfmv"
Aug 01 00:34:40.839957 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:34:40.839872    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tvgkc\" (UniqueName: \"kubernetes.io/projected/12afa0f2-0926-4e52-a74d-bba84664f6c5-kube-api-access-tvgkc\") pod \"12afa0f2-0926-4e52-a74d-bba84664f6c5\" (UID: \"12afa0f2-0926-4e52-a74d-bba84664f6c5\") "
Aug 01 00:34:40.839957 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:34:40.839922    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/12afa0f2-0926-4e52-a74d-bba84664f6c5-proxy-tls\") pod \"12afa0f2-0926-4e52-a74d-bba84664f6c5\" (UID: \"12afa0f2-0926-4e52-a74d-bba84664f6c5\") "
Aug 01 00:34:40.840191 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:34:40.840003    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/12afa0f2-0926-4e52-a74d-bba84664f6c5-kube-rbac-proxy-sar-config\") pod \"12afa0f2-0926-4e52-a74d-bba84664f6c5\" (UID: \"12afa0f2-0926-4e52-a74d-bba84664f6c5\") "
Aug 01 00:34:40.840413 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:34:40.840369    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/12afa0f2-0926-4e52-a74d-bba84664f6c5-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "12afa0f2-0926-4e52-a74d-bba84664f6c5" (UID: "12afa0f2-0926-4e52-a74d-bba84664f6c5"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 01 00:34:40.843180 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:34:40.843118    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/12afa0f2-0926-4e52-a74d-bba84664f6c5-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "12afa0f2-0926-4e52-a74d-bba84664f6c5" (UID: "12afa0f2-0926-4e52-a74d-bba84664f6c5"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 01 00:34:40.844473 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:34:40.844434    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/12afa0f2-0926-4e52-a74d-bba84664f6c5-kube-api-access-tvgkc" (OuterVolumeSpecName: "kube-api-access-tvgkc") pod "12afa0f2-0926-4e52-a74d-bba84664f6c5" (UID: "12afa0f2-0926-4e52-a74d-bba84664f6c5"). InnerVolumeSpecName "kube-api-access-tvgkc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 01 00:34:40.940883 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:34:40.940842    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tvgkc\" (UniqueName: \"kubernetes.io/projected/12afa0f2-0926-4e52-a74d-bba84664f6c5-kube-api-access-tvgkc\") on node \"ip-10-0-128-75.ec2.internal\" DevicePath \"\""
Aug 01 00:34:40.940883 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:34:40.940878    2564 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/12afa0f2-0926-4e52-a74d-bba84664f6c5-proxy-tls\") on node \"ip-10-0-128-75.ec2.internal\" DevicePath \"\""
Aug 01 00:34:40.940883 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:34:40.940891    2564 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/12afa0f2-0926-4e52-a74d-bba84664f6c5-kube-rbac-proxy-sar-config\") on node \"ip-10-0-128-75.ec2.internal\" DevicePath \"\""
Aug 01 00:34:41.606415 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:34:41.606373    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-bb639-predictor-7bbf8d789b-6tfmv" event={"ID":"12afa0f2-0926-4e52-a74d-bba84664f6c5","Type":"ContainerDied","Data":"4a246787baaa4460223234c4f6c8986586ab307ffd334011df4e684583199355"}
Aug 01 00:34:41.607419 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:34:41.607012    2564 scope.go:117] "RemoveContainer" containerID="dac9c5a3de64a7d5753a8bf7b74440ca09b46c6440ba1967094d1d54c1e5fa2b"
Aug 01 00:34:41.607419 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:34:41.607238    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-bb639-predictor-7bbf8d789b-6tfmv"
Aug 01 00:34:41.630992 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:34:41.630959    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-bb639-predictor-7bbf8d789b-6tfmv"]
Aug 01 00:34:41.632669 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:34:41.632642    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-bb639-predictor-7bbf8d789b-6tfmv"]
Aug 01 00:34:42.421880 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:34:42.421835    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="12afa0f2-0926-4e52-a74d-bba84664f6c5" path="/var/lib/kubelet/pods/12afa0f2-0926-4e52-a74d-bba84664f6c5/volumes"
Aug 01 00:34:44.241812 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:34:44.241551    2564 scope.go:117] "RemoveContainer" containerID="572c16a74ccbefc29ad0475b05c04ca8a7031e749eb13d8cbc297f9ef1a46090"
Aug 01 00:34:45.627606 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:34:45.627564    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-lvfql/must-gather-zpmqg" event={"ID":"d348aa25-45c0-49fa-ad49-d0a429ba68ef","Type":"ContainerStarted","Data":"5fefb3eace306332a2e7c776fa841e08083a78f8da8a17962c8f10c5a3daa918"}
Aug 01 00:34:45.628017 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:34:45.627620    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-lvfql/must-gather-zpmqg" event={"ID":"d348aa25-45c0-49fa-ad49-d0a429ba68ef","Type":"ContainerStarted","Data":"983ca0f2ab21ceeda0bade4836087366944f1a427e7a356e14915d8f5896b38f"}
Aug 01 00:34:45.644247 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:34:45.644141    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-lvfql/must-gather-zpmqg" podStartSLOduration=1.446439241 podStartE2EDuration="7.644123533s" totalImagesPullingTime="6.197684292s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-01 00:34:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-01 00:34:45.643817279 +0000 UTC m=+2903.755944967" watchObservedRunningTime="2026-08-01 00:34:45.644123533 +0000 UTC m=+2903.756251208"
Aug 01 00:35:06.718322 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:35:06.718282    2564 generic.go:356] "Generic (PLEG): container finished" podID="d348aa25-45c0-49fa-ad49-d0a429ba68ef" containerID="983ca0f2ab21ceeda0bade4836087366944f1a427e7a356e14915d8f5896b38f" exitCode=0
Aug 01 00:35:06.718874 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:35:06.718327    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-lvfql/must-gather-zpmqg" event={"ID":"d348aa25-45c0-49fa-ad49-d0a429ba68ef","Type":"ContainerDied","Data":"983ca0f2ab21ceeda0bade4836087366944f1a427e7a356e14915d8f5896b38f"}
Aug 01 00:35:06.718874 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:35:06.718746    2564 scope.go:117] "RemoveContainer" containerID="983ca0f2ab21ceeda0bade4836087366944f1a427e7a356e14915d8f5896b38f"
Aug 01 00:35:07.629502 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:35:07.629465    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-lvfql_must-gather-zpmqg_d348aa25-45c0-49fa-ad49-d0a429ba68ef/gather/0.log"
Aug 01 00:35:08.280296 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:35:08.280260    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-pmldc/must-gather-gwl87"]
Aug 01 00:35:08.280711 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:35:08.280564    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="12afa0f2-0926-4e52-a74d-bba84664f6c5" containerName="kserve-container"
Aug 01 00:35:08.280711 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:35:08.280576    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="12afa0f2-0926-4e52-a74d-bba84664f6c5" containerName="kserve-container"
Aug 01 00:35:08.280711 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:35:08.280595    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="12afa0f2-0926-4e52-a74d-bba84664f6c5" containerName="kube-rbac-proxy"
Aug 01 00:35:08.280711 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:35:08.280601    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="12afa0f2-0926-4e52-a74d-bba84664f6c5" containerName="kube-rbac-proxy"
Aug 01 00:35:08.280711 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:35:08.280661    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="12afa0f2-0926-4e52-a74d-bba84664f6c5" containerName="kserve-container"
Aug 01 00:35:08.280711 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:35:08.280674    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="12afa0f2-0926-4e52-a74d-bba84664f6c5" containerName="kube-rbac-proxy"
Aug 01 00:35:08.282735 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:35:08.282717    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-pmldc/must-gather-gwl87"
Aug 01 00:35:08.285542 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:35:08.285513    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-pmldc\"/\"openshift-service-ca.crt\""
Aug 01 00:35:08.285691 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:35:08.285613    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-pmldc\"/\"kube-root-ca.crt\""
Aug 01 00:35:08.286734 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:35:08.286716    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-pmldc\"/\"default-dockercfg-t8758\""
Aug 01 00:35:08.292640 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:35:08.292599    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-pmldc/must-gather-gwl87"]
Aug 01 00:35:08.372587 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:35:08.372542    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/ca1d57bb-1fd8-4792-abe9-ce68b976117f-must-gather-output\") pod \"must-gather-gwl87\" (UID: \"ca1d57bb-1fd8-4792-abe9-ce68b976117f\") " pod="openshift-must-gather-pmldc/must-gather-gwl87"
Aug 01 00:35:08.372587 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:35:08.372586    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qqwtm\" (UniqueName: \"kubernetes.io/projected/ca1d57bb-1fd8-4792-abe9-ce68b976117f-kube-api-access-qqwtm\") pod \"must-gather-gwl87\" (UID: \"ca1d57bb-1fd8-4792-abe9-ce68b976117f\") " pod="openshift-must-gather-pmldc/must-gather-gwl87"
Aug 01 00:35:08.473454 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:35:08.473410    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/ca1d57bb-1fd8-4792-abe9-ce68b976117f-must-gather-output\") pod \"must-gather-gwl87\" (UID: \"ca1d57bb-1fd8-4792-abe9-ce68b976117f\") " pod="openshift-must-gather-pmldc/must-gather-gwl87"
Aug 01 00:35:08.473454 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:35:08.473452    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qqwtm\" (UniqueName: \"kubernetes.io/projected/ca1d57bb-1fd8-4792-abe9-ce68b976117f-kube-api-access-qqwtm\") pod \"must-gather-gwl87\" (UID: \"ca1d57bb-1fd8-4792-abe9-ce68b976117f\") " pod="openshift-must-gather-pmldc/must-gather-gwl87"
Aug 01 00:35:08.473832 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:35:08.473814    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/ca1d57bb-1fd8-4792-abe9-ce68b976117f-must-gather-output\") pod \"must-gather-gwl87\" (UID: \"ca1d57bb-1fd8-4792-abe9-ce68b976117f\") " pod="openshift-must-gather-pmldc/must-gather-gwl87"
Aug 01 00:35:08.481838 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:35:08.481812    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qqwtm\" (UniqueName: \"kubernetes.io/projected/ca1d57bb-1fd8-4792-abe9-ce68b976117f-kube-api-access-qqwtm\") pod \"must-gather-gwl87\" (UID: \"ca1d57bb-1fd8-4792-abe9-ce68b976117f\") " pod="openshift-must-gather-pmldc/must-gather-gwl87"
Aug 01 00:35:08.592561 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:35:08.592472    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-pmldc/must-gather-gwl87"
Aug 01 00:35:08.722521 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:35:08.722477    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-pmldc/must-gather-gwl87"]
Aug 01 00:35:08.726230 ip-10-0-128-75 kubenswrapper[2564]: W0801 00:35:08.726203    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podca1d57bb_1fd8_4792_abe9_ce68b976117f.slice/crio-5eb0be3eef2dde357c41e465f7fb16a1ee00aa979f8c1eff20c9995ad2389bb5 WatchSource:0}: Error finding container 5eb0be3eef2dde357c41e465f7fb16a1ee00aa979f8c1eff20c9995ad2389bb5: Status 404 returned error can't find the container with id 5eb0be3eef2dde357c41e465f7fb16a1ee00aa979f8c1eff20c9995ad2389bb5
Aug 01 00:35:09.731922 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:35:09.731891    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-pmldc/must-gather-gwl87" event={"ID":"ca1d57bb-1fd8-4792-abe9-ce68b976117f","Type":"ContainerStarted","Data":"5ec3d95ae74344ea3d41bf87b211e1aef0e386bb67b3818ba450825fa4cc9675"}
Aug 01 00:35:09.732331 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:35:09.731937    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-pmldc/must-gather-gwl87" event={"ID":"ca1d57bb-1fd8-4792-abe9-ce68b976117f","Type":"ContainerStarted","Data":"5eb0be3eef2dde357c41e465f7fb16a1ee00aa979f8c1eff20c9995ad2389bb5"}
Aug 01 00:35:10.740038 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:35:10.740000    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-pmldc/must-gather-gwl87" event={"ID":"ca1d57bb-1fd8-4792-abe9-ce68b976117f","Type":"ContainerStarted","Data":"772895b04f1ae732cf9feb4d92264ce9fc63a02e19f7267c1a1dfee43259f6bb"}
Aug 01 00:35:10.756858 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:35:10.756790    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-pmldc/must-gather-gwl87" podStartSLOduration=1.859461725 podStartE2EDuration="2.756765351s" totalImagesPullingTime="897.303626ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-01 00:35:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-01 00:35:10.754849333 +0000 UTC m=+2928.866977036" watchObservedRunningTime="2026-08-01 00:35:10.756765351 +0000 UTC m=+2928.868893025"
Aug 01 00:35:11.051875 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:35:11.051785    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-h45s5_69e78750-573b-4dbd-a585-cf0d88056f4f/global-pull-secret-syncer/0.log"
Aug 01 00:35:11.187027 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:35:11.186988    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-x2dtl_fd82faec-581a-491a-8515-4217ac8cac14/konnectivity-agent/0.log"
Aug 01 00:35:11.209882 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:35:11.209829    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-128-75.ec2.internal_187e741f1e261812822fd24432933695/haproxy/0.log"
Aug 01 00:35:13.109954 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:35:13.109921    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-must-gather-lvfql/must-gather-zpmqg"]
Aug 01 00:35:13.110696 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:35:13.110666    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-must-gather-lvfql/must-gather-zpmqg" podUID="d348aa25-45c0-49fa-ad49-d0a429ba68ef" containerName="copy" containerID="cri-o://5fefb3eace306332a2e7c776fa841e08083a78f8da8a17962c8f10c5a3daa918" gracePeriod=2
Aug 01 00:35:13.112201 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:35:13.112173    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-must-gather-lvfql/must-gather-zpmqg"]
Aug 01 00:35:13.127339 ip-10-0-128-75 kubenswrapper[2564]: E0801 00:35:13.122238    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-zpmqg\" is forbidden: User \"system:node:ip-10-0-128-75.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-lvfql\": no relationship found between node 'ip-10-0-128-75.ec2.internal' and this object" podUID="d348aa25-45c0-49fa-ad49-d0a429ba68ef" pod="openshift-must-gather-lvfql/must-gather-zpmqg"
Aug 01 00:35:13.521763 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:35:13.521715    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-lvfql_must-gather-zpmqg_d348aa25-45c0-49fa-ad49-d0a429ba68ef/copy/0.log"
Aug 01 00:35:13.528095 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:35:13.527804    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-lvfql/must-gather-zpmqg"
Aug 01 00:35:13.533680 ip-10-0-128-75 kubenswrapper[2564]: E0801 00:35:13.533484    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-zpmqg\" is forbidden: User \"system:node:ip-10-0-128-75.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-lvfql\": no relationship found between node 'ip-10-0-128-75.ec2.internal' and this object" podUID="d348aa25-45c0-49fa-ad49-d0a429ba68ef" pod="openshift-must-gather-lvfql/must-gather-zpmqg"
Aug 01 00:35:13.620686 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:35:13.620053    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/d348aa25-45c0-49fa-ad49-d0a429ba68ef-must-gather-output\") pod \"d348aa25-45c0-49fa-ad49-d0a429ba68ef\" (UID: \"d348aa25-45c0-49fa-ad49-d0a429ba68ef\") "
Aug 01 00:35:13.620686 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:35:13.620114    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p6vs2\" (UniqueName: \"kubernetes.io/projected/d348aa25-45c0-49fa-ad49-d0a429ba68ef-kube-api-access-p6vs2\") pod \"d348aa25-45c0-49fa-ad49-d0a429ba68ef\" (UID: \"d348aa25-45c0-49fa-ad49-d0a429ba68ef\") "
Aug 01 00:35:13.622735 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:35:13.622656    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d348aa25-45c0-49fa-ad49-d0a429ba68ef-must-gather-output" (OuterVolumeSpecName: "must-gather-output") pod "d348aa25-45c0-49fa-ad49-d0a429ba68ef" (UID: "d348aa25-45c0-49fa-ad49-d0a429ba68ef"). InnerVolumeSpecName "must-gather-output". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 01 00:35:13.633781 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:35:13.633047    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d348aa25-45c0-49fa-ad49-d0a429ba68ef-kube-api-access-p6vs2" (OuterVolumeSpecName: "kube-api-access-p6vs2") pod "d348aa25-45c0-49fa-ad49-d0a429ba68ef" (UID: "d348aa25-45c0-49fa-ad49-d0a429ba68ef"). InnerVolumeSpecName "kube-api-access-p6vs2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 01 00:35:13.721750 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:35:13.721684    2564 reconciler_common.go:299] "Volume detached for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/d348aa25-45c0-49fa-ad49-d0a429ba68ef-must-gather-output\") on node \"ip-10-0-128-75.ec2.internal\" DevicePath \"\""
Aug 01 00:35:13.721750 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:35:13.721721    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p6vs2\" (UniqueName: \"kubernetes.io/projected/d348aa25-45c0-49fa-ad49-d0a429ba68ef-kube-api-access-p6vs2\") on node \"ip-10-0-128-75.ec2.internal\" DevicePath \"\""
Aug 01 00:35:13.757919 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:35:13.757842    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-lvfql_must-gather-zpmqg_d348aa25-45c0-49fa-ad49-d0a429ba68ef/copy/0.log"
Aug 01 00:35:13.761658 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:35:13.758591    2564 generic.go:356] "Generic (PLEG): container finished" podID="d348aa25-45c0-49fa-ad49-d0a429ba68ef" containerID="5fefb3eace306332a2e7c776fa841e08083a78f8da8a17962c8f10c5a3daa918" exitCode=143
Aug 01 00:35:13.761658 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:35:13.758737    2564 scope.go:117] "RemoveContainer" containerID="5fefb3eace306332a2e7c776fa841e08083a78f8da8a17962c8f10c5a3daa918"
Aug 01 00:35:13.761658 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:35:13.758939    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-lvfql/must-gather-zpmqg"
Aug 01 00:35:13.764491 ip-10-0-128-75 kubenswrapper[2564]: E0801 00:35:13.764450    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-zpmqg\" is forbidden: User \"system:node:ip-10-0-128-75.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-lvfql\": no relationship found between node 'ip-10-0-128-75.ec2.internal' and this object" podUID="d348aa25-45c0-49fa-ad49-d0a429ba68ef" pod="openshift-must-gather-lvfql/must-gather-zpmqg"
Aug 01 00:35:13.776920 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:35:13.776897    2564 scope.go:117] "RemoveContainer" containerID="983ca0f2ab21ceeda0bade4836087366944f1a427e7a356e14915d8f5896b38f"
Aug 01 00:35:13.777121 ip-10-0-128-75 kubenswrapper[2564]: E0801 00:35:13.777007    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-zpmqg\" is forbidden: User \"system:node:ip-10-0-128-75.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-lvfql\": no relationship found between node 'ip-10-0-128-75.ec2.internal' and this object" podUID="d348aa25-45c0-49fa-ad49-d0a429ba68ef" pod="openshift-must-gather-lvfql/must-gather-zpmqg"
Aug 01 00:35:13.807247 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:35:13.807203    2564 scope.go:117] "RemoveContainer" containerID="5fefb3eace306332a2e7c776fa841e08083a78f8da8a17962c8f10c5a3daa918"
Aug 01 00:35:13.807982 ip-10-0-128-75 kubenswrapper[2564]: E0801 00:35:13.807944    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5fefb3eace306332a2e7c776fa841e08083a78f8da8a17962c8f10c5a3daa918\": container with ID starting with 5fefb3eace306332a2e7c776fa841e08083a78f8da8a17962c8f10c5a3daa918 not found: ID does not exist" containerID="5fefb3eace306332a2e7c776fa841e08083a78f8da8a17962c8f10c5a3daa918"
Aug 01 00:35:13.808212 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:35:13.808164    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5fefb3eace306332a2e7c776fa841e08083a78f8da8a17962c8f10c5a3daa918"} err="failed to get container status \"5fefb3eace306332a2e7c776fa841e08083a78f8da8a17962c8f10c5a3daa918\": rpc error: code = NotFound desc = could not find container \"5fefb3eace306332a2e7c776fa841e08083a78f8da8a17962c8f10c5a3daa918\": container with ID starting with 5fefb3eace306332a2e7c776fa841e08083a78f8da8a17962c8f10c5a3daa918 not found: ID does not exist"
Aug 01 00:35:13.808354 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:35:13.808341    2564 scope.go:117] "RemoveContainer" containerID="983ca0f2ab21ceeda0bade4836087366944f1a427e7a356e14915d8f5896b38f"
Aug 01 00:35:13.808940 ip-10-0-128-75 kubenswrapper[2564]: E0801 00:35:13.808911    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"983ca0f2ab21ceeda0bade4836087366944f1a427e7a356e14915d8f5896b38f\": container with ID starting with 983ca0f2ab21ceeda0bade4836087366944f1a427e7a356e14915d8f5896b38f not found: ID does not exist" containerID="983ca0f2ab21ceeda0bade4836087366944f1a427e7a356e14915d8f5896b38f"
Aug 01 00:35:13.809161 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:35:13.809131    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"983ca0f2ab21ceeda0bade4836087366944f1a427e7a356e14915d8f5896b38f"} err="failed to get container status \"983ca0f2ab21ceeda0bade4836087366944f1a427e7a356e14915d8f5896b38f\": rpc error: code = NotFound desc = could not find container \"983ca0f2ab21ceeda0bade4836087366944f1a427e7a356e14915d8f5896b38f\": container with ID starting with 983ca0f2ab21ceeda0bade4836087366944f1a427e7a356e14915d8f5896b38f not found: ID does not exist"
Aug 01 00:35:14.426067 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:35:14.426027    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d348aa25-45c0-49fa-ad49-d0a429ba68ef" path="/var/lib/kubelet/pods/d348aa25-45c0-49fa-ad49-d0a429ba68ef/volumes"
Aug 01 00:35:16.110678 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:35:16.110643    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-6bd9664c4-n7p8z_5b8598a5-98e5-4e88-b538-c47bbc69cf7f/kube-state-metrics/0.log"
Aug 01 00:35:16.136262 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:35:16.136229    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-6bd9664c4-n7p8z_5b8598a5-98e5-4e88-b538-c47bbc69cf7f/kube-rbac-proxy-main/0.log"
Aug 01 00:35:16.159353 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:35:16.159315    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-6bd9664c4-n7p8z_5b8598a5-98e5-4e88-b538-c47bbc69cf7f/kube-rbac-proxy-self/0.log"
Aug 01 00:35:16.335178 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:35:16.335096    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-jnmzc_019f1fa4-9e80-41f5-b48d-82e064824829/node-exporter/0.log"
Aug 01 00:35:16.354873 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:35:16.354844    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-jnmzc_019f1fa4-9e80-41f5-b48d-82e064824829/kube-rbac-proxy/0.log"
Aug 01 00:35:16.374375 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:35:16.374335    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-jnmzc_019f1fa4-9e80-41f5-b48d-82e064824829/init-textfile/0.log"
Aug 01 00:35:16.471546 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:35:16.471518    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-6bb4d7b4dd-wz649_c647d292-b5cd-4840-a69c-8c0e9ae41793/kube-rbac-proxy-main/0.log"
Aug 01 00:35:16.493552 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:35:16.493514    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-6bb4d7b4dd-wz649_c647d292-b5cd-4840-a69c-8c0e9ae41793/kube-rbac-proxy-self/0.log"
Aug 01 00:35:16.517708 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:35:16.517157    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-6bb4d7b4dd-wz649_c647d292-b5cd-4840-a69c-8c0e9ae41793/openshift-state-metrics/0.log"
Aug 01 00:35:16.567096 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:35:16.566929    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_136d0bc8-ebce-4191-9680-948f869e4b26/prometheus/0.log"
Aug 01 00:35:16.583367 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:35:16.583338    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_136d0bc8-ebce-4191-9680-948f869e4b26/config-reloader/0.log"
Aug 01 00:35:16.607328 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:35:16.607304    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_136d0bc8-ebce-4191-9680-948f869e4b26/thanos-sidecar/0.log"
Aug 01 00:35:16.633682 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:35:16.633652    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_136d0bc8-ebce-4191-9680-948f869e4b26/kube-rbac-proxy-web/0.log"
Aug 01 00:35:16.656755 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:35:16.656727    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_136d0bc8-ebce-4191-9680-948f869e4b26/kube-rbac-proxy/0.log"
Aug 01 00:35:16.682396 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:35:16.682357    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_136d0bc8-ebce-4191-9680-948f869e4b26/kube-rbac-proxy-thanos/0.log"
Aug 01 00:35:16.705243 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:35:16.705211    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_136d0bc8-ebce-4191-9680-948f869e4b26/init-config-reloader/0.log"
Aug 01 00:35:16.775953 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:35:16.775921    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-66685ff6d4-bngc8_d7ef80c3-fd39-4aae-a789-db9c906ff3fe/prometheus-operator-admission-webhook/0.log"
Aug 01 00:35:19.596825 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:35:19.596790    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-pmldc/perf-node-gather-daemonset-6lh6h"]
Aug 01 00:35:19.597408 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:35:19.597058    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d348aa25-45c0-49fa-ad49-d0a429ba68ef" containerName="gather"
Aug 01 00:35:19.597408 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:35:19.597068    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="d348aa25-45c0-49fa-ad49-d0a429ba68ef" containerName="gather"
Aug 01 00:35:19.597408 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:35:19.597079    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d348aa25-45c0-49fa-ad49-d0a429ba68ef" containerName="copy"
Aug 01 00:35:19.597408 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:35:19.597085    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="d348aa25-45c0-49fa-ad49-d0a429ba68ef" containerName="copy"
Aug 01 00:35:19.597408 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:35:19.597141    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="d348aa25-45c0-49fa-ad49-d0a429ba68ef" containerName="gather"
Aug 01 00:35:19.597408 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:35:19.597153    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="d348aa25-45c0-49fa-ad49-d0a429ba68ef" containerName="copy"
Aug 01 00:35:19.602767 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:35:19.602734    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-pmldc/perf-node-gather-daemonset-6lh6h"
Aug 01 00:35:19.612482 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:35:19.611753    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-pmldc/perf-node-gather-daemonset-6lh6h"]
Aug 01 00:35:19.681033 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:35:19.680995    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e892a626-3b63-48bf-bc64-23fe9e2ab7ee-lib-modules\") pod \"perf-node-gather-daemonset-6lh6h\" (UID: \"e892a626-3b63-48bf-bc64-23fe9e2ab7ee\") " pod="openshift-must-gather-pmldc/perf-node-gather-daemonset-6lh6h"
Aug 01 00:35:19.681205 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:35:19.681063    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5pdcr\" (UniqueName: \"kubernetes.io/projected/e892a626-3b63-48bf-bc64-23fe9e2ab7ee-kube-api-access-5pdcr\") pod \"perf-node-gather-daemonset-6lh6h\" (UID: \"e892a626-3b63-48bf-bc64-23fe9e2ab7ee\") " pod="openshift-must-gather-pmldc/perf-node-gather-daemonset-6lh6h"
Aug 01 00:35:19.681205 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:35:19.681147    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e892a626-3b63-48bf-bc64-23fe9e2ab7ee-proc\") pod \"perf-node-gather-daemonset-6lh6h\" (UID: \"e892a626-3b63-48bf-bc64-23fe9e2ab7ee\") " pod="openshift-must-gather-pmldc/perf-node-gather-daemonset-6lh6h"
Aug 01 00:35:19.681205 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:35:19.681192    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e892a626-3b63-48bf-bc64-23fe9e2ab7ee-sys\") pod \"perf-node-gather-daemonset-6lh6h\" (UID: \"e892a626-3b63-48bf-bc64-23fe9e2ab7ee\") " pod="openshift-must-gather-pmldc/perf-node-gather-daemonset-6lh6h"
Aug 01 00:35:19.681405 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:35:19.681223    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e892a626-3b63-48bf-bc64-23fe9e2ab7ee-podres\") pod \"perf-node-gather-daemonset-6lh6h\" (UID: \"e892a626-3b63-48bf-bc64-23fe9e2ab7ee\") " pod="openshift-must-gather-pmldc/perf-node-gather-daemonset-6lh6h"
Aug 01 00:35:19.782451 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:35:19.782407    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5pdcr\" (UniqueName: \"kubernetes.io/projected/e892a626-3b63-48bf-bc64-23fe9e2ab7ee-kube-api-access-5pdcr\") pod \"perf-node-gather-daemonset-6lh6h\" (UID: \"e892a626-3b63-48bf-bc64-23fe9e2ab7ee\") " pod="openshift-must-gather-pmldc/perf-node-gather-daemonset-6lh6h"
Aug 01 00:35:19.782695 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:35:19.782476    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e892a626-3b63-48bf-bc64-23fe9e2ab7ee-proc\") pod \"perf-node-gather-daemonset-6lh6h\" (UID: \"e892a626-3b63-48bf-bc64-23fe9e2ab7ee\") " pod="openshift-must-gather-pmldc/perf-node-gather-daemonset-6lh6h"
Aug 01 00:35:19.782695 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:35:19.782515    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e892a626-3b63-48bf-bc64-23fe9e2ab7ee-sys\") pod \"perf-node-gather-daemonset-6lh6h\" (UID: \"e892a626-3b63-48bf-bc64-23fe9e2ab7ee\") " pod="openshift-must-gather-pmldc/perf-node-gather-daemonset-6lh6h"
Aug 01 00:35:19.782695 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:35:19.782598    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e892a626-3b63-48bf-bc64-23fe9e2ab7ee-sys\") pod \"perf-node-gather-daemonset-6lh6h\" (UID: \"e892a626-3b63-48bf-bc64-23fe9e2ab7ee\") " pod="openshift-must-gather-pmldc/perf-node-gather-daemonset-6lh6h"
Aug 01 00:35:19.782695 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:35:19.782607    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e892a626-3b63-48bf-bc64-23fe9e2ab7ee-proc\") pod \"perf-node-gather-daemonset-6lh6h\" (UID: \"e892a626-3b63-48bf-bc64-23fe9e2ab7ee\") " pod="openshift-must-gather-pmldc/perf-node-gather-daemonset-6lh6h"
Aug 01 00:35:19.782695 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:35:19.782654    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e892a626-3b63-48bf-bc64-23fe9e2ab7ee-podres\") pod \"perf-node-gather-daemonset-6lh6h\" (UID: \"e892a626-3b63-48bf-bc64-23fe9e2ab7ee\") " pod="openshift-must-gather-pmldc/perf-node-gather-daemonset-6lh6h"
Aug 01 00:35:19.782988 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:35:19.782769    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e892a626-3b63-48bf-bc64-23fe9e2ab7ee-lib-modules\") pod \"perf-node-gather-daemonset-6lh6h\" (UID: \"e892a626-3b63-48bf-bc64-23fe9e2ab7ee\") " pod="openshift-must-gather-pmldc/perf-node-gather-daemonset-6lh6h"
Aug 01 00:35:19.782988 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:35:19.782806    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e892a626-3b63-48bf-bc64-23fe9e2ab7ee-podres\") pod \"perf-node-gather-daemonset-6lh6h\" (UID: \"e892a626-3b63-48bf-bc64-23fe9e2ab7ee\") " pod="openshift-must-gather-pmldc/perf-node-gather-daemonset-6lh6h"
Aug 01 00:35:19.782988 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:35:19.782897    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e892a626-3b63-48bf-bc64-23fe9e2ab7ee-lib-modules\") pod \"perf-node-gather-daemonset-6lh6h\" (UID: \"e892a626-3b63-48bf-bc64-23fe9e2ab7ee\") " pod="openshift-must-gather-pmldc/perf-node-gather-daemonset-6lh6h"
Aug 01 00:35:19.793718 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:35:19.793684    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5pdcr\" (UniqueName: \"kubernetes.io/projected/e892a626-3b63-48bf-bc64-23fe9e2ab7ee-kube-api-access-5pdcr\") pod \"perf-node-gather-daemonset-6lh6h\" (UID: \"e892a626-3b63-48bf-bc64-23fe9e2ab7ee\") " pod="openshift-must-gather-pmldc/perf-node-gather-daemonset-6lh6h"
Aug 01 00:35:19.918313 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:35:19.918278    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-pmldc/perf-node-gather-daemonset-6lh6h"
Aug 01 00:35:20.093491 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:35:20.093368    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-pmldc/perf-node-gather-daemonset-6lh6h"]
Aug 01 00:35:20.096416 ip-10-0-128-75 kubenswrapper[2564]: W0801 00:35:20.096374    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pode892a626_3b63_48bf_bc64_23fe9e2ab7ee.slice/crio-28a00bc91ae0ba88d4ac7ebe476de14d57abdde12f1c84e75a61985fdbd5665f WatchSource:0}: Error finding container 28a00bc91ae0ba88d4ac7ebe476de14d57abdde12f1c84e75a61985fdbd5665f: Status 404 returned error can't find the container with id 28a00bc91ae0ba88d4ac7ebe476de14d57abdde12f1c84e75a61985fdbd5665f
Aug 01 00:35:20.572022 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:35:20.571933    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-8gxhj_22c17973-7031-4c9a-bbea-39d64598f867/dns/0.log"
Aug 01 00:35:20.595107 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:35:20.595079    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-8gxhj_22c17973-7031-4c9a-bbea-39d64598f867/kube-rbac-proxy/0.log"
Aug 01 00:35:20.750026 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:35:20.749951    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-wxz8k_816ea407-9c93-4455-8202-8229340dc7db/dns-node-resolver/0.log"
Aug 01 00:35:20.799528 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:35:20.799486    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-pmldc/perf-node-gather-daemonset-6lh6h" event={"ID":"e892a626-3b63-48bf-bc64-23fe9e2ab7ee","Type":"ContainerStarted","Data":"342a184e7c82bf7685d0dec699b3a941970ed069e1232ede663e69641e0016ec"}
Aug 01 00:35:20.799750 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:35:20.799556    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-pmldc/perf-node-gather-daemonset-6lh6h" event={"ID":"e892a626-3b63-48bf-bc64-23fe9e2ab7ee","Type":"ContainerStarted","Data":"28a00bc91ae0ba88d4ac7ebe476de14d57abdde12f1c84e75a61985fdbd5665f"}
Aug 01 00:35:20.799907 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:35:20.799883    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-pmldc/perf-node-gather-daemonset-6lh6h"
Aug 01 00:35:21.186797 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:35:21.186774    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-pruner-29759040-k5csx_3cabb612-38bd-4e6e-b904-284efbe28a18/image-pruner/0.log"
Aug 01 00:35:21.226652 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:35:21.226590    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-774c7c8d4c-hxx2p_219c8401-f8b7-474e-ac64-98a06039633f/registry/0.log"
Aug 01 00:35:21.266983 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:35:21.266954    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-ls2gt_d98fe0c2-0a27-408e-8131-252e3d469281/node-ca/0.log"
Aug 01 00:35:22.385546 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:35:22.385512    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-j4bjx_3e93b652-caf7-4e5f-9c0a-ef498fa18697/serve-healthcheck-canary/0.log"
Aug 01 00:35:22.879386 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:35:22.879354    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-x5njs_9e3db559-1129-4c9d-b1b8-3861fba9a115/kube-rbac-proxy/0.log"
Aug 01 00:35:22.898979 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:35:22.898946    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-x5njs_9e3db559-1129-4c9d-b1b8-3861fba9a115/exporter/0.log"
Aug 01 00:35:22.919298 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:35:22.919268    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-x5njs_9e3db559-1129-4c9d-b1b8-3861fba9a115/extractor/0.log"
Aug 01 00:35:25.329179 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:35:25.329145    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_s3-init-phkrf_08ae5adf-ffd8-4f08-a500-4cbf4dadce15/s3-init/0.log"
Aug 01 00:35:25.344849 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:35:25.344813    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_s3-init-phkrf_08ae5adf-ffd8-4f08-a500-4cbf4dadce15/download-hf-model/0.log"
Aug 01 00:35:25.375435 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:35:25.375406    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_seaweedfs-64568bcd49-ssq4m_10fb8451-20c5-45c7-8367-2902bddab100/seaweedfs/0.log"
Aug 01 00:35:26.815593 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:35:26.815550    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-pmldc/perf-node-gather-daemonset-6lh6h"
Aug 01 00:35:26.831872 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:35:26.831656    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-pmldc/perf-node-gather-daemonset-6lh6h" podStartSLOduration=7.831614701 podStartE2EDuration="7.831614701s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-01 00:35:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-01 00:35:20.817367406 +0000 UTC m=+2938.929495080" watchObservedRunningTime="2026-08-01 00:35:26.831614701 +0000 UTC m=+2944.943742378"
Aug 01 00:35:30.575792 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:35:30.575761    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-jtsl2_4e0b1a1a-3553-464e-8094-45a9dbf1cb66/kube-multus-additional-cni-plugins/0.log"
Aug 01 00:35:30.595606 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:35:30.595573    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-jtsl2_4e0b1a1a-3553-464e-8094-45a9dbf1cb66/egress-router-binary-copy/0.log"
Aug 01 00:35:30.615608 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:35:30.615574    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-jtsl2_4e0b1a1a-3553-464e-8094-45a9dbf1cb66/cni-plugins/0.log"
Aug 01 00:35:30.644044 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:35:30.643998    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-jtsl2_4e0b1a1a-3553-464e-8094-45a9dbf1cb66/bond-cni-plugin/0.log"
Aug 01 00:35:30.666028 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:35:30.665991    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-jtsl2_4e0b1a1a-3553-464e-8094-45a9dbf1cb66/routeoverride-cni/0.log"
Aug 01 00:35:30.685938 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:35:30.685901    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-jtsl2_4e0b1a1a-3553-464e-8094-45a9dbf1cb66/whereabouts-cni-bincopy/0.log"
Aug 01 00:35:30.706757 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:35:30.706725    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-jtsl2_4e0b1a1a-3553-464e-8094-45a9dbf1cb66/whereabouts-cni/0.log"
Aug 01 00:35:30.936368 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:35:30.936334    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-jw7ld_b0541f63-78c6-4ae2-b6aa-3e7ad98eb151/kube-multus/0.log"
Aug 01 00:35:30.955153 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:35:30.955107    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-j97wn_815ee183-a99c-4636-9937-da826945f531/network-metrics-daemon/0.log"
Aug 01 00:35:30.972071 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:35:30.972034    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-j97wn_815ee183-a99c-4636-9937-da826945f531/kube-rbac-proxy/0.log"
Aug 01 00:35:31.943381 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:35:31.943350    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-j5545_2bd960e3-3638-49f5-bd70-96891131423e/ovn-controller/0.log"
Aug 01 00:35:31.967387 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:35:31.967359    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-j5545_2bd960e3-3638-49f5-bd70-96891131423e/ovn-acl-logging/0.log"
Aug 01 00:35:31.989099 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:35:31.989068    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-j5545_2bd960e3-3638-49f5-bd70-96891131423e/kube-rbac-proxy-node/0.log"
Aug 01 00:35:32.010008 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:35:32.009977    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-j5545_2bd960e3-3638-49f5-bd70-96891131423e/kube-rbac-proxy-ovn-metrics/0.log"
Aug 01 00:35:32.047300 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:35:32.047273    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-j5545_2bd960e3-3638-49f5-bd70-96891131423e/northd/0.log"
Aug 01 00:35:32.110761 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:35:32.110734    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-j5545_2bd960e3-3638-49f5-bd70-96891131423e/nbdb/0.log"
Aug 01 00:35:32.133166 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:35:32.133138    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-j5545_2bd960e3-3638-49f5-bd70-96891131423e/sbdb/0.log"
Aug 01 00:35:32.171076 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:35:32.171044    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-j5545_2bd960e3-3638-49f5-bd70-96891131423e/ovnkube-controller/0.log"
Aug 01 00:35:33.354837 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:35:33.354808    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-7n4sw_2887ad3f-73c9-4fce-8359-c86e96300a84/network-check-target-container/0.log"
Aug 01 00:35:34.223874 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:35:34.223847    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-7ttkn_c0bfad85-f7bf-4457-8523-6a1aa966a280/iptables-alerter/0.log"
Aug 01 00:35:34.860596 ip-10-0-128-75 kubenswrapper[2564]: I0801 00:35:34.860566    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-node-tuning-operator_tuned-h5jbv_d997aba4-5c6e-49ad-81bd-95efee82e1ff/tuned/0.log"