Jul 02 13:43:54.716122 ip-10-0-133-209 systemd[1]: Starting Kubernetes Kubelet...
Jul 02 13:43:55.062690 ip-10-0-133-209 kubenswrapper[2558]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 02 13:43:55.062690 ip-10-0-133-209 kubenswrapper[2558]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 02 13:43:55.062690 ip-10-0-133-209 kubenswrapper[2558]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 02 13:43:55.062690 ip-10-0-133-209 kubenswrapper[2558]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 02 13:43:55.062690 ip-10-0-133-209 kubenswrapper[2558]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 02 13:43:55.063943 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.063855    2558 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 02 13:43:55.068314 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.068290    2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 02 13:43:55.068314 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.068310    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 02 13:43:55.068314 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.068313    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 02 13:43:55.068314 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.068317    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 02 13:43:55.068314 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.068320    2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 02 13:43:55.068314 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.068323    2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 02 13:43:55.068530 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.068326    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 02 13:43:55.068530 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.068329    2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 02 13:43:55.068530 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.068331    2558 feature_gate.go:328] unrecognized feature gate: Example
Jul 02 13:43:55.068530 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.068334    2558 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 02 13:43:55.068530 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.068337    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 02 13:43:55.068530 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.068339    2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 02 13:43:55.068530 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.068342    2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 02 13:43:55.068530 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.068345    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 02 13:43:55.068530 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.068347    2558 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 02 13:43:55.068530 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.068350    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 02 13:43:55.068530 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.068352    2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 02 13:43:55.068530 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.068355    2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 02 13:43:55.068530 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.068357    2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 02 13:43:55.068530 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.068360    2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 02 13:43:55.068530 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.068362    2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 02 13:43:55.068530 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.068365    2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 02 13:43:55.068530 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.068367    2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 02 13:43:55.068530 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.068370    2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 02 13:43:55.068530 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.068372    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 02 13:43:55.068530 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.068375    2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 02 13:43:55.069038 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.068377    2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 02 13:43:55.069038 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.068380    2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 02 13:43:55.069038 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.068382    2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 02 13:43:55.069038 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.068385    2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 02 13:43:55.069038 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.068387    2558 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 02 13:43:55.069038 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.068394    2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 02 13:43:55.069038 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.068397    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 02 13:43:55.069038 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.068399    2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 02 13:43:55.069038 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.068402    2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 02 13:43:55.069038 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.068404    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 02 13:43:55.069038 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.068408    2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 02 13:43:55.069038 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.068410    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 02 13:43:55.069038 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.068412    2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 02 13:43:55.069038 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.068416    2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 02 13:43:55.069038 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.068419    2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 02 13:43:55.069038 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.068422    2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 02 13:43:55.069038 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.068424    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 02 13:43:55.069038 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.068427    2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 02 13:43:55.069038 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.068429    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 02 13:43:55.069038 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.068432    2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 02 13:43:55.069520 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.068436    2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 02 13:43:55.069520 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.068441    2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 02 13:43:55.069520 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.068444    2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 02 13:43:55.069520 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.068446    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 02 13:43:55.069520 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.068449    2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 02 13:43:55.069520 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.068452    2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 02 13:43:55.069520 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.068455    2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 02 13:43:55.069520 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.068458    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 02 13:43:55.069520 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.068460    2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 02 13:43:55.069520 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.068463    2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 02 13:43:55.069520 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.068466    2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 02 13:43:55.069520 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.068469    2558 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 02 13:43:55.069520 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.068472    2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 02 13:43:55.069520 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.068476    2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 02 13:43:55.069520 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.068482    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 02 13:43:55.069520 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.068485    2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 02 13:43:55.069520 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.068488    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 02 13:43:55.069520 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.068491    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 02 13:43:55.069520 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.068493    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 02 13:43:55.070042 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.068496    2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 02 13:43:55.070042 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.068499    2558 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 02 13:43:55.070042 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.068501    2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 02 13:43:55.070042 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.068504    2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 02 13:43:55.070042 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.068506    2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 02 13:43:55.070042 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.068509    2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 02 13:43:55.070042 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.068512    2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 02 13:43:55.070042 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.068515    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 02 13:43:55.070042 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.068517    2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 02 13:43:55.070042 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.068520    2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 02 13:43:55.070042 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.068522    2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 02 13:43:55.070042 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.068525    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 02 13:43:55.070042 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.068527    2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 02 13:43:55.070042 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.068530    2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 02 13:43:55.070042 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.068532    2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 02 13:43:55.070042 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.068535    2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 02 13:43:55.070042 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.068537    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 02 13:43:55.070042 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.068540    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 02 13:43:55.070042 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.068543    2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 02 13:43:55.070042 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.068546    2558 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 02 13:43:55.070526 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.068548    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 02 13:43:55.070526 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.068551    2558 feature_gate.go:328] unrecognized feature gate: Example2
Jul 02 13:43:55.070526 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.068553    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 02 13:43:55.070526 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.068556    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 02 13:43:55.070526 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.068558    2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 02 13:43:55.070526 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.068561    2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 02 13:43:55.070526 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.068563    2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 02 13:43:55.070526 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.068566    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 02 13:43:55.070526 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.068568    2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 02 13:43:55.070526 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.068570    2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 02 13:43:55.070526 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.068574    2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 02 13:43:55.070526 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.068577    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 02 13:43:55.070526 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.068581    2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 02 13:43:55.070526 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.068584    2558 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 02 13:43:55.070526 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.069305    2558 feature_gate.go:328] unrecognized feature gate: Example2
Jul 02 13:43:55.070526 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.069312    2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 02 13:43:55.070526 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.069315    2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 02 13:43:55.070526 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.069319    2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 02 13:43:55.070526 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.069322    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 02 13:43:55.070526 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.069326    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 02 13:43:55.071062 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.069329    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 02 13:43:55.071062 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.069332    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 02 13:43:55.071062 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.069335    2558 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 02 13:43:55.071062 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.069338    2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 02 13:43:55.071062 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.069341    2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 02 13:43:55.071062 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.069344    2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 02 13:43:55.071062 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.069346    2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 02 13:43:55.071062 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.069349    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 02 13:43:55.071062 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.069351    2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 02 13:43:55.071062 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.069354    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 02 13:43:55.071062 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.069356    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 02 13:43:55.071062 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.069361    2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 02 13:43:55.071062 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.069365    2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 02 13:43:55.071062 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.069368    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 02 13:43:55.071062 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.069371    2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 02 13:43:55.071062 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.069374    2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 02 13:43:55.071062 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.069376    2558 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 02 13:43:55.071062 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.069379    2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 02 13:43:55.071062 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.069382    2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 02 13:43:55.071528 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.069385    2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 02 13:43:55.071528 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.069387    2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 02 13:43:55.071528 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.069390    2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 02 13:43:55.071528 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.069393    2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 02 13:43:55.071528 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.069396    2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 02 13:43:55.071528 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.069399    2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 02 13:43:55.071528 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.069401    2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 02 13:43:55.071528 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.069404    2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 02 13:43:55.071528 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.069406    2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 02 13:43:55.071528 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.069409    2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 02 13:43:55.071528 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.069412    2558 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 02 13:43:55.071528 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.069415    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 02 13:43:55.071528 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.069417    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 02 13:43:55.071528 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.069420    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 02 13:43:55.071528 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.069422    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 02 13:43:55.071528 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.069425    2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 02 13:43:55.071528 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.069428    2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 02 13:43:55.071528 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.069430    2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 02 13:43:55.071528 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.069433    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 02 13:43:55.071528 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.069435    2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 02 13:43:55.072073 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.069438    2558 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 02 13:43:55.072073 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.069440    2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 02 13:43:55.072073 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.069443    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 02 13:43:55.072073 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.069445    2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 02 13:43:55.072073 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.069450    2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 02 13:43:55.072073 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.069453    2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 02 13:43:55.072073 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.069456    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 02 13:43:55.072073 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.069459    2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 02 13:43:55.072073 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.069461    2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 02 13:43:55.072073 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.069464    2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 02 13:43:55.072073 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.069467    2558 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 02 13:43:55.072073 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.069470    2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 02 13:43:55.072073 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.069472    2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 02 13:43:55.072073 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.069475    2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 02 13:43:55.072073 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.069479    2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 02 13:43:55.072073 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.069482    2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 02 13:43:55.072073 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.069484    2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 02 13:43:55.072073 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.069487    2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 02 13:43:55.072073 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.069489    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 02 13:43:55.072535 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.069493    2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 02 13:43:55.072535 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.069496    2558 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 02 13:43:55.072535 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.069498    2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 02 13:43:55.072535 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.069501    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 02 13:43:55.072535 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.069504    2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 02 13:43:55.072535 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.069507    2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 02 13:43:55.072535 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.069509    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 02 13:43:55.072535 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.069512    2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 02 13:43:55.072535 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.069515    2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 02 13:43:55.072535 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.069517    2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 02 13:43:55.072535 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.069520    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 02 13:43:55.072535 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.069523    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 02 13:43:55.072535 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.069525    2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 02 13:43:55.072535 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.069528    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 02 13:43:55.072535 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.069532    2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 02 13:43:55.072535 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.069534    2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 02 13:43:55.072535 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.069537    2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 02 13:43:55.072535 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.069539    2558 feature_gate.go:328] unrecognized feature gate: Example
Jul 02 13:43:55.072535 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.069542    2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 02 13:43:55.072535 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.069546    2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 02 13:43:55.073044 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.069548    2558 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 02 13:43:55.073044 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.069551    2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 02 13:43:55.073044 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.069553    2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 02 13:43:55.073044 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.069556    2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 02 13:43:55.073044 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.069558    2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 02 13:43:55.073044 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.069561    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 02 13:43:55.073044 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.069564    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 02 13:43:55.073044 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.069566    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 02 13:43:55.073044 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.069569    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 02 13:43:55.073044 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.069571    2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 02 13:43:55.073044 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.069574    2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 02 13:43:55.073044 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.069576    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 02 13:43:55.073044 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.069579    2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 02 13:43:55.073044 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.069582    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 02 13:43:55.073044 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.069584    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 02 13:43:55.073044 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.069976    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 02 13:43:55.073044 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.069982    2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 02 13:43:55.073044 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.069984    2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 02 13:43:55.073044 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.069987    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 02 13:43:55.073508 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.069990    2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 02 13:43:55.073508 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.069992    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 02 13:43:55.073508 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.069995    2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 02 13:43:55.073508 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.069999    2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 02 13:43:55.073508 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.070002    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 02 13:43:55.073508 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.070005    2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 02 13:43:55.073508 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.070009    2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 02 13:43:55.073508 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.070013    2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 02 13:43:55.073508 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.070016    2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 02 13:43:55.073508 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.070019    2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 02 13:43:55.073508 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.070022    2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 02 13:43:55.073508 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.070025    2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 02 13:43:55.073508 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.070028    2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 02 13:43:55.073508 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.070031    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 02 13:43:55.073508 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.070033    2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 02 13:43:55.073508 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.070036    2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 02 13:43:55.073508 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.070038    2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 02 13:43:55.073508 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.070041    2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 02 13:43:55.073508 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.070044    2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 02 13:43:55.073508 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.070046    2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 02 13:43:55.074016 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.070049    2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 02 13:43:55.074016 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.070052    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 02 13:43:55.074016 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.070055    2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 02 13:43:55.074016 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.070058    2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 02 13:43:55.074016 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.070060    2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 02 13:43:55.074016 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.070063    2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 02 13:43:55.074016 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.070066    2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 02 13:43:55.074016 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.070068    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 02 13:43:55.074016 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.070071    2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 02 13:43:55.074016 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.070073    2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 02 13:43:55.074016 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.070076    2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 02 13:43:55.074016 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.070079    2558 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 02 13:43:55.074016 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.070082    2558 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 02 13:43:55.074016 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.070084    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 02 13:43:55.074016 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.070087    2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 02 13:43:55.074016 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.070089    2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 02 13:43:55.074016 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.070092    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 02 13:43:55.074016 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.070095    2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 02 13:43:55.074016 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.070098    2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 02 13:43:55.074016 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.070100    2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 02 13:43:55.074519 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.070103    2558 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 02 13:43:55.074519 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.070105    2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 02 13:43:55.074519 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.070108    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 02 13:43:55.074519 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.070110    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 02 13:43:55.074519 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.070113    2558 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 02 13:43:55.074519 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.070116    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 02 13:43:55.074519 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.070126    2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 02 13:43:55.074519 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.070129    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 02 13:43:55.074519 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.070131    2558 feature_gate.go:328] unrecognized feature gate: Example2
Jul 02 13:43:55.074519 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.070134    2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 02 13:43:55.074519 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.070136    2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 02 13:43:55.074519 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.070138    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 02 13:43:55.074519 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.070141    2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 02 13:43:55.074519 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.070143    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 02 13:43:55.074519 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.070146    2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 02 13:43:55.074519 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.070149    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 02 13:43:55.074519 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.070151    2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 02 13:43:55.074519 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.070154    2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 02 13:43:55.074519 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.070156    2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 02 13:43:55.074519 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.070159    2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 02 13:43:55.075028 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.070161    2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 02 13:43:55.075028 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.070164    2558 feature_gate.go:328] unrecognized feature gate: Example
Jul 02 13:43:55.075028 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.070166    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 02 13:43:55.075028 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.070169    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 02 13:43:55.075028 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.070171    2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 02 13:43:55.075028 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.070174    2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 02 13:43:55.075028 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.070176    2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 02 13:43:55.075028 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.070179    2558 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 02 13:43:55.075028 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.070182    2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 02 13:43:55.075028 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.070184    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 02 13:43:55.075028 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.070187    2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 02 13:43:55.075028 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.070189    2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 02 13:43:55.075028 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.070191    2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 02 13:43:55.075028 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.070194    2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 02 13:43:55.075028 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.070197    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 02 13:43:55.075028 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.070199    2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 02 13:43:55.075028 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.070201    2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 02 13:43:55.075028 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.070204    2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 02 13:43:55.075028 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.070207    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 02 13:43:55.075028 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.070210    2558 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 02 13:43:55.075516 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.070214    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 02 13:43:55.075516 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.070216    2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 02 13:43:55.075516 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.070219    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 02 13:43:55.075516 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.070221    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 02 13:43:55.075516 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.070224    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 02 13:43:55.075516 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.070226    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 02 13:43:55.075516 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.070229    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 02 13:43:55.075516 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.070232    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 02 13:43:55.075516 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.070236    2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 02 13:43:55.075516 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.070239    2558 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 02 13:43:55.075516 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.070243    2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 02 13:43:55.075516 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.070246    2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 02 13:43:55.075516 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.070249    2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 02 13:43:55.075516 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.070251    2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 02 13:43:55.075516 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.070254    2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 02 13:43:55.075516 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070324    2558 flags.go:64] FLAG: --address="0.0.0.0"
Jul 02 13:43:55.075516 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070331    2558 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 02 13:43:55.075516 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070337    2558 flags.go:64] FLAG: --anonymous-auth="true"
Jul 02 13:43:55.075516 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070341    2558 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 02 13:43:55.075516 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070345    2558 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 02 13:43:55.076038 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070349    2558 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 02 13:43:55.076038 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070352    2558 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 02 13:43:55.076038 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070356    2558 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 02 13:43:55.076038 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070359    2558 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 02 13:43:55.076038 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070362    2558 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 02 13:43:55.076038 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070366    2558 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 02 13:43:55.076038 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070369    2558 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 02 13:43:55.076038 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070372    2558 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 02 13:43:55.076038 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070375    2558 flags.go:64] FLAG: --cgroup-root=""
Jul 02 13:43:55.076038 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070378    2558 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 02 13:43:55.076038 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070381    2558 flags.go:64] FLAG: --client-ca-file=""
Jul 02 13:43:55.076038 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070391    2558 flags.go:64] FLAG: --cloud-provider="external"
Jul 02 13:43:55.076038 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070394    2558 flags.go:64] FLAG: --cluster-dns="[]"
Jul 02 13:43:55.076038 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070398    2558 flags.go:64] FLAG: --cluster-domain=""
Jul 02 13:43:55.076038 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070401    2558 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 02 13:43:55.076038 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070404    2558 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 02 13:43:55.076038 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070407    2558 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 02 13:43:55.076038 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070410    2558 flags.go:64] FLAG: --container-log-max-files="5"
Jul 02 13:43:55.076038 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070417    2558 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 02 13:43:55.076038 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070420    2558 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 02 13:43:55.076038 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070423    2558 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 02 13:43:55.076038 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070426    2558 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 02 13:43:55.076038 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070430    2558 flags.go:64] FLAG: --contention-profiling="false"
Jul 02 13:43:55.076038 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070433    2558 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 02 13:43:55.076621 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070435    2558 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 02 13:43:55.076621 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070439    2558 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 02 13:43:55.076621 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070441    2558 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 02 13:43:55.076621 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070445    2558 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 02 13:43:55.076621 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070448    2558 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 02 13:43:55.076621 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070451    2558 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 02 13:43:55.076621 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070454    2558 flags.go:64] FLAG: --enable-load-reader="false"
Jul 02 13:43:55.076621 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070459    2558 flags.go:64] FLAG: --enable-server="true"
Jul 02 13:43:55.076621 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070462    2558 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 02 13:43:55.076621 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070467    2558 flags.go:64] FLAG: --event-burst="100"
Jul 02 13:43:55.076621 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070469    2558 flags.go:64] FLAG: --event-qps="50"
Jul 02 13:43:55.076621 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070472    2558 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 02 13:43:55.076621 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070475    2558 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 02 13:43:55.076621 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070478    2558 flags.go:64] FLAG: --eviction-hard=""
Jul 02 13:43:55.076621 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070482    2558 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 02 13:43:55.076621 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070485    2558 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 02 13:43:55.076621 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070489    2558 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 02 13:43:55.076621 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070492    2558 flags.go:64] FLAG: --eviction-soft=""
Jul 02 13:43:55.076621 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070495    2558 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 02 13:43:55.076621 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070497    2558 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 02 13:43:55.076621 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070500    2558 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 02 13:43:55.076621 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070503    2558 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 02 13:43:55.076621 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070506    2558 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 02 13:43:55.076621 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070509    2558 flags.go:64] FLAG: --fail-swap-on="true"
Jul 02 13:43:55.076621 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070511    2558 flags.go:64] FLAG: --feature-gates=""
Jul 02 13:43:55.077304 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070515    2558 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 02 13:43:55.077304 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070522    2558 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 02 13:43:55.077304 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070525    2558 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 02 13:43:55.077304 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070528    2558 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 02 13:43:55.077304 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070531    2558 flags.go:64] FLAG: --healthz-port="10248"
Jul 02 13:43:55.077304 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070534    2558 flags.go:64] FLAG: --help="false"
Jul 02 13:43:55.077304 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070537    2558 flags.go:64] FLAG: --hostname-override="ip-10-0-133-209.ec2.internal"
Jul 02 13:43:55.077304 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070540    2558 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 02 13:43:55.077304 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070543    2558 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 02 13:43:55.077304 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070546    2558 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 02 13:43:55.077304 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070550    2558 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 02 13:43:55.077304 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070553    2558 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 02 13:43:55.077304 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070556    2558 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 02 13:43:55.077304 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070559    2558 flags.go:64] FLAG: --image-service-endpoint=""
Jul 02 13:43:55.077304 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070563    2558 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 02 13:43:55.077304 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070566    2558 flags.go:64] FLAG: --kube-api-burst="100"
Jul 02 13:43:55.077304 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070569    2558 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 02 13:43:55.077304 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070572    2558 flags.go:64] FLAG: --kube-api-qps="50"
Jul 02 13:43:55.077304 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070575    2558 flags.go:64] FLAG: --kube-reserved=""
Jul 02 13:43:55.077304 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070577    2558 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 02 13:43:55.077304 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070580    2558 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 02 13:43:55.077304 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070583    2558 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 02 13:43:55.077304 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070586    2558 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 02 13:43:55.077304 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070589    2558 flags.go:64] FLAG: --lock-file=""
Jul 02 13:43:55.077914 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070591    2558 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 02 13:43:55.077914 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070594    2558 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 02 13:43:55.077914 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070597    2558 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 02 13:43:55.077914 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070602    2558 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 02 13:43:55.077914 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070606    2558 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 02 13:43:55.077914 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070609    2558 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 02 13:43:55.077914 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070611    2558 flags.go:64] FLAG: --logging-format="text"
Jul 02 13:43:55.077914 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070614    2558 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 02 13:43:55.077914 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070617    2558 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 02 13:43:55.077914 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070622    2558 flags.go:64] FLAG: --manifest-url=""
Jul 02 13:43:55.077914 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070624    2558 flags.go:64] FLAG: --manifest-url-header=""
Jul 02 13:43:55.077914 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070629    2558 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 02 13:43:55.077914 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070631    2558 flags.go:64] FLAG: --max-open-files="1000000"
Jul 02 13:43:55.077914 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070635    2558 flags.go:64] FLAG: --max-pods="110"
Jul 02 13:43:55.077914 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070638    2558 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 02 13:43:55.077914 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070641    2558 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 02 13:43:55.077914 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070644    2558 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 02 13:43:55.077914 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070647    2558 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 02 13:43:55.077914 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070650    2558 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 02 13:43:55.077914 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070653    2558 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 02 13:43:55.077914 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070656    2558 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 02 13:43:55.077914 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070662    2558 flags.go:64] FLAG: --node-status-max-images="50"
Jul 02 13:43:55.077914 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070666    2558 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 02 13:43:55.077914 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070670    2558 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 02 13:43:55.077914 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070673    2558 flags.go:64] FLAG: --pod-cidr=""
Jul 02 13:43:55.078535 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070676    2558 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8a2632cd31d78133be7b5fb6999469b2acf9b98d077c7f6ae5d0d94083d76b04"
Jul 02 13:43:55.078535 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070681    2558 flags.go:64] FLAG: --pod-manifest-path=""
Jul 02 13:43:55.078535 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070684    2558 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 02 13:43:55.078535 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070687    2558 flags.go:64] FLAG: --pods-per-core="0"
Jul 02 13:43:55.078535 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070689    2558 flags.go:64] FLAG: --port="10250"
Jul 02 13:43:55.078535 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070693    2558 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 02 13:43:55.078535 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070695    2558 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0f5331035cdd7e3a2"
Jul 02 13:43:55.078535 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070698    2558 flags.go:64] FLAG: --qos-reserved=""
Jul 02 13:43:55.078535 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070701    2558 flags.go:64] FLAG: --read-only-port="10255"
Jul 02 13:43:55.078535 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070704    2558 flags.go:64] FLAG: --register-node="true"
Jul 02 13:43:55.078535 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070707    2558 flags.go:64] FLAG: --register-with-taints=""
Jul 02 13:43:55.078535 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070711    2558 flags.go:64] FLAG: --registry-burst="10"
Jul 02 13:43:55.078535 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070714    2558 flags.go:64] FLAG: --registry-qps="5"
Jul 02 13:43:55.078535 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070717    2558 flags.go:64] FLAG: --reserved-cpus=""
Jul 02 13:43:55.078535 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070719    2558 flags.go:64] FLAG: --reserved-memory=""
Jul 02 13:43:55.078535 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070723    2558 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 02 13:43:55.078535 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070726    2558 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 02 13:43:55.078535 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070730    2558 flags.go:64] FLAG: --rotate-certificates="false"
Jul 02 13:43:55.078535 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070733    2558 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 02 13:43:55.078535 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070735    2558 flags.go:64] FLAG: --runonce="false"
Jul 02 13:43:55.078535 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070738    2558 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 02 13:43:55.078535 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070741    2558 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 02 13:43:55.078535 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070744    2558 flags.go:64] FLAG: --seccomp-default="false"
Jul 02 13:43:55.078535 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070747    2558 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 02 13:43:55.078535 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070749    2558 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 02 13:43:55.079166 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070752    2558 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 02 13:43:55.079166 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070755    2558 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 02 13:43:55.079166 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070758    2558 flags.go:64] FLAG: --storage-driver-password="root"
Jul 02 13:43:55.079166 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070761    2558 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 02 13:43:55.079166 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070765    2558 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 02 13:43:55.079166 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070768    2558 flags.go:64] FLAG: --storage-driver-user="root"
Jul 02 13:43:55.079166 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070771    2558 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 02 13:43:55.079166 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070773    2558 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 02 13:43:55.079166 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070776    2558 flags.go:64] FLAG: --system-cgroups=""
Jul 02 13:43:55.079166 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070779    2558 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 02 13:43:55.079166 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070784    2558 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 02 13:43:55.079166 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070787    2558 flags.go:64] FLAG: --tls-cert-file=""
Jul 02 13:43:55.079166 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070790    2558 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 02 13:43:55.079166 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070793    2558 flags.go:64] FLAG: --tls-min-version=""
Jul 02 13:43:55.079166 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070796    2558 flags.go:64] FLAG: --tls-private-key-file=""
Jul 02 13:43:55.079166 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070799    2558 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 02 13:43:55.079166 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070802    2558 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 02 13:43:55.079166 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070804    2558 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 02 13:43:55.079166 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070808    2558 flags.go:64] FLAG: --v="2"
Jul 02 13:43:55.079166 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070812    2558 flags.go:64] FLAG: --version="false"
Jul 02 13:43:55.079166 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070816    2558 flags.go:64] FLAG: --vmodule=""
Jul 02 13:43:55.079166 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070832    2558 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 02 13:43:55.079166 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.070836    2558 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 02 13:43:55.079166 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.070947    2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 02 13:43:55.079166 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.070952    2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 02 13:43:55.079766 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.070955    2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 02 13:43:55.079766 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.070958    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 02 13:43:55.079766 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.070961    2558 feature_gate.go:328] unrecognized feature gate: Example2
Jul 02 13:43:55.079766 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.070963    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 02 13:43:55.079766 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.070966    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 02 13:43:55.079766 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.070968    2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 02 13:43:55.079766 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.070971    2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 02 13:43:55.079766 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.070974    2558 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 02 13:43:55.079766 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.070976    2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 02 13:43:55.079766 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.070979    2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 02 13:43:55.079766 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.070981    2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 02 13:43:55.079766 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.070986    2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 02 13:43:55.079766 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.070988    2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 02 13:43:55.079766 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.070991    2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 02 13:43:55.079766 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.070993    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 02 13:43:55.079766 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.070996    2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 02 13:43:55.079766 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.070998    2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 02 13:43:55.079766 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.071001    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 02 13:43:55.079766 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.071004    2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 02 13:43:55.079766 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.071007    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 02 13:43:55.080270 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.071009    2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 02 13:43:55.080270 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.071012    2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 02 13:43:55.080270 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.071014    2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 02 13:43:55.080270 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.071017    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 02 13:43:55.080270 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.071019    2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 02 13:43:55.080270 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.071022    2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 02 13:43:55.080270 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.071025    2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 02 13:43:55.080270 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.071029    2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 02 13:43:55.080270 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.071032    2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 02 13:43:55.080270 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.071034    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 02 13:43:55.080270 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.071037    2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 02 13:43:55.080270 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.071040    2558 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 02 13:43:55.080270 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.071043    2558 feature_gate.go:328] unrecognized feature gate: Example
Jul 02 13:43:55.080270 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.071046    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 02 13:43:55.080270 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.071050    2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 02 13:43:55.080270 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.071053    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 02 13:43:55.080270 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.071056    2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 02 13:43:55.080270 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.071058    2558 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 02 13:43:55.080270 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.071061    2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 02 13:43:55.080741 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.071064    2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 02 13:43:55.080741 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.071067    2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 02 13:43:55.080741 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.071071    2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 02 13:43:55.080741 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.071075    2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 02 13:43:55.080741 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.071079    2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 02 13:43:55.080741 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.071082    2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 02 13:43:55.080741 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.071085    2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 02 13:43:55.080741 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.071088    2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 02 13:43:55.080741 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.071091    2558 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 02 13:43:55.080741 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.071094    2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 02 13:43:55.080741 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.071097    2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 02 13:43:55.080741 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.071099    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 02 13:43:55.080741 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.071102    2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 02 13:43:55.080741 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.071105    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 02 13:43:55.080741 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.071107    2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 02 13:43:55.080741 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.071110    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 02 13:43:55.080741 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.071113    2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 02 13:43:55.080741 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.071116    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 02 13:43:55.080741 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.071118    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 02 13:43:55.081222 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.071121    2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 02 13:43:55.081222 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.071123    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 02 13:43:55.081222 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.071126    2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 02 13:43:55.081222 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.071129    2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 02 13:43:55.081222 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.071131    2558 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 02 13:43:55.081222 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.071135    2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 02 13:43:55.081222 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.071138    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 02 13:43:55.081222 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.071140    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 02 13:43:55.081222 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.071143    2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 02 13:43:55.081222 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.071146    2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 02 13:43:55.081222 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.071148    2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 02 13:43:55.081222 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.071151    2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 02 13:43:55.081222 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.071153    2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 02 13:43:55.081222 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.071156    2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 02 13:43:55.081222 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.071158    2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 02 13:43:55.081222 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.071161    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 02 13:43:55.081222 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.071164    2558 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 02 13:43:55.081222 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.071168    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 02 13:43:55.081222 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.071170    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 02 13:43:55.081222 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.071173    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 02 13:43:55.081721 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.071175    2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 02 13:43:55.081721 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.071178    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 02 13:43:55.081721 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.071180    2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 02 13:43:55.081721 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.071182    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 02 13:43:55.081721 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.071185    2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 02 13:43:55.081721 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.071187    2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 02 13:43:55.081721 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.071190    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 02 13:43:55.081721 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.071192    2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 02 13:43:55.081721 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.071195    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 02 13:43:55.081721 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.071197    2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 02 13:43:55.081721 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.071200    2558 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 02 13:43:55.081721 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.071203    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 02 13:43:55.081721 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.071205    2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 02 13:43:55.081721 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.071208    2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 02 13:43:55.081721 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.071210    2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 02 13:43:55.081721 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.071212    2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 02 13:43:55.081721 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.071215    2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 02 13:43:55.081721 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.071220    2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 02 13:43:55.081721 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.071223    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 02 13:43:55.082323 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.071766    2558 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 02 13:43:55.082323 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.078398    2558 server.go:529] "Kubelet version" kubeletVersion="v1.34.8"
Jul 02 13:43:55.082323 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.078414    2558 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 02 13:43:55.082323 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.078426    2558 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 02 13:43:55.082323 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.078432    2558 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 02 13:43:55.082323 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.078506    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 02 13:43:55.082323 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.078513    2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 02 13:43:55.082323 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.078517    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 02 13:43:55.082323 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.078520    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 02 13:43:55.082323 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.078524    2558 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 02 13:43:55.082323 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.078527    2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 02 13:43:55.082323 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.078530    2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 02 13:43:55.082323 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.078533    2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 02 13:43:55.082323 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.078535    2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 02 13:43:55.082759 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.078538    2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 02 13:43:55.082759 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.078541    2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 02 13:43:55.082759 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.078543    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 02 13:43:55.082759 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.078546    2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 02 13:43:55.082759 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.078548    2558 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 02 13:43:55.082759 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.078553    2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 02 13:43:55.082759 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.078556    2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 02 13:43:55.082759 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.078559    2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 02 13:43:55.082759 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.078562    2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 02 13:43:55.082759 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.078565    2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 02 13:43:55.082759 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.078568    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 02 13:43:55.082759 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.078571    2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 02 13:43:55.082759 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.078574    2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 02 13:43:55.082759 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.078576    2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 02 13:43:55.082759 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.078579    2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 02 13:43:55.082759 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.078582    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 02 13:43:55.082759 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.078585    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 02 13:43:55.082759 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.078588    2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 02 13:43:55.082759 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.078591    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 02 13:43:55.083236 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.078593    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 02 13:43:55.083236 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.078596    2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 02 13:43:55.083236 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.078599    2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 02 13:43:55.083236 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.078602    2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 02 13:43:55.083236 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.078604    2558 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 02 13:43:55.083236 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.078607    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 02 13:43:55.083236 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.078609    2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 02 13:43:55.083236 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.078612    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 02 13:43:55.083236 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.078614    2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 02 13:43:55.083236 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.078616    2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 02 13:43:55.083236 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.078619    2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 02 13:43:55.083236 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.078621    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 02 13:43:55.083236 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.078624    2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 02 13:43:55.083236 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.078627    2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 02 13:43:55.083236 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.078629    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 02 13:43:55.083236 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.078632    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 02 13:43:55.083236 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.078634    2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 02 13:43:55.083236 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.078636    2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 02 13:43:55.083236 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.078639    2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 02 13:43:55.083236 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.078641    2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 02 13:43:55.083714 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.078644    2558 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 02 13:43:55.083714 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.078646    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 02 13:43:55.083714 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.078649    2558 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 02 13:43:55.083714 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.078651    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 02 13:43:55.083714 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.078653    2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 02 13:43:55.083714 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.078656    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 02 13:43:55.083714 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.078659    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 02 13:43:55.083714 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.078662    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 02 13:43:55.083714 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.078664    2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 02 13:43:55.083714 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.078672    2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 02 13:43:55.083714 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.078675    2558 feature_gate.go:328] unrecognized feature gate: Example2
Jul 02 13:43:55.083714 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.078678    2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 02 13:43:55.083714 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.078680    2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 02 13:43:55.083714 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.078683    2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 02 13:43:55.083714 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.078688    2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 02 13:43:55.083714 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.078691    2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 02 13:43:55.083714 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.078694    2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 02 13:43:55.083714 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.078697    2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 02 13:43:55.083714 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.078699    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 02 13:43:55.084176 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.078702    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 02 13:43:55.084176 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.078704    2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 02 13:43:55.084176 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.078707    2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 02 13:43:55.084176 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.078709    2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 02 13:43:55.084176 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.078712    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 02 13:43:55.084176 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.078715    2558 feature_gate.go:328] unrecognized feature gate: Example
Jul 02 13:43:55.084176 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.078717    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 02 13:43:55.084176 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.078721    2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 02 13:43:55.084176 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.078724    2558 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 02 13:43:55.084176 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.078727    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 02 13:43:55.084176 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.078730    2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 02 13:43:55.084176 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.078732    2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 02 13:43:55.084176 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.078735    2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 02 13:43:55.084176 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.078737    2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 02 13:43:55.084176 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.078740    2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 02 13:43:55.084176 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.078742    2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 02 13:43:55.084176 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.078745    2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 02 13:43:55.084176 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.078747    2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 02 13:43:55.084176 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.078750    2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 02 13:43:55.084658 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.078752    2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 02 13:43:55.084658 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.078754    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 02 13:43:55.084658 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.078757    2558 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 02 13:43:55.084658 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.078759    2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 02 13:43:55.084658 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.078767    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 02 13:43:55.084658 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.078769    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 02 13:43:55.084658 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.078772    2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 02 13:43:55.084658 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.078775    2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 02 13:43:55.084658 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.078778    2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 02 13:43:55.084658 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.078781    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 02 13:43:55.084658 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.078783    2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 02 13:43:55.084658 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.078785    2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 02 13:43:55.084658 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.078788    2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 02 13:43:55.084658 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.078793    2558 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 02 13:43:55.084658 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.078967    2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 02 13:43:55.085117 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.078972    2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 02 13:43:55.085117 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.078975    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 02 13:43:55.085117 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.078977    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 02 13:43:55.085117 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.078980    2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 02 13:43:55.085117 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.078982    2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 02 13:43:55.085117 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.078985    2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 02 13:43:55.085117 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.078987    2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 02 13:43:55.085117 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.078989    2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 02 13:43:55.085117 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.078992    2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 02 13:43:55.085117 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.078994    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 02 13:43:55.085117 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.078997    2558 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 02 13:43:55.085117 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.078999    2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 02 13:43:55.085117 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.079002    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 02 13:43:55.085117 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.079004    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 02 13:43:55.085117 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.079006    2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 02 13:43:55.085117 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.079009    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 02 13:43:55.085117 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.079011    2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 02 13:43:55.085117 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.079014    2558 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 02 13:43:55.085117 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.079016    2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 02 13:43:55.085574 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.079018    2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 02 13:43:55.085574 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.079021    2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 02 13:43:55.085574 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.079023    2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 02 13:43:55.085574 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.079031    2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 02 13:43:55.085574 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.079034    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 02 13:43:55.085574 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.079037    2558 feature_gate.go:328] unrecognized feature gate: Example
Jul 02 13:43:55.085574 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.079039    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 02 13:43:55.085574 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.079042    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 02 13:43:55.085574 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.079044    2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 02 13:43:55.085574 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.079047    2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 02 13:43:55.085574 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.079049    2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 02 13:43:55.085574 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.079052    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 02 13:43:55.085574 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.079054    2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 02 13:43:55.085574 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.079056    2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 02 13:43:55.085574 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.079059    2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 02 13:43:55.085574 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.079061    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 02 13:43:55.085574 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.079064    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 02 13:43:55.085574 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.079066    2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 02 13:43:55.085574 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.079068    2558 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 02 13:43:55.085574 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.079071    2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 02 13:43:55.086206 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.079074    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 02 13:43:55.086206 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.079076    2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 02 13:43:55.086206 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.079079    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 02 13:43:55.086206 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.079081    2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 02 13:43:55.086206 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.079083    2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 02 13:43:55.086206 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.079086    2558 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 02 13:43:55.086206 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.079088    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 02 13:43:55.086206 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.079091    2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 02 13:43:55.086206 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.079093    2558 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 02 13:43:55.086206 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.079096    2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 02 13:43:55.086206 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.079098    2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 02 13:43:55.086206 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.079100    2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 02 13:43:55.086206 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.079103    2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 02 13:43:55.086206 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.079105    2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 02 13:43:55.086206 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.079108    2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 02 13:43:55.086206 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.079110    2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 02 13:43:55.086206 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.079122    2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 02 13:43:55.086206 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.079125    2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 02 13:43:55.086206 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.079128    2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 02 13:43:55.086206 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.079131    2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 02 13:43:55.086772 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.079134    2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 02 13:43:55.086772 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.079136    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 02 13:43:55.086772 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.079139    2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 02 13:43:55.086772 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.079141    2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 02 13:43:55.086772 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.079144    2558 feature_gate.go:328] unrecognized feature gate: Example2
Jul 02 13:43:55.086772 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.079146    2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 02 13:43:55.086772 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.079150    2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 02 13:43:55.086772 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.079153    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 02 13:43:55.086772 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.079156    2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 02 13:43:55.086772 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.079159    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 02 13:43:55.086772 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.079162    2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 02 13:43:55.086772 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.079164    2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 02 13:43:55.086772 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.079167    2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 02 13:43:55.086772 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.079170    2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 02 13:43:55.086772 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.079172    2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 02 13:43:55.086772 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.079175    2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 02 13:43:55.086772 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.079178    2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 02 13:43:55.086772 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.079181    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 02 13:43:55.086772 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.079184    2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 02 13:43:55.087257 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.079187    2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 02 13:43:55.087257 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.079189    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 02 13:43:55.087257 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.079192    2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 02 13:43:55.087257 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.079195    2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 02 13:43:55.087257 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.079198    2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 02 13:43:55.087257 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.079200    2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 02 13:43:55.087257 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.079203    2558 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 02 13:43:55.087257 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.079205    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 02 13:43:55.087257 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.079207    2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 02 13:43:55.087257 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.079210    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 02 13:43:55.087257 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.079219    2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 02 13:43:55.087257 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.079222    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 02 13:43:55.087257 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.079225    2558 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 02 13:43:55.087257 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.079227    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 02 13:43:55.087257 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.079230    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 02 13:43:55.087257 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.079232    2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 02 13:43:55.087257 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.079234    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 02 13:43:55.087257 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.079237    2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 02 13:43:55.087257 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.079239    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 02 13:43:55.087257 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:55.079242    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 02 13:43:55.087740 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.079247    2558 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 02 13:43:55.087740 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.079369    2558 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 02 13:43:55.087740 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.081265    2558 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 02 13:43:55.087740 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.082484    2558 server.go:1019] "Starting client certificate rotation"
Jul 02 13:43:55.087740 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.082590    2558 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 02 13:43:55.087740 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.082625    2558 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 02 13:43:55.087740 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.084868    2558 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 02 13:43:55.089961 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.089946    2558 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 02 13:43:55.099479 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.099462    2558 log.go:25] "Validated CRI v1 runtime API"
Jul 02 13:43:55.103790 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.103773    2558 log.go:25] "Validated CRI v1 image API"
Jul 02 13:43:55.104640 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.104617    2558 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 02 13:43:55.107631 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.107591    2558 fs.go:135] Filesystem UUIDs: map[792f69a6-2fca-4d6c-9b9d-011152d13d9f:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 a7b30b85-37cf-4c4d-9d96-76b4b399ccb3:/dev/nvme0n1p4]
Jul 02 13:43:55.107709 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.107629    2558 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 02 13:43:55.113075 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.112967    2558 manager.go:216] Machine: {Timestamp:2026-07-02 13:43:55.11246002 +0000 UTC m=+0.290922195 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3121226 MemoryCapacity:32812167168 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec27cca2bab83609ccc832f66d16672f SystemUUID:ec27cca2-bab8-3609-ccc8-32f66d16672f BootID:093d0494-332d-435b-a4e1-5bbb6a61aef5 Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406085632 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406081536 Type:vfs Inodes:4005391 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:1f:3d:37:5e:13 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:1f:3d:37:5e:13 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:da:c2:5d:4d:d4:88 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812167168 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 02 13:43:55.113075 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.113070    2558 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 02 13:43:55.113187 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.113156    2558 manager.go:232] Version: {KernelVersion:5.14.0-570.124.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260623-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 02 13:43:55.114294 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.114278    2558 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 02 13:43:55.117275 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.117246    2558 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 02 13:43:55.117409 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.117279    2558 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-133-209.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 02 13:43:55.117931 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.117921    2558 topology_manager.go:138] "Creating topology manager with none policy"
Jul 02 13:43:55.117962 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.117934    2558 container_manager_linux.go:309] "Creating device plugin manager"
Jul 02 13:43:55.117962 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.117941    2558 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 02 13:43:55.118540 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.118529    2558 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 02 13:43:55.119219 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.119210    2558 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 02 13:43:55.121901 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.121888    2558 state_mem.go:36] "Initialized new in-memory state store"
Jul 02 13:43:55.122021 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.122012    2558 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 02 13:43:55.122056 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.122043    2558 kubelet.go:486] "Attempting to sync node with API server"
Jul 02 13:43:55.122083 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.122060    2558 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 02 13:43:55.122083 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.122079    2558 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 02 13:43:55.122152 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.122088    2558 kubelet.go:398] "Adding apiserver pod source"
Jul 02 13:43:55.122152 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.122099    2558 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 02 13:43:55.122509 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.122497    2558 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 02 13:43:55.122568 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.122516    2558 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 02 13:43:55.124325 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.124298    2558 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1"
Jul 02 13:43:55.125518 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.125504    2558 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 02 13:43:55.125591 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.125535    2558 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 02 13:43:55.126159 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.126147    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 02 13:43:55.126212 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.126164    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 02 13:43:55.126212 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.126170    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 02 13:43:55.126212 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.126176    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 02 13:43:55.126212 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.126185    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 02 13:43:55.126212 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.126190    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 02 13:43:55.126212 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.126199    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 02 13:43:55.126212 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.126204    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 02 13:43:55.126212 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.126211    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 02 13:43:55.126212 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.126216    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 02 13:43:55.126493 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.126225    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 02 13:43:55.126493 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.126230    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 02 13:43:55.126863 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.126851    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 02 13:43:55.126920 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.126866    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 02 13:43:55.133111 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.133043    2558 server.go:1268] "Started kubelet"
Jul 02 13:43:55.133501 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.133474    2558 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 02 13:43:55.134227 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.134175    2558 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 02 13:43:55.134302 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.134258    2558 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 02 13:43:55.135231 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.135197    2558 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 02 13:43:55.135613 ip-10-0-133-209 systemd[1]: Started Kubernetes Kubelet.
Jul 02 13:43:55.136752 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:43:55.136648    2558 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-133-209.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 02 13:43:55.136752 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:43:55.136712    2558 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 02 13:43:55.136752 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.136722    2558 server.go:310] "Adding debug handlers to kubelet server"
Jul 02 13:43:55.136985 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.136765    2558 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-133-209.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 02 13:43:55.138877 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:43:55.138845    2558 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 02 13:43:55.139669 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.139655    2558 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-blh8m"
Jul 02 13:43:55.142070 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.142050    2558 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 02 13:43:55.142166 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.142056    2558 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 02 13:43:55.142232 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:43:55.142188    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-209.ec2.internal\" not found"
Jul 02 13:43:55.142752 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.142735    2558 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 02 13:43:55.142752 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.142749    2558 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 02 13:43:55.142957 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.142763    2558 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 02 13:43:55.142957 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.142873    2558 reconstruct.go:97] "Volume reconstruction finished"
Jul 02 13:43:55.142957 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.142883    2558 reconciler.go:29] "Reconciler: start to sync state"
Jul 02 13:43:55.143187 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.143158    2558 factory.go:55] Registering systemd factory
Jul 02 13:43:55.143187 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.143174    2558 factory.go:223] Registration of the systemd container factory successfully
Jul 02 13:43:55.143398 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.143381    2558 factory.go:153] Registering CRI-O factory
Jul 02 13:43:55.143398 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.143397    2558 factory.go:223] Registration of the crio container factory successfully
Jul 02 13:43:55.143505 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.143450    2558 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 02 13:43:55.143505 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.143473    2558 factory.go:103] Registering Raw factory
Jul 02 13:43:55.143505 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.143486    2558 manager.go:1196] Started watching for new ooms in manager
Jul 02 13:43:55.143896 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.143881    2558 manager.go:349] Starting recovery of all containers
Jul 02 13:43:55.144818 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:43:55.144068    2558 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-209.ec2.internal.18be7ce73f809419  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-209.ec2.internal,UID:ip-10-0-133-209.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-133-209.ec2.internal,},FirstTimestamp:2026-07-02 13:43:55.129975833 +0000 UTC m=+0.308438007,LastTimestamp:2026-07-02 13:43:55.129975833 +0000 UTC m=+0.308438007,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-209.ec2.internal,}"
Jul 02 13:43:55.151412 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:43:55.151383    2558 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-133-209.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 02 13:43:55.151511 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:43:55.151429    2558 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 02 13:43:55.154274 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.154255    2558 manager.go:354] Recovery completed
Jul 02 13:43:55.155748 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:43:55.155728    2558 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory
Jul 02 13:43:55.156167 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.156152    2558 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-blh8m"
Jul 02 13:43:55.158979 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.158955    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 02 13:43:55.161046 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.161028    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-209.ec2.internal" event="NodeHasSufficientMemory"
Jul 02 13:43:55.161097 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.161062    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-209.ec2.internal" event="NodeHasNoDiskPressure"
Jul 02 13:43:55.161097 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.161082    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-209.ec2.internal" event="NodeHasSufficientPID"
Jul 02 13:43:55.161636 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.161621    2558 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 02 13:43:55.161672 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.161636    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 02 13:43:55.161710 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.161637    2558 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 02 13:43:55.161710 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.161697    2558 state_mem.go:36] "Initialized new in-memory state store"
Jul 02 13:43:55.162321 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.162308    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-209.ec2.internal" event="NodeHasSufficientMemory"
Jul 02 13:43:55.162384 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.162333    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-209.ec2.internal" event="NodeHasNoDiskPressure"
Jul 02 13:43:55.162384 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.162343    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-209.ec2.internal" event="NodeHasSufficientPID"
Jul 02 13:43:55.162722 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.162711    2558 policy_none.go:49] "None policy: Start"
Jul 02 13:43:55.162758 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.162726    2558 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 02 13:43:55.162758 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.162736    2558 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 02 13:43:55.163682 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.163658    2558 policy_none.go:47] "Start"
Jul 02 13:43:55.163741 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:43:55.163660    2558 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-209.ec2.internal.18be7ce7415ab248  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-209.ec2.internal,UID:ip-10-0-133-209.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-133-209.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-133-209.ec2.internal,},FirstTimestamp:2026-07-02 13:43:55.161047624 +0000 UTC m=+0.339509799,LastTimestamp:2026-07-02 13:43:55.161047624 +0000 UTC m=+0.339509799,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-209.ec2.internal,}"
Jul 02 13:43:55.199510 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.199488    2558 manager.go:337] "Starting Device Plugin manager"
Jul 02 13:43:55.214455 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:43:55.199536    2558 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 02 13:43:55.214455 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.199549    2558 server.go:87] "Starting device plugin registration server"
Jul 02 13:43:55.214455 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.199802    2558 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 02 13:43:55.214455 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.199812    2558 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 02 13:43:55.214455 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.199911    2558 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 02 13:43:55.214455 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.200037    2558 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 02 13:43:55.214455 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.200046    2558 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 02 13:43:55.214455 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:43:55.200520    2558 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 02 13:43:55.214455 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:43:55.200565    2558 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-133-209.ec2.internal\" not found"
Jul 02 13:43:55.214455 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.209960    2558 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 02 13:43:55.214455 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.211262    2558 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 02 13:43:55.214455 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.211301    2558 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 02 13:43:55.214455 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.211321    2558 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 02 13:43:55.214455 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:43:55.211353    2558 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 02 13:43:55.218253 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.218233    2558 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 02 13:43:55.300331 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.300286    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 02 13:43:55.301350 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.301330    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-209.ec2.internal" event="NodeHasSufficientMemory"
Jul 02 13:43:55.301428 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.301370    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-209.ec2.internal" event="NodeHasNoDiskPressure"
Jul 02 13:43:55.301428 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.301386    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-209.ec2.internal" event="NodeHasSufficientPID"
Jul 02 13:43:55.301428 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.301417    2558 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-133-209.ec2.internal"
Jul 02 13:43:55.310090 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.310069    2558 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-133-209.ec2.internal"
Jul 02 13:43:55.310142 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:43:55.310096    2558 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-133-209.ec2.internal\": node \"ip-10-0-133-209.ec2.internal\" not found"
Jul 02 13:43:55.311426 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.311406    2558 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-209.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-133-209.ec2.internal"]
Jul 02 13:43:55.311478 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.311469    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 02 13:43:55.312315 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.312301    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-209.ec2.internal" event="NodeHasSufficientMemory"
Jul 02 13:43:55.312384 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.312330    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-209.ec2.internal" event="NodeHasNoDiskPressure"
Jul 02 13:43:55.312384 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.312339    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-209.ec2.internal" event="NodeHasSufficientPID"
Jul 02 13:43:55.313554 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.313491    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 02 13:43:55.313554 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.313544    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-209.ec2.internal"
Jul 02 13:43:55.313636 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.313571    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 02 13:43:55.314228 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.314210    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-209.ec2.internal" event="NodeHasSufficientMemory"
Jul 02 13:43:55.314321 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.314228    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-209.ec2.internal" event="NodeHasSufficientMemory"
Jul 02 13:43:55.314321 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.314240    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-209.ec2.internal" event="NodeHasNoDiskPressure"
Jul 02 13:43:55.314321 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.314249    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-209.ec2.internal" event="NodeHasNoDiskPressure"
Jul 02 13:43:55.314321 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.314251    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-209.ec2.internal" event="NodeHasSufficientPID"
Jul 02 13:43:55.314321 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.314262    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-209.ec2.internal" event="NodeHasSufficientPID"
Jul 02 13:43:55.315322 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.315307    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-209.ec2.internal"
Jul 02 13:43:55.315411 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.315332    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 02 13:43:55.315948 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.315931    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-209.ec2.internal" event="NodeHasSufficientMemory"
Jul 02 13:43:55.316023 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.315952    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-209.ec2.internal" event="NodeHasNoDiskPressure"
Jul 02 13:43:55.316023 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.315962    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-209.ec2.internal" event="NodeHasSufficientPID"
Jul 02 13:43:55.325490 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:43:55.325472    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-209.ec2.internal\" not found"
Jul 02 13:43:55.339619 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:43:55.339601    2558 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-209.ec2.internal\" not found" node="ip-10-0-133-209.ec2.internal"
Jul 02 13:43:55.344122 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.344104    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/44e46b482fd93a3b06e0f6824e9467c5-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-209.ec2.internal\" (UID: \"44e46b482fd93a3b06e0f6824e9467c5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-209.ec2.internal"
Jul 02 13:43:55.344192 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.344128    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/44e46b482fd93a3b06e0f6824e9467c5-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-209.ec2.internal\" (UID: \"44e46b482fd93a3b06e0f6824e9467c5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-209.ec2.internal"
Jul 02 13:43:55.344192 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.344145    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4d8a265dbb955726a248c0c5b7ee4114-config\") pod \"kube-apiserver-proxy-ip-10-0-133-209.ec2.internal\" (UID: \"4d8a265dbb955726a248c0c5b7ee4114\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-209.ec2.internal"
Jul 02 13:43:55.344253 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:43:55.344242    2558 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-209.ec2.internal\" not found" node="ip-10-0-133-209.ec2.internal"
Jul 02 13:43:55.425972 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:43:55.425948    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-209.ec2.internal\" not found"
Jul 02 13:43:55.444506 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.444482    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4d8a265dbb955726a248c0c5b7ee4114-config\") pod \"kube-apiserver-proxy-ip-10-0-133-209.ec2.internal\" (UID: \"4d8a265dbb955726a248c0c5b7ee4114\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-209.ec2.internal"
Jul 02 13:43:55.444581 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.444516    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/44e46b482fd93a3b06e0f6824e9467c5-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-209.ec2.internal\" (UID: \"44e46b482fd93a3b06e0f6824e9467c5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-209.ec2.internal"
Jul 02 13:43:55.444581 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.444539    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/44e46b482fd93a3b06e0f6824e9467c5-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-209.ec2.internal\" (UID: \"44e46b482fd93a3b06e0f6824e9467c5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-209.ec2.internal"
Jul 02 13:43:55.444642 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.444589    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/44e46b482fd93a3b06e0f6824e9467c5-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-209.ec2.internal\" (UID: \"44e46b482fd93a3b06e0f6824e9467c5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-209.ec2.internal"
Jul 02 13:43:55.444642 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.444589    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4d8a265dbb955726a248c0c5b7ee4114-config\") pod \"kube-apiserver-proxy-ip-10-0-133-209.ec2.internal\" (UID: \"4d8a265dbb955726a248c0c5b7ee4114\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-209.ec2.internal"
Jul 02 13:43:55.444642 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.444589    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/44e46b482fd93a3b06e0f6824e9467c5-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-209.ec2.internal\" (UID: \"44e46b482fd93a3b06e0f6824e9467c5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-209.ec2.internal"
Jul 02 13:43:55.526648 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:43:55.526598    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-209.ec2.internal\" not found"
Jul 02 13:43:55.627499 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:43:55.627417    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-209.ec2.internal\" not found"
Jul 02 13:43:55.642637 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.642610    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-209.ec2.internal"
Jul 02 13:43:55.646807 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:55.646667    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-209.ec2.internal"
Jul 02 13:43:55.727771 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:43:55.727722    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-209.ec2.internal\" not found"
Jul 02 13:43:55.828280 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:43:55.828245    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-209.ec2.internal\" not found"
Jul 02 13:43:55.928933 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:43:55.928853    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-209.ec2.internal\" not found"
Jul 02 13:43:56.029453 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:43:56.029423    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-209.ec2.internal\" not found"
Jul 02 13:43:56.082753 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:56.082724    2558 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 02 13:43:56.083312 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:56.082934    2558 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 02 13:43:56.130412 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:43:56.130360    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-209.ec2.internal\" not found"
Jul 02 13:43:56.159920 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:56.159866    2558 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-01 13:38:55 +0000 UTC" deadline="2028-04-10 09:42:00.66395719 +0000 UTC"
Jul 02 13:43:56.159920 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:56.159906    2558 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15547h58m4.504055177s"
Jul 02 13:43:56.181530 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:56.181455    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d8a265dbb955726a248c0c5b7ee4114.slice/crio-bfc0bb96cd7824261529e64db2ceb1bb55795f6e174d0ff4b017e01d17d68dc8 WatchSource:0}: Error finding container bfc0bb96cd7824261529e64db2ceb1bb55795f6e174d0ff4b017e01d17d68dc8: Status 404 returned error can't find the container with id bfc0bb96cd7824261529e64db2ceb1bb55795f6e174d0ff4b017e01d17d68dc8
Jul 02 13:43:56.181906 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:56.181875    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44e46b482fd93a3b06e0f6824e9467c5.slice/crio-2c07d78bc85c3d27c2ea424f4a788b7cb6f3812890cdeafe60cf926907c79b37 WatchSource:0}: Error finding container 2c07d78bc85c3d27c2ea424f4a788b7cb6f3812890cdeafe60cf926907c79b37: Status 404 returned error can't find the container with id 2c07d78bc85c3d27c2ea424f4a788b7cb6f3812890cdeafe60cf926907c79b37
Jul 02 13:43:56.186738 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:56.186718    2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 02 13:43:56.189912 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:56.189804    2558 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 02 13:43:56.215598 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:56.215544    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-209.ec2.internal" event={"ID":"44e46b482fd93a3b06e0f6824e9467c5","Type":"ContainerStarted","Data":"2c07d78bc85c3d27c2ea424f4a788b7cb6f3812890cdeafe60cf926907c79b37"}
Jul 02 13:43:56.216679 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:56.216657    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-209.ec2.internal" event={"ID":"4d8a265dbb955726a248c0c5b7ee4114","Type":"ContainerStarted","Data":"bfc0bb96cd7824261529e64db2ceb1bb55795f6e174d0ff4b017e01d17d68dc8"}
Jul 02 13:43:56.230618 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:43:56.230568    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-209.ec2.internal\" not found"
Jul 02 13:43:56.320403 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:56.320378    2558 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 02 13:43:56.343042 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:56.343014    2558 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-209.ec2.internal"
Jul 02 13:43:56.354891 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:56.354872    2558 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 02 13:43:56.356581 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:56.356564    2558 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-209.ec2.internal"
Jul 02 13:43:56.360337 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:56.359022    2558 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 02 13:43:56.369757 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:56.369736    2558 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 02 13:43:57.124115 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.124078    2558 apiserver.go:52] "Watching apiserver"
Jul 02 13:43:57.135491 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.135457    2558 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 02 13:43:57.135971 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.135933    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-operator/iptables-alerter-lxhdr","openshift-ovn-kubernetes/ovnkube-node-8f2ll","kube-system/global-pull-secret-syncer-2tjcl","kube-system/konnectivity-agent-m4ndl","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tvfdz","openshift-dns/node-resolver-g9pll","openshift-image-registry/node-ca-t99mm","openshift-multus/multus-976zg","openshift-multus/multus-additional-cni-plugins-s85vc","openshift-multus/network-metrics-daemon-29552","kube-system/kube-apiserver-proxy-ip-10-0-133-209.ec2.internal","openshift-cluster-node-tuning-operator/tuned-scnm4","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-209.ec2.internal","openshift-network-diagnostics/network-check-target-q9w6j"]
Jul 02 13:43:57.138272 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.138249    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-t99mm"
Jul 02 13:43:57.140492 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.140468    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-scnm4"
Jul 02 13:43:57.141086 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.141067    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 02 13:43:57.141249 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.141229    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-4tz9n\""
Jul 02 13:43:57.141373 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.141357    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 02 13:43:57.141501 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.141483    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 02 13:43:57.142301 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.142285    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q9w6j"
Jul 02 13:43:57.142415 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:43:57.142393    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q9w6j" podUID="06e4af50-04c6-4c58-af10-0e0a28c1a261"
Jul 02 13:43:57.142529 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.142515    2558 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 02 13:43:57.143157 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.143121    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 02 13:43:57.143617 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.143595    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 02 13:43:57.143737 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.143715    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-6dxlx\""
Jul 02 13:43:57.144156 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.144137    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-m4ndl"
Jul 02 13:43:57.146558 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.146322    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tvfdz"
Jul 02 13:43:57.146688 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.146667    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 02 13:43:57.147267 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.147245    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 02 13:43:57.147517 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.147499    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-xv2cj\""
Jul 02 13:43:57.149007 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.148986    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 02 13:43:57.149245 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.149229    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 02 13:43:57.149482 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.149459    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-pqvtt\""
Jul 02 13:43:57.149552 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.149489    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 02 13:43:57.149771 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.149749    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-g9pll"
Jul 02 13:43:57.152936 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.152891    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-vrrgz\""
Jul 02 13:43:57.153139 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.153118    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 02 13:43:57.153442 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.153265    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 02 13:43:57.153442 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.153353    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-s85vc"
Jul 02 13:43:57.155510 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.155482    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m888j\" (UniqueName: \"kubernetes.io/projected/de0396fd-b15d-4e52-b0af-7e40954f53db-kube-api-access-m888j\") pod \"node-ca-t99mm\" (UID: \"de0396fd-b15d-4e52-b0af-7e40954f53db\") " pod="openshift-image-registry/node-ca-t99mm"
Jul 02 13:43:57.155631 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.155536    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/823f515b-2ffa-4001-a325-1acc4a714ca3-var-lib-kubelet\") pod \"tuned-scnm4\" (UID: \"823f515b-2ffa-4001-a325-1acc4a714ca3\") " pod="openshift-cluster-node-tuning-operator/tuned-scnm4"
Jul 02 13:43:57.155631 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.155575    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cq8jf\" (UniqueName: \"kubernetes.io/projected/06e4af50-04c6-4c58-af10-0e0a28c1a261-kube-api-access-cq8jf\") pod \"network-check-target-q9w6j\" (UID: \"06e4af50-04c6-4c58-af10-0e0a28c1a261\") " pod="openshift-network-diagnostics/network-check-target-q9w6j"
Jul 02 13:43:57.155631 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.155609    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/23638fc3-c572-4760-8e60-e82de1c6f290-agent-certs\") pod \"konnectivity-agent-m4ndl\" (UID: \"23638fc3-c572-4760-8e60-e82de1c6f290\") " pod="kube-system/konnectivity-agent-m4ndl"
Jul 02 13:43:57.155794 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.155656    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/de0396fd-b15d-4e52-b0af-7e40954f53db-serviceca\") pod \"node-ca-t99mm\" (UID: \"de0396fd-b15d-4e52-b0af-7e40954f53db\") " pod="openshift-image-registry/node-ca-t99mm"
Jul 02 13:43:57.155794 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.155703    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/823f515b-2ffa-4001-a325-1acc4a714ca3-etc-sysconfig\") pod \"tuned-scnm4\" (UID: \"823f515b-2ffa-4001-a325-1acc4a714ca3\") " pod="openshift-cluster-node-tuning-operator/tuned-scnm4"
Jul 02 13:43:57.155794 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.155738    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/823f515b-2ffa-4001-a325-1acc4a714ca3-etc-sysctl-d\") pod \"tuned-scnm4\" (UID: \"823f515b-2ffa-4001-a325-1acc4a714ca3\") " pod="openshift-cluster-node-tuning-operator/tuned-scnm4"
Jul 02 13:43:57.155794 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.155770    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/823f515b-2ffa-4001-a325-1acc4a714ca3-etc-sysctl-conf\") pod \"tuned-scnm4\" (UID: \"823f515b-2ffa-4001-a325-1acc4a714ca3\") " pod="openshift-cluster-node-tuning-operator/tuned-scnm4"
Jul 02 13:43:57.156074 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.155803    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/823f515b-2ffa-4001-a325-1acc4a714ca3-etc-systemd\") pod \"tuned-scnm4\" (UID: \"823f515b-2ffa-4001-a325-1acc4a714ca3\") " pod="openshift-cluster-node-tuning-operator/tuned-scnm4"
Jul 02 13:43:57.156074 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.155829    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/823f515b-2ffa-4001-a325-1acc4a714ca3-run\") pod \"tuned-scnm4\" (UID: \"823f515b-2ffa-4001-a325-1acc4a714ca3\") " pod="openshift-cluster-node-tuning-operator/tuned-scnm4"
Jul 02 13:43:57.156074 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.155882    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/823f515b-2ffa-4001-a325-1acc4a714ca3-etc-tuned\") pod \"tuned-scnm4\" (UID: \"823f515b-2ffa-4001-a325-1acc4a714ca3\") " pod="openshift-cluster-node-tuning-operator/tuned-scnm4"
Jul 02 13:43:57.156074 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.155919    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/de0396fd-b15d-4e52-b0af-7e40954f53db-host\") pod \"node-ca-t99mm\" (UID: \"de0396fd-b15d-4e52-b0af-7e40954f53db\") " pod="openshift-image-registry/node-ca-t99mm"
Jul 02 13:43:57.156074 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.155923    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 02 13:43:57.156074 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.155953    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/823f515b-2ffa-4001-a325-1acc4a714ca3-etc-modprobe-d\") pod \"tuned-scnm4\" (UID: \"823f515b-2ffa-4001-a325-1acc4a714ca3\") " pod="openshift-cluster-node-tuning-operator/tuned-scnm4"
Jul 02 13:43:57.156074 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.155988    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/823f515b-2ffa-4001-a325-1acc4a714ca3-etc-kubernetes\") pod \"tuned-scnm4\" (UID: \"823f515b-2ffa-4001-a325-1acc4a714ca3\") " pod="openshift-cluster-node-tuning-operator/tuned-scnm4"
Jul 02 13:43:57.156074 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.156013    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/823f515b-2ffa-4001-a325-1acc4a714ca3-host\") pod \"tuned-scnm4\" (UID: \"823f515b-2ffa-4001-a325-1acc4a714ca3\") " pod="openshift-cluster-node-tuning-operator/tuned-scnm4"
Jul 02 13:43:57.156074 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.156044    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/823f515b-2ffa-4001-a325-1acc4a714ca3-tmp\") pod \"tuned-scnm4\" (UID: \"823f515b-2ffa-4001-a325-1acc4a714ca3\") " pod="openshift-cluster-node-tuning-operator/tuned-scnm4"
Jul 02 13:43:57.156566 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.156081    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dzzmz\" (UniqueName: \"kubernetes.io/projected/823f515b-2ffa-4001-a325-1acc4a714ca3-kube-api-access-dzzmz\") pod \"tuned-scnm4\" (UID: \"823f515b-2ffa-4001-a325-1acc4a714ca3\") " pod="openshift-cluster-node-tuning-operator/tuned-scnm4"
Jul 02 13:43:57.156566 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.156116    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/823f515b-2ffa-4001-a325-1acc4a714ca3-sys\") pod \"tuned-scnm4\" (UID: \"823f515b-2ffa-4001-a325-1acc4a714ca3\") " pod="openshift-cluster-node-tuning-operator/tuned-scnm4"
Jul 02 13:43:57.156566 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.156148    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/823f515b-2ffa-4001-a325-1acc4a714ca3-lib-modules\") pod \"tuned-scnm4\" (UID: \"823f515b-2ffa-4001-a325-1acc4a714ca3\") " pod="openshift-cluster-node-tuning-operator/tuned-scnm4"
Jul 02 13:43:57.156566 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.156179    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/23638fc3-c572-4760-8e60-e82de1c6f290-konnectivity-ca\") pod \"konnectivity-agent-m4ndl\" (UID: \"23638fc3-c572-4760-8e60-e82de1c6f290\") " pod="kube-system/konnectivity-agent-m4ndl"
Jul 02 13:43:57.156566 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.156554    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 02 13:43:57.156826 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.156583    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-sxmmt\""
Jul 02 13:43:57.156826 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.156609    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 02 13:43:57.156952 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.156938    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 02 13:43:57.157438 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.157412    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-lxhdr"
Jul 02 13:43:57.159582 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.159563    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 02 13:43:57.160751 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.160732    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-8f2ll"
Jul 02 13:43:57.162393 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.162371    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29552"
Jul 02 13:43:57.162494 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:43:57.162462    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-29552" podUID="ad2afabf-42cb-4619-b249-a2b011e32565"
Jul 02 13:43:57.162693 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.162670    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 02 13:43:57.162800 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.162785    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-gkj8m\""
Jul 02 13:43:57.162895 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.162820    2558 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 02 13:43:57.162972 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.162951    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 02 13:43:57.163151 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.163134    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 02 13:43:57.164108 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.164089    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 02 13:43:57.164313 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.164295    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 02 13:43:57.164487 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.164298    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-f2c66\""
Jul 02 13:43:57.164730 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.164710    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 02 13:43:57.164975 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.164957    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 02 13:43:57.165364 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.165345    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 02 13:43:57.165465 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.165398    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 02 13:43:57.165619 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.165600    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-976zg"
Jul 02 13:43:57.167837 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.167818    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2tjcl"
Jul 02 13:43:57.167959 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:43:57.167940    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2tjcl" podUID="6ebfc25f-6722-4f73-98b8-18b416460437"
Jul 02 13:43:57.168185 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.168165    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 02 13:43:57.168284 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.168208    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-lztbz\""
Jul 02 13:43:57.188134 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.188111    2558 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-m97db"
Jul 02 13:43:57.197664 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.197381    2558 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-m97db"
Jul 02 13:43:57.244846 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.244809    2558 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 02 13:43:57.257194 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.257155    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/355d744e-81d0-41f9-864f-34b8d7b981af-kubelet-dir\") pod \"aws-ebs-csi-driver-node-tvfdz\" (UID: \"355d744e-81d0-41f9-864f-34b8d7b981af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tvfdz"
Jul 02 13:43:57.257361 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.257283    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/355d744e-81d0-41f9-864f-34b8d7b981af-registration-dir\") pod \"aws-ebs-csi-driver-node-tvfdz\" (UID: \"355d744e-81d0-41f9-864f-34b8d7b981af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tvfdz"
Jul 02 13:43:57.257361 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.257317    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3310d1f7-1643-4017-9edf-71fa702d2615-host-cni-bin\") pod \"ovnkube-node-8f2ll\" (UID: \"3310d1f7-1643-4017-9edf-71fa702d2615\") " pod="openshift-ovn-kubernetes/ovnkube-node-8f2ll"
Jul 02 13:43:57.257361 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.257343    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ffqs2\" (UniqueName: \"kubernetes.io/projected/ad2afabf-42cb-4619-b249-a2b011e32565-kube-api-access-ffqs2\") pod \"network-metrics-daemon-29552\" (UID: \"ad2afabf-42cb-4619-b249-a2b011e32565\") " pod="openshift-multus/network-metrics-daemon-29552"
Jul 02 13:43:57.257495 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.257370    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/7e1c7379-372d-48f3-a865-d0510d6c27a9-iptables-alerter-script\") pod \"iptables-alerter-lxhdr\" (UID: \"7e1c7379-372d-48f3-a865-d0510d6c27a9\") " pod="openshift-network-operator/iptables-alerter-lxhdr"
Jul 02 13:43:57.257495 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.257395    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/3310d1f7-1643-4017-9edf-71fa702d2615-run-systemd\") pod \"ovnkube-node-8f2ll\" (UID: \"3310d1f7-1643-4017-9edf-71fa702d2615\") " pod="openshift-ovn-kubernetes/ovnkube-node-8f2ll"
Jul 02 13:43:57.257495 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.257414    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/3310d1f7-1643-4017-9edf-71fa702d2615-ovn-node-metrics-cert\") pod \"ovnkube-node-8f2ll\" (UID: \"3310d1f7-1643-4017-9edf-71fa702d2615\") " pod="openshift-ovn-kubernetes/ovnkube-node-8f2ll"
Jul 02 13:43:57.257495 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.257438    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/364e6172-44ee-4b85-a48c-a0f29679abce-host-run-k8s-cni-cncf-io\") pod \"multus-976zg\" (UID: \"364e6172-44ee-4b85-a48c-a0f29679abce\") " pod="openshift-multus/multus-976zg"
Jul 02 13:43:57.257495 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.257461    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/3310d1f7-1643-4017-9edf-71fa702d2615-run-ovn\") pod \"ovnkube-node-8f2ll\" (UID: \"3310d1f7-1643-4017-9edf-71fa702d2615\") " pod="openshift-ovn-kubernetes/ovnkube-node-8f2ll"
Jul 02 13:43:57.257721 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.257503    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/3310d1f7-1643-4017-9edf-71fa702d2615-node-log\") pod \"ovnkube-node-8f2ll\" (UID: \"3310d1f7-1643-4017-9edf-71fa702d2615\") " pod="openshift-ovn-kubernetes/ovnkube-node-8f2ll"
Jul 02 13:43:57.257721 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.257545    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cdjzl\" (UniqueName: \"kubernetes.io/projected/3310d1f7-1643-4017-9edf-71fa702d2615-kube-api-access-cdjzl\") pod \"ovnkube-node-8f2ll\" (UID: \"3310d1f7-1643-4017-9edf-71fa702d2615\") " pod="openshift-ovn-kubernetes/ovnkube-node-8f2ll"
Jul 02 13:43:57.257721 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.257595    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/823f515b-2ffa-4001-a325-1acc4a714ca3-etc-kubernetes\") pod \"tuned-scnm4\" (UID: \"823f515b-2ffa-4001-a325-1acc4a714ca3\") " pod="openshift-cluster-node-tuning-operator/tuned-scnm4"
Jul 02 13:43:57.257721 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.257622    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/364e6172-44ee-4b85-a48c-a0f29679abce-cnibin\") pod \"multus-976zg\" (UID: \"364e6172-44ee-4b85-a48c-a0f29679abce\") " pod="openshift-multus/multus-976zg"
Jul 02 13:43:57.257721 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.257645    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/3310d1f7-1643-4017-9edf-71fa702d2615-host-cni-netd\") pod \"ovnkube-node-8f2ll\" (UID: \"3310d1f7-1643-4017-9edf-71fa702d2615\") " pod="openshift-ovn-kubernetes/ovnkube-node-8f2ll"
Jul 02 13:43:57.257721 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.257668    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4mvm5\" (UniqueName: \"kubernetes.io/projected/859dd06a-dc53-48ca-a2ea-9fea2bad0253-kube-api-access-4mvm5\") pod \"node-resolver-g9pll\" (UID: \"859dd06a-dc53-48ca-a2ea-9fea2bad0253\") " pod="openshift-dns/node-resolver-g9pll"
Jul 02 13:43:57.257721 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.257688    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/823f515b-2ffa-4001-a325-1acc4a714ca3-sys\") pod \"tuned-scnm4\" (UID: \"823f515b-2ffa-4001-a325-1acc4a714ca3\") " pod="openshift-cluster-node-tuning-operator/tuned-scnm4"
Jul 02 13:43:57.257721 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.257703    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/364e6172-44ee-4b85-a48c-a0f29679abce-host-run-netns\") pod \"multus-976zg\" (UID: \"364e6172-44ee-4b85-a48c-a0f29679abce\") " pod="openshift-multus/multus-976zg"
Jul 02 13:43:57.257721 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.257722    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/364e6172-44ee-4b85-a48c-a0f29679abce-multus-conf-dir\") pod \"multus-976zg\" (UID: \"364e6172-44ee-4b85-a48c-a0f29679abce\") " pod="openshift-multus/multus-976zg"
Jul 02 13:43:57.258162 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.257738    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p66bz\" (UniqueName: \"kubernetes.io/projected/355d744e-81d0-41f9-864f-34b8d7b981af-kube-api-access-p66bz\") pod \"aws-ebs-csi-driver-node-tvfdz\" (UID: \"355d744e-81d0-41f9-864f-34b8d7b981af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tvfdz"
Jul 02 13:43:57.258162 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.257787    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/3310d1f7-1643-4017-9edf-71fa702d2615-log-socket\") pod \"ovnkube-node-8f2ll\" (UID: \"3310d1f7-1643-4017-9edf-71fa702d2615\") " pod="openshift-ovn-kubernetes/ovnkube-node-8f2ll"
Jul 02 13:43:57.258162 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.257843    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/823f515b-2ffa-4001-a325-1acc4a714ca3-etc-kubernetes\") pod \"tuned-scnm4\" (UID: \"823f515b-2ffa-4001-a325-1acc4a714ca3\") " pod="openshift-cluster-node-tuning-operator/tuned-scnm4"
Jul 02 13:43:57.258162 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.257913    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/823f515b-2ffa-4001-a325-1acc4a714ca3-sys\") pod \"tuned-scnm4\" (UID: \"823f515b-2ffa-4001-a325-1acc4a714ca3\") " pod="openshift-cluster-node-tuning-operator/tuned-scnm4"
Jul 02 13:43:57.258162 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.257949    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cq8jf\" (UniqueName: \"kubernetes.io/projected/06e4af50-04c6-4c58-af10-0e0a28c1a261-kube-api-access-cq8jf\") pod \"network-check-target-q9w6j\" (UID: \"06e4af50-04c6-4c58-af10-0e0a28c1a261\") " pod="openshift-network-diagnostics/network-check-target-q9w6j"
Jul 02 13:43:57.258162 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.257981    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/364e6172-44ee-4b85-a48c-a0f29679abce-etc-kubernetes\") pod \"multus-976zg\" (UID: \"364e6172-44ee-4b85-a48c-a0f29679abce\") " pod="openshift-multus/multus-976zg"
Jul 02 13:43:57.258162 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.258030    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6ebfc25f-6722-4f73-98b8-18b416460437-kubelet-config\") pod \"global-pull-secret-syncer-2tjcl\" (UID: \"6ebfc25f-6722-4f73-98b8-18b416460437\") " pod="kube-system/global-pull-secret-syncer-2tjcl"
Jul 02 13:43:57.258162 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.258059    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/355d744e-81d0-41f9-864f-34b8d7b981af-etc-selinux\") pod \"aws-ebs-csi-driver-node-tvfdz\" (UID: \"355d744e-81d0-41f9-864f-34b8d7b981af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tvfdz"
Jul 02 13:43:57.258162 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.258107    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/823f515b-2ffa-4001-a325-1acc4a714ca3-etc-sysctl-d\") pod \"tuned-scnm4\" (UID: \"823f515b-2ffa-4001-a325-1acc4a714ca3\") " pod="openshift-cluster-node-tuning-operator/tuned-scnm4"
Jul 02 13:43:57.258162 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.258139    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/823f515b-2ffa-4001-a325-1acc4a714ca3-etc-sysctl-conf\") pod \"tuned-scnm4\" (UID: \"823f515b-2ffa-4001-a325-1acc4a714ca3\") " pod="openshift-cluster-node-tuning-operator/tuned-scnm4"
Jul 02 13:43:57.258162 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.258163    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/823f515b-2ffa-4001-a325-1acc4a714ca3-run\") pod \"tuned-scnm4\" (UID: \"823f515b-2ffa-4001-a325-1acc4a714ca3\") " pod="openshift-cluster-node-tuning-operator/tuned-scnm4"
Jul 02 13:43:57.258659 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.258191    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lj9vs\" (UniqueName: \"kubernetes.io/projected/e1521a8d-402a-4bee-bc35-bfa382300084-kube-api-access-lj9vs\") pod \"multus-additional-cni-plugins-s85vc\" (UID: \"e1521a8d-402a-4bee-bc35-bfa382300084\") " pod="openshift-multus/multus-additional-cni-plugins-s85vc"
Jul 02 13:43:57.258659 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.258251    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/355d744e-81d0-41f9-864f-34b8d7b981af-socket-dir\") pod \"aws-ebs-csi-driver-node-tvfdz\" (UID: \"355d744e-81d0-41f9-864f-34b8d7b981af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tvfdz"
Jul 02 13:43:57.258659 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.258278    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-djxlq\" (UniqueName: \"kubernetes.io/projected/7e1c7379-372d-48f3-a865-d0510d6c27a9-kube-api-access-djxlq\") pod \"iptables-alerter-lxhdr\" (UID: \"7e1c7379-372d-48f3-a865-d0510d6c27a9\") " pod="openshift-network-operator/iptables-alerter-lxhdr"
Jul 02 13:43:57.258659 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.258307    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3310d1f7-1643-4017-9edf-71fa702d2615-var-lib-openvswitch\") pod \"ovnkube-node-8f2ll\" (UID: \"3310d1f7-1643-4017-9edf-71fa702d2615\") " pod="openshift-ovn-kubernetes/ovnkube-node-8f2ll"
Jul 02 13:43:57.258659 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.258337    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dzzmz\" (UniqueName: \"kubernetes.io/projected/823f515b-2ffa-4001-a325-1acc4a714ca3-kube-api-access-dzzmz\") pod \"tuned-scnm4\" (UID: \"823f515b-2ffa-4001-a325-1acc4a714ca3\") " pod="openshift-cluster-node-tuning-operator/tuned-scnm4"
Jul 02 13:43:57.258659 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.258364    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/23638fc3-c572-4760-8e60-e82de1c6f290-konnectivity-ca\") pod \"konnectivity-agent-m4ndl\" (UID: \"23638fc3-c572-4760-8e60-e82de1c6f290\") " pod="kube-system/konnectivity-agent-m4ndl"
Jul 02 13:43:57.258659 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.258391    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/3310d1f7-1643-4017-9edf-71fa702d2615-systemd-units\") pod \"ovnkube-node-8f2ll\" (UID: \"3310d1f7-1643-4017-9edf-71fa702d2615\") " pod="openshift-ovn-kubernetes/ovnkube-node-8f2ll"
Jul 02 13:43:57.258659 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.258426    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/823f515b-2ffa-4001-a325-1acc4a714ca3-etc-sysctl-d\") pod \"tuned-scnm4\" (UID: \"823f515b-2ffa-4001-a325-1acc4a714ca3\") " pod="openshift-cluster-node-tuning-operator/tuned-scnm4"
Jul 02 13:43:57.258659 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.258468    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/859dd06a-dc53-48ca-a2ea-9fea2bad0253-hosts-file\") pod \"node-resolver-g9pll\" (UID: \"859dd06a-dc53-48ca-a2ea-9fea2bad0253\") " pod="openshift-dns/node-resolver-g9pll"
Jul 02 13:43:57.258659 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.258498    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/364e6172-44ee-4b85-a48c-a0f29679abce-host-var-lib-cni-multus\") pod \"multus-976zg\" (UID: \"364e6172-44ee-4b85-a48c-a0f29679abce\") " pod="openshift-multus/multus-976zg"
Jul 02 13:43:57.258659 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.258525    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m888j\" (UniqueName: \"kubernetes.io/projected/de0396fd-b15d-4e52-b0af-7e40954f53db-kube-api-access-m888j\") pod \"node-ca-t99mm\" (UID: \"de0396fd-b15d-4e52-b0af-7e40954f53db\") " pod="openshift-image-registry/node-ca-t99mm"
Jul 02 13:43:57.258659 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.258574    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/823f515b-2ffa-4001-a325-1acc4a714ca3-etc-sysctl-conf\") pod \"tuned-scnm4\" (UID: \"823f515b-2ffa-4001-a325-1acc4a714ca3\") " pod="openshift-cluster-node-tuning-operator/tuned-scnm4"
Jul 02 13:43:57.259301 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.258748    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/823f515b-2ffa-4001-a325-1acc4a714ca3-run\") pod \"tuned-scnm4\" (UID: \"823f515b-2ffa-4001-a325-1acc4a714ca3\") " pod="openshift-cluster-node-tuning-operator/tuned-scnm4"
Jul 02 13:43:57.259301 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.258769    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/823f515b-2ffa-4001-a325-1acc4a714ca3-var-lib-kubelet\") pod \"tuned-scnm4\" (UID: \"823f515b-2ffa-4001-a325-1acc4a714ca3\") " pod="openshift-cluster-node-tuning-operator/tuned-scnm4"
Jul 02 13:43:57.259301 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.258805    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/23638fc3-c572-4760-8e60-e82de1c6f290-agent-certs\") pod \"konnectivity-agent-m4ndl\" (UID: \"23638fc3-c572-4760-8e60-e82de1c6f290\") " pod="kube-system/konnectivity-agent-m4ndl"
Jul 02 13:43:57.259301 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.258861    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/823f515b-2ffa-4001-a325-1acc4a714ca3-var-lib-kubelet\") pod \"tuned-scnm4\" (UID: \"823f515b-2ffa-4001-a325-1acc4a714ca3\") " pod="openshift-cluster-node-tuning-operator/tuned-scnm4"
Jul 02 13:43:57.259301 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.258894    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e1521a8d-402a-4bee-bc35-bfa382300084-os-release\") pod \"multus-additional-cni-plugins-s85vc\" (UID: \"e1521a8d-402a-4bee-bc35-bfa382300084\") " pod="openshift-multus/multus-additional-cni-plugins-s85vc"
Jul 02 13:43:57.259301 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.258947    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e1521a8d-402a-4bee-bc35-bfa382300084-cni-binary-copy\") pod \"multus-additional-cni-plugins-s85vc\" (UID: \"e1521a8d-402a-4bee-bc35-bfa382300084\") " pod="openshift-multus/multus-additional-cni-plugins-s85vc"
Jul 02 13:43:57.259301 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.258999    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/364e6172-44ee-4b85-a48c-a0f29679abce-os-release\") pod \"multus-976zg\" (UID: \"364e6172-44ee-4b85-a48c-a0f29679abce\") " pod="openshift-multus/multus-976zg"
Jul 02 13:43:57.259301 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.259025    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/364e6172-44ee-4b85-a48c-a0f29679abce-cni-binary-copy\") pod \"multus-976zg\" (UID: \"364e6172-44ee-4b85-a48c-a0f29679abce\") " pod="openshift-multus/multus-976zg"
Jul 02 13:43:57.259301 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.259049    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3310d1f7-1643-4017-9edf-71fa702d2615-host-slash\") pod \"ovnkube-node-8f2ll\" (UID: \"3310d1f7-1643-4017-9edf-71fa702d2615\") " pod="openshift-ovn-kubernetes/ovnkube-node-8f2ll"
Jul 02 13:43:57.259301 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.259096    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/823f515b-2ffa-4001-a325-1acc4a714ca3-etc-sysconfig\") pod \"tuned-scnm4\" (UID: \"823f515b-2ffa-4001-a325-1acc4a714ca3\") " pod="openshift-cluster-node-tuning-operator/tuned-scnm4"
Jul 02 13:43:57.259301 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.259145    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/823f515b-2ffa-4001-a325-1acc4a714ca3-etc-systemd\") pod \"tuned-scnm4\" (UID: \"823f515b-2ffa-4001-a325-1acc4a714ca3\") " pod="openshift-cluster-node-tuning-operator/tuned-scnm4"
Jul 02 13:43:57.259301 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.259189    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/823f515b-2ffa-4001-a325-1acc4a714ca3-etc-systemd\") pod \"tuned-scnm4\" (UID: \"823f515b-2ffa-4001-a325-1acc4a714ca3\") " pod="openshift-cluster-node-tuning-operator/tuned-scnm4"
Jul 02 13:43:57.259301 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.259230    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/823f515b-2ffa-4001-a325-1acc4a714ca3-etc-tuned\") pod \"tuned-scnm4\" (UID: \"823f515b-2ffa-4001-a325-1acc4a714ca3\") " pod="openshift-cluster-node-tuning-operator/tuned-scnm4"
Jul 02 13:43:57.259301 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.259286    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e1521a8d-402a-4bee-bc35-bfa382300084-cnibin\") pod \"multus-additional-cni-plugins-s85vc\" (UID: \"e1521a8d-402a-4bee-bc35-bfa382300084\") " pod="openshift-multus/multus-additional-cni-plugins-s85vc"
Jul 02 13:43:57.260005 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.259317    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/23638fc3-c572-4760-8e60-e82de1c6f290-konnectivity-ca\") pod \"konnectivity-agent-m4ndl\" (UID: \"23638fc3-c572-4760-8e60-e82de1c6f290\") " pod="kube-system/konnectivity-agent-m4ndl"
Jul 02 13:43:57.260005 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.259289    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/823f515b-2ffa-4001-a325-1acc4a714ca3-etc-sysconfig\") pod \"tuned-scnm4\" (UID: \"823f515b-2ffa-4001-a325-1acc4a714ca3\") " pod="openshift-cluster-node-tuning-operator/tuned-scnm4"
Jul 02 13:43:57.260005 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.259331    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/364e6172-44ee-4b85-a48c-a0f29679abce-multus-socket-dir-parent\") pod \"multus-976zg\" (UID: \"364e6172-44ee-4b85-a48c-a0f29679abce\") " pod="openshift-multus/multus-976zg"
Jul 02 13:43:57.260005 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.259322    2558 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 02 13:43:57.260005 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.259378    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/364e6172-44ee-4b85-a48c-a0f29679abce-hostroot\") pod \"multus-976zg\" (UID: \"364e6172-44ee-4b85-a48c-a0f29679abce\") " pod="openshift-multus/multus-976zg"
Jul 02 13:43:57.260005 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.259400    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6ebfc25f-6722-4f73-98b8-18b416460437-dbus\") pod \"global-pull-secret-syncer-2tjcl\" (UID: \"6ebfc25f-6722-4f73-98b8-18b416460437\") " pod="kube-system/global-pull-secret-syncer-2tjcl"
Jul 02 13:43:57.260005 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.259461    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6ebfc25f-6722-4f73-98b8-18b416460437-original-pull-secret\") pod \"global-pull-secret-syncer-2tjcl\" (UID: \"6ebfc25f-6722-4f73-98b8-18b416460437\") " pod="kube-system/global-pull-secret-syncer-2tjcl"
Jul 02 13:43:57.260005 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.259510    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/de0396fd-b15d-4e52-b0af-7e40954f53db-host\") pod \"node-ca-t99mm\" (UID: \"de0396fd-b15d-4e52-b0af-7e40954f53db\") " pod="openshift-image-registry/node-ca-t99mm"
Jul 02 13:43:57.260005 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.259532    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/de0396fd-b15d-4e52-b0af-7e40954f53db-host\") pod \"node-ca-t99mm\" (UID: \"de0396fd-b15d-4e52-b0af-7e40954f53db\") " pod="openshift-image-registry/node-ca-t99mm"
Jul 02 13:43:57.260005 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.259550    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/823f515b-2ffa-4001-a325-1acc4a714ca3-etc-modprobe-d\") pod \"tuned-scnm4\" (UID: \"823f515b-2ffa-4001-a325-1acc4a714ca3\") " pod="openshift-cluster-node-tuning-operator/tuned-scnm4"
Jul 02 13:43:57.260005 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.259608    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/823f515b-2ffa-4001-a325-1acc4a714ca3-host\") pod \"tuned-scnm4\" (UID: \"823f515b-2ffa-4001-a325-1acc4a714ca3\") " pod="openshift-cluster-node-tuning-operator/tuned-scnm4"
Jul 02 13:43:57.260005 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.259650    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ad2afabf-42cb-4619-b249-a2b011e32565-metrics-certs\") pod \"network-metrics-daemon-29552\" (UID: \"ad2afabf-42cb-4619-b249-a2b011e32565\") " pod="openshift-multus/network-metrics-daemon-29552"
Jul 02 13:43:57.260005 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.259687    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/823f515b-2ffa-4001-a325-1acc4a714ca3-etc-modprobe-d\") pod \"tuned-scnm4\" (UID: \"823f515b-2ffa-4001-a325-1acc4a714ca3\") " pod="openshift-cluster-node-tuning-operator/tuned-scnm4"
Jul 02 13:43:57.260005 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.259691    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/823f515b-2ffa-4001-a325-1acc4a714ca3-host\") pod \"tuned-scnm4\" (UID: \"823f515b-2ffa-4001-a325-1acc4a714ca3\") " pod="openshift-cluster-node-tuning-operator/tuned-scnm4"
Jul 02 13:43:57.260005 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.259738    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/355d744e-81d0-41f9-864f-34b8d7b981af-sys-fs\") pod \"aws-ebs-csi-driver-node-tvfdz\" (UID: \"355d744e-81d0-41f9-864f-34b8d7b981af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tvfdz"
Jul 02 13:43:57.260005 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.259759    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/3310d1f7-1643-4017-9edf-71fa702d2615-host-kubelet\") pod \"ovnkube-node-8f2ll\" (UID: \"3310d1f7-1643-4017-9edf-71fa702d2615\") " pod="openshift-ovn-kubernetes/ovnkube-node-8f2ll"
Jul 02 13:43:57.260005 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.259783    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3310d1f7-1643-4017-9edf-71fa702d2615-host-run-netns\") pod \"ovnkube-node-8f2ll\" (UID: \"3310d1f7-1643-4017-9edf-71fa702d2615\") " pod="openshift-ovn-kubernetes/ovnkube-node-8f2ll"
Jul 02 13:43:57.260005 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.259814    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/823f515b-2ffa-4001-a325-1acc4a714ca3-lib-modules\") pod \"tuned-scnm4\" (UID: \"823f515b-2ffa-4001-a325-1acc4a714ca3\") " pod="openshift-cluster-node-tuning-operator/tuned-scnm4"
Jul 02 13:43:57.260833 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.259857    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e1521a8d-402a-4bee-bc35-bfa382300084-system-cni-dir\") pod \"multus-additional-cni-plugins-s85vc\" (UID: \"e1521a8d-402a-4bee-bc35-bfa382300084\") " pod="openshift-multus/multus-additional-cni-plugins-s85vc"
Jul 02 13:43:57.260833 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.259886    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e1521a8d-402a-4bee-bc35-bfa382300084-tuning-conf-dir\") pod \"multus-additional-cni-plugins-s85vc\" (UID: \"e1521a8d-402a-4bee-bc35-bfa382300084\") " pod="openshift-multus/multus-additional-cni-plugins-s85vc"
Jul 02 13:43:57.260833 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.259912    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/364e6172-44ee-4b85-a48c-a0f29679abce-multus-cni-dir\") pod \"multus-976zg\" (UID: \"364e6172-44ee-4b85-a48c-a0f29679abce\") " pod="openshift-multus/multus-976zg"
Jul 02 13:43:57.260833 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.259936    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/364e6172-44ee-4b85-a48c-a0f29679abce-host-run-multus-certs\") pod \"multus-976zg\" (UID: \"364e6172-44ee-4b85-a48c-a0f29679abce\") " pod="openshift-multus/multus-976zg"
Jul 02 13:43:57.260833 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.259954    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/823f515b-2ffa-4001-a325-1acc4a714ca3-lib-modules\") pod \"tuned-scnm4\" (UID: \"823f515b-2ffa-4001-a325-1acc4a714ca3\") " pod="openshift-cluster-node-tuning-operator/tuned-scnm4"
Jul 02 13:43:57.260833 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.259985    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4s5dk\" (UniqueName: \"kubernetes.io/projected/364e6172-44ee-4b85-a48c-a0f29679abce-kube-api-access-4s5dk\") pod \"multus-976zg\" (UID: \"364e6172-44ee-4b85-a48c-a0f29679abce\") " pod="openshift-multus/multus-976zg"
Jul 02 13:43:57.260833 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.260027    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/355d744e-81d0-41f9-864f-34b8d7b981af-device-dir\") pod \"aws-ebs-csi-driver-node-tvfdz\" (UID: \"355d744e-81d0-41f9-864f-34b8d7b981af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tvfdz"
Jul 02 13:43:57.260833 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.260051    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7e1c7379-372d-48f3-a865-d0510d6c27a9-host-slash\") pod \"iptables-alerter-lxhdr\" (UID: \"7e1c7379-372d-48f3-a865-d0510d6c27a9\") " pod="openshift-network-operator/iptables-alerter-lxhdr"
Jul 02 13:43:57.260833 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.260074    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/364e6172-44ee-4b85-a48c-a0f29679abce-system-cni-dir\") pod \"multus-976zg\" (UID: \"364e6172-44ee-4b85-a48c-a0f29679abce\") " pod="openshift-multus/multus-976zg"
Jul 02 13:43:57.260833 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.260094    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/364e6172-44ee-4b85-a48c-a0f29679abce-multus-daemon-config\") pod \"multus-976zg\" (UID: \"364e6172-44ee-4b85-a48c-a0f29679abce\") " pod="openshift-multus/multus-976zg"
Jul 02 13:43:57.260833 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.260117    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3310d1f7-1643-4017-9edf-71fa702d2615-run-openvswitch\") pod \"ovnkube-node-8f2ll\" (UID: \"3310d1f7-1643-4017-9edf-71fa702d2615\") " pod="openshift-ovn-kubernetes/ovnkube-node-8f2ll"
Jul 02 13:43:57.260833 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.260140    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3310d1f7-1643-4017-9edf-71fa702d2615-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-8f2ll\" (UID: \"3310d1f7-1643-4017-9edf-71fa702d2615\") " pod="openshift-ovn-kubernetes/ovnkube-node-8f2ll"
Jul 02 13:43:57.260833 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.260164    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/3310d1f7-1643-4017-9edf-71fa702d2615-ovnkube-script-lib\") pod \"ovnkube-node-8f2ll\" (UID: \"3310d1f7-1643-4017-9edf-71fa702d2615\") " pod="openshift-ovn-kubernetes/ovnkube-node-8f2ll"
Jul 02 13:43:57.260833 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.260187    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/de0396fd-b15d-4e52-b0af-7e40954f53db-serviceca\") pod \"node-ca-t99mm\" (UID: \"de0396fd-b15d-4e52-b0af-7e40954f53db\") " pod="openshift-image-registry/node-ca-t99mm"
Jul 02 13:43:57.260833 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.260214    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e1521a8d-402a-4bee-bc35-bfa382300084-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-s85vc\" (UID: \"e1521a8d-402a-4bee-bc35-bfa382300084\") " pod="openshift-multus/multus-additional-cni-plugins-s85vc"
Jul 02 13:43:57.260833 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.260240    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/823f515b-2ffa-4001-a325-1acc4a714ca3-tmp\") pod \"tuned-scnm4\" (UID: \"823f515b-2ffa-4001-a325-1acc4a714ca3\") " pod="openshift-cluster-node-tuning-operator/tuned-scnm4"
Jul 02 13:43:57.261486 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.260261    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/364e6172-44ee-4b85-a48c-a0f29679abce-host-var-lib-cni-bin\") pod \"multus-976zg\" (UID: \"364e6172-44ee-4b85-a48c-a0f29679abce\") " pod="openshift-multus/multus-976zg"
Jul 02 13:43:57.261486 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.260284    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/364e6172-44ee-4b85-a48c-a0f29679abce-host-var-lib-kubelet\") pod \"multus-976zg\" (UID: \"364e6172-44ee-4b85-a48c-a0f29679abce\") " pod="openshift-multus/multus-976zg"
Jul 02 13:43:57.261486 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.260305    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3310d1f7-1643-4017-9edf-71fa702d2615-etc-openvswitch\") pod \"ovnkube-node-8f2ll\" (UID: \"3310d1f7-1643-4017-9edf-71fa702d2615\") " pod="openshift-ovn-kubernetes/ovnkube-node-8f2ll"
Jul 02 13:43:57.261486 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.260327    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3310d1f7-1643-4017-9edf-71fa702d2615-host-run-ovn-kubernetes\") pod \"ovnkube-node-8f2ll\" (UID: \"3310d1f7-1643-4017-9edf-71fa702d2615\") " pod="openshift-ovn-kubernetes/ovnkube-node-8f2ll"
Jul 02 13:43:57.261486 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.260372    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/3310d1f7-1643-4017-9edf-71fa702d2615-ovnkube-config\") pod \"ovnkube-node-8f2ll\" (UID: \"3310d1f7-1643-4017-9edf-71fa702d2615\") " pod="openshift-ovn-kubernetes/ovnkube-node-8f2ll"
Jul 02 13:43:57.261486 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.260392    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/3310d1f7-1643-4017-9edf-71fa702d2615-env-overrides\") pod \"ovnkube-node-8f2ll\" (UID: \"3310d1f7-1643-4017-9edf-71fa702d2615\") " pod="openshift-ovn-kubernetes/ovnkube-node-8f2ll"
Jul 02 13:43:57.261486 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.260415    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/859dd06a-dc53-48ca-a2ea-9fea2bad0253-tmp-dir\") pod \"node-resolver-g9pll\" (UID: \"859dd06a-dc53-48ca-a2ea-9fea2bad0253\") " pod="openshift-dns/node-resolver-g9pll"
Jul 02 13:43:57.261486 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.260435    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e1521a8d-402a-4bee-bc35-bfa382300084-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-s85vc\" (UID: \"e1521a8d-402a-4bee-bc35-bfa382300084\") " pod="openshift-multus/multus-additional-cni-plugins-s85vc"
Jul 02 13:43:57.261486 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.260571    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/de0396fd-b15d-4e52-b0af-7e40954f53db-serviceca\") pod \"node-ca-t99mm\" (UID: \"de0396fd-b15d-4e52-b0af-7e40954f53db\") " pod="openshift-image-registry/node-ca-t99mm"
Jul 02 13:43:57.263867 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.263840    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/823f515b-2ffa-4001-a325-1acc4a714ca3-etc-tuned\") pod \"tuned-scnm4\" (UID: \"823f515b-2ffa-4001-a325-1acc4a714ca3\") " pod="openshift-cluster-node-tuning-operator/tuned-scnm4"
Jul 02 13:43:57.264277 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.264255    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/23638fc3-c572-4760-8e60-e82de1c6f290-agent-certs\") pod \"konnectivity-agent-m4ndl\" (UID: \"23638fc3-c572-4760-8e60-e82de1c6f290\") " pod="kube-system/konnectivity-agent-m4ndl"
Jul 02 13:43:57.264808 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.264776    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/823f515b-2ffa-4001-a325-1acc4a714ca3-tmp\") pod \"tuned-scnm4\" (UID: \"823f515b-2ffa-4001-a325-1acc4a714ca3\") " pod="openshift-cluster-node-tuning-operator/tuned-scnm4"
Jul 02 13:43:57.266177 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:43:57.266153    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 02 13:43:57.266295 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:43:57.266185    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 02 13:43:57.266295 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:43:57.266202    2558 projected.go:196] Error preparing data for projected volume kube-api-access-cq8jf for pod openshift-network-diagnostics/network-check-target-q9w6j: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 02 13:43:57.266295 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:43:57.266273    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/06e4af50-04c6-4c58-af10-0e0a28c1a261-kube-api-access-cq8jf podName:06e4af50-04c6-4c58-af10-0e0a28c1a261 nodeName:}" failed. No retries permitted until 2026-07-02 13:43:57.766248729 +0000 UTC m=+2.944710912 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-cq8jf" (UniqueName: "kubernetes.io/projected/06e4af50-04c6-4c58-af10-0e0a28c1a261-kube-api-access-cq8jf") pod "network-check-target-q9w6j" (UID: "06e4af50-04c6-4c58-af10-0e0a28c1a261") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 02 13:43:57.269362 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.269341    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m888j\" (UniqueName: \"kubernetes.io/projected/de0396fd-b15d-4e52-b0af-7e40954f53db-kube-api-access-m888j\") pod \"node-ca-t99mm\" (UID: \"de0396fd-b15d-4e52-b0af-7e40954f53db\") " pod="openshift-image-registry/node-ca-t99mm"
Jul 02 13:43:57.270078 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.270052    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dzzmz\" (UniqueName: \"kubernetes.io/projected/823f515b-2ffa-4001-a325-1acc4a714ca3-kube-api-access-dzzmz\") pod \"tuned-scnm4\" (UID: \"823f515b-2ffa-4001-a325-1acc4a714ca3\") " pod="openshift-cluster-node-tuning-operator/tuned-scnm4"
Jul 02 13:43:57.360791 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.360753    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e1521a8d-402a-4bee-bc35-bfa382300084-cnibin\") pod \"multus-additional-cni-plugins-s85vc\" (UID: \"e1521a8d-402a-4bee-bc35-bfa382300084\") " pod="openshift-multus/multus-additional-cni-plugins-s85vc"
Jul 02 13:43:57.360791 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.360802    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/364e6172-44ee-4b85-a48c-a0f29679abce-multus-socket-dir-parent\") pod \"multus-976zg\" (UID: \"364e6172-44ee-4b85-a48c-a0f29679abce\") " pod="openshift-multus/multus-976zg"
Jul 02 13:43:57.360791 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.360821    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/364e6172-44ee-4b85-a48c-a0f29679abce-hostroot\") pod \"multus-976zg\" (UID: \"364e6172-44ee-4b85-a48c-a0f29679abce\") " pod="openshift-multus/multus-976zg"
Jul 02 13:43:57.361088 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.360859    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6ebfc25f-6722-4f73-98b8-18b416460437-dbus\") pod \"global-pull-secret-syncer-2tjcl\" (UID: \"6ebfc25f-6722-4f73-98b8-18b416460437\") " pod="kube-system/global-pull-secret-syncer-2tjcl"
Jul 02 13:43:57.361088 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.360889    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6ebfc25f-6722-4f73-98b8-18b416460437-original-pull-secret\") pod \"global-pull-secret-syncer-2tjcl\" (UID: \"6ebfc25f-6722-4f73-98b8-18b416460437\") " pod="kube-system/global-pull-secret-syncer-2tjcl"
Jul 02 13:43:57.361088 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.360907    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e1521a8d-402a-4bee-bc35-bfa382300084-cnibin\") pod \"multus-additional-cni-plugins-s85vc\" (UID: \"e1521a8d-402a-4bee-bc35-bfa382300084\") " pod="openshift-multus/multus-additional-cni-plugins-s85vc"
Jul 02 13:43:57.361088 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.360919    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ad2afabf-42cb-4619-b249-a2b011e32565-metrics-certs\") pod \"network-metrics-daemon-29552\" (UID: \"ad2afabf-42cb-4619-b249-a2b011e32565\") " pod="openshift-multus/network-metrics-daemon-29552"
Jul 02 13:43:57.361088 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.360929    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/364e6172-44ee-4b85-a48c-a0f29679abce-multus-socket-dir-parent\") pod \"multus-976zg\" (UID: \"364e6172-44ee-4b85-a48c-a0f29679abce\") " pod="openshift-multus/multus-976zg"
Jul 02 13:43:57.361088 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.360931    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/364e6172-44ee-4b85-a48c-a0f29679abce-hostroot\") pod \"multus-976zg\" (UID: \"364e6172-44ee-4b85-a48c-a0f29679abce\") " pod="openshift-multus/multus-976zg"
Jul 02 13:43:57.361088 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.360958    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/355d744e-81d0-41f9-864f-34b8d7b981af-sys-fs\") pod \"aws-ebs-csi-driver-node-tvfdz\" (UID: \"355d744e-81d0-41f9-864f-34b8d7b981af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tvfdz"
Jul 02 13:43:57.361088 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.360985    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/3310d1f7-1643-4017-9edf-71fa702d2615-host-kubelet\") pod \"ovnkube-node-8f2ll\" (UID: \"3310d1f7-1643-4017-9edf-71fa702d2615\") " pod="openshift-ovn-kubernetes/ovnkube-node-8f2ll"
Jul 02 13:43:57.361088 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.361008    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3310d1f7-1643-4017-9edf-71fa702d2615-host-run-netns\") pod \"ovnkube-node-8f2ll\" (UID: \"3310d1f7-1643-4017-9edf-71fa702d2615\") " pod="openshift-ovn-kubernetes/ovnkube-node-8f2ll"
Jul 02 13:43:57.361088 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:43:57.361020    2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 02 13:43:57.361088 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.361062    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/3310d1f7-1643-4017-9edf-71fa702d2615-host-kubelet\") pod \"ovnkube-node-8f2ll\" (UID: \"3310d1f7-1643-4017-9edf-71fa702d2615\") " pod="openshift-ovn-kubernetes/ovnkube-node-8f2ll"
Jul 02 13:43:57.361088 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:43:57.361093    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ebfc25f-6722-4f73-98b8-18b416460437-original-pull-secret podName:6ebfc25f-6722-4f73-98b8-18b416460437 nodeName:}" failed. No retries permitted until 2026-07-02 13:43:57.861071307 +0000 UTC m=+3.039533499 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6ebfc25f-6722-4f73-98b8-18b416460437-original-pull-secret") pod "global-pull-secret-syncer-2tjcl" (UID: "6ebfc25f-6722-4f73-98b8-18b416460437") : object "kube-system"/"original-pull-secret" not registered
Jul 02 13:43:57.361088 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.361092    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6ebfc25f-6722-4f73-98b8-18b416460437-dbus\") pod \"global-pull-secret-syncer-2tjcl\" (UID: \"6ebfc25f-6722-4f73-98b8-18b416460437\") " pod="kube-system/global-pull-secret-syncer-2tjcl"
Jul 02 13:43:57.361702 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.361097    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/355d744e-81d0-41f9-864f-34b8d7b981af-sys-fs\") pod \"aws-ebs-csi-driver-node-tvfdz\" (UID: \"355d744e-81d0-41f9-864f-34b8d7b981af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tvfdz"
Jul 02 13:43:57.361702 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:43:57.361021    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 02 13:43:57.361702 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.361117    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e1521a8d-402a-4bee-bc35-bfa382300084-system-cni-dir\") pod \"multus-additional-cni-plugins-s85vc\" (UID: \"e1521a8d-402a-4bee-bc35-bfa382300084\") " pod="openshift-multus/multus-additional-cni-plugins-s85vc"
Jul 02 13:43:57.361702 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.361143    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e1521a8d-402a-4bee-bc35-bfa382300084-system-cni-dir\") pod \"multus-additional-cni-plugins-s85vc\" (UID: \"e1521a8d-402a-4bee-bc35-bfa382300084\") " pod="openshift-multus/multus-additional-cni-plugins-s85vc"
Jul 02 13:43:57.361702 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.361156    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e1521a8d-402a-4bee-bc35-bfa382300084-tuning-conf-dir\") pod \"multus-additional-cni-plugins-s85vc\" (UID: \"e1521a8d-402a-4bee-bc35-bfa382300084\") " pod="openshift-multus/multus-additional-cni-plugins-s85vc"
Jul 02 13:43:57.361702 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.361099    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3310d1f7-1643-4017-9edf-71fa702d2615-host-run-netns\") pod \"ovnkube-node-8f2ll\" (UID: \"3310d1f7-1643-4017-9edf-71fa702d2615\") " pod="openshift-ovn-kubernetes/ovnkube-node-8f2ll"
Jul 02 13:43:57.361702 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:43:57.361178    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad2afabf-42cb-4619-b249-a2b011e32565-metrics-certs podName:ad2afabf-42cb-4619-b249-a2b011e32565 nodeName:}" failed. No retries permitted until 2026-07-02 13:43:57.861158515 +0000 UTC m=+3.039620697 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ad2afabf-42cb-4619-b249-a2b011e32565-metrics-certs") pod "network-metrics-daemon-29552" (UID: "ad2afabf-42cb-4619-b249-a2b011e32565") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 02 13:43:57.361702 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.361202    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/364e6172-44ee-4b85-a48c-a0f29679abce-multus-cni-dir\") pod \"multus-976zg\" (UID: \"364e6172-44ee-4b85-a48c-a0f29679abce\") " pod="openshift-multus/multus-976zg"
Jul 02 13:43:57.361702 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.361225    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/364e6172-44ee-4b85-a48c-a0f29679abce-host-run-multus-certs\") pod \"multus-976zg\" (UID: \"364e6172-44ee-4b85-a48c-a0f29679abce\") " pod="openshift-multus/multus-976zg"
Jul 02 13:43:57.361702 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.361254    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4s5dk\" (UniqueName: \"kubernetes.io/projected/364e6172-44ee-4b85-a48c-a0f29679abce-kube-api-access-4s5dk\") pod \"multus-976zg\" (UID: \"364e6172-44ee-4b85-a48c-a0f29679abce\") " pod="openshift-multus/multus-976zg"
Jul 02 13:43:57.361702 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.361282    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/355d744e-81d0-41f9-864f-34b8d7b981af-device-dir\") pod \"aws-ebs-csi-driver-node-tvfdz\" (UID: \"355d744e-81d0-41f9-864f-34b8d7b981af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tvfdz"
Jul 02 13:43:57.361702 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.361302    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/364e6172-44ee-4b85-a48c-a0f29679abce-host-run-multus-certs\") pod \"multus-976zg\" (UID: \"364e6172-44ee-4b85-a48c-a0f29679abce\") " pod="openshift-multus/multus-976zg"
Jul 02 13:43:57.361702 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.361308    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7e1c7379-372d-48f3-a865-d0510d6c27a9-host-slash\") pod \"iptables-alerter-lxhdr\" (UID: \"7e1c7379-372d-48f3-a865-d0510d6c27a9\") " pod="openshift-network-operator/iptables-alerter-lxhdr"
Jul 02 13:43:57.361702 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.361311    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/364e6172-44ee-4b85-a48c-a0f29679abce-multus-cni-dir\") pod \"multus-976zg\" (UID: \"364e6172-44ee-4b85-a48c-a0f29679abce\") " pod="openshift-multus/multus-976zg"
Jul 02 13:43:57.361702 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.361337    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e1521a8d-402a-4bee-bc35-bfa382300084-tuning-conf-dir\") pod \"multus-additional-cni-plugins-s85vc\" (UID: \"e1521a8d-402a-4bee-bc35-bfa382300084\") " pod="openshift-multus/multus-additional-cni-plugins-s85vc"
Jul 02 13:43:57.361702 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.361347    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/364e6172-44ee-4b85-a48c-a0f29679abce-system-cni-dir\") pod \"multus-976zg\" (UID: \"364e6172-44ee-4b85-a48c-a0f29679abce\") " pod="openshift-multus/multus-976zg"
Jul 02 13:43:57.361702 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.361363    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/355d744e-81d0-41f9-864f-34b8d7b981af-device-dir\") pod \"aws-ebs-csi-driver-node-tvfdz\" (UID: \"355d744e-81d0-41f9-864f-34b8d7b981af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tvfdz"
Jul 02 13:43:57.362569 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.361348    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7e1c7379-372d-48f3-a865-d0510d6c27a9-host-slash\") pod \"iptables-alerter-lxhdr\" (UID: \"7e1c7379-372d-48f3-a865-d0510d6c27a9\") " pod="openshift-network-operator/iptables-alerter-lxhdr"
Jul 02 13:43:57.362569 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.361377    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/364e6172-44ee-4b85-a48c-a0f29679abce-multus-daemon-config\") pod \"multus-976zg\" (UID: \"364e6172-44ee-4b85-a48c-a0f29679abce\") " pod="openshift-multus/multus-976zg"
Jul 02 13:43:57.362569 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.361392    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/364e6172-44ee-4b85-a48c-a0f29679abce-system-cni-dir\") pod \"multus-976zg\" (UID: \"364e6172-44ee-4b85-a48c-a0f29679abce\") " pod="openshift-multus/multus-976zg"
Jul 02 13:43:57.362569 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.361403    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3310d1f7-1643-4017-9edf-71fa702d2615-run-openvswitch\") pod \"ovnkube-node-8f2ll\" (UID: \"3310d1f7-1643-4017-9edf-71fa702d2615\") " pod="openshift-ovn-kubernetes/ovnkube-node-8f2ll"
Jul 02 13:43:57.362569 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.361433    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3310d1f7-1643-4017-9edf-71fa702d2615-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-8f2ll\" (UID: \"3310d1f7-1643-4017-9edf-71fa702d2615\") " pod="openshift-ovn-kubernetes/ovnkube-node-8f2ll"
Jul 02 13:43:57.362569 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.361469    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/3310d1f7-1643-4017-9edf-71fa702d2615-ovnkube-script-lib\") pod \"ovnkube-node-8f2ll\" (UID: \"3310d1f7-1643-4017-9edf-71fa702d2615\") " pod="openshift-ovn-kubernetes/ovnkube-node-8f2ll"
Jul 02 13:43:57.362569 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.361491    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3310d1f7-1643-4017-9edf-71fa702d2615-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-8f2ll\" (UID: \"3310d1f7-1643-4017-9edf-71fa702d2615\") " pod="openshift-ovn-kubernetes/ovnkube-node-8f2ll"
Jul 02 13:43:57.362569 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.361501    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e1521a8d-402a-4bee-bc35-bfa382300084-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-s85vc\" (UID: \"e1521a8d-402a-4bee-bc35-bfa382300084\") " pod="openshift-multus/multus-additional-cni-plugins-s85vc"
Jul 02 13:43:57.362569 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.361529    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/364e6172-44ee-4b85-a48c-a0f29679abce-host-var-lib-cni-bin\") pod \"multus-976zg\" (UID: \"364e6172-44ee-4b85-a48c-a0f29679abce\") " pod="openshift-multus/multus-976zg"
Jul 02 13:43:57.362569 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.361539    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3310d1f7-1643-4017-9edf-71fa702d2615-run-openvswitch\") pod \"ovnkube-node-8f2ll\" (UID: \"3310d1f7-1643-4017-9edf-71fa702d2615\") " pod="openshift-ovn-kubernetes/ovnkube-node-8f2ll"
Jul 02 13:43:57.362569 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.361584    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/364e6172-44ee-4b85-a48c-a0f29679abce-host-var-lib-cni-bin\") pod \"multus-976zg\" (UID: \"364e6172-44ee-4b85-a48c-a0f29679abce\") " pod="openshift-multus/multus-976zg"
Jul 02 13:43:57.362569 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.361634    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/364e6172-44ee-4b85-a48c-a0f29679abce-host-var-lib-kubelet\") pod \"multus-976zg\" (UID: \"364e6172-44ee-4b85-a48c-a0f29679abce\") " pod="openshift-multus/multus-976zg"
Jul 02 13:43:57.362569 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.361663    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3310d1f7-1643-4017-9edf-71fa702d2615-etc-openvswitch\") pod \"ovnkube-node-8f2ll\" (UID: \"3310d1f7-1643-4017-9edf-71fa702d2615\") " pod="openshift-ovn-kubernetes/ovnkube-node-8f2ll"
Jul 02 13:43:57.362569 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.361676    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/364e6172-44ee-4b85-a48c-a0f29679abce-host-var-lib-kubelet\") pod \"multus-976zg\" (UID: \"364e6172-44ee-4b85-a48c-a0f29679abce\") " pod="openshift-multus/multus-976zg"
Jul 02 13:43:57.362569 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.361690    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3310d1f7-1643-4017-9edf-71fa702d2615-host-run-ovn-kubernetes\") pod \"ovnkube-node-8f2ll\" (UID: \"3310d1f7-1643-4017-9edf-71fa702d2615\") " pod="openshift-ovn-kubernetes/ovnkube-node-8f2ll"
Jul 02 13:43:57.362569 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.361716    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/3310d1f7-1643-4017-9edf-71fa702d2615-ovnkube-config\") pod \"ovnkube-node-8f2ll\" (UID: \"3310d1f7-1643-4017-9edf-71fa702d2615\") " pod="openshift-ovn-kubernetes/ovnkube-node-8f2ll"
Jul 02 13:43:57.362569 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.361730    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3310d1f7-1643-4017-9edf-71fa702d2615-etc-openvswitch\") pod \"ovnkube-node-8f2ll\" (UID: \"3310d1f7-1643-4017-9edf-71fa702d2615\") " pod="openshift-ovn-kubernetes/ovnkube-node-8f2ll"
Jul 02 13:43:57.363514 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.361749    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/3310d1f7-1643-4017-9edf-71fa702d2615-env-overrides\") pod \"ovnkube-node-8f2ll\" (UID: \"3310d1f7-1643-4017-9edf-71fa702d2615\") " pod="openshift-ovn-kubernetes/ovnkube-node-8f2ll"
Jul 02 13:43:57.363514 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.361775    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/859dd06a-dc53-48ca-a2ea-9fea2bad0253-tmp-dir\") pod \"node-resolver-g9pll\" (UID: \"859dd06a-dc53-48ca-a2ea-9fea2bad0253\") " pod="openshift-dns/node-resolver-g9pll"
Jul 02 13:43:57.363514 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.361779    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3310d1f7-1643-4017-9edf-71fa702d2615-host-run-ovn-kubernetes\") pod \"ovnkube-node-8f2ll\" (UID: \"3310d1f7-1643-4017-9edf-71fa702d2615\") " pod="openshift-ovn-kubernetes/ovnkube-node-8f2ll"
Jul 02 13:43:57.363514 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.361802    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e1521a8d-402a-4bee-bc35-bfa382300084-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-s85vc\" (UID: \"e1521a8d-402a-4bee-bc35-bfa382300084\") " pod="openshift-multus/multus-additional-cni-plugins-s85vc"
Jul 02 13:43:57.363514 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.361828    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/355d744e-81d0-41f9-864f-34b8d7b981af-kubelet-dir\") pod \"aws-ebs-csi-driver-node-tvfdz\" (UID: \"355d744e-81d0-41f9-864f-34b8d7b981af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tvfdz"
Jul 02 13:43:57.363514 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.361858    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/355d744e-81d0-41f9-864f-34b8d7b981af-registration-dir\") pod \"aws-ebs-csi-driver-node-tvfdz\" (UID: \"355d744e-81d0-41f9-864f-34b8d7b981af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tvfdz"
Jul 02 13:43:57.363514 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.361901    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3310d1f7-1643-4017-9edf-71fa702d2615-host-cni-bin\") pod \"ovnkube-node-8f2ll\" (UID: \"3310d1f7-1643-4017-9edf-71fa702d2615\") " pod="openshift-ovn-kubernetes/ovnkube-node-8f2ll"
Jul 02 13:43:57.363514 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.361929    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ffqs2\" (UniqueName: \"kubernetes.io/projected/ad2afabf-42cb-4619-b249-a2b011e32565-kube-api-access-ffqs2\") pod \"network-metrics-daemon-29552\" (UID: \"ad2afabf-42cb-4619-b249-a2b011e32565\") " pod="openshift-multus/network-metrics-daemon-29552"
Jul 02 13:43:57.363514 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.361956    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/7e1c7379-372d-48f3-a865-d0510d6c27a9-iptables-alerter-script\") pod \"iptables-alerter-lxhdr\" (UID: \"7e1c7379-372d-48f3-a865-d0510d6c27a9\") " pod="openshift-network-operator/iptables-alerter-lxhdr"
Jul 02 13:43:57.363514 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.361981    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/3310d1f7-1643-4017-9edf-71fa702d2615-run-systemd\") pod \"ovnkube-node-8f2ll\" (UID: \"3310d1f7-1643-4017-9edf-71fa702d2615\") " pod="openshift-ovn-kubernetes/ovnkube-node-8f2ll"
Jul 02 13:43:57.363514 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.362005    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/3310d1f7-1643-4017-9edf-71fa702d2615-ovn-node-metrics-cert\") pod \"ovnkube-node-8f2ll\" (UID: \"3310d1f7-1643-4017-9edf-71fa702d2615\") " pod="openshift-ovn-kubernetes/ovnkube-node-8f2ll"
Jul 02 13:43:57.363514 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.362026    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/364e6172-44ee-4b85-a48c-a0f29679abce-host-run-k8s-cni-cncf-io\") pod \"multus-976zg\" (UID: \"364e6172-44ee-4b85-a48c-a0f29679abce\") " pod="openshift-multus/multus-976zg"
Jul 02 13:43:57.363514 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.362042    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/3310d1f7-1643-4017-9edf-71fa702d2615-run-ovn\") pod \"ovnkube-node-8f2ll\" (UID: \"3310d1f7-1643-4017-9edf-71fa702d2615\") " pod="openshift-ovn-kubernetes/ovnkube-node-8f2ll"
Jul 02 13:43:57.363514 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.362058    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/3310d1f7-1643-4017-9edf-71fa702d2615-node-log\") pod \"ovnkube-node-8f2ll\" (UID: \"3310d1f7-1643-4017-9edf-71fa702d2615\") " pod="openshift-ovn-kubernetes/ovnkube-node-8f2ll"
Jul 02 13:43:57.363514 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.362073    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cdjzl\" (UniqueName: \"kubernetes.io/projected/3310d1f7-1643-4017-9edf-71fa702d2615-kube-api-access-cdjzl\") pod \"ovnkube-node-8f2ll\" (UID: \"3310d1f7-1643-4017-9edf-71fa702d2615\") " pod="openshift-ovn-kubernetes/ovnkube-node-8f2ll"
Jul 02 13:43:57.363514 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.362090    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/364e6172-44ee-4b85-a48c-a0f29679abce-cnibin\") pod \"multus-976zg\" (UID: \"364e6172-44ee-4b85-a48c-a0f29679abce\") " pod="openshift-multus/multus-976zg"
Jul 02 13:43:57.363514 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.362104    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/3310d1f7-1643-4017-9edf-71fa702d2615-host-cni-netd\") pod \"ovnkube-node-8f2ll\" (UID: \"3310d1f7-1643-4017-9edf-71fa702d2615\") " pod="openshift-ovn-kubernetes/ovnkube-node-8f2ll"
Jul 02 13:43:57.364147 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.362121    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4mvm5\" (UniqueName: \"kubernetes.io/projected/859dd06a-dc53-48ca-a2ea-9fea2bad0253-kube-api-access-4mvm5\") pod \"node-resolver-g9pll\" (UID: \"859dd06a-dc53-48ca-a2ea-9fea2bad0253\") " pod="openshift-dns/node-resolver-g9pll"
Jul 02 13:43:57.364147 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.362140    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/364e6172-44ee-4b85-a48c-a0f29679abce-host-run-netns\") pod \"multus-976zg\" (UID: \"364e6172-44ee-4b85-a48c-a0f29679abce\") " pod="openshift-multus/multus-976zg"
Jul 02 13:43:57.364147 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.362135    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/364e6172-44ee-4b85-a48c-a0f29679abce-multus-daemon-config\") pod \"multus-976zg\" (UID: \"364e6172-44ee-4b85-a48c-a0f29679abce\") " pod="openshift-multus/multus-976zg"
Jul 02 13:43:57.364147 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.362161    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/364e6172-44ee-4b85-a48c-a0f29679abce-multus-conf-dir\") pod \"multus-976zg\" (UID: \"364e6172-44ee-4b85-a48c-a0f29679abce\") " pod="openshift-multus/multus-976zg"
Jul 02 13:43:57.364147 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.362178    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p66bz\" (UniqueName: \"kubernetes.io/projected/355d744e-81d0-41f9-864f-34b8d7b981af-kube-api-access-p66bz\") pod \"aws-ebs-csi-driver-node-tvfdz\" (UID: \"355d744e-81d0-41f9-864f-34b8d7b981af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tvfdz"
Jul 02 13:43:57.364147 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.362183    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/859dd06a-dc53-48ca-a2ea-9fea2bad0253-tmp-dir\") pod \"node-resolver-g9pll\" (UID: \"859dd06a-dc53-48ca-a2ea-9fea2bad0253\") " pod="openshift-dns/node-resolver-g9pll"
Jul 02 13:43:57.364147 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.362189    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/3310d1f7-1643-4017-9edf-71fa702d2615-ovnkube-script-lib\") pod \"ovnkube-node-8f2ll\" (UID: \"3310d1f7-1643-4017-9edf-71fa702d2615\") " pod="openshift-ovn-kubernetes/ovnkube-node-8f2ll"
Jul 02 13:43:57.364147 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.362240    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/3310d1f7-1643-4017-9edf-71fa702d2615-env-overrides\") pod \"ovnkube-node-8f2ll\" (UID: \"3310d1f7-1643-4017-9edf-71fa702d2615\") " pod="openshift-ovn-kubernetes/ovnkube-node-8f2ll"
Jul 02 13:43:57.364147 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.362249    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/3310d1f7-1643-4017-9edf-71fa702d2615-log-socket\") pod \"ovnkube-node-8f2ll\" (UID: \"3310d1f7-1643-4017-9edf-71fa702d2615\") " pod="openshift-ovn-kubernetes/ovnkube-node-8f2ll"
Jul 02 13:43:57.364147 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.362259    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/355d744e-81d0-41f9-864f-34b8d7b981af-kubelet-dir\") pod \"aws-ebs-csi-driver-node-tvfdz\" (UID: \"355d744e-81d0-41f9-864f-34b8d7b981af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tvfdz"
Jul 02 13:43:57.364147 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.362195    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/3310d1f7-1643-4017-9edf-71fa702d2615-log-socket\") pod \"ovnkube-node-8f2ll\" (UID: \"3310d1f7-1643-4017-9edf-71fa702d2615\") " pod="openshift-ovn-kubernetes/ovnkube-node-8f2ll"
Jul 02 13:43:57.364147 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.362271    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/364e6172-44ee-4b85-a48c-a0f29679abce-multus-conf-dir\") pod \"multus-976zg\" (UID: \"364e6172-44ee-4b85-a48c-a0f29679abce\") " pod="openshift-multus/multus-976zg"
Jul 02 13:43:57.364147 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.362302    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/3310d1f7-1643-4017-9edf-71fa702d2615-run-ovn\") pod \"ovnkube-node-8f2ll\" (UID: \"3310d1f7-1643-4017-9edf-71fa702d2615\") " pod="openshift-ovn-kubernetes/ovnkube-node-8f2ll"
Jul 02 13:43:57.364147 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.362313    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/364e6172-44ee-4b85-a48c-a0f29679abce-etc-kubernetes\") pod \"multus-976zg\" (UID: \"364e6172-44ee-4b85-a48c-a0f29679abce\") " pod="openshift-multus/multus-976zg"
Jul 02 13:43:57.364147 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.362317    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/3310d1f7-1643-4017-9edf-71fa702d2615-run-systemd\") pod \"ovnkube-node-8f2ll\" (UID: \"3310d1f7-1643-4017-9edf-71fa702d2615\") " pod="openshift-ovn-kubernetes/ovnkube-node-8f2ll"
Jul 02 13:43:57.364147 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.362340    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6ebfc25f-6722-4f73-98b8-18b416460437-kubelet-config\") pod \"global-pull-secret-syncer-2tjcl\" (UID: \"6ebfc25f-6722-4f73-98b8-18b416460437\") " pod="kube-system/global-pull-secret-syncer-2tjcl"
Jul 02 13:43:57.364147 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.362366    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/355d744e-81d0-41f9-864f-34b8d7b981af-etc-selinux\") pod \"aws-ebs-csi-driver-node-tvfdz\" (UID: \"355d744e-81d0-41f9-864f-34b8d7b981af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tvfdz"
Jul 02 13:43:57.364147 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.362395    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lj9vs\" (UniqueName: \"kubernetes.io/projected/e1521a8d-402a-4bee-bc35-bfa382300084-kube-api-access-lj9vs\") pod \"multus-additional-cni-plugins-s85vc\" (UID: \"e1521a8d-402a-4bee-bc35-bfa382300084\") " pod="openshift-multus/multus-additional-cni-plugins-s85vc"
Jul 02 13:43:57.364850 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.362421    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/355d744e-81d0-41f9-864f-34b8d7b981af-socket-dir\") pod \"aws-ebs-csi-driver-node-tvfdz\" (UID: \"355d744e-81d0-41f9-864f-34b8d7b981af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tvfdz"
Jul 02 13:43:57.364850 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.362451    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-djxlq\" (UniqueName: \"kubernetes.io/projected/7e1c7379-372d-48f3-a865-d0510d6c27a9-kube-api-access-djxlq\") pod \"iptables-alerter-lxhdr\" (UID: \"7e1c7379-372d-48f3-a865-d0510d6c27a9\") " pod="openshift-network-operator/iptables-alerter-lxhdr"
Jul 02 13:43:57.364850 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.362469    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/364e6172-44ee-4b85-a48c-a0f29679abce-host-run-k8s-cni-cncf-io\") pod \"multus-976zg\" (UID: \"364e6172-44ee-4b85-a48c-a0f29679abce\") " pod="openshift-multus/multus-976zg"
Jul 02 13:43:57.364850 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.362477    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3310d1f7-1643-4017-9edf-71fa702d2615-var-lib-openvswitch\") pod \"ovnkube-node-8f2ll\" (UID: \"3310d1f7-1643-4017-9edf-71fa702d2615\") " pod="openshift-ovn-kubernetes/ovnkube-node-8f2ll"
Jul 02 13:43:57.364850 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.362500    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/364e6172-44ee-4b85-a48c-a0f29679abce-etc-kubernetes\") pod \"multus-976zg\" (UID: \"364e6172-44ee-4b85-a48c-a0f29679abce\") " pod="openshift-multus/multus-976zg"
Jul 02 13:43:57.364850 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.362509    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/3310d1f7-1643-4017-9edf-71fa702d2615-systemd-units\") pod \"ovnkube-node-8f2ll\" (UID: \"3310d1f7-1643-4017-9edf-71fa702d2615\") " pod="openshift-ovn-kubernetes/ovnkube-node-8f2ll"
Jul 02 13:43:57.364850 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.362551    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/3310d1f7-1643-4017-9edf-71fa702d2615-systemd-units\") pod \"ovnkube-node-8f2ll\" (UID: \"3310d1f7-1643-4017-9edf-71fa702d2615\") " pod="openshift-ovn-kubernetes/ovnkube-node-8f2ll"
Jul 02 13:43:57.364850 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.362565    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/3310d1f7-1643-4017-9edf-71fa702d2615-node-log\") pod \"ovnkube-node-8f2ll\" (UID: \"3310d1f7-1643-4017-9edf-71fa702d2615\") " pod="openshift-ovn-kubernetes/ovnkube-node-8f2ll"
Jul 02 13:43:57.364850 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.362599    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6ebfc25f-6722-4f73-98b8-18b416460437-kubelet-config\") pod \"global-pull-secret-syncer-2tjcl\" (UID: \"6ebfc25f-6722-4f73-98b8-18b416460437\") " pod="kube-system/global-pull-secret-syncer-2tjcl"
Jul 02 13:43:57.364850 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.362614    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/3310d1f7-1643-4017-9edf-71fa702d2615-host-cni-netd\") pod \"ovnkube-node-8f2ll\" (UID: \"3310d1f7-1643-4017-9edf-71fa702d2615\") " pod="openshift-ovn-kubernetes/ovnkube-node-8f2ll"
Jul 02 13:43:57.364850 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.362248    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/364e6172-44ee-4b85-a48c-a0f29679abce-host-run-netns\") pod \"multus-976zg\" (UID: \"364e6172-44ee-4b85-a48c-a0f29679abce\") " pod="openshift-multus/multus-976zg"
Jul 02 13:43:57.364850 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.362250    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3310d1f7-1643-4017-9edf-71fa702d2615-host-cni-bin\") pod \"ovnkube-node-8f2ll\" (UID: \"3310d1f7-1643-4017-9edf-71fa702d2615\") " pod="openshift-ovn-kubernetes/ovnkube-node-8f2ll"
Jul 02 13:43:57.364850 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.362189    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e1521a8d-402a-4bee-bc35-bfa382300084-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-s85vc\" (UID: \"e1521a8d-402a-4bee-bc35-bfa382300084\") " pod="openshift-multus/multus-additional-cni-plugins-s85vc"
Jul 02 13:43:57.364850 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.362689    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/355d744e-81d0-41f9-864f-34b8d7b981af-registration-dir\") pod \"aws-ebs-csi-driver-node-tvfdz\" (UID: \"355d744e-81d0-41f9-864f-34b8d7b981af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tvfdz"
Jul 02 13:43:57.364850 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.362689    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/3310d1f7-1643-4017-9edf-71fa702d2615-ovnkube-config\") pod \"ovnkube-node-8f2ll\" (UID: \"3310d1f7-1643-4017-9edf-71fa702d2615\") " pod="openshift-ovn-kubernetes/ovnkube-node-8f2ll"
Jul 02 13:43:57.364850 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.362705    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/355d744e-81d0-41f9-864f-34b8d7b981af-etc-selinux\") pod \"aws-ebs-csi-driver-node-tvfdz\" (UID: \"355d744e-81d0-41f9-864f-34b8d7b981af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tvfdz"
Jul 02 13:43:57.364850 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.362714    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/859dd06a-dc53-48ca-a2ea-9fea2bad0253-hosts-file\") pod \"node-resolver-g9pll\" (UID: \"859dd06a-dc53-48ca-a2ea-9fea2bad0253\") " pod="openshift-dns/node-resolver-g9pll"
Jul 02 13:43:57.365761 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.362717    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e1521a8d-402a-4bee-bc35-bfa382300084-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-s85vc\" (UID: \"e1521a8d-402a-4bee-bc35-bfa382300084\") " pod="openshift-multus/multus-additional-cni-plugins-s85vc"
Jul 02 13:43:57.365761 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.362745    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/364e6172-44ee-4b85-a48c-a0f29679abce-host-var-lib-cni-multus\") pod \"multus-976zg\" (UID: \"364e6172-44ee-4b85-a48c-a0f29679abce\") " pod="openshift-multus/multus-976zg"
Jul 02 13:43:57.365761 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.362778    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e1521a8d-402a-4bee-bc35-bfa382300084-os-release\") pod \"multus-additional-cni-plugins-s85vc\" (UID: \"e1521a8d-402a-4bee-bc35-bfa382300084\") " pod="openshift-multus/multus-additional-cni-plugins-s85vc"
Jul 02 13:43:57.365761 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.362784    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/364e6172-44ee-4b85-a48c-a0f29679abce-cnibin\") pod \"multus-976zg\" (UID: \"364e6172-44ee-4b85-a48c-a0f29679abce\") " pod="openshift-multus/multus-976zg"
Jul 02 13:43:57.365761 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.362804    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e1521a8d-402a-4bee-bc35-bfa382300084-cni-binary-copy\") pod \"multus-additional-cni-plugins-s85vc\" (UID: \"e1521a8d-402a-4bee-bc35-bfa382300084\") " pod="openshift-multus/multus-additional-cni-plugins-s85vc"
Jul 02 13:43:57.365761 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.362860    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/859dd06a-dc53-48ca-a2ea-9fea2bad0253-hosts-file\") pod \"node-resolver-g9pll\" (UID: \"859dd06a-dc53-48ca-a2ea-9fea2bad0253\") " pod="openshift-dns/node-resolver-g9pll"
Jul 02 13:43:57.365761 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.362951    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/364e6172-44ee-4b85-a48c-a0f29679abce-os-release\") pod \"multus-976zg\" (UID: \"364e6172-44ee-4b85-a48c-a0f29679abce\") " pod="openshift-multus/multus-976zg"
Jul 02 13:43:57.365761 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.362976    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/364e6172-44ee-4b85-a48c-a0f29679abce-cni-binary-copy\") pod \"multus-976zg\" (UID: \"364e6172-44ee-4b85-a48c-a0f29679abce\") " pod="openshift-multus/multus-976zg"
Jul 02 13:43:57.365761 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.362992    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3310d1f7-1643-4017-9edf-71fa702d2615-host-slash\") pod \"ovnkube-node-8f2ll\" (UID: \"3310d1f7-1643-4017-9edf-71fa702d2615\") " pod="openshift-ovn-kubernetes/ovnkube-node-8f2ll"
Jul 02 13:43:57.365761 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.363042    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e1521a8d-402a-4bee-bc35-bfa382300084-os-release\") pod \"multus-additional-cni-plugins-s85vc\" (UID: \"e1521a8d-402a-4bee-bc35-bfa382300084\") " pod="openshift-multus/multus-additional-cni-plugins-s85vc"
Jul 02 13:43:57.365761 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.363108    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/364e6172-44ee-4b85-a48c-a0f29679abce-host-var-lib-cni-multus\") pod \"multus-976zg\" (UID: \"364e6172-44ee-4b85-a48c-a0f29679abce\") " pod="openshift-multus/multus-976zg"
Jul 02 13:43:57.365761 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.363124    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/355d744e-81d0-41f9-864f-34b8d7b981af-socket-dir\") pod \"aws-ebs-csi-driver-node-tvfdz\" (UID: \"355d744e-81d0-41f9-864f-34b8d7b981af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tvfdz"
Jul 02 13:43:57.365761 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.363154    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3310d1f7-1643-4017-9edf-71fa702d2615-var-lib-openvswitch\") pod \"ovnkube-node-8f2ll\" (UID: \"3310d1f7-1643-4017-9edf-71fa702d2615\") " pod="openshift-ovn-kubernetes/ovnkube-node-8f2ll"
Jul 02 13:43:57.365761 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.363158    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/364e6172-44ee-4b85-a48c-a0f29679abce-os-release\") pod \"multus-976zg\" (UID: \"364e6172-44ee-4b85-a48c-a0f29679abce\") " pod="openshift-multus/multus-976zg"
Jul 02 13:43:57.365761 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.363191    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3310d1f7-1643-4017-9edf-71fa702d2615-host-slash\") pod \"ovnkube-node-8f2ll\" (UID: \"3310d1f7-1643-4017-9edf-71fa702d2615\") " pod="openshift-ovn-kubernetes/ovnkube-node-8f2ll"
Jul 02 13:43:57.365761 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.363264    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/7e1c7379-372d-48f3-a865-d0510d6c27a9-iptables-alerter-script\") pod \"iptables-alerter-lxhdr\" (UID: \"7e1c7379-372d-48f3-a865-d0510d6c27a9\") " pod="openshift-network-operator/iptables-alerter-lxhdr"
Jul 02 13:43:57.365761 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.363286    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e1521a8d-402a-4bee-bc35-bfa382300084-cni-binary-copy\") pod \"multus-additional-cni-plugins-s85vc\" (UID: \"e1521a8d-402a-4bee-bc35-bfa382300084\") " pod="openshift-multus/multus-additional-cni-plugins-s85vc"
Jul 02 13:43:57.365761 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.363627    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/364e6172-44ee-4b85-a48c-a0f29679abce-cni-binary-copy\") pod \"multus-976zg\" (UID: \"364e6172-44ee-4b85-a48c-a0f29679abce\") " pod="openshift-multus/multus-976zg"
Jul 02 13:43:57.366707 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.365127    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/3310d1f7-1643-4017-9edf-71fa702d2615-ovn-node-metrics-cert\") pod \"ovnkube-node-8f2ll\" (UID: \"3310d1f7-1643-4017-9edf-71fa702d2615\") " pod="openshift-ovn-kubernetes/ovnkube-node-8f2ll"
Jul 02 13:43:57.372784 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.372758    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4s5dk\" (UniqueName: \"kubernetes.io/projected/364e6172-44ee-4b85-a48c-a0f29679abce-kube-api-access-4s5dk\") pod \"multus-976zg\" (UID: \"364e6172-44ee-4b85-a48c-a0f29679abce\") " pod="openshift-multus/multus-976zg"
Jul 02 13:43:57.373794 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.373753    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-djxlq\" (UniqueName: \"kubernetes.io/projected/7e1c7379-372d-48f3-a865-d0510d6c27a9-kube-api-access-djxlq\") pod \"iptables-alerter-lxhdr\" (UID: \"7e1c7379-372d-48f3-a865-d0510d6c27a9\") " pod="openshift-network-operator/iptables-alerter-lxhdr"
Jul 02 13:43:57.373991 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.373804    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4mvm5\" (UniqueName: \"kubernetes.io/projected/859dd06a-dc53-48ca-a2ea-9fea2bad0253-kube-api-access-4mvm5\") pod \"node-resolver-g9pll\" (UID: \"859dd06a-dc53-48ca-a2ea-9fea2bad0253\") " pod="openshift-dns/node-resolver-g9pll"
Jul 02 13:43:57.373991 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.373931    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cdjzl\" (UniqueName: \"kubernetes.io/projected/3310d1f7-1643-4017-9edf-71fa702d2615-kube-api-access-cdjzl\") pod \"ovnkube-node-8f2ll\" (UID: \"3310d1f7-1643-4017-9edf-71fa702d2615\") " pod="openshift-ovn-kubernetes/ovnkube-node-8f2ll"
Jul 02 13:43:57.376306 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.376251    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lj9vs\" (UniqueName: \"kubernetes.io/projected/e1521a8d-402a-4bee-bc35-bfa382300084-kube-api-access-lj9vs\") pod \"multus-additional-cni-plugins-s85vc\" (UID: \"e1521a8d-402a-4bee-bc35-bfa382300084\") " pod="openshift-multus/multus-additional-cni-plugins-s85vc"
Jul 02 13:43:57.379988 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.379964    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p66bz\" (UniqueName: \"kubernetes.io/projected/355d744e-81d0-41f9-864f-34b8d7b981af-kube-api-access-p66bz\") pod \"aws-ebs-csi-driver-node-tvfdz\" (UID: \"355d744e-81d0-41f9-864f-34b8d7b981af\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tvfdz"
Jul 02 13:43:57.380266 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.380243    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ffqs2\" (UniqueName: \"kubernetes.io/projected/ad2afabf-42cb-4619-b249-a2b011e32565-kube-api-access-ffqs2\") pod \"network-metrics-daemon-29552\" (UID: \"ad2afabf-42cb-4619-b249-a2b011e32565\") " pod="openshift-multus/network-metrics-daemon-29552"
Jul 02 13:43:57.452064 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.452031    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-t99mm"
Jul 02 13:43:57.466579 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.466552    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-scnm4"
Jul 02 13:43:57.474547 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.474520    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-m4ndl"
Jul 02 13:43:57.482593 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.482563    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tvfdz"
Jul 02 13:43:57.491174 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.491146    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-g9pll"
Jul 02 13:43:57.498216 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.498194    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-s85vc"
Jul 02 13:43:57.507814 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.507791    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-lxhdr"
Jul 02 13:43:57.519079 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.519038    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-8f2ll"
Jul 02 13:43:57.524453 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.524433    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-976zg"
Jul 02 13:43:57.670115 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.670039    2558 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 02 13:43:57.767387 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.767350    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cq8jf\" (UniqueName: \"kubernetes.io/projected/06e4af50-04c6-4c58-af10-0e0a28c1a261-kube-api-access-cq8jf\") pod \"network-check-target-q9w6j\" (UID: \"06e4af50-04c6-4c58-af10-0e0a28c1a261\") " pod="openshift-network-diagnostics/network-check-target-q9w6j"
Jul 02 13:43:57.767563 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:43:57.767520    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 02 13:43:57.767563 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:43:57.767540    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 02 13:43:57.767563 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:43:57.767550    2558 projected.go:196] Error preparing data for projected volume kube-api-access-cq8jf for pod openshift-network-diagnostics/network-check-target-q9w6j: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 02 13:43:57.767693 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:43:57.767605    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/06e4af50-04c6-4c58-af10-0e0a28c1a261-kube-api-access-cq8jf podName:06e4af50-04c6-4c58-af10-0e0a28c1a261 nodeName:}" failed. No retries permitted until 2026-07-02 13:43:58.767588354 +0000 UTC m=+3.946050540 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-cq8jf" (UniqueName: "kubernetes.io/projected/06e4af50-04c6-4c58-af10-0e0a28c1a261-kube-api-access-cq8jf") pod "network-check-target-q9w6j" (UID: "06e4af50-04c6-4c58-af10-0e0a28c1a261") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 02 13:43:57.867990 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.867953    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6ebfc25f-6722-4f73-98b8-18b416460437-original-pull-secret\") pod \"global-pull-secret-syncer-2tjcl\" (UID: \"6ebfc25f-6722-4f73-98b8-18b416460437\") " pod="kube-system/global-pull-secret-syncer-2tjcl"
Jul 02 13:43:57.868167 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:57.867998    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ad2afabf-42cb-4619-b249-a2b011e32565-metrics-certs\") pod \"network-metrics-daemon-29552\" (UID: \"ad2afabf-42cb-4619-b249-a2b011e32565\") " pod="openshift-multus/network-metrics-daemon-29552"
Jul 02 13:43:57.868167 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:43:57.868097    2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 02 13:43:57.868167 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:43:57.868129    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 02 13:43:57.868313 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:43:57.868184    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad2afabf-42cb-4619-b249-a2b011e32565-metrics-certs podName:ad2afabf-42cb-4619-b249-a2b011e32565 nodeName:}" failed. No retries permitted until 2026-07-02 13:43:58.86816839 +0000 UTC m=+4.046630555 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ad2afabf-42cb-4619-b249-a2b011e32565-metrics-certs") pod "network-metrics-daemon-29552" (UID: "ad2afabf-42cb-4619-b249-a2b011e32565") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 02 13:43:57.868313 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:43:57.868200    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ebfc25f-6722-4f73-98b8-18b416460437-original-pull-secret podName:6ebfc25f-6722-4f73-98b8-18b416460437 nodeName:}" failed. No retries permitted until 2026-07-02 13:43:58.868193746 +0000 UTC m=+4.046655909 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6ebfc25f-6722-4f73-98b8-18b416460437-original-pull-secret") pod "global-pull-secret-syncer-2tjcl" (UID: "6ebfc25f-6722-4f73-98b8-18b416460437") : object "kube-system"/"original-pull-secret" not registered
Jul 02 13:43:58.201170 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:58.201129    2558 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-01 13:38:57 +0000 UTC" deadline="2027-11-27 01:17:09.05361961 +0000 UTC"
Jul 02 13:43:58.201170 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:58.201163    2558 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12299h33m10.852459846s"
Jul 02 13:43:58.269946 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:58.269883    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1521a8d_402a_4bee_bc35_bfa382300084.slice/crio-558604b9586c088b39bd66f7e191ccf1d1acb6c778a0fbd1b2df0313714202ed WatchSource:0}: Error finding container 558604b9586c088b39bd66f7e191ccf1d1acb6c778a0fbd1b2df0313714202ed: Status 404 returned error can't find the container with id 558604b9586c088b39bd66f7e191ccf1d1acb6c778a0fbd1b2df0313714202ed
Jul 02 13:43:58.278133 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:58.278101    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod859dd06a_dc53_48ca_a2ea_9fea2bad0253.slice/crio-eef1c9bae4918140b53de2a6d511953cf0d83525bea90e50586847d33016e3f8 WatchSource:0}: Error finding container eef1c9bae4918140b53de2a6d511953cf0d83525bea90e50586847d33016e3f8: Status 404 returned error can't find the container with id eef1c9bae4918140b53de2a6d511953cf0d83525bea90e50586847d33016e3f8
Jul 02 13:43:58.278389 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:58.278359    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod364e6172_44ee_4b85_a48c_a0f29679abce.slice/crio-83ebb9ee84b0368d6debe831219b8863ae69dce811cb166dd8f6cee6e2c985e5 WatchSource:0}: Error finding container 83ebb9ee84b0368d6debe831219b8863ae69dce811cb166dd8f6cee6e2c985e5: Status 404 returned error can't find the container with id 83ebb9ee84b0368d6debe831219b8863ae69dce811cb166dd8f6cee6e2c985e5
Jul 02 13:43:58.279299 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:58.279261    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde0396fd_b15d_4e52_b0af_7e40954f53db.slice/crio-34e8a799ef74cbf1d641b37eba93cb3c656f076e26ea6ea3feeae26913dcf6c6 WatchSource:0}: Error finding container 34e8a799ef74cbf1d641b37eba93cb3c656f076e26ea6ea3feeae26913dcf6c6: Status 404 returned error can't find the container with id 34e8a799ef74cbf1d641b37eba93cb3c656f076e26ea6ea3feeae26913dcf6c6
Jul 02 13:43:58.284135 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:43:58.283952    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod355d744e_81d0_41f9_864f_34b8d7b981af.slice/crio-1f421be7ebfb3d2a414ec24880148a8623e181c4621bf28330efea7727566fb6 WatchSource:0}: Error finding container 1f421be7ebfb3d2a414ec24880148a8623e181c4621bf28330efea7727566fb6: Status 404 returned error can't find the container with id 1f421be7ebfb3d2a414ec24880148a8623e181c4621bf28330efea7727566fb6
Jul 02 13:43:58.775009 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:58.774633    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cq8jf\" (UniqueName: \"kubernetes.io/projected/06e4af50-04c6-4c58-af10-0e0a28c1a261-kube-api-access-cq8jf\") pod \"network-check-target-q9w6j\" (UID: \"06e4af50-04c6-4c58-af10-0e0a28c1a261\") " pod="openshift-network-diagnostics/network-check-target-q9w6j"
Jul 02 13:43:58.775170 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:43:58.774841    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 02 13:43:58.775170 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:43:58.775077    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 02 13:43:58.775170 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:43:58.775094    2558 projected.go:196] Error preparing data for projected volume kube-api-access-cq8jf for pod openshift-network-diagnostics/network-check-target-q9w6j: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 02 13:43:58.775170 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:43:58.775161    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/06e4af50-04c6-4c58-af10-0e0a28c1a261-kube-api-access-cq8jf podName:06e4af50-04c6-4c58-af10-0e0a28c1a261 nodeName:}" failed. No retries permitted until 2026-07-02 13:44:00.775138673 +0000 UTC m=+5.953600858 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-cq8jf" (UniqueName: "kubernetes.io/projected/06e4af50-04c6-4c58-af10-0e0a28c1a261-kube-api-access-cq8jf") pod "network-check-target-q9w6j" (UID: "06e4af50-04c6-4c58-af10-0e0a28c1a261") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 02 13:43:58.876034 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:58.875997    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6ebfc25f-6722-4f73-98b8-18b416460437-original-pull-secret\") pod \"global-pull-secret-syncer-2tjcl\" (UID: \"6ebfc25f-6722-4f73-98b8-18b416460437\") " pod="kube-system/global-pull-secret-syncer-2tjcl"
Jul 02 13:43:58.876237 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:58.876050    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ad2afabf-42cb-4619-b249-a2b011e32565-metrics-certs\") pod \"network-metrics-daemon-29552\" (UID: \"ad2afabf-42cb-4619-b249-a2b011e32565\") " pod="openshift-multus/network-metrics-daemon-29552"
Jul 02 13:43:58.876237 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:43:58.876173    2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 02 13:43:58.876237 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:43:58.876204    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 02 13:43:58.876400 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:43:58.876267    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ebfc25f-6722-4f73-98b8-18b416460437-original-pull-secret podName:6ebfc25f-6722-4f73-98b8-18b416460437 nodeName:}" failed. No retries permitted until 2026-07-02 13:44:00.876242718 +0000 UTC m=+6.054704883 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6ebfc25f-6722-4f73-98b8-18b416460437-original-pull-secret") pod "global-pull-secret-syncer-2tjcl" (UID: "6ebfc25f-6722-4f73-98b8-18b416460437") : object "kube-system"/"original-pull-secret" not registered
Jul 02 13:43:58.876400 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:43:58.876299    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad2afabf-42cb-4619-b249-a2b011e32565-metrics-certs podName:ad2afabf-42cb-4619-b249-a2b011e32565 nodeName:}" failed. No retries permitted until 2026-07-02 13:44:00.87627883 +0000 UTC m=+6.054740999 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ad2afabf-42cb-4619-b249-a2b011e32565-metrics-certs") pod "network-metrics-daemon-29552" (UID: "ad2afabf-42cb-4619-b249-a2b011e32565") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 02 13:43:59.201930 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:59.201706    2558 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-01 13:38:57 +0000 UTC" deadline="2028-03-05 11:00:03.493016151 +0000 UTC"
Jul 02 13:43:59.201930 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:59.201765    2558 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14685h16m4.291255412s"
Jul 02 13:43:59.211910 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:59.211875    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29552"
Jul 02 13:43:59.212084 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:43:59.212053    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-29552" podUID="ad2afabf-42cb-4619-b249-a2b011e32565"
Jul 02 13:43:59.212480 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:59.212455    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2tjcl"
Jul 02 13:43:59.212588 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:43:59.212563    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2tjcl" podUID="6ebfc25f-6722-4f73-98b8-18b416460437"
Jul 02 13:43:59.212649 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:59.212629    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q9w6j"
Jul 02 13:43:59.212720 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:43:59.212701    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q9w6j" podUID="06e4af50-04c6-4c58-af10-0e0a28c1a261"
Jul 02 13:43:59.236545 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:59.236427    2558 generic.go:356] "Generic (PLEG): container finished" podID="44e46b482fd93a3b06e0f6824e9467c5" containerID="6d6c155ce4d51587405ab0f23a18da4c0d910ec1cd0182d3e47e02a40f135c38" exitCode=0
Jul 02 13:43:59.236545 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:59.236538    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-209.ec2.internal" event={"ID":"44e46b482fd93a3b06e0f6824e9467c5","Type":"ContainerDied","Data":"6d6c155ce4d51587405ab0f23a18da4c0d910ec1cd0182d3e47e02a40f135c38"}
Jul 02 13:43:59.243466 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:59.243432    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-lxhdr" event={"ID":"7e1c7379-372d-48f3-a865-d0510d6c27a9","Type":"ContainerStarted","Data":"2cdadb8be40c822c686a74ac34850aa35ed70716b4de56af3c50dde616e44630"}
Jul 02 13:43:59.253246 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:59.253199    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-t99mm" event={"ID":"de0396fd-b15d-4e52-b0af-7e40954f53db","Type":"ContainerStarted","Data":"34e8a799ef74cbf1d641b37eba93cb3c656f076e26ea6ea3feeae26913dcf6c6"}
Jul 02 13:43:59.256487 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:59.256453    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-976zg" event={"ID":"364e6172-44ee-4b85-a48c-a0f29679abce","Type":"ContainerStarted","Data":"83ebb9ee84b0368d6debe831219b8863ae69dce811cb166dd8f6cee6e2c985e5"}
Jul 02 13:43:59.270848 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:59.270805    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-s85vc" event={"ID":"e1521a8d-402a-4bee-bc35-bfa382300084","Type":"ContainerStarted","Data":"558604b9586c088b39bd66f7e191ccf1d1acb6c778a0fbd1b2df0313714202ed"}
Jul 02 13:43:59.284232 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:59.284173    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-209.ec2.internal" event={"ID":"4d8a265dbb955726a248c0c5b7ee4114","Type":"ContainerStarted","Data":"44c52031b57cd3663c292e0e3a87b39bea285166263bb95212763f2a48a81eb7"}
Jul 02 13:43:59.295191 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:59.295154    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-m4ndl" event={"ID":"23638fc3-c572-4760-8e60-e82de1c6f290","Type":"ContainerStarted","Data":"f4c664dd472aa9c22a11561b293ab047bc7f02910283ff279ac09e7dfff447e6"}
Jul 02 13:43:59.314865 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:59.314823    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tvfdz" event={"ID":"355d744e-81d0-41f9-864f-34b8d7b981af","Type":"ContainerStarted","Data":"1f421be7ebfb3d2a414ec24880148a8623e181c4621bf28330efea7727566fb6"}
Jul 02 13:43:59.356158 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:59.356067    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-g9pll" event={"ID":"859dd06a-dc53-48ca-a2ea-9fea2bad0253","Type":"ContainerStarted","Data":"eef1c9bae4918140b53de2a6d511953cf0d83525bea90e50586847d33016e3f8"}
Jul 02 13:43:59.374127 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:59.373645    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8f2ll" event={"ID":"3310d1f7-1643-4017-9edf-71fa702d2615","Type":"ContainerStarted","Data":"fabbfbfd9dd8d5706e55465632d061de1415eb13f9d429554870bf6ea26c3392"}
Jul 02 13:43:59.393933 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:43:59.393870    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-scnm4" event={"ID":"823f515b-2ffa-4001-a325-1acc4a714ca3","Type":"ContainerStarted","Data":"9b3ff3077c735d9905ffa14f520ae567e30c63a4cf48d7bad8722d30219ff9a3"}
Jul 02 13:44:00.424020 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:00.423977    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-209.ec2.internal" event={"ID":"44e46b482fd93a3b06e0f6824e9467c5","Type":"ContainerStarted","Data":"57c93212ff48e898cbde47ff54c243eb221bb2225f106b213c45defec6493e48"}
Jul 02 13:44:00.443971 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:00.443874    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-209.ec2.internal" podStartSLOduration=4.443850898 podStartE2EDuration="4.443850898s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 13:43:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:43:59.310616654 +0000 UTC m=+4.489078852" watchObservedRunningTime="2026-07-02 13:44:00.443850898 +0000 UTC m=+5.622313088"
Jul 02 13:44:00.796617 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:00.796524    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cq8jf\" (UniqueName: \"kubernetes.io/projected/06e4af50-04c6-4c58-af10-0e0a28c1a261-kube-api-access-cq8jf\") pod \"network-check-target-q9w6j\" (UID: \"06e4af50-04c6-4c58-af10-0e0a28c1a261\") " pod="openshift-network-diagnostics/network-check-target-q9w6j"
Jul 02 13:44:00.796788 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:44:00.796755    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 02 13:44:00.796788 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:44:00.796780    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 02 13:44:00.796887 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:44:00.796793    2558 projected.go:196] Error preparing data for projected volume kube-api-access-cq8jf for pod openshift-network-diagnostics/network-check-target-q9w6j: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 02 13:44:00.796887 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:44:00.796857    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/06e4af50-04c6-4c58-af10-0e0a28c1a261-kube-api-access-cq8jf podName:06e4af50-04c6-4c58-af10-0e0a28c1a261 nodeName:}" failed. No retries permitted until 2026-07-02 13:44:04.796837304 +0000 UTC m=+9.975299493 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-cq8jf" (UniqueName: "kubernetes.io/projected/06e4af50-04c6-4c58-af10-0e0a28c1a261-kube-api-access-cq8jf") pod "network-check-target-q9w6j" (UID: "06e4af50-04c6-4c58-af10-0e0a28c1a261") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 02 13:44:00.897375 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:00.897158    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6ebfc25f-6722-4f73-98b8-18b416460437-original-pull-secret\") pod \"global-pull-secret-syncer-2tjcl\" (UID: \"6ebfc25f-6722-4f73-98b8-18b416460437\") " pod="kube-system/global-pull-secret-syncer-2tjcl"
Jul 02 13:44:00.897375 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:00.897216    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ad2afabf-42cb-4619-b249-a2b011e32565-metrics-certs\") pod \"network-metrics-daemon-29552\" (UID: \"ad2afabf-42cb-4619-b249-a2b011e32565\") " pod="openshift-multus/network-metrics-daemon-29552"
Jul 02 13:44:00.897375 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:44:00.897300    2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 02 13:44:00.897375 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:44:00.897324    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 02 13:44:00.897375 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:44:00.897366    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ebfc25f-6722-4f73-98b8-18b416460437-original-pull-secret podName:6ebfc25f-6722-4f73-98b8-18b416460437 nodeName:}" failed. No retries permitted until 2026-07-02 13:44:04.897349064 +0000 UTC m=+10.075811251 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6ebfc25f-6722-4f73-98b8-18b416460437-original-pull-secret") pod "global-pull-secret-syncer-2tjcl" (UID: "6ebfc25f-6722-4f73-98b8-18b416460437") : object "kube-system"/"original-pull-secret" not registered
Jul 02 13:44:00.897375 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:44:00.897384    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad2afabf-42cb-4619-b249-a2b011e32565-metrics-certs podName:ad2afabf-42cb-4619-b249-a2b011e32565 nodeName:}" failed. No retries permitted until 2026-07-02 13:44:04.89737561 +0000 UTC m=+10.075837773 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ad2afabf-42cb-4619-b249-a2b011e32565-metrics-certs") pod "network-metrics-daemon-29552" (UID: "ad2afabf-42cb-4619-b249-a2b011e32565") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 02 13:44:01.214127 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:01.214044    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2tjcl"
Jul 02 13:44:01.214307 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:44:01.214180    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2tjcl" podUID="6ebfc25f-6722-4f73-98b8-18b416460437"
Jul 02 13:44:01.214637 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:01.214586    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q9w6j"
Jul 02 13:44:01.214749 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:44:01.214687    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q9w6j" podUID="06e4af50-04c6-4c58-af10-0e0a28c1a261"
Jul 02 13:44:01.214942 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:01.214922    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29552"
Jul 02 13:44:01.215058 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:44:01.215035    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-29552" podUID="ad2afabf-42cb-4619-b249-a2b011e32565"
Jul 02 13:44:03.213009 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:03.212931    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q9w6j"
Jul 02 13:44:03.213470 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:44:03.213062    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q9w6j" podUID="06e4af50-04c6-4c58-af10-0e0a28c1a261"
Jul 02 13:44:03.213470 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:03.213452    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29552"
Jul 02 13:44:03.213570 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:44:03.213555    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-29552" podUID="ad2afabf-42cb-4619-b249-a2b011e32565"
Jul 02 13:44:03.213805 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:03.213676    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2tjcl"
Jul 02 13:44:03.213805 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:44:03.213754    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2tjcl" podUID="6ebfc25f-6722-4f73-98b8-18b416460437"
Jul 02 13:44:04.833437 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:04.833397    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cq8jf\" (UniqueName: \"kubernetes.io/projected/06e4af50-04c6-4c58-af10-0e0a28c1a261-kube-api-access-cq8jf\") pod \"network-check-target-q9w6j\" (UID: \"06e4af50-04c6-4c58-af10-0e0a28c1a261\") " pod="openshift-network-diagnostics/network-check-target-q9w6j"
Jul 02 13:44:04.833865 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:44:04.833543    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 02 13:44:04.833865 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:44:04.833566    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 02 13:44:04.833865 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:44:04.833579    2558 projected.go:196] Error preparing data for projected volume kube-api-access-cq8jf for pod openshift-network-diagnostics/network-check-target-q9w6j: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 02 13:44:04.833865 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:44:04.833635    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/06e4af50-04c6-4c58-af10-0e0a28c1a261-kube-api-access-cq8jf podName:06e4af50-04c6-4c58-af10-0e0a28c1a261 nodeName:}" failed. No retries permitted until 2026-07-02 13:44:12.833617718 +0000 UTC m=+18.012079897 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-cq8jf" (UniqueName: "kubernetes.io/projected/06e4af50-04c6-4c58-af10-0e0a28c1a261-kube-api-access-cq8jf") pod "network-check-target-q9w6j" (UID: "06e4af50-04c6-4c58-af10-0e0a28c1a261") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 02 13:44:04.934321 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:04.934275    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6ebfc25f-6722-4f73-98b8-18b416460437-original-pull-secret\") pod \"global-pull-secret-syncer-2tjcl\" (UID: \"6ebfc25f-6722-4f73-98b8-18b416460437\") " pod="kube-system/global-pull-secret-syncer-2tjcl"
Jul 02 13:44:04.934488 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:04.934339    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ad2afabf-42cb-4619-b249-a2b011e32565-metrics-certs\") pod \"network-metrics-daemon-29552\" (UID: \"ad2afabf-42cb-4619-b249-a2b011e32565\") " pod="openshift-multus/network-metrics-daemon-29552"
Jul 02 13:44:04.934575 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:44:04.934525    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 02 13:44:04.934642 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:44:04.934596    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad2afabf-42cb-4619-b249-a2b011e32565-metrics-certs podName:ad2afabf-42cb-4619-b249-a2b011e32565 nodeName:}" failed. No retries permitted until 2026-07-02 13:44:12.934573582 +0000 UTC m=+18.113035763 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ad2afabf-42cb-4619-b249-a2b011e32565-metrics-certs") pod "network-metrics-daemon-29552" (UID: "ad2afabf-42cb-4619-b249-a2b011e32565") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 02 13:44:04.935057 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:44:04.935031    2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 02 13:44:04.935179 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:44:04.935096    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ebfc25f-6722-4f73-98b8-18b416460437-original-pull-secret podName:6ebfc25f-6722-4f73-98b8-18b416460437 nodeName:}" failed. No retries permitted until 2026-07-02 13:44:12.935079449 +0000 UTC m=+18.113541629 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6ebfc25f-6722-4f73-98b8-18b416460437-original-pull-secret") pod "global-pull-secret-syncer-2tjcl" (UID: "6ebfc25f-6722-4f73-98b8-18b416460437") : object "kube-system"/"original-pull-secret" not registered
Jul 02 13:44:05.212686 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:05.212602    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q9w6j"
Jul 02 13:44:05.213570 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:05.213055    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2tjcl"
Jul 02 13:44:05.213570 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:44:05.213187    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2tjcl" podUID="6ebfc25f-6722-4f73-98b8-18b416460437"
Jul 02 13:44:05.213993 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:05.212941    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29552"
Jul 02 13:44:05.213993 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:44:05.213960    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-29552" podUID="ad2afabf-42cb-4619-b249-a2b011e32565"
Jul 02 13:44:05.214608 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:44:05.214586    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q9w6j" podUID="06e4af50-04c6-4c58-af10-0e0a28c1a261"
Jul 02 13:44:07.211946 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:07.211916    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q9w6j"
Jul 02 13:44:07.212374 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:07.211925    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29552"
Jul 02 13:44:07.212374 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:44:07.212042    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q9w6j" podUID="06e4af50-04c6-4c58-af10-0e0a28c1a261"
Jul 02 13:44:07.212374 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:07.212059    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2tjcl"
Jul 02 13:44:07.212374 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:44:07.212148    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-29552" podUID="ad2afabf-42cb-4619-b249-a2b011e32565"
Jul 02 13:44:07.212374 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:44:07.212201    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2tjcl" podUID="6ebfc25f-6722-4f73-98b8-18b416460437"
Jul 02 13:44:09.212121 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:09.212085    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2tjcl"
Jul 02 13:44:09.212605 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:09.212084    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29552"
Jul 02 13:44:09.212605 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:09.212227    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q9w6j"
Jul 02 13:44:09.212605 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:44:09.212224    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2tjcl" podUID="6ebfc25f-6722-4f73-98b8-18b416460437"
Jul 02 13:44:09.212605 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:44:09.212317    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-29552" podUID="ad2afabf-42cb-4619-b249-a2b011e32565"
Jul 02 13:44:09.212605 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:44:09.212370    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q9w6j" podUID="06e4af50-04c6-4c58-af10-0e0a28c1a261"
Jul 02 13:44:11.211906 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:11.211864    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q9w6j"
Jul 02 13:44:11.212369 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:11.211911    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2tjcl"
Jul 02 13:44:11.212369 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:11.211911    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29552"
Jul 02 13:44:11.212369 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:44:11.211979    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q9w6j" podUID="06e4af50-04c6-4c58-af10-0e0a28c1a261"
Jul 02 13:44:11.212369 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:44:11.212105    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-29552" podUID="ad2afabf-42cb-4619-b249-a2b011e32565"
Jul 02 13:44:11.212369 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:44:11.212170    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2tjcl" podUID="6ebfc25f-6722-4f73-98b8-18b416460437"
Jul 02 13:44:12.899348 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:12.899310    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cq8jf\" (UniqueName: \"kubernetes.io/projected/06e4af50-04c6-4c58-af10-0e0a28c1a261-kube-api-access-cq8jf\") pod \"network-check-target-q9w6j\" (UID: \"06e4af50-04c6-4c58-af10-0e0a28c1a261\") " pod="openshift-network-diagnostics/network-check-target-q9w6j"
Jul 02 13:44:12.899696 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:44:12.899496    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 02 13:44:12.899696 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:44:12.899518    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 02 13:44:12.899696 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:44:12.899528    2558 projected.go:196] Error preparing data for projected volume kube-api-access-cq8jf for pod openshift-network-diagnostics/network-check-target-q9w6j: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 02 13:44:12.899696 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:44:12.899580    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/06e4af50-04c6-4c58-af10-0e0a28c1a261-kube-api-access-cq8jf podName:06e4af50-04c6-4c58-af10-0e0a28c1a261 nodeName:}" failed. No retries permitted until 2026-07-02 13:44:28.899567229 +0000 UTC m=+34.078029393 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-cq8jf" (UniqueName: "kubernetes.io/projected/06e4af50-04c6-4c58-af10-0e0a28c1a261-kube-api-access-cq8jf") pod "network-check-target-q9w6j" (UID: "06e4af50-04c6-4c58-af10-0e0a28c1a261") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 02 13:44:12.999819 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:12.999778    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6ebfc25f-6722-4f73-98b8-18b416460437-original-pull-secret\") pod \"global-pull-secret-syncer-2tjcl\" (UID: \"6ebfc25f-6722-4f73-98b8-18b416460437\") " pod="kube-system/global-pull-secret-syncer-2tjcl"
Jul 02 13:44:12.999819 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:12.999839    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ad2afabf-42cb-4619-b249-a2b011e32565-metrics-certs\") pod \"network-metrics-daemon-29552\" (UID: \"ad2afabf-42cb-4619-b249-a2b011e32565\") " pod="openshift-multus/network-metrics-daemon-29552"
Jul 02 13:44:13.000060 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:44:12.999929    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 02 13:44:13.000060 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:44:12.999937    2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 02 13:44:13.000060 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:44:12.999981    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad2afabf-42cb-4619-b249-a2b011e32565-metrics-certs podName:ad2afabf-42cb-4619-b249-a2b011e32565 nodeName:}" failed. No retries permitted until 2026-07-02 13:44:28.999967696 +0000 UTC m=+34.178429859 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ad2afabf-42cb-4619-b249-a2b011e32565-metrics-certs") pod "network-metrics-daemon-29552" (UID: "ad2afabf-42cb-4619-b249-a2b011e32565") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 02 13:44:13.000060 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:44:13.000006    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ebfc25f-6722-4f73-98b8-18b416460437-original-pull-secret podName:6ebfc25f-6722-4f73-98b8-18b416460437 nodeName:}" failed. No retries permitted until 2026-07-02 13:44:28.999987372 +0000 UTC m=+34.178449549 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6ebfc25f-6722-4f73-98b8-18b416460437-original-pull-secret") pod "global-pull-secret-syncer-2tjcl" (UID: "6ebfc25f-6722-4f73-98b8-18b416460437") : object "kube-system"/"original-pull-secret" not registered
Jul 02 13:44:13.212198 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:13.212163    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29552"
Jul 02 13:44:13.212354 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:44:13.212301    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-29552" podUID="ad2afabf-42cb-4619-b249-a2b011e32565"
Jul 02 13:44:13.212752 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:13.212726    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2tjcl"
Jul 02 13:44:13.212887 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:44:13.212866    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2tjcl" podUID="6ebfc25f-6722-4f73-98b8-18b416460437"
Jul 02 13:44:13.212988 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:13.212965    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q9w6j"
Jul 02 13:44:13.213074 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:44:13.213056    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q9w6j" podUID="06e4af50-04c6-4c58-af10-0e0a28c1a261"
Jul 02 13:44:15.212983 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:15.212955    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29552"
Jul 02 13:44:15.213508 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:44:15.213055    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-29552" podUID="ad2afabf-42cb-4619-b249-a2b011e32565"
Jul 02 13:44:15.213508 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:15.213128    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2tjcl"
Jul 02 13:44:15.213508 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:44:15.213222    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2tjcl" podUID="6ebfc25f-6722-4f73-98b8-18b416460437"
Jul 02 13:44:15.213508 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:15.213258    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q9w6j"
Jul 02 13:44:15.213508 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:44:15.213323    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q9w6j" podUID="06e4af50-04c6-4c58-af10-0e0a28c1a261"
Jul 02 13:44:17.213059 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:17.212305    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q9w6j"
Jul 02 13:44:17.213059 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:17.212322    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2tjcl"
Jul 02 13:44:17.213059 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:17.212323    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29552"
Jul 02 13:44:17.213059 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:44:17.212438    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q9w6j" podUID="06e4af50-04c6-4c58-af10-0e0a28c1a261"
Jul 02 13:44:17.213059 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:44:17.212544    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-29552" podUID="ad2afabf-42cb-4619-b249-a2b011e32565"
Jul 02 13:44:17.213059 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:44:17.212625    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2tjcl" podUID="6ebfc25f-6722-4f73-98b8-18b416460437"
Jul 02 13:44:18.464398 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:18.463768    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-t99mm" event={"ID":"de0396fd-b15d-4e52-b0af-7e40954f53db","Type":"ContainerStarted","Data":"07a8e3bf6765d569ce45e6fc4e241eba6bbd38bc25446ef69bc1c18d806be95e"}
Jul 02 13:44:18.465216 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:18.465109    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-976zg" event={"ID":"364e6172-44ee-4b85-a48c-a0f29679abce","Type":"ContainerStarted","Data":"0d27b8409610e194e478035b78e3e20331fa3bb56ac25b115a01b4a7d28d6378"}
Jul 02 13:44:18.466689 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:18.466667    2558 generic.go:356] "Generic (PLEG): container finished" podID="e1521a8d-402a-4bee-bc35-bfa382300084" containerID="f59d09bd68c9cca39a618b0962eb5e220bad5d4d972458b8758dc407dadcf3fa" exitCode=0
Jul 02 13:44:18.466802 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:18.466728    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-s85vc" event={"ID":"e1521a8d-402a-4bee-bc35-bfa382300084","Type":"ContainerDied","Data":"f59d09bd68c9cca39a618b0962eb5e220bad5d4d972458b8758dc407dadcf3fa"}
Jul 02 13:44:18.468563 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:18.468534    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-m4ndl" event={"ID":"23638fc3-c572-4760-8e60-e82de1c6f290","Type":"ContainerStarted","Data":"d8a2c8b73f56beea99537c78e2bd31bdcbc6ec61014f6a1d506bea1030e886cc"}
Jul 02 13:44:18.470019 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:18.469999    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tvfdz" event={"ID":"355d744e-81d0-41f9-864f-34b8d7b981af","Type":"ContainerStarted","Data":"cb7ce1fb1ba7439ec5619507c51c91f1688f175fa72f642bcacc5a70255eb192"}
Jul 02 13:44:18.471423 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:18.471404    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-g9pll" event={"ID":"859dd06a-dc53-48ca-a2ea-9fea2bad0253","Type":"ContainerStarted","Data":"50145bb8c4d38e5066a5654db822560a3cded355729962b11f5b0474ae050535"}
Jul 02 13:44:18.474005 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:18.473988    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8f2ll_3310d1f7-1643-4017-9edf-71fa702d2615/ovn-acl-logging/0.log"
Jul 02 13:44:18.474343 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:18.474326    2558 generic.go:356] "Generic (PLEG): container finished" podID="3310d1f7-1643-4017-9edf-71fa702d2615" containerID="bc7074cdb1f3898a71c09f258b5a6aec671412caae00742fd8e52ea179710fb5" exitCode=1
Jul 02 13:44:18.474413 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:18.474388    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8f2ll" event={"ID":"3310d1f7-1643-4017-9edf-71fa702d2615","Type":"ContainerStarted","Data":"2df22eb9f16289096c7d2f4b3aa493f6b02611c6de04c49ffb4d26793c132986"}
Jul 02 13:44:18.474519 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:18.474503    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8f2ll" event={"ID":"3310d1f7-1643-4017-9edf-71fa702d2615","Type":"ContainerStarted","Data":"e2bb73458a03406b029b15bdcadce5fa9e9e4fcaba60241c2ac51601df5cd92b"}
Jul 02 13:44:18.474589 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:18.474562    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8f2ll" event={"ID":"3310d1f7-1643-4017-9edf-71fa702d2615","Type":"ContainerStarted","Data":"013e82b84d742bbb6ca496fd891f097c960ea5e2472e30728a7cc2d4b3844289"}
Jul 02 13:44:18.474645 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:18.474614    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8f2ll" event={"ID":"3310d1f7-1643-4017-9edf-71fa702d2615","Type":"ContainerStarted","Data":"00d113cb0a0a59661c1436e973640dee8f0d0cd92d70a208d1b912af9401a22f"}
Jul 02 13:44:18.474703 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:18.474663    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8f2ll" event={"ID":"3310d1f7-1643-4017-9edf-71fa702d2615","Type":"ContainerDied","Data":"bc7074cdb1f3898a71c09f258b5a6aec671412caae00742fd8e52ea179710fb5"}
Jul 02 13:44:18.474751 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:18.474714    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8f2ll" event={"ID":"3310d1f7-1643-4017-9edf-71fa702d2615","Type":"ContainerStarted","Data":"7f5dec48f9cf8e6a43db3496f593dda2881bc39a7a181524c9fe77f487a726da"}
Jul 02 13:44:18.475814 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:18.475791    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-scnm4" event={"ID":"823f515b-2ffa-4001-a325-1acc4a714ca3","Type":"ContainerStarted","Data":"a538699d9568483ab5cd49b7d5055d56d5ba1f0fbd1755bd1227563d5f14f26a"}
Jul 02 13:44:18.482425 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:18.482389    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-209.ec2.internal" podStartSLOduration=22.48237844 podStartE2EDuration="22.48237844s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 13:43:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:44:00.444800984 +0000 UTC m=+5.623263184" watchObservedRunningTime="2026-07-02 13:44:18.48237844 +0000 UTC m=+23.660840623"
Jul 02 13:44:18.482881 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:18.482846    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-t99mm" podStartSLOduration=4.355731905 podStartE2EDuration="23.482816967s" totalImagesPullingTime="19.127085062s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 13:43:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:44:18.482220945 +0000 UTC m=+23.660683140" watchObservedRunningTime="2026-07-02 13:44:18.482816967 +0000 UTC m=+23.661279151"
Jul 02 13:44:18.505641 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:18.505589    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-scnm4" podStartSLOduration=4.364222014 podStartE2EDuration="23.505573729s" totalImagesPullingTime="19.141351715s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 13:43:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:44:18.505459678 +0000 UTC m=+23.683921862" watchObservedRunningTime="2026-07-02 13:44:18.505573729 +0000 UTC m=+23.684035913"
Jul 02 13:44:18.525138 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:18.525083    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-g9pll" podStartSLOduration=4.414360598 podStartE2EDuration="23.525071141s" totalImagesPullingTime="19.110710543s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 13:43:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:44:18.524610368 +0000 UTC m=+23.703072569" watchObservedRunningTime="2026-07-02 13:44:18.525071141 +0000 UTC m=+23.703533324"
Jul 02 13:44:18.545259 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:18.545200    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-976zg" podStartSLOduration=4.020478347 podStartE2EDuration="23.545158267s" totalImagesPullingTime="19.52467992s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 13:43:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:44:18.544391719 +0000 UTC m=+23.722853905" watchObservedRunningTime="2026-07-02 13:44:18.545158267 +0000 UTC m=+23.723620451"
Jul 02 13:44:18.590172 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:18.590119    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-m4ndl" podStartSLOduration=8.736966787 podStartE2EDuration="23.590104323s" totalImagesPullingTime="14.853137536s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 13:43:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:44:18.560941035 +0000 UTC m=+23.739403219" watchObservedRunningTime="2026-07-02 13:44:18.590104323 +0000 UTC m=+23.768566558"
Jul 02 13:44:18.711011 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:18.710985    2558 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 02 13:44:19.211745 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:19.211713    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2tjcl"
Jul 02 13:44:19.211976 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:19.211714    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29552"
Jul 02 13:44:19.211976 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:44:19.211860    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2tjcl" podUID="6ebfc25f-6722-4f73-98b8-18b416460437"
Jul 02 13:44:19.211976 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:19.211785    2558 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-02T13:44:18.71100626Z","UUID":"102ec3a4-9995-4fb0-9207-81bcf35fe54f","Handler":null,"Name":"","Endpoint":""}
Jul 02 13:44:19.211976 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:19.211893    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q9w6j"
Jul 02 13:44:19.212184 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:44:19.211978    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-29552" podUID="ad2afabf-42cb-4619-b249-a2b011e32565"
Jul 02 13:44:19.212184 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:44:19.212034    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q9w6j" podUID="06e4af50-04c6-4c58-af10-0e0a28c1a261"
Jul 02 13:44:19.213708 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:19.213690    2558 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 02 13:44:19.213859 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:19.213730    2558 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 02 13:44:19.487509 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:19.487432    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-lxhdr" event={"ID":"7e1c7379-372d-48f3-a865-d0510d6c27a9","Type":"ContainerStarted","Data":"cd7f5d5687c7955be2f54a75ea0fca3cbc5b546a5526db15178869230444ac64"}
Jul 02 13:44:19.489545 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:19.489465    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tvfdz" event={"ID":"355d744e-81d0-41f9-864f-34b8d7b981af","Type":"ContainerStarted","Data":"e5e4bb614ae681a633c1c00c3d8068ec9b7712498ee55df35092058dbe8013cd"}
Jul 02 13:44:19.516561 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:19.516501    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-lxhdr" podStartSLOduration=5.3895171 podStartE2EDuration="24.516480948s" totalImagesPullingTime="19.126963848s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 13:43:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:44:19.51630712 +0000 UTC m=+24.694769305" watchObservedRunningTime="2026-07-02 13:44:19.516480948 +0000 UTC m=+24.694943327"
Jul 02 13:44:20.382621 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:20.382583    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-m4ndl"
Jul 02 13:44:20.383334 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:20.383308    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-m4ndl"
Jul 02 13:44:20.493769 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:20.493726    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tvfdz" event={"ID":"355d744e-81d0-41f9-864f-34b8d7b981af","Type":"ContainerStarted","Data":"45f8811c4467076620a3ee935d1f8a973fe7bcd5074d2bad92beaaaadecf304e"}
Jul 02 13:44:20.494352 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:20.494327    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-m4ndl"
Jul 02 13:44:20.494847 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:20.494804    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-m4ndl"
Jul 02 13:44:20.516696 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:20.516641    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tvfdz" podStartSLOduration=4.394033831 podStartE2EDuration="25.516621191s" totalImagesPullingTime="21.12258736s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 13:43:55 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:44:20.516209078 +0000 UTC m=+25.694671263" watchObservedRunningTime="2026-07-02 13:44:20.516621191 +0000 UTC m=+25.695083376"
Jul 02 13:44:21.212314 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:21.212284    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2tjcl"
Jul 02 13:44:21.212508 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:21.212415    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q9w6j"
Jul 02 13:44:21.212508 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:21.212447    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29552"
Jul 02 13:44:21.212618 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:44:21.212522    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q9w6j" podUID="06e4af50-04c6-4c58-af10-0e0a28c1a261"
Jul 02 13:44:21.212618 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:44:21.212416    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2tjcl" podUID="6ebfc25f-6722-4f73-98b8-18b416460437"
Jul 02 13:44:21.212618 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:44:21.212611    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-29552" podUID="ad2afabf-42cb-4619-b249-a2b011e32565"
Jul 02 13:44:21.500253 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:21.500113    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8f2ll_3310d1f7-1643-4017-9edf-71fa702d2615/ovn-acl-logging/0.log"
Jul 02 13:44:21.501356 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:21.501326    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8f2ll" event={"ID":"3310d1f7-1643-4017-9edf-71fa702d2615","Type":"ContainerStarted","Data":"26502b4fe29851642b92ab3b9cb1c9e669f35a77a05b0ab2821107c5a237dec3"}
Jul 02 13:44:23.211758 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:23.211521    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q9w6j"
Jul 02 13:44:23.212519 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:23.211531    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29552"
Jul 02 13:44:23.212519 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:44:23.211779    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q9w6j" podUID="06e4af50-04c6-4c58-af10-0e0a28c1a261"
Jul 02 13:44:23.212519 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:23.211542    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2tjcl"
Jul 02 13:44:23.212519 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:44:23.211972    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-29552" podUID="ad2afabf-42cb-4619-b249-a2b011e32565"
Jul 02 13:44:23.212519 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:44:23.211988    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2tjcl" podUID="6ebfc25f-6722-4f73-98b8-18b416460437"
Jul 02 13:44:23.508090 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:23.507843    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8f2ll_3310d1f7-1643-4017-9edf-71fa702d2615/ovn-acl-logging/0.log"
Jul 02 13:44:23.508362 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:23.508338    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8f2ll" event={"ID":"3310d1f7-1643-4017-9edf-71fa702d2615","Type":"ContainerStarted","Data":"74938734d7c605d6cc17b3efcd3e8d5e08e559b5ea560b30914288e5441bab83"}
Jul 02 13:44:23.508617 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:23.508592    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-8f2ll"
Jul 02 13:44:23.508816 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:23.508801    2558 scope.go:117] "RemoveContainer" containerID="bc7074cdb1f3898a71c09f258b5a6aec671412caae00742fd8e52ea179710fb5"
Jul 02 13:44:23.510303 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:23.510273    2558 generic.go:356] "Generic (PLEG): container finished" podID="e1521a8d-402a-4bee-bc35-bfa382300084" containerID="f384c6b29388231f52849a7a15723b48bf4a1153cbfceaec5f2889209c14994b" exitCode=0
Jul 02 13:44:23.510480 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:23.510306    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-s85vc" event={"ID":"e1521a8d-402a-4bee-bc35-bfa382300084","Type":"ContainerDied","Data":"f384c6b29388231f52849a7a15723b48bf4a1153cbfceaec5f2889209c14994b"}
Jul 02 13:44:23.529435 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:23.529402    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-8f2ll"
Jul 02 13:44:23.650536 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:23.650498    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-8f2ll"
Jul 02 13:44:24.515082 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:24.515006    2558 generic.go:356] "Generic (PLEG): container finished" podID="e1521a8d-402a-4bee-bc35-bfa382300084" containerID="389bb862d3a782256f09350d3be261cb4a068f62d7a37075008d464968d242dd" exitCode=0
Jul 02 13:44:24.515458 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:24.515086    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-s85vc" event={"ID":"e1521a8d-402a-4bee-bc35-bfa382300084","Type":"ContainerDied","Data":"389bb862d3a782256f09350d3be261cb4a068f62d7a37075008d464968d242dd"}
Jul 02 13:44:24.518931 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:24.518915    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8f2ll_3310d1f7-1643-4017-9edf-71fa702d2615/ovn-acl-logging/0.log"
Jul 02 13:44:24.519280 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:24.519251    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8f2ll" event={"ID":"3310d1f7-1643-4017-9edf-71fa702d2615","Type":"ContainerStarted","Data":"8a9f889c902647c5f52f02d7a8207d686a70d5ceae152a455e78dc5204f138ba"}
Jul 02 13:44:24.519482 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:24.519466    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-8f2ll"
Jul 02 13:44:24.536210 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:24.536186    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-8f2ll"
Jul 02 13:44:24.576725 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:24.576683    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-8f2ll" podStartSLOduration=10.167833057 podStartE2EDuration="29.576667739s" totalImagesPullingTime="19.408834682s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 13:43:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:44:24.576524617 +0000 UTC m=+29.754986802" watchObservedRunningTime="2026-07-02 13:44:24.576667739 +0000 UTC m=+29.755129922"
Jul 02 13:44:24.863641 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:24.863562    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-2tjcl"]
Jul 02 13:44:24.863776 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:24.863666    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2tjcl"
Jul 02 13:44:24.863844 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:44:24.863801    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2tjcl" podUID="6ebfc25f-6722-4f73-98b8-18b416460437"
Jul 02 13:44:24.867262 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:24.867023    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-q9w6j"]
Jul 02 13:44:24.867262 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:24.867132    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q9w6j"
Jul 02 13:44:24.867262 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:44:24.867219    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q9w6j" podUID="06e4af50-04c6-4c58-af10-0e0a28c1a261"
Jul 02 13:44:24.867884 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:24.867863    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-29552"]
Jul 02 13:44:24.867992 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:24.867944    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29552"
Jul 02 13:44:24.868040 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:44:24.868023    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-29552" podUID="ad2afabf-42cb-4619-b249-a2b011e32565"
Jul 02 13:44:25.524541 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:25.524288    2558 generic.go:356] "Generic (PLEG): container finished" podID="e1521a8d-402a-4bee-bc35-bfa382300084" containerID="0a911d7070a72934a1c3c31aed257110758a8b25667beaaf9840f6fae74b381f" exitCode=0
Jul 02 13:44:25.524541 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:25.524375    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-s85vc" event={"ID":"e1521a8d-402a-4bee-bc35-bfa382300084","Type":"ContainerDied","Data":"0a911d7070a72934a1c3c31aed257110758a8b25667beaaf9840f6fae74b381f"}
Jul 02 13:44:26.212016 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:26.211981    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q9w6j"
Jul 02 13:44:26.212178 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:26.212019    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29552"
Jul 02 13:44:26.212178 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:26.212019    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2tjcl"
Jul 02 13:44:26.212178 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:44:26.212121    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q9w6j" podUID="06e4af50-04c6-4c58-af10-0e0a28c1a261"
Jul 02 13:44:26.212338 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:44:26.212201    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-29552" podUID="ad2afabf-42cb-4619-b249-a2b011e32565"
Jul 02 13:44:26.212338 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:44:26.212301    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2tjcl" podUID="6ebfc25f-6722-4f73-98b8-18b416460437"
Jul 02 13:44:28.211992 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:28.211962    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q9w6j"
Jul 02 13:44:28.211992 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:28.211983    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29552"
Jul 02 13:44:28.211992 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:28.211995    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2tjcl"
Jul 02 13:44:28.212743 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:44:28.212090    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q9w6j" podUID="06e4af50-04c6-4c58-af10-0e0a28c1a261"
Jul 02 13:44:28.212743 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:44:28.212180    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-29552" podUID="ad2afabf-42cb-4619-b249-a2b011e32565"
Jul 02 13:44:28.212743 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:44:28.212280    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2tjcl" podUID="6ebfc25f-6722-4f73-98b8-18b416460437"
Jul 02 13:44:28.919456 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:28.919413    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cq8jf\" (UniqueName: \"kubernetes.io/projected/06e4af50-04c6-4c58-af10-0e0a28c1a261-kube-api-access-cq8jf\") pod \"network-check-target-q9w6j\" (UID: \"06e4af50-04c6-4c58-af10-0e0a28c1a261\") " pod="openshift-network-diagnostics/network-check-target-q9w6j"
Jul 02 13:44:28.919709 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:44:28.919566    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 02 13:44:28.919709 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:44:28.919591    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 02 13:44:28.919709 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:44:28.919604    2558 projected.go:196] Error preparing data for projected volume kube-api-access-cq8jf for pod openshift-network-diagnostics/network-check-target-q9w6j: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 02 13:44:28.919709 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:44:28.919675    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/06e4af50-04c6-4c58-af10-0e0a28c1a261-kube-api-access-cq8jf podName:06e4af50-04c6-4c58-af10-0e0a28c1a261 nodeName:}" failed. No retries permitted until 2026-07-02 13:45:00.919654203 +0000 UTC m=+66.098116379 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-cq8jf" (UniqueName: "kubernetes.io/projected/06e4af50-04c6-4c58-af10-0e0a28c1a261-kube-api-access-cq8jf") pod "network-check-target-q9w6j" (UID: "06e4af50-04c6-4c58-af10-0e0a28c1a261") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 02 13:44:29.020103 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:29.020065    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6ebfc25f-6722-4f73-98b8-18b416460437-original-pull-secret\") pod \"global-pull-secret-syncer-2tjcl\" (UID: \"6ebfc25f-6722-4f73-98b8-18b416460437\") " pod="kube-system/global-pull-secret-syncer-2tjcl"
Jul 02 13:44:29.020103 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:29.020115    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ad2afabf-42cb-4619-b249-a2b011e32565-metrics-certs\") pod \"network-metrics-daemon-29552\" (UID: \"ad2afabf-42cb-4619-b249-a2b011e32565\") " pod="openshift-multus/network-metrics-daemon-29552"
Jul 02 13:44:29.020375 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:44:29.020242    2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 02 13:44:29.020375 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:44:29.020257    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 02 13:44:29.020375 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:44:29.020311    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ebfc25f-6722-4f73-98b8-18b416460437-original-pull-secret podName:6ebfc25f-6722-4f73-98b8-18b416460437 nodeName:}" failed. No retries permitted until 2026-07-02 13:45:01.020294544 +0000 UTC m=+66.198756706 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6ebfc25f-6722-4f73-98b8-18b416460437-original-pull-secret") pod "global-pull-secret-syncer-2tjcl" (UID: "6ebfc25f-6722-4f73-98b8-18b416460437") : object "kube-system"/"original-pull-secret" not registered
Jul 02 13:44:29.020375 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:44:29.020328    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad2afabf-42cb-4619-b249-a2b011e32565-metrics-certs podName:ad2afabf-42cb-4619-b249-a2b011e32565 nodeName:}" failed. No retries permitted until 2026-07-02 13:45:01.020320204 +0000 UTC m=+66.198782366 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ad2afabf-42cb-4619-b249-a2b011e32565-metrics-certs") pod "network-metrics-daemon-29552" (UID: "ad2afabf-42cb-4619-b249-a2b011e32565") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 02 13:44:30.212287 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:30.212223    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q9w6j"
Jul 02 13:44:30.212721 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:44:30.212316    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q9w6j" podUID="06e4af50-04c6-4c58-af10-0e0a28c1a261"
Jul 02 13:44:30.212721 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:30.212230    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29552"
Jul 02 13:44:30.212721 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:30.212361    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2tjcl"
Jul 02 13:44:30.212721 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:44:30.212457    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-29552" podUID="ad2afabf-42cb-4619-b249-a2b011e32565"
Jul 02 13:44:30.212721 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:44:30.212552    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2tjcl" podUID="6ebfc25f-6722-4f73-98b8-18b416460437"
Jul 02 13:44:30.539999 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:30.539913    2558 generic.go:356] "Generic (PLEG): container finished" podID="e1521a8d-402a-4bee-bc35-bfa382300084" containerID="0c4a5cf3e81c02674636bfced6db2f97b7032fe6ee4af6728bd39380c892f830" exitCode=0
Jul 02 13:44:30.539999 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:30.539958    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-s85vc" event={"ID":"e1521a8d-402a-4bee-bc35-bfa382300084","Type":"ContainerDied","Data":"0c4a5cf3e81c02674636bfced6db2f97b7032fe6ee4af6728bd39380c892f830"}
Jul 02 13:44:31.545179 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:31.545097    2558 generic.go:356] "Generic (PLEG): container finished" podID="e1521a8d-402a-4bee-bc35-bfa382300084" containerID="6dcf296a7b5f7f62e442ce4df20135896c5403fc3c1bdf7104e52cbbc3b19e1e" exitCode=0
Jul 02 13:44:31.545179 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:31.545132    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-s85vc" event={"ID":"e1521a8d-402a-4bee-bc35-bfa382300084","Type":"ContainerDied","Data":"6dcf296a7b5f7f62e442ce4df20135896c5403fc3c1bdf7104e52cbbc3b19e1e"}
Jul 02 13:44:31.665190 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:31.665162    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-209.ec2.internal" event="NodeReady"
Jul 02 13:44:31.665343 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:31.665328    2558 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 02 13:44:31.730554 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:31.728988    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-zcqdj"]
Jul 02 13:44:31.752534 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:31.752506    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-gcr27"]
Jul 02 13:44:31.752681 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:31.752635    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-zcqdj"
Jul 02 13:44:31.756470 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:31.756450    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-rwl8t\""
Jul 02 13:44:31.756588 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:31.756477    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 02 13:44:31.756682 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:31.756664    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 02 13:44:31.774774 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:31.774746    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-zcqdj"]
Jul 02 13:44:31.774921 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:31.774786    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-gcr27"]
Jul 02 13:44:31.774921 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:31.774892    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-gcr27"
Jul 02 13:44:31.778086 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:31.778064    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-srjlp\""
Jul 02 13:44:31.778086 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:31.778080    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 02 13:44:31.778271 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:31.778102    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 02 13:44:31.778365 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:31.778352    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 02 13:44:31.843461 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:31.843430    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xccvs\" (UniqueName: \"kubernetes.io/projected/2bf21a2c-20d4-47ce-922b-b48612f3c438-kube-api-access-xccvs\") pod \"dns-default-zcqdj\" (UID: \"2bf21a2c-20d4-47ce-922b-b48612f3c438\") " pod="openshift-dns/dns-default-zcqdj"
Jul 02 13:44:31.843461 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:31.843465    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2bf21a2c-20d4-47ce-922b-b48612f3c438-metrics-tls\") pod \"dns-default-zcqdj\" (UID: \"2bf21a2c-20d4-47ce-922b-b48612f3c438\") " pod="openshift-dns/dns-default-zcqdj"
Jul 02 13:44:31.843665 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:31.843494    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2bf21a2c-20d4-47ce-922b-b48612f3c438-config-volume\") pod \"dns-default-zcqdj\" (UID: \"2bf21a2c-20d4-47ce-922b-b48612f3c438\") " pod="openshift-dns/dns-default-zcqdj"
Jul 02 13:44:31.843665 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:31.843544    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2bf21a2c-20d4-47ce-922b-b48612f3c438-tmp-dir\") pod \"dns-default-zcqdj\" (UID: \"2bf21a2c-20d4-47ce-922b-b48612f3c438\") " pod="openshift-dns/dns-default-zcqdj"
Jul 02 13:44:31.944889 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:31.944850    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xccvs\" (UniqueName: \"kubernetes.io/projected/2bf21a2c-20d4-47ce-922b-b48612f3c438-kube-api-access-xccvs\") pod \"dns-default-zcqdj\" (UID: \"2bf21a2c-20d4-47ce-922b-b48612f3c438\") " pod="openshift-dns/dns-default-zcqdj"
Jul 02 13:44:31.944889 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:31.944889    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7b4fd04b-30e6-4599-b877-2719d6d08231-cert\") pod \"ingress-canary-gcr27\" (UID: \"7b4fd04b-30e6-4599-b877-2719d6d08231\") " pod="openshift-ingress-canary/ingress-canary-gcr27"
Jul 02 13:44:31.945075 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:31.944908    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2bf21a2c-20d4-47ce-922b-b48612f3c438-metrics-tls\") pod \"dns-default-zcqdj\" (UID: \"2bf21a2c-20d4-47ce-922b-b48612f3c438\") " pod="openshift-dns/dns-default-zcqdj"
Jul 02 13:44:31.945075 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:31.944958    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2bf21a2c-20d4-47ce-922b-b48612f3c438-config-volume\") pod \"dns-default-zcqdj\" (UID: \"2bf21a2c-20d4-47ce-922b-b48612f3c438\") " pod="openshift-dns/dns-default-zcqdj"
Jul 02 13:44:31.945143 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:31.945094    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dmdb4\" (UniqueName: \"kubernetes.io/projected/7b4fd04b-30e6-4599-b877-2719d6d08231-kube-api-access-dmdb4\") pod \"ingress-canary-gcr27\" (UID: \"7b4fd04b-30e6-4599-b877-2719d6d08231\") " pod="openshift-ingress-canary/ingress-canary-gcr27"
Jul 02 13:44:31.945143 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:44:31.945133    2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 02 13:44:31.945214 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:31.945158    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2bf21a2c-20d4-47ce-922b-b48612f3c438-tmp-dir\") pod \"dns-default-zcqdj\" (UID: \"2bf21a2c-20d4-47ce-922b-b48612f3c438\") " pod="openshift-dns/dns-default-zcqdj"
Jul 02 13:44:31.945214 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:44:31.945191    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2bf21a2c-20d4-47ce-922b-b48612f3c438-metrics-tls podName:2bf21a2c-20d4-47ce-922b-b48612f3c438 nodeName:}" failed. No retries permitted until 2026-07-02 13:44:32.445173544 +0000 UTC m=+37.623635781 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2bf21a2c-20d4-47ce-922b-b48612f3c438-metrics-tls") pod "dns-default-zcqdj" (UID: "2bf21a2c-20d4-47ce-922b-b48612f3c438") : secret "dns-default-metrics-tls" not found
Jul 02 13:44:31.945489 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:31.945474    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2bf21a2c-20d4-47ce-922b-b48612f3c438-config-volume\") pod \"dns-default-zcqdj\" (UID: \"2bf21a2c-20d4-47ce-922b-b48612f3c438\") " pod="openshift-dns/dns-default-zcqdj"
Jul 02 13:44:31.945528 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:31.945485    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2bf21a2c-20d4-47ce-922b-b48612f3c438-tmp-dir\") pod \"dns-default-zcqdj\" (UID: \"2bf21a2c-20d4-47ce-922b-b48612f3c438\") " pod="openshift-dns/dns-default-zcqdj"
Jul 02 13:44:31.957497 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:31.957375    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xccvs\" (UniqueName: \"kubernetes.io/projected/2bf21a2c-20d4-47ce-922b-b48612f3c438-kube-api-access-xccvs\") pod \"dns-default-zcqdj\" (UID: \"2bf21a2c-20d4-47ce-922b-b48612f3c438\") " pod="openshift-dns/dns-default-zcqdj"
Jul 02 13:44:32.045552 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:32.045512    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7b4fd04b-30e6-4599-b877-2719d6d08231-cert\") pod \"ingress-canary-gcr27\" (UID: \"7b4fd04b-30e6-4599-b877-2719d6d08231\") " pod="openshift-ingress-canary/ingress-canary-gcr27"
Jul 02 13:44:32.045698 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:32.045565    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dmdb4\" (UniqueName: \"kubernetes.io/projected/7b4fd04b-30e6-4599-b877-2719d6d08231-kube-api-access-dmdb4\") pod \"ingress-canary-gcr27\" (UID: \"7b4fd04b-30e6-4599-b877-2719d6d08231\") " pod="openshift-ingress-canary/ingress-canary-gcr27"
Jul 02 13:44:32.048340 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:32.048319    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7b4fd04b-30e6-4599-b877-2719d6d08231-cert\") pod \"ingress-canary-gcr27\" (UID: \"7b4fd04b-30e6-4599-b877-2719d6d08231\") " pod="openshift-ingress-canary/ingress-canary-gcr27"
Jul 02 13:44:32.055616 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:32.055593    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dmdb4\" (UniqueName: \"kubernetes.io/projected/7b4fd04b-30e6-4599-b877-2719d6d08231-kube-api-access-dmdb4\") pod \"ingress-canary-gcr27\" (UID: \"7b4fd04b-30e6-4599-b877-2719d6d08231\") " pod="openshift-ingress-canary/ingress-canary-gcr27"
Jul 02 13:44:32.082616 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:32.082581    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-gcr27"
Jul 02 13:44:32.212103 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:32.212068    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29552"
Jul 02 13:44:32.212892 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:32.212547    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q9w6j"
Jul 02 13:44:32.212892 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:32.212551    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2tjcl"
Jul 02 13:44:32.215972 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:32.215648    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 02 13:44:32.215972 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:32.215732    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 02 13:44:32.215972 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:32.215849    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 02 13:44:32.216204 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:32.216032    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-bsszs\""
Jul 02 13:44:32.216490 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:32.216472    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 02 13:44:32.216574 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:32.216512    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-f4vwh\""
Jul 02 13:44:32.290267 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:32.290241    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-gcr27"]
Jul 02 13:44:32.294146 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:44:32.294116    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b4fd04b_30e6_4599_b877_2719d6d08231.slice/crio-485f621b3ac8972669285cc2fdbbdd2ce8bf44b26b94d2d6149496a2c6f5ac76 WatchSource:0}: Error finding container 485f621b3ac8972669285cc2fdbbdd2ce8bf44b26b94d2d6149496a2c6f5ac76: Status 404 returned error can't find the container with id 485f621b3ac8972669285cc2fdbbdd2ce8bf44b26b94d2d6149496a2c6f5ac76
Jul 02 13:44:32.449128 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:32.449044    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2bf21a2c-20d4-47ce-922b-b48612f3c438-metrics-tls\") pod \"dns-default-zcqdj\" (UID: \"2bf21a2c-20d4-47ce-922b-b48612f3c438\") " pod="openshift-dns/dns-default-zcqdj"
Jul 02 13:44:32.449270 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:44:32.449170    2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 02 13:44:32.449270 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:44:32.449228    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2bf21a2c-20d4-47ce-922b-b48612f3c438-metrics-tls podName:2bf21a2c-20d4-47ce-922b-b48612f3c438 nodeName:}" failed. No retries permitted until 2026-07-02 13:44:33.449211734 +0000 UTC m=+38.627673915 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2bf21a2c-20d4-47ce-922b-b48612f3c438-metrics-tls") pod "dns-default-zcqdj" (UID: "2bf21a2c-20d4-47ce-922b-b48612f3c438") : secret "dns-default-metrics-tls" not found
Jul 02 13:44:32.548699 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:32.548656    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-gcr27" event={"ID":"7b4fd04b-30e6-4599-b877-2719d6d08231","Type":"ContainerStarted","Data":"485f621b3ac8972669285cc2fdbbdd2ce8bf44b26b94d2d6149496a2c6f5ac76"}
Jul 02 13:44:32.552055 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:32.552032    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-s85vc" event={"ID":"e1521a8d-402a-4bee-bc35-bfa382300084","Type":"ContainerStarted","Data":"09318b53cc17f79e17daf6628c0817a7de6c3e0f5bf762ab1e82634b804491d8"}
Jul 02 13:44:32.578120 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:32.578070    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-s85vc" podStartSLOduration=8.789632559 podStartE2EDuration="37.578054947s" totalImagesPullingTime="28.788422388s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 13:43:55 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:44:32.577425466 +0000 UTC m=+37.755887651" watchObservedRunningTime="2026-07-02 13:44:32.578054947 +0000 UTC m=+37.756517130"
Jul 02 13:44:33.455655 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:33.455620    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2bf21a2c-20d4-47ce-922b-b48612f3c438-metrics-tls\") pod \"dns-default-zcqdj\" (UID: \"2bf21a2c-20d4-47ce-922b-b48612f3c438\") " pod="openshift-dns/dns-default-zcqdj"
Jul 02 13:44:33.455873 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:44:33.455811    2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 02 13:44:33.455947 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:44:33.455930    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2bf21a2c-20d4-47ce-922b-b48612f3c438-metrics-tls podName:2bf21a2c-20d4-47ce-922b-b48612f3c438 nodeName:}" failed. No retries permitted until 2026-07-02 13:44:35.455907675 +0000 UTC m=+40.634369846 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2bf21a2c-20d4-47ce-922b-b48612f3c438-metrics-tls") pod "dns-default-zcqdj" (UID: "2bf21a2c-20d4-47ce-922b-b48612f3c438") : secret "dns-default-metrics-tls" not found
Jul 02 13:44:35.476417 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:35.476232    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2bf21a2c-20d4-47ce-922b-b48612f3c438-metrics-tls\") pod \"dns-default-zcqdj\" (UID: \"2bf21a2c-20d4-47ce-922b-b48612f3c438\") " pod="openshift-dns/dns-default-zcqdj"
Jul 02 13:44:35.476941 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:44:35.476379    2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 02 13:44:35.476941 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:44:35.476482    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2bf21a2c-20d4-47ce-922b-b48612f3c438-metrics-tls podName:2bf21a2c-20d4-47ce-922b-b48612f3c438 nodeName:}" failed. No retries permitted until 2026-07-02 13:44:39.476466329 +0000 UTC m=+44.654928491 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2bf21a2c-20d4-47ce-922b-b48612f3c438-metrics-tls") pod "dns-default-zcqdj" (UID: "2bf21a2c-20d4-47ce-922b-b48612f3c438") : secret "dns-default-metrics-tls" not found
Jul 02 13:44:35.560695 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:35.560665    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-gcr27" event={"ID":"7b4fd04b-30e6-4599-b877-2719d6d08231","Type":"ContainerStarted","Data":"e0c8f288b5acd07b6d71126b1209530134fdf40783379a42145e385b7c795bc5"}
Jul 02 13:44:35.580049 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:35.580006    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-gcr27" podStartSLOduration=2.2730489240000002 podStartE2EDuration="4.579991845s" totalImagesPullingTime="2.306942921s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 13:44:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:44:35.578360142 +0000 UTC m=+40.756822326" watchObservedRunningTime="2026-07-02 13:44:35.579991845 +0000 UTC m=+40.758454032"
Jul 02 13:44:39.502947 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:39.502906    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2bf21a2c-20d4-47ce-922b-b48612f3c438-metrics-tls\") pod \"dns-default-zcqdj\" (UID: \"2bf21a2c-20d4-47ce-922b-b48612f3c438\") " pod="openshift-dns/dns-default-zcqdj"
Jul 02 13:44:39.503579 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:44:39.503073    2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 02 13:44:39.503579 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:44:39.503162    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2bf21a2c-20d4-47ce-922b-b48612f3c438-metrics-tls podName:2bf21a2c-20d4-47ce-922b-b48612f3c438 nodeName:}" failed. No retries permitted until 2026-07-02 13:44:47.503142109 +0000 UTC m=+52.681604287 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2bf21a2c-20d4-47ce-922b-b48612f3c438-metrics-tls") pod "dns-default-zcqdj" (UID: "2bf21a2c-20d4-47ce-922b-b48612f3c438") : secret "dns-default-metrics-tls" not found
Jul 02 13:44:47.557279 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:47.557238    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2bf21a2c-20d4-47ce-922b-b48612f3c438-metrics-tls\") pod \"dns-default-zcqdj\" (UID: \"2bf21a2c-20d4-47ce-922b-b48612f3c438\") " pod="openshift-dns/dns-default-zcqdj"
Jul 02 13:44:47.557805 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:44:47.557383    2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 02 13:44:47.557805 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:44:47.557441    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2bf21a2c-20d4-47ce-922b-b48612f3c438-metrics-tls podName:2bf21a2c-20d4-47ce-922b-b48612f3c438 nodeName:}" failed. No retries permitted until 2026-07-02 13:45:03.557424366 +0000 UTC m=+68.735886528 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2bf21a2c-20d4-47ce-922b-b48612f3c438-metrics-tls") pod "dns-default-zcqdj" (UID: "2bf21a2c-20d4-47ce-922b-b48612f3c438") : secret "dns-default-metrics-tls" not found
Jul 02 13:44:56.539237 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:44:56.539201    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-8f2ll"
Jul 02 13:45:00.950917 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:00.950878    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cq8jf\" (UniqueName: \"kubernetes.io/projected/06e4af50-04c6-4c58-af10-0e0a28c1a261-kube-api-access-cq8jf\") pod \"network-check-target-q9w6j\" (UID: \"06e4af50-04c6-4c58-af10-0e0a28c1a261\") " pod="openshift-network-diagnostics/network-check-target-q9w6j"
Jul 02 13:45:00.954223 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:00.954203    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 02 13:45:00.964443 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:00.964420    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 02 13:45:00.975298 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:00.975274    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cq8jf\" (UniqueName: \"kubernetes.io/projected/06e4af50-04c6-4c58-af10-0e0a28c1a261-kube-api-access-cq8jf\") pod \"network-check-target-q9w6j\" (UID: \"06e4af50-04c6-4c58-af10-0e0a28c1a261\") " pod="openshift-network-diagnostics/network-check-target-q9w6j"
Jul 02 13:45:01.035266 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:01.035238    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-bsszs\""
Jul 02 13:45:01.042681 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:01.042659    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q9w6j"
Jul 02 13:45:01.052129 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:01.052102    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6ebfc25f-6722-4f73-98b8-18b416460437-original-pull-secret\") pod \"global-pull-secret-syncer-2tjcl\" (UID: \"6ebfc25f-6722-4f73-98b8-18b416460437\") " pod="kube-system/global-pull-secret-syncer-2tjcl"
Jul 02 13:45:01.052235 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:01.052134    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ad2afabf-42cb-4619-b249-a2b011e32565-metrics-certs\") pod \"network-metrics-daemon-29552\" (UID: \"ad2afabf-42cb-4619-b249-a2b011e32565\") " pod="openshift-multus/network-metrics-daemon-29552"
Jul 02 13:45:01.055274 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:01.055256    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 02 13:45:01.055333 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:01.055273    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 02 13:45:01.063288 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:45:01.063271    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 02 13:45:01.063352 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:45:01.063345    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad2afabf-42cb-4619-b249-a2b011e32565-metrics-certs podName:ad2afabf-42cb-4619-b249-a2b011e32565 nodeName:}" failed. No retries permitted until 2026-07-02 13:46:05.063327726 +0000 UTC m=+130.241789888 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ad2afabf-42cb-4619-b249-a2b011e32565-metrics-certs") pod "network-metrics-daemon-29552" (UID: "ad2afabf-42cb-4619-b249-a2b011e32565") : secret "metrics-daemon-secret" not found
Jul 02 13:45:01.065579 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:01.065552    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6ebfc25f-6722-4f73-98b8-18b416460437-original-pull-secret\") pod \"global-pull-secret-syncer-2tjcl\" (UID: \"6ebfc25f-6722-4f73-98b8-18b416460437\") " pod="kube-system/global-pull-secret-syncer-2tjcl"
Jul 02 13:45:01.199692 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:01.199661    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-q9w6j"]
Jul 02 13:45:01.203034 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:45:01.203007    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06e4af50_04c6_4c58_af10_0e0a28c1a261.slice/crio-596e2bf2ab3f2948a525369474fa5564cf99a87a7d2a6bd09c26abd1e90b405c WatchSource:0}: Error finding container 596e2bf2ab3f2948a525369474fa5564cf99a87a7d2a6bd09c26abd1e90b405c: Status 404 returned error can't find the container with id 596e2bf2ab3f2948a525369474fa5564cf99a87a7d2a6bd09c26abd1e90b405c
Jul 02 13:45:01.337043 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:01.337006    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2tjcl"
Jul 02 13:45:01.479112 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:01.479035    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-2tjcl"]
Jul 02 13:45:01.482663 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:45:01.482636    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ebfc25f_6722_4f73_98b8_18b416460437.slice/crio-8365ba5987f506f4da4c19dd92b589c6fba4c9ccd2d98d35004631c852d2536f WatchSource:0}: Error finding container 8365ba5987f506f4da4c19dd92b589c6fba4c9ccd2d98d35004631c852d2536f: Status 404 returned error can't find the container with id 8365ba5987f506f4da4c19dd92b589c6fba4c9ccd2d98d35004631c852d2536f
Jul 02 13:45:01.622852 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:01.622792    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-q9w6j" event={"ID":"06e4af50-04c6-4c58-af10-0e0a28c1a261","Type":"ContainerStarted","Data":"596e2bf2ab3f2948a525369474fa5564cf99a87a7d2a6bd09c26abd1e90b405c"}
Jul 02 13:45:01.623875 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:01.623852    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-2tjcl" event={"ID":"6ebfc25f-6722-4f73-98b8-18b416460437","Type":"ContainerStarted","Data":"8365ba5987f506f4da4c19dd92b589c6fba4c9ccd2d98d35004631c852d2536f"}
Jul 02 13:45:03.570134 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:03.570093    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2bf21a2c-20d4-47ce-922b-b48612f3c438-metrics-tls\") pod \"dns-default-zcqdj\" (UID: \"2bf21a2c-20d4-47ce-922b-b48612f3c438\") " pod="openshift-dns/dns-default-zcqdj"
Jul 02 13:45:03.570612 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:45:03.570253    2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 02 13:45:03.570612 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:45:03.570319    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2bf21a2c-20d4-47ce-922b-b48612f3c438-metrics-tls podName:2bf21a2c-20d4-47ce-922b-b48612f3c438 nodeName:}" failed. No retries permitted until 2026-07-02 13:45:35.570304463 +0000 UTC m=+100.748766647 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2bf21a2c-20d4-47ce-922b-b48612f3c438-metrics-tls") pod "dns-default-zcqdj" (UID: "2bf21a2c-20d4-47ce-922b-b48612f3c438") : secret "dns-default-metrics-tls" not found
Jul 02 13:45:04.383381 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:04.383347    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-69d9f679db-g87ql"]
Jul 02 13:45:04.385127 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:04.385108    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-69d9f679db-g87ql"
Jul 02 13:45:04.388720 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:04.388698    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Jul 02 13:45:04.389661 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:04.389637    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Jul 02 13:45:04.389771 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:04.389636    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-z8cn5\""
Jul 02 13:45:04.395339 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:04.395315    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-69d9f679db-g87ql"]
Jul 02 13:45:04.410710 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:04.410681    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-75dffdcbd7-bsn5q"]
Jul 02 13:45:04.412621 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:04.412593    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-75dffdcbd7-bsn5q"
Jul 02 13:45:04.420251 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:04.420226    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Jul 02 13:45:04.420379 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:04.420335    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Jul 02 13:45:04.420444 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:04.420327    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Jul 02 13:45:04.421518 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:04.421492    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Jul 02 13:45:04.421518 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:04.421506    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-r9scf\""
Jul 02 13:45:04.421681 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:04.421497    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 02 13:45:04.421681 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:04.421560    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 02 13:45:04.435203 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:04.435177    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-75dffdcbd7-bsn5q"]
Jul 02 13:45:04.476530 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:04.476491    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/a4cc908b-b2a6-409e-8377-9ed59a79ff27-default-certificate\") pod \"router-default-75dffdcbd7-bsn5q\" (UID: \"a4cc908b-b2a6-409e-8377-9ed59a79ff27\") " pod="openshift-ingress/router-default-75dffdcbd7-bsn5q"
Jul 02 13:45:04.476714 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:04.476538    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/a4cc908b-b2a6-409e-8377-9ed59a79ff27-stats-auth\") pod \"router-default-75dffdcbd7-bsn5q\" (UID: \"a4cc908b-b2a6-409e-8377-9ed59a79ff27\") " pod="openshift-ingress/router-default-75dffdcbd7-bsn5q"
Jul 02 13:45:04.476714 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:04.476631    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dzps7\" (UniqueName: \"kubernetes.io/projected/a4cc908b-b2a6-409e-8377-9ed59a79ff27-kube-api-access-dzps7\") pod \"router-default-75dffdcbd7-bsn5q\" (UID: \"a4cc908b-b2a6-409e-8377-9ed59a79ff27\") " pod="openshift-ingress/router-default-75dffdcbd7-bsn5q"
Jul 02 13:45:04.476714 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:04.476662    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a4cc908b-b2a6-409e-8377-9ed59a79ff27-metrics-certs\") pod \"router-default-75dffdcbd7-bsn5q\" (UID: \"a4cc908b-b2a6-409e-8377-9ed59a79ff27\") " pod="openshift-ingress/router-default-75dffdcbd7-bsn5q"
Jul 02 13:45:04.476714 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:04.476695    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c4q7f\" (UniqueName: \"kubernetes.io/projected/bf9d29ad-19aa-4844-8742-d5f10adcaad8-kube-api-access-c4q7f\") pod \"volume-data-source-validator-69d9f679db-g87ql\" (UID: \"bf9d29ad-19aa-4844-8742-d5f10adcaad8\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-69d9f679db-g87ql"
Jul 02 13:45:04.476931 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:04.476722    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a4cc908b-b2a6-409e-8377-9ed59a79ff27-service-ca-bundle\") pod \"router-default-75dffdcbd7-bsn5q\" (UID: \"a4cc908b-b2a6-409e-8377-9ed59a79ff27\") " pod="openshift-ingress/router-default-75dffdcbd7-bsn5q"
Jul 02 13:45:04.577731 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:04.577685    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/a4cc908b-b2a6-409e-8377-9ed59a79ff27-stats-auth\") pod \"router-default-75dffdcbd7-bsn5q\" (UID: \"a4cc908b-b2a6-409e-8377-9ed59a79ff27\") " pod="openshift-ingress/router-default-75dffdcbd7-bsn5q"
Jul 02 13:45:04.578217 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:04.577767    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dzps7\" (UniqueName: \"kubernetes.io/projected/a4cc908b-b2a6-409e-8377-9ed59a79ff27-kube-api-access-dzps7\") pod \"router-default-75dffdcbd7-bsn5q\" (UID: \"a4cc908b-b2a6-409e-8377-9ed59a79ff27\") " pod="openshift-ingress/router-default-75dffdcbd7-bsn5q"
Jul 02 13:45:04.578217 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:04.577937    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a4cc908b-b2a6-409e-8377-9ed59a79ff27-metrics-certs\") pod \"router-default-75dffdcbd7-bsn5q\" (UID: \"a4cc908b-b2a6-409e-8377-9ed59a79ff27\") " pod="openshift-ingress/router-default-75dffdcbd7-bsn5q"
Jul 02 13:45:04.578217 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:04.577983    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c4q7f\" (UniqueName: \"kubernetes.io/projected/bf9d29ad-19aa-4844-8742-d5f10adcaad8-kube-api-access-c4q7f\") pod \"volume-data-source-validator-69d9f679db-g87ql\" (UID: \"bf9d29ad-19aa-4844-8742-d5f10adcaad8\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-69d9f679db-g87ql"
Jul 02 13:45:04.578217 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:04.578019    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a4cc908b-b2a6-409e-8377-9ed59a79ff27-service-ca-bundle\") pod \"router-default-75dffdcbd7-bsn5q\" (UID: \"a4cc908b-b2a6-409e-8377-9ed59a79ff27\") " pod="openshift-ingress/router-default-75dffdcbd7-bsn5q"
Jul 02 13:45:04.578217 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:45:04.578079    2558 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 02 13:45:04.578217 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:04.578107    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/a4cc908b-b2a6-409e-8377-9ed59a79ff27-default-certificate\") pod \"router-default-75dffdcbd7-bsn5q\" (UID: \"a4cc908b-b2a6-409e-8377-9ed59a79ff27\") " pod="openshift-ingress/router-default-75dffdcbd7-bsn5q"
Jul 02 13:45:04.578217 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:45:04.578142    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a4cc908b-b2a6-409e-8377-9ed59a79ff27-metrics-certs podName:a4cc908b-b2a6-409e-8377-9ed59a79ff27 nodeName:}" failed. No retries permitted until 2026-07-02 13:45:05.078124387 +0000 UTC m=+70.256586550 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a4cc908b-b2a6-409e-8377-9ed59a79ff27-metrics-certs") pod "router-default-75dffdcbd7-bsn5q" (UID: "a4cc908b-b2a6-409e-8377-9ed59a79ff27") : secret "router-metrics-certs-default" not found
Jul 02 13:45:04.578510 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:45:04.578260    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a4cc908b-b2a6-409e-8377-9ed59a79ff27-service-ca-bundle podName:a4cc908b-b2a6-409e-8377-9ed59a79ff27 nodeName:}" failed. No retries permitted until 2026-07-02 13:45:05.078239514 +0000 UTC m=+70.256701695 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/a4cc908b-b2a6-409e-8377-9ed59a79ff27-service-ca-bundle") pod "router-default-75dffdcbd7-bsn5q" (UID: "a4cc908b-b2a6-409e-8377-9ed59a79ff27") : configmap references non-existent config key: service-ca.crt
Jul 02 13:45:04.580694 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:04.580673    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/a4cc908b-b2a6-409e-8377-9ed59a79ff27-stats-auth\") pod \"router-default-75dffdcbd7-bsn5q\" (UID: \"a4cc908b-b2a6-409e-8377-9ed59a79ff27\") " pod="openshift-ingress/router-default-75dffdcbd7-bsn5q"
Jul 02 13:45:04.581011 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:04.580983    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/a4cc908b-b2a6-409e-8377-9ed59a79ff27-default-certificate\") pod \"router-default-75dffdcbd7-bsn5q\" (UID: \"a4cc908b-b2a6-409e-8377-9ed59a79ff27\") " pod="openshift-ingress/router-default-75dffdcbd7-bsn5q"
Jul 02 13:45:04.594898 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:04.594860    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dzps7\" (UniqueName: \"kubernetes.io/projected/a4cc908b-b2a6-409e-8377-9ed59a79ff27-kube-api-access-dzps7\") pod \"router-default-75dffdcbd7-bsn5q\" (UID: \"a4cc908b-b2a6-409e-8377-9ed59a79ff27\") " pod="openshift-ingress/router-default-75dffdcbd7-bsn5q"
Jul 02 13:45:04.595071 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:04.595046    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c4q7f\" (UniqueName: \"kubernetes.io/projected/bf9d29ad-19aa-4844-8742-d5f10adcaad8-kube-api-access-c4q7f\") pod \"volume-data-source-validator-69d9f679db-g87ql\" (UID: \"bf9d29ad-19aa-4844-8742-d5f10adcaad8\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-69d9f679db-g87ql"
Jul 02 13:45:04.694668 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:04.694577    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-69d9f679db-g87ql"
Jul 02 13:45:05.083123 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:05.083037    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a4cc908b-b2a6-409e-8377-9ed59a79ff27-metrics-certs\") pod \"router-default-75dffdcbd7-bsn5q\" (UID: \"a4cc908b-b2a6-409e-8377-9ed59a79ff27\") " pod="openshift-ingress/router-default-75dffdcbd7-bsn5q"
Jul 02 13:45:05.083123 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:05.083084    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a4cc908b-b2a6-409e-8377-9ed59a79ff27-service-ca-bundle\") pod \"router-default-75dffdcbd7-bsn5q\" (UID: \"a4cc908b-b2a6-409e-8377-9ed59a79ff27\") " pod="openshift-ingress/router-default-75dffdcbd7-bsn5q"
Jul 02 13:45:05.083327 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:45:05.083215    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a4cc908b-b2a6-409e-8377-9ed59a79ff27-service-ca-bundle podName:a4cc908b-b2a6-409e-8377-9ed59a79ff27 nodeName:}" failed. No retries permitted until 2026-07-02 13:45:06.083196658 +0000 UTC m=+71.261658821 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/a4cc908b-b2a6-409e-8377-9ed59a79ff27-service-ca-bundle") pod "router-default-75dffdcbd7-bsn5q" (UID: "a4cc908b-b2a6-409e-8377-9ed59a79ff27") : configmap references non-existent config key: service-ca.crt
Jul 02 13:45:05.083327 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:45:05.083213    2558 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 02 13:45:05.083327 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:45:05.083287    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a4cc908b-b2a6-409e-8377-9ed59a79ff27-metrics-certs podName:a4cc908b-b2a6-409e-8377-9ed59a79ff27 nodeName:}" failed. No retries permitted until 2026-07-02 13:45:06.083269126 +0000 UTC m=+71.261731292 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a4cc908b-b2a6-409e-8377-9ed59a79ff27-metrics-certs") pod "router-default-75dffdcbd7-bsn5q" (UID: "a4cc908b-b2a6-409e-8377-9ed59a79ff27") : secret "router-metrics-certs-default" not found
Jul 02 13:45:06.091104 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:06.091064    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a4cc908b-b2a6-409e-8377-9ed59a79ff27-metrics-certs\") pod \"router-default-75dffdcbd7-bsn5q\" (UID: \"a4cc908b-b2a6-409e-8377-9ed59a79ff27\") " pod="openshift-ingress/router-default-75dffdcbd7-bsn5q"
Jul 02 13:45:06.091500 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:06.091122    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a4cc908b-b2a6-409e-8377-9ed59a79ff27-service-ca-bundle\") pod \"router-default-75dffdcbd7-bsn5q\" (UID: \"a4cc908b-b2a6-409e-8377-9ed59a79ff27\") " pod="openshift-ingress/router-default-75dffdcbd7-bsn5q"
Jul 02 13:45:06.091500 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:45:06.091217    2558 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 02 13:45:06.091500 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:45:06.091262    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a4cc908b-b2a6-409e-8377-9ed59a79ff27-service-ca-bundle podName:a4cc908b-b2a6-409e-8377-9ed59a79ff27 nodeName:}" failed. No retries permitted until 2026-07-02 13:45:08.091244367 +0000 UTC m=+73.269706551 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/a4cc908b-b2a6-409e-8377-9ed59a79ff27-service-ca-bundle") pod "router-default-75dffdcbd7-bsn5q" (UID: "a4cc908b-b2a6-409e-8377-9ed59a79ff27") : configmap references non-existent config key: service-ca.crt
Jul 02 13:45:06.091500 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:45:06.091279    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a4cc908b-b2a6-409e-8377-9ed59a79ff27-metrics-certs podName:a4cc908b-b2a6-409e-8377-9ed59a79ff27 nodeName:}" failed. No retries permitted until 2026-07-02 13:45:08.091271194 +0000 UTC m=+73.269733360 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a4cc908b-b2a6-409e-8377-9ed59a79ff27-metrics-certs") pod "router-default-75dffdcbd7-bsn5q" (UID: "a4cc908b-b2a6-409e-8377-9ed59a79ff27") : secret "router-metrics-certs-default" not found
Jul 02 13:45:06.412160 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:06.412133    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-69d9f679db-g87ql"]
Jul 02 13:45:06.415468 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:45:06.415444    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf9d29ad_19aa_4844_8742_d5f10adcaad8.slice/crio-456b8e1c8fcc2d245c7150dac27c1440cb4da22918bdac5b33eab341cd5507d8 WatchSource:0}: Error finding container 456b8e1c8fcc2d245c7150dac27c1440cb4da22918bdac5b33eab341cd5507d8: Status 404 returned error can't find the container with id 456b8e1c8fcc2d245c7150dac27c1440cb4da22918bdac5b33eab341cd5507d8
Jul 02 13:45:06.639764 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:06.639656    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-2tjcl" event={"ID":"6ebfc25f-6722-4f73-98b8-18b416460437","Type":"ContainerStarted","Data":"29473f4296424dd34bb6d34449d139f9aecdcabf7abcf6ded580f32e5ec72ef9"}
Jul 02 13:45:06.641188 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:06.641163    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-q9w6j" event={"ID":"06e4af50-04c6-4c58-af10-0e0a28c1a261","Type":"ContainerStarted","Data":"ea0c0a911a391efc32658a7b6035eaad2ed604f18b7b6f6e310be4c797204db3"}
Jul 02 13:45:06.641342 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:06.641212    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-q9w6j"
Jul 02 13:45:06.642183 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:06.642163    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-69d9f679db-g87ql" event={"ID":"bf9d29ad-19aa-4844-8742-d5f10adcaad8","Type":"ContainerStarted","Data":"456b8e1c8fcc2d245c7150dac27c1440cb4da22918bdac5b33eab341cd5507d8"}
Jul 02 13:45:06.656663 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:06.656621    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-2tjcl" podStartSLOduration=65.848942524 podStartE2EDuration="1m10.656605153s" totalImagesPullingTime="4.807662629s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 13:43:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:45:06.65608801 +0000 UTC m=+71.834550195" watchObservedRunningTime="2026-07-02 13:45:06.656605153 +0000 UTC m=+71.835067336"
Jul 02 13:45:06.673908 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:06.673854    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-q9w6j" podStartSLOduration=66.597756161 podStartE2EDuration="1m11.673838731s" totalImagesPullingTime="5.07608257s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 13:43:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:45:06.672907327 +0000 UTC m=+71.851369511" watchObservedRunningTime="2026-07-02 13:45:06.673838731 +0000 UTC m=+71.852300906"
Jul 02 13:45:07.645967 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:07.645932    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-69d9f679db-g87ql" event={"ID":"bf9d29ad-19aa-4844-8742-d5f10adcaad8","Type":"ContainerStarted","Data":"1bbf03d2b34a62a10ec16bfd4c2dfbfbd2cf4a78a247d38cd3e0346ee07011f7"}
Jul 02 13:45:07.663710 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:07.663499    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-69d9f679db-g87ql" podStartSLOduration=2.655669704 podStartE2EDuration="3.663475856s" totalImagesPullingTime="1.007806152s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 13:45:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:45:07.663156328 +0000 UTC m=+72.841618504" watchObservedRunningTime="2026-07-02 13:45:07.663475856 +0000 UTC m=+72.841938042"
Jul 02 13:45:08.103511 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:08.103475    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a4cc908b-b2a6-409e-8377-9ed59a79ff27-metrics-certs\") pod \"router-default-75dffdcbd7-bsn5q\" (UID: \"a4cc908b-b2a6-409e-8377-9ed59a79ff27\") " pod="openshift-ingress/router-default-75dffdcbd7-bsn5q"
Jul 02 13:45:08.103511 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:08.103518    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a4cc908b-b2a6-409e-8377-9ed59a79ff27-service-ca-bundle\") pod \"router-default-75dffdcbd7-bsn5q\" (UID: \"a4cc908b-b2a6-409e-8377-9ed59a79ff27\") " pod="openshift-ingress/router-default-75dffdcbd7-bsn5q"
Jul 02 13:45:08.103725 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:45:08.103632    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a4cc908b-b2a6-409e-8377-9ed59a79ff27-service-ca-bundle podName:a4cc908b-b2a6-409e-8377-9ed59a79ff27 nodeName:}" failed. No retries permitted until 2026-07-02 13:45:12.103618444 +0000 UTC m=+77.282080608 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/a4cc908b-b2a6-409e-8377-9ed59a79ff27-service-ca-bundle") pod "router-default-75dffdcbd7-bsn5q" (UID: "a4cc908b-b2a6-409e-8377-9ed59a79ff27") : configmap references non-existent config key: service-ca.crt
Jul 02 13:45:08.103725 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:45:08.103631    2558 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 02 13:45:08.103725 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:45:08.103685    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a4cc908b-b2a6-409e-8377-9ed59a79ff27-metrics-certs podName:a4cc908b-b2a6-409e-8377-9ed59a79ff27 nodeName:}" failed. No retries permitted until 2026-07-02 13:45:12.103672069 +0000 UTC m=+77.282134230 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a4cc908b-b2a6-409e-8377-9ed59a79ff27-metrics-certs") pod "router-default-75dffdcbd7-bsn5q" (UID: "a4cc908b-b2a6-409e-8377-9ed59a79ff27") : secret "router-metrics-certs-default" not found
Jul 02 13:45:09.935705 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:09.935666    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-nv5c5"]
Jul 02 13:45:09.937813 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:09.937791    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-nv5c5"
Jul 02 13:45:09.940693 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:09.940666    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 02 13:45:09.940693 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:09.940683    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 02 13:45:09.940882 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:09.940763    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-k62kw\""
Jul 02 13:45:09.940882 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:09.940686    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 02 13:45:09.940882 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:09.940794    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 02 13:45:09.953782 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:09.953760    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-nv5c5"]
Jul 02 13:45:10.016079 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:10.016033    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a5dbc04b-cfab-4330-b693-5cf3fa6c7263-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nv5c5\" (UID: \"a5dbc04b-cfab-4330-b693-5cf3fa6c7263\") " pod="openshift-insights/insights-runtime-extractor-nv5c5"
Jul 02 13:45:10.016079 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:10.016085    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a5dbc04b-cfab-4330-b693-5cf3fa6c7263-crio-socket\") pod \"insights-runtime-extractor-nv5c5\" (UID: \"a5dbc04b-cfab-4330-b693-5cf3fa6c7263\") " pod="openshift-insights/insights-runtime-extractor-nv5c5"
Jul 02 13:45:10.016337 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:10.016105    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nx8qk\" (UniqueName: \"kubernetes.io/projected/a5dbc04b-cfab-4330-b693-5cf3fa6c7263-kube-api-access-nx8qk\") pod \"insights-runtime-extractor-nv5c5\" (UID: \"a5dbc04b-cfab-4330-b693-5cf3fa6c7263\") " pod="openshift-insights/insights-runtime-extractor-nv5c5"
Jul 02 13:45:10.016337 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:10.016287    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a5dbc04b-cfab-4330-b693-5cf3fa6c7263-data-volume\") pod \"insights-runtime-extractor-nv5c5\" (UID: \"a5dbc04b-cfab-4330-b693-5cf3fa6c7263\") " pod="openshift-insights/insights-runtime-extractor-nv5c5"
Jul 02 13:45:10.016337 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:10.016325    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a5dbc04b-cfab-4330-b693-5cf3fa6c7263-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nv5c5\" (UID: \"a5dbc04b-cfab-4330-b693-5cf3fa6c7263\") " pod="openshift-insights/insights-runtime-extractor-nv5c5"
Jul 02 13:45:10.117006 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:10.116958    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a5dbc04b-cfab-4330-b693-5cf3fa6c7263-data-volume\") pod \"insights-runtime-extractor-nv5c5\" (UID: \"a5dbc04b-cfab-4330-b693-5cf3fa6c7263\") " pod="openshift-insights/insights-runtime-extractor-nv5c5"
Jul 02 13:45:10.117006 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:10.117009    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a5dbc04b-cfab-4330-b693-5cf3fa6c7263-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nv5c5\" (UID: \"a5dbc04b-cfab-4330-b693-5cf3fa6c7263\") " pod="openshift-insights/insights-runtime-extractor-nv5c5"
Jul 02 13:45:10.117183 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:45:10.117162    2558 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 02 13:45:10.117239 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:10.117207    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a5dbc04b-cfab-4330-b693-5cf3fa6c7263-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nv5c5\" (UID: \"a5dbc04b-cfab-4330-b693-5cf3fa6c7263\") " pod="openshift-insights/insights-runtime-extractor-nv5c5"
Jul 02 13:45:10.117239 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:45:10.117235    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a5dbc04b-cfab-4330-b693-5cf3fa6c7263-kube-rbac-proxy-cm podName:a5dbc04b-cfab-4330-b693-5cf3fa6c7263 nodeName:}" failed. No retries permitted until 2026-07-02 13:45:10.61721412 +0000 UTC m=+75.795676291 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a5dbc04b-cfab-4330-b693-5cf3fa6c7263-kube-rbac-proxy-cm") pod "insights-runtime-extractor-nv5c5" (UID: "a5dbc04b-cfab-4330-b693-5cf3fa6c7263") : configmap "kube-rbac-proxy" not found
Jul 02 13:45:10.117422 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:45:10.117272    2558 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 02 13:45:10.117422 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:10.117288    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a5dbc04b-cfab-4330-b693-5cf3fa6c7263-data-volume\") pod \"insights-runtime-extractor-nv5c5\" (UID: \"a5dbc04b-cfab-4330-b693-5cf3fa6c7263\") " pod="openshift-insights/insights-runtime-extractor-nv5c5"
Jul 02 13:45:10.117422 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:45:10.117313    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a5dbc04b-cfab-4330-b693-5cf3fa6c7263-insights-runtime-extractor-tls podName:a5dbc04b-cfab-4330-b693-5cf3fa6c7263 nodeName:}" failed. No retries permitted until 2026-07-02 13:45:10.617301876 +0000 UTC m=+75.795764043 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a5dbc04b-cfab-4330-b693-5cf3fa6c7263-insights-runtime-extractor-tls") pod "insights-runtime-extractor-nv5c5" (UID: "a5dbc04b-cfab-4330-b693-5cf3fa6c7263") : secret "insights-runtime-extractor-tls" not found
Jul 02 13:45:10.117422 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:10.117288    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a5dbc04b-cfab-4330-b693-5cf3fa6c7263-crio-socket\") pod \"insights-runtime-extractor-nv5c5\" (UID: \"a5dbc04b-cfab-4330-b693-5cf3fa6c7263\") " pod="openshift-insights/insights-runtime-extractor-nv5c5"
Jul 02 13:45:10.117422 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:10.117340    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a5dbc04b-cfab-4330-b693-5cf3fa6c7263-crio-socket\") pod \"insights-runtime-extractor-nv5c5\" (UID: \"a5dbc04b-cfab-4330-b693-5cf3fa6c7263\") " pod="openshift-insights/insights-runtime-extractor-nv5c5"
Jul 02 13:45:10.117422 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:10.117344    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nx8qk\" (UniqueName: \"kubernetes.io/projected/a5dbc04b-cfab-4330-b693-5cf3fa6c7263-kube-api-access-nx8qk\") pod \"insights-runtime-extractor-nv5c5\" (UID: \"a5dbc04b-cfab-4330-b693-5cf3fa6c7263\") " pod="openshift-insights/insights-runtime-extractor-nv5c5"
Jul 02 13:45:10.128107 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:10.128072    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nx8qk\" (UniqueName: \"kubernetes.io/projected/a5dbc04b-cfab-4330-b693-5cf3fa6c7263-kube-api-access-nx8qk\") pod \"insights-runtime-extractor-nv5c5\" (UID: \"a5dbc04b-cfab-4330-b693-5cf3fa6c7263\") " pod="openshift-insights/insights-runtime-extractor-nv5c5"
Jul 02 13:45:10.621751 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:10.621694    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a5dbc04b-cfab-4330-b693-5cf3fa6c7263-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nv5c5\" (UID: \"a5dbc04b-cfab-4330-b693-5cf3fa6c7263\") " pod="openshift-insights/insights-runtime-extractor-nv5c5"
Jul 02 13:45:10.621986 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:10.621773    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a5dbc04b-cfab-4330-b693-5cf3fa6c7263-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nv5c5\" (UID: \"a5dbc04b-cfab-4330-b693-5cf3fa6c7263\") " pod="openshift-insights/insights-runtime-extractor-nv5c5"
Jul 02 13:45:10.621986 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:45:10.621853    2558 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 02 13:45:10.621986 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:45:10.621909    2558 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 02 13:45:10.621986 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:45:10.621931    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a5dbc04b-cfab-4330-b693-5cf3fa6c7263-kube-rbac-proxy-cm podName:a5dbc04b-cfab-4330-b693-5cf3fa6c7263 nodeName:}" failed. No retries permitted until 2026-07-02 13:45:11.621914494 +0000 UTC m=+76.800376657 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a5dbc04b-cfab-4330-b693-5cf3fa6c7263-kube-rbac-proxy-cm") pod "insights-runtime-extractor-nv5c5" (UID: "a5dbc04b-cfab-4330-b693-5cf3fa6c7263") : configmap "kube-rbac-proxy" not found
Jul 02 13:45:10.621986 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:45:10.621964    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a5dbc04b-cfab-4330-b693-5cf3fa6c7263-insights-runtime-extractor-tls podName:a5dbc04b-cfab-4330-b693-5cf3fa6c7263 nodeName:}" failed. No retries permitted until 2026-07-02 13:45:11.621951646 +0000 UTC m=+76.800413808 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a5dbc04b-cfab-4330-b693-5cf3fa6c7263-insights-runtime-extractor-tls") pod "insights-runtime-extractor-nv5c5" (UID: "a5dbc04b-cfab-4330-b693-5cf3fa6c7263") : secret "insights-runtime-extractor-tls" not found
Jul 02 13:45:11.630890 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:11.630841    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a5dbc04b-cfab-4330-b693-5cf3fa6c7263-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nv5c5\" (UID: \"a5dbc04b-cfab-4330-b693-5cf3fa6c7263\") " pod="openshift-insights/insights-runtime-extractor-nv5c5"
Jul 02 13:45:11.630890 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:45:11.630898    2558 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 02 13:45:11.631322 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:45:11.630962    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a5dbc04b-cfab-4330-b693-5cf3fa6c7263-kube-rbac-proxy-cm podName:a5dbc04b-cfab-4330-b693-5cf3fa6c7263 nodeName:}" failed. No retries permitted until 2026-07-02 13:45:13.630946316 +0000 UTC m=+78.809408477 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a5dbc04b-cfab-4330-b693-5cf3fa6c7263-kube-rbac-proxy-cm") pod "insights-runtime-extractor-nv5c5" (UID: "a5dbc04b-cfab-4330-b693-5cf3fa6c7263") : configmap "kube-rbac-proxy" not found
Jul 02 13:45:11.631322 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:45:11.630973    2558 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 02 13:45:11.631322 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:45:11.631013    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a5dbc04b-cfab-4330-b693-5cf3fa6c7263-insights-runtime-extractor-tls podName:a5dbc04b-cfab-4330-b693-5cf3fa6c7263 nodeName:}" failed. No retries permitted until 2026-07-02 13:45:13.631002658 +0000 UTC m=+78.809464822 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a5dbc04b-cfab-4330-b693-5cf3fa6c7263-insights-runtime-extractor-tls") pod "insights-runtime-extractor-nv5c5" (UID: "a5dbc04b-cfab-4330-b693-5cf3fa6c7263") : secret "insights-runtime-extractor-tls" not found
Jul 02 13:45:11.631322 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:11.630908    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a5dbc04b-cfab-4330-b693-5cf3fa6c7263-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nv5c5\" (UID: \"a5dbc04b-cfab-4330-b693-5cf3fa6c7263\") " pod="openshift-insights/insights-runtime-extractor-nv5c5"
Jul 02 13:45:12.134908 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:12.134871    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a4cc908b-b2a6-409e-8377-9ed59a79ff27-metrics-certs\") pod \"router-default-75dffdcbd7-bsn5q\" (UID: \"a4cc908b-b2a6-409e-8377-9ed59a79ff27\") " pod="openshift-ingress/router-default-75dffdcbd7-bsn5q"
Jul 02 13:45:12.135072 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:12.134916    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a4cc908b-b2a6-409e-8377-9ed59a79ff27-service-ca-bundle\") pod \"router-default-75dffdcbd7-bsn5q\" (UID: \"a4cc908b-b2a6-409e-8377-9ed59a79ff27\") " pod="openshift-ingress/router-default-75dffdcbd7-bsn5q"
Jul 02 13:45:12.135072 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:45:12.135020    2558 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 02 13:45:12.135072 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:45:12.135037    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a4cc908b-b2a6-409e-8377-9ed59a79ff27-service-ca-bundle podName:a4cc908b-b2a6-409e-8377-9ed59a79ff27 nodeName:}" failed. No retries permitted until 2026-07-02 13:45:20.135021811 +0000 UTC m=+85.313483974 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/a4cc908b-b2a6-409e-8377-9ed59a79ff27-service-ca-bundle") pod "router-default-75dffdcbd7-bsn5q" (UID: "a4cc908b-b2a6-409e-8377-9ed59a79ff27") : configmap references non-existent config key: service-ca.crt
Jul 02 13:45:12.135072 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:45:12.135070    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a4cc908b-b2a6-409e-8377-9ed59a79ff27-metrics-certs podName:a4cc908b-b2a6-409e-8377-9ed59a79ff27 nodeName:}" failed. No retries permitted until 2026-07-02 13:45:20.135057795 +0000 UTC m=+85.313519956 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a4cc908b-b2a6-409e-8377-9ed59a79ff27-metrics-certs") pod "router-default-75dffdcbd7-bsn5q" (UID: "a4cc908b-b2a6-409e-8377-9ed59a79ff27") : secret "router-metrics-certs-default" not found
Jul 02 13:45:12.731229 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:12.731200    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-g9pll_859dd06a-dc53-48ca-a2ea-9fea2bad0253/dns-node-resolver/0.log"
Jul 02 13:45:13.645631 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:13.645599    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a5dbc04b-cfab-4330-b693-5cf3fa6c7263-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nv5c5\" (UID: \"a5dbc04b-cfab-4330-b693-5cf3fa6c7263\") " pod="openshift-insights/insights-runtime-extractor-nv5c5"
Jul 02 13:45:13.645842 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:13.645656    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a5dbc04b-cfab-4330-b693-5cf3fa6c7263-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nv5c5\" (UID: \"a5dbc04b-cfab-4330-b693-5cf3fa6c7263\") " pod="openshift-insights/insights-runtime-extractor-nv5c5"
Jul 02 13:45:13.645842 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:45:13.645729    2558 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 02 13:45:13.645842 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:45:13.645772    2558 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 02 13:45:13.645842 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:45:13.645795    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a5dbc04b-cfab-4330-b693-5cf3fa6c7263-kube-rbac-proxy-cm podName:a5dbc04b-cfab-4330-b693-5cf3fa6c7263 nodeName:}" failed. No retries permitted until 2026-07-02 13:45:17.645779334 +0000 UTC m=+82.824241496 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a5dbc04b-cfab-4330-b693-5cf3fa6c7263-kube-rbac-proxy-cm") pod "insights-runtime-extractor-nv5c5" (UID: "a5dbc04b-cfab-4330-b693-5cf3fa6c7263") : configmap "kube-rbac-proxy" not found
Jul 02 13:45:13.646054 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:45:13.645868    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a5dbc04b-cfab-4330-b693-5cf3fa6c7263-insights-runtime-extractor-tls podName:a5dbc04b-cfab-4330-b693-5cf3fa6c7263 nodeName:}" failed. No retries permitted until 2026-07-02 13:45:17.645807534 +0000 UTC m=+82.824269712 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a5dbc04b-cfab-4330-b693-5cf3fa6c7263-insights-runtime-extractor-tls") pod "insights-runtime-extractor-nv5c5" (UID: "a5dbc04b-cfab-4330-b693-5cf3fa6c7263") : secret "insights-runtime-extractor-tls" not found
Jul 02 13:45:13.945093 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:13.945016    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-t99mm_de0396fd-b15d-4e52-b0af-7e40954f53db/node-ca/0.log"
Jul 02 13:45:14.778680 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:14.778653    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-gcr27_7b4fd04b-30e6-4599-b877-2719d6d08231/serve-healthcheck-canary/0.log"
Jul 02 13:45:17.677189 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:17.677077    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a5dbc04b-cfab-4330-b693-5cf3fa6c7263-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nv5c5\" (UID: \"a5dbc04b-cfab-4330-b693-5cf3fa6c7263\") " pod="openshift-insights/insights-runtime-extractor-nv5c5"
Jul 02 13:45:17.677189 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:17.677124    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a5dbc04b-cfab-4330-b693-5cf3fa6c7263-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nv5c5\" (UID: \"a5dbc04b-cfab-4330-b693-5cf3fa6c7263\") " pod="openshift-insights/insights-runtime-extractor-nv5c5"
Jul 02 13:45:17.677589 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:45:17.677201    2558 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 02 13:45:17.677589 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:45:17.677266    2558 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 02 13:45:17.677589 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:45:17.677275    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a5dbc04b-cfab-4330-b693-5cf3fa6c7263-kube-rbac-proxy-cm podName:a5dbc04b-cfab-4330-b693-5cf3fa6c7263 nodeName:}" failed. No retries permitted until 2026-07-02 13:45:25.677253769 +0000 UTC m=+90.855715950 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a5dbc04b-cfab-4330-b693-5cf3fa6c7263-kube-rbac-proxy-cm") pod "insights-runtime-extractor-nv5c5" (UID: "a5dbc04b-cfab-4330-b693-5cf3fa6c7263") : configmap "kube-rbac-proxy" not found
Jul 02 13:45:17.677589 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:45:17.677304    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a5dbc04b-cfab-4330-b693-5cf3fa6c7263-insights-runtime-extractor-tls podName:a5dbc04b-cfab-4330-b693-5cf3fa6c7263 nodeName:}" failed. No retries permitted until 2026-07-02 13:45:25.677292474 +0000 UTC m=+90.855754636 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a5dbc04b-cfab-4330-b693-5cf3fa6c7263-insights-runtime-extractor-tls") pod "insights-runtime-extractor-nv5c5" (UID: "a5dbc04b-cfab-4330-b693-5cf3fa6c7263") : secret "insights-runtime-extractor-tls" not found
Jul 02 13:45:20.197910 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:20.197856    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a4cc908b-b2a6-409e-8377-9ed59a79ff27-metrics-certs\") pod \"router-default-75dffdcbd7-bsn5q\" (UID: \"a4cc908b-b2a6-409e-8377-9ed59a79ff27\") " pod="openshift-ingress/router-default-75dffdcbd7-bsn5q"
Jul 02 13:45:20.197910 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:20.197923    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a4cc908b-b2a6-409e-8377-9ed59a79ff27-service-ca-bundle\") pod \"router-default-75dffdcbd7-bsn5q\" (UID: \"a4cc908b-b2a6-409e-8377-9ed59a79ff27\") " pod="openshift-ingress/router-default-75dffdcbd7-bsn5q"
Jul 02 13:45:20.198392 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:45:20.197996    2558 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 02 13:45:20.198392 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:45:20.198071    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a4cc908b-b2a6-409e-8377-9ed59a79ff27-metrics-certs podName:a4cc908b-b2a6-409e-8377-9ed59a79ff27 nodeName:}" failed. No retries permitted until 2026-07-02 13:45:36.198053847 +0000 UTC m=+101.376516020 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a4cc908b-b2a6-409e-8377-9ed59a79ff27-metrics-certs") pod "router-default-75dffdcbd7-bsn5q" (UID: "a4cc908b-b2a6-409e-8377-9ed59a79ff27") : secret "router-metrics-certs-default" not found
Jul 02 13:45:20.198392 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:45:20.198111    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a4cc908b-b2a6-409e-8377-9ed59a79ff27-service-ca-bundle podName:a4cc908b-b2a6-409e-8377-9ed59a79ff27 nodeName:}" failed. No retries permitted until 2026-07-02 13:45:36.198092542 +0000 UTC m=+101.376554726 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/a4cc908b-b2a6-409e-8377-9ed59a79ff27-service-ca-bundle") pod "router-default-75dffdcbd7-bsn5q" (UID: "a4cc908b-b2a6-409e-8377-9ed59a79ff27") : configmap references non-existent config key: service-ca.crt
Jul 02 13:45:25.744423 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:25.744380    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a5dbc04b-cfab-4330-b693-5cf3fa6c7263-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nv5c5\" (UID: \"a5dbc04b-cfab-4330-b693-5cf3fa6c7263\") " pod="openshift-insights/insights-runtime-extractor-nv5c5"
Jul 02 13:45:25.744927 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:25.744442    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a5dbc04b-cfab-4330-b693-5cf3fa6c7263-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nv5c5\" (UID: \"a5dbc04b-cfab-4330-b693-5cf3fa6c7263\") " pod="openshift-insights/insights-runtime-extractor-nv5c5"
Jul 02 13:45:25.744927 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:45:25.744577    2558 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 02 13:45:25.744927 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:45:25.744656    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a5dbc04b-cfab-4330-b693-5cf3fa6c7263-insights-runtime-extractor-tls podName:a5dbc04b-cfab-4330-b693-5cf3fa6c7263 nodeName:}" failed. No retries permitted until 2026-07-02 13:45:41.74463718 +0000 UTC m=+106.923099361 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a5dbc04b-cfab-4330-b693-5cf3fa6c7263-insights-runtime-extractor-tls") pod "insights-runtime-extractor-nv5c5" (UID: "a5dbc04b-cfab-4330-b693-5cf3fa6c7263") : secret "insights-runtime-extractor-tls" not found
Jul 02 13:45:25.744927 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:25.744919    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a5dbc04b-cfab-4330-b693-5cf3fa6c7263-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nv5c5\" (UID: \"a5dbc04b-cfab-4330-b693-5cf3fa6c7263\") " pod="openshift-insights/insights-runtime-extractor-nv5c5"
Jul 02 13:45:35.618713 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:35.618673    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2bf21a2c-20d4-47ce-922b-b48612f3c438-metrics-tls\") pod \"dns-default-zcqdj\" (UID: \"2bf21a2c-20d4-47ce-922b-b48612f3c438\") " pod="openshift-dns/dns-default-zcqdj"
Jul 02 13:45:35.621162 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:35.621140    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2bf21a2c-20d4-47ce-922b-b48612f3c438-metrics-tls\") pod \"dns-default-zcqdj\" (UID: \"2bf21a2c-20d4-47ce-922b-b48612f3c438\") " pod="openshift-dns/dns-default-zcqdj"
Jul 02 13:45:35.664112 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:35.664074    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-rwl8t\""
Jul 02 13:45:35.671302 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:35.671280    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-zcqdj"
Jul 02 13:45:35.802428 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:35.802394    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-zcqdj"]
Jul 02 13:45:35.805800 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:45:35.805771    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2bf21a2c_20d4_47ce_922b_b48612f3c438.slice/crio-34d615b1783c64141a51d43c6ab651dfb9b3bb7dbe8301a9f07daa339109b145 WatchSource:0}: Error finding container 34d615b1783c64141a51d43c6ab651dfb9b3bb7dbe8301a9f07daa339109b145: Status 404 returned error can't find the container with id 34d615b1783c64141a51d43c6ab651dfb9b3bb7dbe8301a9f07daa339109b145
Jul 02 13:45:36.224005 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:36.223950    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a4cc908b-b2a6-409e-8377-9ed59a79ff27-service-ca-bundle\") pod \"router-default-75dffdcbd7-bsn5q\" (UID: \"a4cc908b-b2a6-409e-8377-9ed59a79ff27\") " pod="openshift-ingress/router-default-75dffdcbd7-bsn5q"
Jul 02 13:45:36.224215 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:36.224074    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a4cc908b-b2a6-409e-8377-9ed59a79ff27-metrics-certs\") pod \"router-default-75dffdcbd7-bsn5q\" (UID: \"a4cc908b-b2a6-409e-8377-9ed59a79ff27\") " pod="openshift-ingress/router-default-75dffdcbd7-bsn5q"
Jul 02 13:45:36.224635 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:36.224610    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a4cc908b-b2a6-409e-8377-9ed59a79ff27-service-ca-bundle\") pod \"router-default-75dffdcbd7-bsn5q\" (UID: \"a4cc908b-b2a6-409e-8377-9ed59a79ff27\") " pod="openshift-ingress/router-default-75dffdcbd7-bsn5q"
Jul 02 13:45:36.226519 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:36.226496    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a4cc908b-b2a6-409e-8377-9ed59a79ff27-metrics-certs\") pod \"router-default-75dffdcbd7-bsn5q\" (UID: \"a4cc908b-b2a6-409e-8377-9ed59a79ff27\") " pod="openshift-ingress/router-default-75dffdcbd7-bsn5q"
Jul 02 13:45:36.523806 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:36.523718    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-75dffdcbd7-bsn5q"
Jul 02 13:45:36.689090 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:36.689046    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-75dffdcbd7-bsn5q"]
Jul 02 13:45:36.692949 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:45:36.692920    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4cc908b_b2a6_409e_8377_9ed59a79ff27.slice/crio-f293b9631bc81b35f1a4f10fcaff24e4b3d05ff5dda0e6964bee99758d5e5975 WatchSource:0}: Error finding container f293b9631bc81b35f1a4f10fcaff24e4b3d05ff5dda0e6964bee99758d5e5975: Status 404 returned error can't find the container with id f293b9631bc81b35f1a4f10fcaff24e4b3d05ff5dda0e6964bee99758d5e5975
Jul 02 13:45:36.717145 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:36.717118    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-zcqdj" event={"ID":"2bf21a2c-20d4-47ce-922b-b48612f3c438","Type":"ContainerStarted","Data":"34d615b1783c64141a51d43c6ab651dfb9b3bb7dbe8301a9f07daa339109b145"}
Jul 02 13:45:36.718320 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:36.718294    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-75dffdcbd7-bsn5q" event={"ID":"a4cc908b-b2a6-409e-8377-9ed59a79ff27","Type":"ContainerStarted","Data":"f293b9631bc81b35f1a4f10fcaff24e4b3d05ff5dda0e6964bee99758d5e5975"}
Jul 02 13:45:37.221694 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:37.221661    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-96fbb6bb4-zncpd"]
Jul 02 13:45:37.224722 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:37.224700    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-96fbb6bb4-zncpd"
Jul 02 13:45:37.228072 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:37.228048    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 02 13:45:37.229258 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:37.229235    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Jul 02 13:45:37.229373 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:37.229333    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 02 13:45:37.229502 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:37.229485    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 02 13:45:37.231257 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:37.231235    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-xv4r9\""
Jul 02 13:45:37.263844 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:37.263791    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56997cfdcc-tcntm"]
Jul 02 13:45:37.266906 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:37.266879    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56997cfdcc-tcntm"
Jul 02 13:45:37.274436 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:45:37.274397    2558 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"cluster-proxy-ca\" is forbidden: User \"system:node:ip-10-0-133-209.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"open-cluster-management-agent-addon\": no relationship found between node 'ip-10-0-133-209.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" type="*v1.Secret"
Jul 02 13:45:37.274607 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:45:37.274583    2558 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"cluster-proxy-hub-kubeconfig\" is forbidden: User \"system:node:ip-10-0-133-209.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"open-cluster-management-agent-addon\": no relationship found between node 'ip-10-0-133-209.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" type="*v1.Secret"
Jul 02 13:45:37.275233 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:45:37.275201    2558 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\" is forbidden: User \"system:node:ip-10-0-133-209.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"open-cluster-management-agent-addon\": no relationship found between node 'ip-10-0-133-209.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" type="*v1.Secret"
Jul 02 13:45:37.275364 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:45:37.275276    2558 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"cluster-proxy-service-proxy-server-certificates\" is forbidden: User \"system:node:ip-10-0-133-209.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"open-cluster-management-agent-addon\": no relationship found between node 'ip-10-0-133-209.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" type="*v1.Secret"
Jul 02 13:45:37.279099 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:37.279073    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-96fbb6bb4-zncpd"]
Jul 02 13:45:37.333626 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:37.333587    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/fb2ddc29-ff17-4e22-9202-1fe56d7b0a82-hub\") pod \"cluster-proxy-proxy-agent-56997cfdcc-tcntm\" (UID: \"fb2ddc29-ff17-4e22-9202-1fe56d7b0a82\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56997cfdcc-tcntm"
Jul 02 13:45:37.333805 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:37.333783    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/fb2ddc29-ff17-4e22-9202-1fe56d7b0a82-ca\") pod \"cluster-proxy-proxy-agent-56997cfdcc-tcntm\" (UID: \"fb2ddc29-ff17-4e22-9202-1fe56d7b0a82\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56997cfdcc-tcntm"
Jul 02 13:45:37.333907 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:37.333836    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/fb2ddc29-ff17-4e22-9202-1fe56d7b0a82-additional-service-ca\") pod \"cluster-proxy-proxy-agent-56997cfdcc-tcntm\" (UID: \"fb2ddc29-ff17-4e22-9202-1fe56d7b0a82\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56997cfdcc-tcntm"
Jul 02 13:45:37.333907 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:37.333869    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/fb2ddc29-ff17-4e22-9202-1fe56d7b0a82-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-56997cfdcc-tcntm\" (UID: \"fb2ddc29-ff17-4e22-9202-1fe56d7b0a82\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56997cfdcc-tcntm"
Jul 02 13:45:37.333907 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:37.333895    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nqgvc\" (UniqueName: \"kubernetes.io/projected/fb2ddc29-ff17-4e22-9202-1fe56d7b0a82-kube-api-access-nqgvc\") pod \"cluster-proxy-proxy-agent-56997cfdcc-tcntm\" (UID: \"fb2ddc29-ff17-4e22-9202-1fe56d7b0a82\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56997cfdcc-tcntm"
Jul 02 13:45:37.334061 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:37.333962    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1466dcec-e91c-4542-9334-ddd6e2b8a8e9-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-96fbb6bb4-zncpd\" (UID: \"1466dcec-e91c-4542-9334-ddd6e2b8a8e9\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-96fbb6bb4-zncpd"
Jul 02 13:45:37.334061 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:37.334034    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/fb2ddc29-ff17-4e22-9202-1fe56d7b0a82-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-56997cfdcc-tcntm\" (UID: \"fb2ddc29-ff17-4e22-9202-1fe56d7b0a82\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56997cfdcc-tcntm"
Jul 02 13:45:37.334159 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:37.334061    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-229f2\" (UniqueName: \"kubernetes.io/projected/1466dcec-e91c-4542-9334-ddd6e2b8a8e9-kube-api-access-229f2\") pod \"managed-serviceaccount-addon-agent-96fbb6bb4-zncpd\" (UID: \"1466dcec-e91c-4542-9334-ddd6e2b8a8e9\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-96fbb6bb4-zncpd"
Jul 02 13:45:37.398764 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:37.398733    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56997cfdcc-tcntm"]
Jul 02 13:45:37.434437 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:37.434400    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/fb2ddc29-ff17-4e22-9202-1fe56d7b0a82-hub\") pod \"cluster-proxy-proxy-agent-56997cfdcc-tcntm\" (UID: \"fb2ddc29-ff17-4e22-9202-1fe56d7b0a82\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56997cfdcc-tcntm"
Jul 02 13:45:37.434620 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:37.434481    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/fb2ddc29-ff17-4e22-9202-1fe56d7b0a82-ca\") pod \"cluster-proxy-proxy-agent-56997cfdcc-tcntm\" (UID: \"fb2ddc29-ff17-4e22-9202-1fe56d7b0a82\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56997cfdcc-tcntm"
Jul 02 13:45:37.434620 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:37.434503    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/fb2ddc29-ff17-4e22-9202-1fe56d7b0a82-additional-service-ca\") pod \"cluster-proxy-proxy-agent-56997cfdcc-tcntm\" (UID: \"fb2ddc29-ff17-4e22-9202-1fe56d7b0a82\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56997cfdcc-tcntm"
Jul 02 13:45:37.434620 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:37.434536    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/fb2ddc29-ff17-4e22-9202-1fe56d7b0a82-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-56997cfdcc-tcntm\" (UID: \"fb2ddc29-ff17-4e22-9202-1fe56d7b0a82\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56997cfdcc-tcntm"
Jul 02 13:45:37.434620 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:37.434558    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nqgvc\" (UniqueName: \"kubernetes.io/projected/fb2ddc29-ff17-4e22-9202-1fe56d7b0a82-kube-api-access-nqgvc\") pod \"cluster-proxy-proxy-agent-56997cfdcc-tcntm\" (UID: \"fb2ddc29-ff17-4e22-9202-1fe56d7b0a82\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56997cfdcc-tcntm"
Jul 02 13:45:37.434620 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:37.434586    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1466dcec-e91c-4542-9334-ddd6e2b8a8e9-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-96fbb6bb4-zncpd\" (UID: \"1466dcec-e91c-4542-9334-ddd6e2b8a8e9\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-96fbb6bb4-zncpd"
Jul 02 13:45:37.434904 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:37.434629    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/fb2ddc29-ff17-4e22-9202-1fe56d7b0a82-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-56997cfdcc-tcntm\" (UID: \"fb2ddc29-ff17-4e22-9202-1fe56d7b0a82\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56997cfdcc-tcntm"
Jul 02 13:45:37.434904 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:37.434655    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-229f2\" (UniqueName: \"kubernetes.io/projected/1466dcec-e91c-4542-9334-ddd6e2b8a8e9-kube-api-access-229f2\") pod \"managed-serviceaccount-addon-agent-96fbb6bb4-zncpd\" (UID: \"1466dcec-e91c-4542-9334-ddd6e2b8a8e9\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-96fbb6bb4-zncpd"
Jul 02 13:45:37.435259 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:37.435233    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/fb2ddc29-ff17-4e22-9202-1fe56d7b0a82-additional-service-ca\") pod \"cluster-proxy-proxy-agent-56997cfdcc-tcntm\" (UID: \"fb2ddc29-ff17-4e22-9202-1fe56d7b0a82\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56997cfdcc-tcntm"
Jul 02 13:45:37.438163 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:37.438136    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1466dcec-e91c-4542-9334-ddd6e2b8a8e9-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-96fbb6bb4-zncpd\" (UID: \"1466dcec-e91c-4542-9334-ddd6e2b8a8e9\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-96fbb6bb4-zncpd"
Jul 02 13:45:37.503687 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:37.503609    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-229f2\" (UniqueName: \"kubernetes.io/projected/1466dcec-e91c-4542-9334-ddd6e2b8a8e9-kube-api-access-229f2\") pod \"managed-serviceaccount-addon-agent-96fbb6bb4-zncpd\" (UID: \"1466dcec-e91c-4542-9334-ddd6e2b8a8e9\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-96fbb6bb4-zncpd"
Jul 02 13:45:37.510275 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:37.510212    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nqgvc\" (UniqueName: \"kubernetes.io/projected/fb2ddc29-ff17-4e22-9202-1fe56d7b0a82-kube-api-access-nqgvc\") pod \"cluster-proxy-proxy-agent-56997cfdcc-tcntm\" (UID: \"fb2ddc29-ff17-4e22-9202-1fe56d7b0a82\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56997cfdcc-tcntm"
Jul 02 13:45:37.550601 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:37.550570    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-96fbb6bb4-zncpd"
Jul 02 13:45:37.648523 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:37.648497    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-q9w6j"
Jul 02 13:45:37.723789 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:37.723756    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-75dffdcbd7-bsn5q" event={"ID":"a4cc908b-b2a6-409e-8377-9ed59a79ff27","Type":"ContainerStarted","Data":"0b1774539b82f90f16c45f58e177711df28c3393215b06e87dec626cd2d240e8"}
Jul 02 13:45:37.725677 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:37.725639    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-zcqdj" event={"ID":"2bf21a2c-20d4-47ce-922b-b48612f3c438","Type":"ContainerStarted","Data":"ca4d9351a5917d828ceab1a386ade2f899d18cd6ea8fc1b2a51d103562139a0c"}
Jul 02 13:45:37.862813 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:37.862778    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-96fbb6bb4-zncpd"]
Jul 02 13:45:37.868674 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:45:37.868648    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1466dcec_e91c_4542_9334_ddd6e2b8a8e9.slice/crio-e602a666e470ae954652482ac0372723898b78b40f7e06fd318670063f4b166d WatchSource:0}: Error finding container e602a666e470ae954652482ac0372723898b78b40f7e06fd318670063f4b166d: Status 404 returned error can't find the container with id e602a666e470ae954652482ac0372723898b78b40f7e06fd318670063f4b166d
Jul 02 13:45:37.959884 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:37.959810    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-75dffdcbd7-bsn5q" podStartSLOduration=33.959794601 podStartE2EDuration="33.959794601s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 13:45:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:45:37.957543483 +0000 UTC m=+103.136005668" watchObservedRunningTime="2026-07-02 13:45:37.959794601 +0000 UTC m=+103.138256782"
Jul 02 13:45:38.198131 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:38.198102    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Jul 02 13:45:38.207882 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:38.207856    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/fb2ddc29-ff17-4e22-9202-1fe56d7b0a82-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-56997cfdcc-tcntm\" (UID: \"fb2ddc29-ff17-4e22-9202-1fe56d7b0a82\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56997cfdcc-tcntm"
Jul 02 13:45:38.375844 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:38.375792    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Jul 02 13:45:38.387888 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:38.387850    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/fb2ddc29-ff17-4e22-9202-1fe56d7b0a82-hub\") pod \"cluster-proxy-proxy-agent-56997cfdcc-tcntm\" (UID: \"fb2ddc29-ff17-4e22-9202-1fe56d7b0a82\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56997cfdcc-tcntm"
Jul 02 13:45:38.435529 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:45:38.435494    2558 secret.go:189] Couldn't get secret open-cluster-management-agent-addon/cluster-proxy-ca: failed to sync secret cache: timed out waiting for the condition
Jul 02 13:45:38.435529 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:45:38.435508    2558 secret.go:189] Couldn't get secret open-cluster-management-agent-addon/cluster-proxy-hub-kubeconfig: failed to sync secret cache: timed out waiting for the condition
Jul 02 13:45:38.435746 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:45:38.435577    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fb2ddc29-ff17-4e22-9202-1fe56d7b0a82-ca podName:fb2ddc29-ff17-4e22-9202-1fe56d7b0a82 nodeName:}" failed. No retries permitted until 2026-07-02 13:45:38.935561023 +0000 UTC m=+104.114023186 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "ca" (UniqueName: "kubernetes.io/secret/fb2ddc29-ff17-4e22-9202-1fe56d7b0a82-ca") pod "cluster-proxy-proxy-agent-56997cfdcc-tcntm" (UID: "fb2ddc29-ff17-4e22-9202-1fe56d7b0a82") : failed to sync secret cache: timed out waiting for the condition
Jul 02 13:45:38.435746 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:45:38.435597    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fb2ddc29-ff17-4e22-9202-1fe56d7b0a82-hub-kubeconfig podName:fb2ddc29-ff17-4e22-9202-1fe56d7b0a82 nodeName:}" failed. No retries permitted until 2026-07-02 13:45:38.935587268 +0000 UTC m=+104.114049430 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "hub-kubeconfig" (UniqueName: "kubernetes.io/secret/fb2ddc29-ff17-4e22-9202-1fe56d7b0a82-hub-kubeconfig") pod "cluster-proxy-proxy-agent-56997cfdcc-tcntm" (UID: "fb2ddc29-ff17-4e22-9202-1fe56d7b0a82") : failed to sync secret cache: timed out waiting for the condition
Jul 02 13:45:38.497642 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:38.497547    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Jul 02 13:45:38.524567 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:38.524500    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-75dffdcbd7-bsn5q"
Jul 02 13:45:38.527687 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:38.527661    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-75dffdcbd7-bsn5q"
Jul 02 13:45:38.598365 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:38.598321    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Jul 02 13:45:38.732287 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:38.732243    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-zcqdj" event={"ID":"2bf21a2c-20d4-47ce-922b-b48612f3c438","Type":"ContainerStarted","Data":"71cb1dcd4327ab1d4aa94c4712cf367a138a1f787dcf01be3d07fe839676914a"}
Jul 02 13:45:38.732718 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:38.732348    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-zcqdj"
Jul 02 13:45:38.734026 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:38.734006    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-96fbb6bb4-zncpd" event={"ID":"1466dcec-e91c-4542-9334-ddd6e2b8a8e9","Type":"ContainerStarted","Data":"e602a666e470ae954652482ac0372723898b78b40f7e06fd318670063f4b166d"}
Jul 02 13:45:38.734597 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:38.734463    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-75dffdcbd7-bsn5q"
Jul 02 13:45:38.735947 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:38.735924    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-75dffdcbd7-bsn5q"
Jul 02 13:45:38.758757 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:38.758621    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-zcqdj" podStartSLOduration=65.986478659 podStartE2EDuration="1m7.758599978s" totalImagesPullingTime="1.772121319s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 13:44:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:45:38.75500446 +0000 UTC m=+103.933466647" watchObservedRunningTime="2026-07-02 13:45:38.758599978 +0000 UTC m=+103.937062164"
Jul 02 13:45:38.948070 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:38.948030    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/fb2ddc29-ff17-4e22-9202-1fe56d7b0a82-ca\") pod \"cluster-proxy-proxy-agent-56997cfdcc-tcntm\" (UID: \"fb2ddc29-ff17-4e22-9202-1fe56d7b0a82\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56997cfdcc-tcntm"
Jul 02 13:45:38.948255 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:38.948113    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/fb2ddc29-ff17-4e22-9202-1fe56d7b0a82-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-56997cfdcc-tcntm\" (UID: \"fb2ddc29-ff17-4e22-9202-1fe56d7b0a82\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56997cfdcc-tcntm"
Jul 02 13:45:38.951521 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:38.951494    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/fb2ddc29-ff17-4e22-9202-1fe56d7b0a82-ca\") pod \"cluster-proxy-proxy-agent-56997cfdcc-tcntm\" (UID: \"fb2ddc29-ff17-4e22-9202-1fe56d7b0a82\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56997cfdcc-tcntm"
Jul 02 13:45:38.951658 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:38.951495    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/fb2ddc29-ff17-4e22-9202-1fe56d7b0a82-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-56997cfdcc-tcntm\" (UID: \"fb2ddc29-ff17-4e22-9202-1fe56d7b0a82\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56997cfdcc-tcntm"
Jul 02 13:45:39.076627 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:39.076530    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56997cfdcc-tcntm"
Jul 02 13:45:39.217654 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:39.217623    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56997cfdcc-tcntm"]
Jul 02 13:45:40.474707 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:45:40.474675    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfb2ddc29_ff17_4e22_9202_1fe56d7b0a82.slice/crio-3f9ee0a8173dcaad7915d36cfaa3cbe6b1225ec457f360fc2a3439be8e45aee0 WatchSource:0}: Error finding container 3f9ee0a8173dcaad7915d36cfaa3cbe6b1225ec457f360fc2a3439be8e45aee0: Status 404 returned error can't find the container with id 3f9ee0a8173dcaad7915d36cfaa3cbe6b1225ec457f360fc2a3439be8e45aee0
Jul 02 13:45:40.741507 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:40.741429    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56997cfdcc-tcntm" event={"ID":"fb2ddc29-ff17-4e22-9202-1fe56d7b0a82","Type":"ContainerStarted","Data":"3f9ee0a8173dcaad7915d36cfaa3cbe6b1225ec457f360fc2a3439be8e45aee0"}
Jul 02 13:45:41.747538 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:41.747498    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-96fbb6bb4-zncpd" event={"ID":"1466dcec-e91c-4542-9334-ddd6e2b8a8e9","Type":"ContainerStarted","Data":"4ec03ad19a27228ce83341492a6903fcef733faeb9df36002942f4db86f92443"}
Jul 02 13:45:41.766491 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:41.766432    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-96fbb6bb4-zncpd" podStartSLOduration=1.774885711 podStartE2EDuration="4.766407909s" totalImagesPullingTime="2.991522198s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 13:45:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:45:41.765674391 +0000 UTC m=+106.944136574" watchObservedRunningTime="2026-07-02 13:45:41.766407909 +0000 UTC m=+106.944870095"
Jul 02 13:45:41.769127 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:41.769100    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a5dbc04b-cfab-4330-b693-5cf3fa6c7263-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nv5c5\" (UID: \"a5dbc04b-cfab-4330-b693-5cf3fa6c7263\") " pod="openshift-insights/insights-runtime-extractor-nv5c5"
Jul 02 13:45:41.771720 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:41.771701    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a5dbc04b-cfab-4330-b693-5cf3fa6c7263-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nv5c5\" (UID: \"a5dbc04b-cfab-4330-b693-5cf3fa6c7263\") " pod="openshift-insights/insights-runtime-extractor-nv5c5"
Jul 02 13:45:42.047400 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:42.047304    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-nv5c5"
Jul 02 13:45:42.216080 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:42.216047    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-nv5c5"]
Jul 02 13:45:42.219617 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:45:42.219596    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5dbc04b_cfab_4330_b693_5cf3fa6c7263.slice/crio-c07a85cbb8104ade038e49969f7a69bab949ce11b8830c07b29c95ea3e786490 WatchSource:0}: Error finding container c07a85cbb8104ade038e49969f7a69bab949ce11b8830c07b29c95ea3e786490: Status 404 returned error can't find the container with id c07a85cbb8104ade038e49969f7a69bab949ce11b8830c07b29c95ea3e786490
Jul 02 13:45:42.752081 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:42.752040    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-nv5c5" event={"ID":"a5dbc04b-cfab-4330-b693-5cf3fa6c7263","Type":"ContainerStarted","Data":"c07a85cbb8104ade038e49969f7a69bab949ce11b8830c07b29c95ea3e786490"}
Jul 02 13:45:47.769612 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:47.769569    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56997cfdcc-tcntm" event={"ID":"fb2ddc29-ff17-4e22-9202-1fe56d7b0a82","Type":"ContainerStarted","Data":"d67223820d23696b726fd74a51e17053d0b330533bf68fb225f5560d52fe1840"}
Jul 02 13:45:47.770127 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:47.769627    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56997cfdcc-tcntm" event={"ID":"fb2ddc29-ff17-4e22-9202-1fe56d7b0a82","Type":"ContainerStarted","Data":"c92c9c6b9384f0ba6279188b1f238ddaa308459ee16b3d1fb93c418009ef3813"}
Jul 02 13:45:47.770127 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:47.769645    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56997cfdcc-tcntm" event={"ID":"fb2ddc29-ff17-4e22-9202-1fe56d7b0a82","Type":"ContainerStarted","Data":"d6cdb313e1343be4731fe147b5bb26adc71ff95c8d118430922069cec25e7f02"}
Jul 02 13:45:47.771006 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:47.770984    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-nv5c5" event={"ID":"a5dbc04b-cfab-4330-b693-5cf3fa6c7263","Type":"ContainerStarted","Data":"21891cfc699c8b4c81795608aab28c6cb55a828c9761a3f9c39def9962a0b4d3"}
Jul 02 13:45:47.803572 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:47.803526    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56997cfdcc-tcntm" podStartSLOduration=4.149459056 podStartE2EDuration="10.803511622s" totalImagesPullingTime="6.654052566s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 13:45:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:45:47.803234277 +0000 UTC m=+112.981696462" watchObservedRunningTime="2026-07-02 13:45:47.803511622 +0000 UTC m=+112.981973806"
Jul 02 13:45:48.739653 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:48.739624    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-zcqdj"
Jul 02 13:45:48.775880 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:48.775847    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-nv5c5" event={"ID":"a5dbc04b-cfab-4330-b693-5cf3fa6c7263","Type":"ContainerStarted","Data":"951864ecf5836232eec10a97940f7c8dd20efacc289e8ded5dc5cdb9cc702aa4"}
Jul 02 13:45:50.998747 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:50.998712    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-bmljc"]
Jul 02 13:45:51.009406 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:51.009371    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-bmljc"
Jul 02 13:45:51.016408 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:51.016374    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 02 13:45:51.016555 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:51.016428    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 02 13:45:51.016854 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:51.016685    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 02 13:45:51.017556 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:51.017415    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 02 13:45:51.017556 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:51.017417    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-9mx2c\""
Jul 02 13:45:51.017556 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:51.017504    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 02 13:45:51.017800 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:51.017688    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 02 13:45:51.144725 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:51.144695    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4f2e0e73-0517-440f-95b7-28364750732f-node-exporter-textfile\") pod \"node-exporter-bmljc\" (UID: \"4f2e0e73-0517-440f-95b7-28364750732f\") " pod="openshift-monitoring/node-exporter-bmljc"
Jul 02 13:45:51.144855 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:51.144727    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4f2e0e73-0517-440f-95b7-28364750732f-node-exporter-wtmp\") pod \"node-exporter-bmljc\" (UID: \"4f2e0e73-0517-440f-95b7-28364750732f\") " pod="openshift-monitoring/node-exporter-bmljc"
Jul 02 13:45:51.144855 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:51.144746    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5mx6f\" (UniqueName: \"kubernetes.io/projected/4f2e0e73-0517-440f-95b7-28364750732f-kube-api-access-5mx6f\") pod \"node-exporter-bmljc\" (UID: \"4f2e0e73-0517-440f-95b7-28364750732f\") " pod="openshift-monitoring/node-exporter-bmljc"
Jul 02 13:45:51.144855 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:51.144766    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4f2e0e73-0517-440f-95b7-28364750732f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-bmljc\" (UID: \"4f2e0e73-0517-440f-95b7-28364750732f\") " pod="openshift-monitoring/node-exporter-bmljc"
Jul 02 13:45:51.144981 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:51.144860    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4f2e0e73-0517-440f-95b7-28364750732f-root\") pod \"node-exporter-bmljc\" (UID: \"4f2e0e73-0517-440f-95b7-28364750732f\") " pod="openshift-monitoring/node-exporter-bmljc"
Jul 02 13:45:51.144981 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:51.144900    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4f2e0e73-0517-440f-95b7-28364750732f-metrics-client-ca\") pod \"node-exporter-bmljc\" (UID: \"4f2e0e73-0517-440f-95b7-28364750732f\") " pod="openshift-monitoring/node-exporter-bmljc"
Jul 02 13:45:51.144981 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:51.144958    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4f2e0e73-0517-440f-95b7-28364750732f-node-exporter-tls\") pod \"node-exporter-bmljc\" (UID: \"4f2e0e73-0517-440f-95b7-28364750732f\") " pod="openshift-monitoring/node-exporter-bmljc"
Jul 02 13:45:51.144981 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:51.144978    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4f2e0e73-0517-440f-95b7-28364750732f-node-exporter-accelerators-collector-config\") pod \"node-exporter-bmljc\" (UID: \"4f2e0e73-0517-440f-95b7-28364750732f\") " pod="openshift-monitoring/node-exporter-bmljc"
Jul 02 13:45:51.145112 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:51.144998    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4f2e0e73-0517-440f-95b7-28364750732f-sys\") pod \"node-exporter-bmljc\" (UID: \"4f2e0e73-0517-440f-95b7-28364750732f\") " pod="openshift-monitoring/node-exporter-bmljc"
Jul 02 13:45:51.246111 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:51.246081    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4f2e0e73-0517-440f-95b7-28364750732f-root\") pod \"node-exporter-bmljc\" (UID: \"4f2e0e73-0517-440f-95b7-28364750732f\") " pod="openshift-monitoring/node-exporter-bmljc"
Jul 02 13:45:51.246111 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:51.246116    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4f2e0e73-0517-440f-95b7-28364750732f-metrics-client-ca\") pod \"node-exporter-bmljc\" (UID: \"4f2e0e73-0517-440f-95b7-28364750732f\") " pod="openshift-monitoring/node-exporter-bmljc"
Jul 02 13:45:51.246379 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:51.246154    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4f2e0e73-0517-440f-95b7-28364750732f-node-exporter-tls\") pod \"node-exporter-bmljc\" (UID: \"4f2e0e73-0517-440f-95b7-28364750732f\") " pod="openshift-monitoring/node-exporter-bmljc"
Jul 02 13:45:51.246379 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:51.246186    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4f2e0e73-0517-440f-95b7-28364750732f-node-exporter-accelerators-collector-config\") pod \"node-exporter-bmljc\" (UID: \"4f2e0e73-0517-440f-95b7-28364750732f\") " pod="openshift-monitoring/node-exporter-bmljc"
Jul 02 13:45:51.246379 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:51.246198    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4f2e0e73-0517-440f-95b7-28364750732f-root\") pod \"node-exporter-bmljc\" (UID: \"4f2e0e73-0517-440f-95b7-28364750732f\") " pod="openshift-monitoring/node-exporter-bmljc"
Jul 02 13:45:51.246379 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:51.246211    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4f2e0e73-0517-440f-95b7-28364750732f-sys\") pod \"node-exporter-bmljc\" (UID: \"4f2e0e73-0517-440f-95b7-28364750732f\") " pod="openshift-monitoring/node-exporter-bmljc"
Jul 02 13:45:51.246379 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:51.246252    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4f2e0e73-0517-440f-95b7-28364750732f-sys\") pod \"node-exporter-bmljc\" (UID: \"4f2e0e73-0517-440f-95b7-28364750732f\") " pod="openshift-monitoring/node-exporter-bmljc"
Jul 02 13:45:51.246379 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:51.246262    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4f2e0e73-0517-440f-95b7-28364750732f-node-exporter-textfile\") pod \"node-exporter-bmljc\" (UID: \"4f2e0e73-0517-440f-95b7-28364750732f\") " pod="openshift-monitoring/node-exporter-bmljc"
Jul 02 13:45:51.246379 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:51.246285    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4f2e0e73-0517-440f-95b7-28364750732f-node-exporter-wtmp\") pod \"node-exporter-bmljc\" (UID: \"4f2e0e73-0517-440f-95b7-28364750732f\") " pod="openshift-monitoring/node-exporter-bmljc"
Jul 02 13:45:51.246379 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:51.246304    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5mx6f\" (UniqueName: \"kubernetes.io/projected/4f2e0e73-0517-440f-95b7-28364750732f-kube-api-access-5mx6f\") pod \"node-exporter-bmljc\" (UID: \"4f2e0e73-0517-440f-95b7-28364750732f\") " pod="openshift-monitoring/node-exporter-bmljc"
Jul 02 13:45:51.246379 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:51.246337    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4f2e0e73-0517-440f-95b7-28364750732f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-bmljc\" (UID: \"4f2e0e73-0517-440f-95b7-28364750732f\") " pod="openshift-monitoring/node-exporter-bmljc"
Jul 02 13:45:51.246815 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:51.246472    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4f2e0e73-0517-440f-95b7-28364750732f-node-exporter-wtmp\") pod \"node-exporter-bmljc\" (UID: \"4f2e0e73-0517-440f-95b7-28364750732f\") " pod="openshift-monitoring/node-exporter-bmljc"
Jul 02 13:45:51.246815 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:51.246653    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4f2e0e73-0517-440f-95b7-28364750732f-node-exporter-textfile\") pod \"node-exporter-bmljc\" (UID: \"4f2e0e73-0517-440f-95b7-28364750732f\") " pod="openshift-monitoring/node-exporter-bmljc"
Jul 02 13:45:51.246954 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:51.246854    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4f2e0e73-0517-440f-95b7-28364750732f-node-exporter-accelerators-collector-config\") pod \"node-exporter-bmljc\" (UID: \"4f2e0e73-0517-440f-95b7-28364750732f\") " pod="openshift-monitoring/node-exporter-bmljc"
Jul 02 13:45:51.246954 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:51.246873    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4f2e0e73-0517-440f-95b7-28364750732f-metrics-client-ca\") pod \"node-exporter-bmljc\" (UID: \"4f2e0e73-0517-440f-95b7-28364750732f\") " pod="openshift-monitoring/node-exporter-bmljc"
Jul 02 13:45:51.248733 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:51.248717    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4f2e0e73-0517-440f-95b7-28364750732f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-bmljc\" (UID: \"4f2e0e73-0517-440f-95b7-28364750732f\") " pod="openshift-monitoring/node-exporter-bmljc"
Jul 02 13:45:51.248852 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:51.248801    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4f2e0e73-0517-440f-95b7-28364750732f-node-exporter-tls\") pod \"node-exporter-bmljc\" (UID: \"4f2e0e73-0517-440f-95b7-28364750732f\") " pod="openshift-monitoring/node-exporter-bmljc"
Jul 02 13:45:51.255695 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:51.255676    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5mx6f\" (UniqueName: \"kubernetes.io/projected/4f2e0e73-0517-440f-95b7-28364750732f-kube-api-access-5mx6f\") pod \"node-exporter-bmljc\" (UID: \"4f2e0e73-0517-440f-95b7-28364750732f\") " pod="openshift-monitoring/node-exporter-bmljc"
Jul 02 13:45:51.319776 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:51.319738    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-bmljc"
Jul 02 13:45:51.329921 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:45:51.329893    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f2e0e73_0517_440f_95b7_28364750732f.slice/crio-14cfe2b267cff857ef7d34772fe2729d7b5d19c3a9c646fb1ed31f49f607b316 WatchSource:0}: Error finding container 14cfe2b267cff857ef7d34772fe2729d7b5d19c3a9c646fb1ed31f49f607b316: Status 404 returned error can't find the container with id 14cfe2b267cff857ef7d34772fe2729d7b5d19c3a9c646fb1ed31f49f607b316
Jul 02 13:45:51.789554 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:51.789517    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-bmljc" event={"ID":"4f2e0e73-0517-440f-95b7-28364750732f","Type":"ContainerStarted","Data":"14cfe2b267cff857ef7d34772fe2729d7b5d19c3a9c646fb1ed31f49f607b316"}
Jul 02 13:45:51.791871 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:51.791844    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-nv5c5" event={"ID":"a5dbc04b-cfab-4330-b693-5cf3fa6c7263","Type":"ContainerStarted","Data":"3ff97ee780b7b0f252e50b76378b9a3008af34722db9900319707b776f5fe8a7"}
Jul 02 13:45:51.822727 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:51.822671    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-nv5c5" podStartSLOduration=34.043451565 podStartE2EDuration="42.822652366s" totalImagesPullingTime="8.779200801s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 13:45:09 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:45:51.820530832 +0000 UTC m=+116.998993040" watchObservedRunningTime="2026-07-02 13:45:51.822652366 +0000 UTC m=+117.001114551"
Jul 02 13:45:52.797109 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:52.797070    2558 generic.go:356] "Generic (PLEG): container finished" podID="4f2e0e73-0517-440f-95b7-28364750732f" containerID="128c9d26da9d0c264b375255b35db3b7d4d1ed59059ce8e59832a5d5c5113d35" exitCode=0
Jul 02 13:45:52.797587 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:52.797158    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-bmljc" event={"ID":"4f2e0e73-0517-440f-95b7-28364750732f","Type":"ContainerDied","Data":"128c9d26da9d0c264b375255b35db3b7d4d1ed59059ce8e59832a5d5c5113d35"}
Jul 02 13:45:52.996539 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:52.996502    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-57879888b8-q8md5"]
Jul 02 13:45:53.001419 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:53.001389    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-57879888b8-q8md5"
Jul 02 13:45:53.005216 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:53.005187    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Jul 02 13:45:53.005391 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:53.005277    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Jul 02 13:45:53.005391 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:53.005277    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-ldpc9\""
Jul 02 13:45:53.005550 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:53.005536    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-7jl48d0o6s21r\""
Jul 02 13:45:53.005934 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:53.005919    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Jul 02 13:45:53.005979 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:53.005919    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Jul 02 13:45:53.006349 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:53.006331    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Jul 02 13:45:53.016048 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:53.016025    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-57879888b8-q8md5"]
Jul 02 13:45:53.160311 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:53.160273    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/277131f0-f6c3-4190-b01f-dabc843e7403-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-57879888b8-q8md5\" (UID: \"277131f0-f6c3-4190-b01f-dabc843e7403\") " pod="openshift-monitoring/thanos-querier-57879888b8-q8md5"
Jul 02 13:45:53.160517 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:53.160334    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/277131f0-f6c3-4190-b01f-dabc843e7403-secret-thanos-querier-tls\") pod \"thanos-querier-57879888b8-q8md5\" (UID: \"277131f0-f6c3-4190-b01f-dabc843e7403\") " pod="openshift-monitoring/thanos-querier-57879888b8-q8md5"
Jul 02 13:45:53.160517 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:53.160357    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/277131f0-f6c3-4190-b01f-dabc843e7403-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-57879888b8-q8md5\" (UID: \"277131f0-f6c3-4190-b01f-dabc843e7403\") " pod="openshift-monitoring/thanos-querier-57879888b8-q8md5"
Jul 02 13:45:53.160517 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:53.160397    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tvbg9\" (UniqueName: \"kubernetes.io/projected/277131f0-f6c3-4190-b01f-dabc843e7403-kube-api-access-tvbg9\") pod \"thanos-querier-57879888b8-q8md5\" (UID: \"277131f0-f6c3-4190-b01f-dabc843e7403\") " pod="openshift-monitoring/thanos-querier-57879888b8-q8md5"
Jul 02 13:45:53.160517 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:53.160458    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/277131f0-f6c3-4190-b01f-dabc843e7403-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-57879888b8-q8md5\" (UID: \"277131f0-f6c3-4190-b01f-dabc843e7403\") " pod="openshift-monitoring/thanos-querier-57879888b8-q8md5"
Jul 02 13:45:53.160517 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:53.160494    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/277131f0-f6c3-4190-b01f-dabc843e7403-secret-grpc-tls\") pod \"thanos-querier-57879888b8-q8md5\" (UID: \"277131f0-f6c3-4190-b01f-dabc843e7403\") " pod="openshift-monitoring/thanos-querier-57879888b8-q8md5"
Jul 02 13:45:53.160719 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:53.160526    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/277131f0-f6c3-4190-b01f-dabc843e7403-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-57879888b8-q8md5\" (UID: \"277131f0-f6c3-4190-b01f-dabc843e7403\") " pod="openshift-monitoring/thanos-querier-57879888b8-q8md5"
Jul 02 13:45:53.160719 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:53.160571    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/277131f0-f6c3-4190-b01f-dabc843e7403-metrics-client-ca\") pod \"thanos-querier-57879888b8-q8md5\" (UID: \"277131f0-f6c3-4190-b01f-dabc843e7403\") " pod="openshift-monitoring/thanos-querier-57879888b8-q8md5"
Jul 02 13:45:53.261319 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:53.261280    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/277131f0-f6c3-4190-b01f-dabc843e7403-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-57879888b8-q8md5\" (UID: \"277131f0-f6c3-4190-b01f-dabc843e7403\") " pod="openshift-monitoring/thanos-querier-57879888b8-q8md5"
Jul 02 13:45:53.261498 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:53.261343    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/277131f0-f6c3-4190-b01f-dabc843e7403-secret-thanos-querier-tls\") pod \"thanos-querier-57879888b8-q8md5\" (UID: \"277131f0-f6c3-4190-b01f-dabc843e7403\") " pod="openshift-monitoring/thanos-querier-57879888b8-q8md5"
Jul 02 13:45:53.261498 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:53.261368    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/277131f0-f6c3-4190-b01f-dabc843e7403-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-57879888b8-q8md5\" (UID: \"277131f0-f6c3-4190-b01f-dabc843e7403\") " pod="openshift-monitoring/thanos-querier-57879888b8-q8md5"
Jul 02 13:45:53.261498 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:53.261385    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tvbg9\" (UniqueName: \"kubernetes.io/projected/277131f0-f6c3-4190-b01f-dabc843e7403-kube-api-access-tvbg9\") pod \"thanos-querier-57879888b8-q8md5\" (UID: \"277131f0-f6c3-4190-b01f-dabc843e7403\") " pod="openshift-monitoring/thanos-querier-57879888b8-q8md5"
Jul 02 13:45:53.261498 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:53.261404    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/277131f0-f6c3-4190-b01f-dabc843e7403-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-57879888b8-q8md5\" (UID: \"277131f0-f6c3-4190-b01f-dabc843e7403\") " pod="openshift-monitoring/thanos-querier-57879888b8-q8md5"
Jul 02 13:45:53.261498 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:53.261418    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/277131f0-f6c3-4190-b01f-dabc843e7403-secret-grpc-tls\") pod \"thanos-querier-57879888b8-q8md5\" (UID: \"277131f0-f6c3-4190-b01f-dabc843e7403\") " pod="openshift-monitoring/thanos-querier-57879888b8-q8md5"
Jul 02 13:45:53.261780 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:53.261518    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/277131f0-f6c3-4190-b01f-dabc843e7403-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-57879888b8-q8md5\" (UID: \"277131f0-f6c3-4190-b01f-dabc843e7403\") " pod="openshift-monitoring/thanos-querier-57879888b8-q8md5"
Jul 02 13:45:53.261780 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:53.261583    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/277131f0-f6c3-4190-b01f-dabc843e7403-metrics-client-ca\") pod \"thanos-querier-57879888b8-q8md5\" (UID: \"277131f0-f6c3-4190-b01f-dabc843e7403\") " pod="openshift-monitoring/thanos-querier-57879888b8-q8md5"
Jul 02 13:45:53.262585 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:53.262557    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/277131f0-f6c3-4190-b01f-dabc843e7403-metrics-client-ca\") pod \"thanos-querier-57879888b8-q8md5\" (UID: \"277131f0-f6c3-4190-b01f-dabc843e7403\") " pod="openshift-monitoring/thanos-querier-57879888b8-q8md5"
Jul 02 13:45:53.264158 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:53.264136    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/277131f0-f6c3-4190-b01f-dabc843e7403-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-57879888b8-q8md5\" (UID: \"277131f0-f6c3-4190-b01f-dabc843e7403\") " pod="openshift-monitoring/thanos-querier-57879888b8-q8md5"
Jul 02 13:45:53.264382 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:53.264342    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/277131f0-f6c3-4190-b01f-dabc843e7403-secret-thanos-querier-tls\") pod \"thanos-querier-57879888b8-q8md5\" (UID: \"277131f0-f6c3-4190-b01f-dabc843e7403\") " pod="openshift-monitoring/thanos-querier-57879888b8-q8md5"
Jul 02 13:45:53.264525 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:53.264504    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/277131f0-f6c3-4190-b01f-dabc843e7403-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-57879888b8-q8md5\" (UID: \"277131f0-f6c3-4190-b01f-dabc843e7403\") " pod="openshift-monitoring/thanos-querier-57879888b8-q8md5"
Jul 02 13:45:53.264728 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:53.264706    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/277131f0-f6c3-4190-b01f-dabc843e7403-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-57879888b8-q8md5\" (UID: \"277131f0-f6c3-4190-b01f-dabc843e7403\") " pod="openshift-monitoring/thanos-querier-57879888b8-q8md5"
Jul 02 13:45:53.264765 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:53.264737    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/277131f0-f6c3-4190-b01f-dabc843e7403-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-57879888b8-q8md5\" (UID: \"277131f0-f6c3-4190-b01f-dabc843e7403\") " pod="openshift-monitoring/thanos-querier-57879888b8-q8md5"
Jul 02 13:45:53.264798 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:53.264783    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/277131f0-f6c3-4190-b01f-dabc843e7403-secret-grpc-tls\") pod \"thanos-querier-57879888b8-q8md5\" (UID: \"277131f0-f6c3-4190-b01f-dabc843e7403\") " pod="openshift-monitoring/thanos-querier-57879888b8-q8md5"
Jul 02 13:45:53.277138 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:53.277118    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tvbg9\" (UniqueName: \"kubernetes.io/projected/277131f0-f6c3-4190-b01f-dabc843e7403-kube-api-access-tvbg9\") pod \"thanos-querier-57879888b8-q8md5\" (UID: \"277131f0-f6c3-4190-b01f-dabc843e7403\") " pod="openshift-monitoring/thanos-querier-57879888b8-q8md5"
Jul 02 13:45:53.311057 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:53.311018    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-57879888b8-q8md5"
Jul 02 13:45:53.449852 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:53.449702    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-57879888b8-q8md5"]
Jul 02 13:45:53.452372 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:45:53.452334    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod277131f0_f6c3_4190_b01f_dabc843e7403.slice/crio-8b2803322bda2e4126f2775eb4371e37c6930fef6b86e631ff6f1fe00600b337 WatchSource:0}: Error finding container 8b2803322bda2e4126f2775eb4371e37c6930fef6b86e631ff6f1fe00600b337: Status 404 returned error can't find the container with id 8b2803322bda2e4126f2775eb4371e37c6930fef6b86e631ff6f1fe00600b337
Jul 02 13:45:53.802398 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:53.802300    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-bmljc" event={"ID":"4f2e0e73-0517-440f-95b7-28364750732f","Type":"ContainerStarted","Data":"b57ab36632892dd597161536501aaf9e156c2b51502ede88adc817adb5c0341c"}
Jul 02 13:45:53.802398 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:53.802356    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-bmljc" event={"ID":"4f2e0e73-0517-440f-95b7-28364750732f","Type":"ContainerStarted","Data":"937b2ef8b9061d498f4f5ea7263e8ae257106d7554872a80883815be97ad3c6d"}
Jul 02 13:45:53.803525 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:53.803504    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-57879888b8-q8md5" event={"ID":"277131f0-f6c3-4190-b01f-dabc843e7403","Type":"ContainerStarted","Data":"8b2803322bda2e4126f2775eb4371e37c6930fef6b86e631ff6f1fe00600b337"}
Jul 02 13:45:53.823644 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:53.823529    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-bmljc" podStartSLOduration=2.934610331 podStartE2EDuration="3.823512861s" totalImagesPullingTime="815.538837ms" totalInitContainerRuntime="73.363693ms" isStatefulPod=true podCreationTimestamp="2026-07-02 13:45:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:45:53.823197136 +0000 UTC m=+119.001659344" watchObservedRunningTime="2026-07-02 13:45:53.823512861 +0000 UTC m=+119.001975044"
Jul 02 13:45:55.713520 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:55.713489    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-cd8656b8-5v7b8"]
Jul 02 13:45:55.715955 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:55.715938    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-cd8656b8-5v7b8"
Jul 02 13:45:55.718488 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:55.718466    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Jul 02 13:45:55.718488 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:55.718474    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-89zth\""
Jul 02 13:45:55.725020 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:55.724997    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-cd8656b8-5v7b8"]
Jul 02 13:45:55.811840 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:55.811737    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-57879888b8-q8md5" event={"ID":"277131f0-f6c3-4190-b01f-dabc843e7403","Type":"ContainerStarted","Data":"a6f98eb0e0a812003304d343407e9eae9e50fec2231db07897ef59f6db20e089"}
Jul 02 13:45:55.811840 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:55.811805    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-57879888b8-q8md5" event={"ID":"277131f0-f6c3-4190-b01f-dabc843e7403","Type":"ContainerStarted","Data":"a898455088549f68fe4ac8de62831966055c9e529c54f11a8199ab7acb1ef38f"}
Jul 02 13:45:55.812031 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:55.811845    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-57879888b8-q8md5" event={"ID":"277131f0-f6c3-4190-b01f-dabc843e7403","Type":"ContainerStarted","Data":"4934a57448be303b413deadd604e4ef25731b9e274a4b246ef89e75fc983b7bd"}
Jul 02 13:45:55.888180 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:55.888123    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jnwgq\" (UniqueName: \"kubernetes.io/projected/cd5df5ba-88e2-4926-b63c-f9fe0cdb8faa-kube-api-access-jnwgq\") pod \"monitoring-plugin-cd8656b8-5v7b8\" (UID: \"cd5df5ba-88e2-4926-b63c-f9fe0cdb8faa\") " pod="openshift-monitoring/monitoring-plugin-cd8656b8-5v7b8"
Jul 02 13:45:55.888316 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:55.888202    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/cd5df5ba-88e2-4926-b63c-f9fe0cdb8faa-monitoring-plugin-cert\") pod \"monitoring-plugin-cd8656b8-5v7b8\" (UID: \"cd5df5ba-88e2-4926-b63c-f9fe0cdb8faa\") " pod="openshift-monitoring/monitoring-plugin-cd8656b8-5v7b8"
Jul 02 13:45:55.988604 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:55.988572    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jnwgq\" (UniqueName: \"kubernetes.io/projected/cd5df5ba-88e2-4926-b63c-f9fe0cdb8faa-kube-api-access-jnwgq\") pod \"monitoring-plugin-cd8656b8-5v7b8\" (UID: \"cd5df5ba-88e2-4926-b63c-f9fe0cdb8faa\") " pod="openshift-monitoring/monitoring-plugin-cd8656b8-5v7b8"
Jul 02 13:45:55.988717 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:55.988617    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/cd5df5ba-88e2-4926-b63c-f9fe0cdb8faa-monitoring-plugin-cert\") pod \"monitoring-plugin-cd8656b8-5v7b8\" (UID: \"cd5df5ba-88e2-4926-b63c-f9fe0cdb8faa\") " pod="openshift-monitoring/monitoring-plugin-cd8656b8-5v7b8"
Jul 02 13:45:55.988755 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:45:55.988724    2558 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found
Jul 02 13:45:55.988787 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:45:55.988779    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cd5df5ba-88e2-4926-b63c-f9fe0cdb8faa-monitoring-plugin-cert podName:cd5df5ba-88e2-4926-b63c-f9fe0cdb8faa nodeName:}" failed. No retries permitted until 2026-07-02 13:45:56.48876292 +0000 UTC m=+121.667225083 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/cd5df5ba-88e2-4926-b63c-f9fe0cdb8faa-monitoring-plugin-cert") pod "monitoring-plugin-cd8656b8-5v7b8" (UID: "cd5df5ba-88e2-4926-b63c-f9fe0cdb8faa") : secret "monitoring-plugin-cert" not found
Jul 02 13:45:55.999765 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:55.999735    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jnwgq\" (UniqueName: \"kubernetes.io/projected/cd5df5ba-88e2-4926-b63c-f9fe0cdb8faa-kube-api-access-jnwgq\") pod \"monitoring-plugin-cd8656b8-5v7b8\" (UID: \"cd5df5ba-88e2-4926-b63c-f9fe0cdb8faa\") " pod="openshift-monitoring/monitoring-plugin-cd8656b8-5v7b8"
Jul 02 13:45:56.219810 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:56.219781    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-685f676f49-xk5v9"]
Jul 02 13:45:56.222218 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:56.222198    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-685f676f49-xk5v9"
Jul 02 13:45:56.227446 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:56.227423    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Jul 02 13:45:56.227572 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:56.227449    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Jul 02 13:45:56.227572 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:56.227460    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-9rfwj\""
Jul 02 13:45:56.227572 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:56.227452    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Jul 02 13:45:56.227683 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:56.227603    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Jul 02 13:45:56.228312 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:56.228295    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Jul 02 13:45:56.233104 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:56.233085    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Jul 02 13:45:56.250744 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:56.250721    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-685f676f49-xk5v9"]
Jul 02 13:45:56.292667 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:56.292635    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/ce9206ce-a65b-4da4-9391-e64748d0518d-federate-client-tls\") pod \"telemeter-client-685f676f49-xk5v9\" (UID: \"ce9206ce-a65b-4da4-9391-e64748d0518d\") " pod="openshift-monitoring/telemeter-client-685f676f49-xk5v9"
Jul 02 13:45:56.292812 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:56.292683    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ce9206ce-a65b-4da4-9391-e64748d0518d-telemeter-trusted-ca-bundle\") pod \"telemeter-client-685f676f49-xk5v9\" (UID: \"ce9206ce-a65b-4da4-9391-e64748d0518d\") " pod="openshift-monitoring/telemeter-client-685f676f49-xk5v9"
Jul 02 13:45:56.292812 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:56.292747    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/ce9206ce-a65b-4da4-9391-e64748d0518d-secret-telemeter-client\") pod \"telemeter-client-685f676f49-xk5v9\" (UID: \"ce9206ce-a65b-4da4-9391-e64748d0518d\") " pod="openshift-monitoring/telemeter-client-685f676f49-xk5v9"
Jul 02 13:45:56.292812 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:56.292780    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ce9206ce-a65b-4da4-9391-e64748d0518d-serving-certs-ca-bundle\") pod \"telemeter-client-685f676f49-xk5v9\" (UID: \"ce9206ce-a65b-4da4-9391-e64748d0518d\") " pod="openshift-monitoring/telemeter-client-685f676f49-xk5v9"
Jul 02 13:45:56.292812 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:56.292798    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/ce9206ce-a65b-4da4-9391-e64748d0518d-telemeter-client-tls\") pod \"telemeter-client-685f676f49-xk5v9\" (UID: \"ce9206ce-a65b-4da4-9391-e64748d0518d\") " pod="openshift-monitoring/telemeter-client-685f676f49-xk5v9"
Jul 02 13:45:56.293025 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:56.292816    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ce9206ce-a65b-4da4-9391-e64748d0518d-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-685f676f49-xk5v9\" (UID: \"ce9206ce-a65b-4da4-9391-e64748d0518d\") " pod="openshift-monitoring/telemeter-client-685f676f49-xk5v9"
Jul 02 13:45:56.293025 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:56.292880    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-74jfw\" (UniqueName: \"kubernetes.io/projected/ce9206ce-a65b-4da4-9391-e64748d0518d-kube-api-access-74jfw\") pod \"telemeter-client-685f676f49-xk5v9\" (UID: \"ce9206ce-a65b-4da4-9391-e64748d0518d\") " pod="openshift-monitoring/telemeter-client-685f676f49-xk5v9"
Jul 02 13:45:56.293025 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:56.292897    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ce9206ce-a65b-4da4-9391-e64748d0518d-metrics-client-ca\") pod \"telemeter-client-685f676f49-xk5v9\" (UID: \"ce9206ce-a65b-4da4-9391-e64748d0518d\") " pod="openshift-monitoring/telemeter-client-685f676f49-xk5v9"
Jul 02 13:45:56.393590 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:56.393557    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/ce9206ce-a65b-4da4-9391-e64748d0518d-secret-telemeter-client\") pod \"telemeter-client-685f676f49-xk5v9\" (UID: \"ce9206ce-a65b-4da4-9391-e64748d0518d\") " pod="openshift-monitoring/telemeter-client-685f676f49-xk5v9"
Jul 02 13:45:56.393590 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:56.393596    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ce9206ce-a65b-4da4-9391-e64748d0518d-serving-certs-ca-bundle\") pod \"telemeter-client-685f676f49-xk5v9\" (UID: \"ce9206ce-a65b-4da4-9391-e64748d0518d\") " pod="openshift-monitoring/telemeter-client-685f676f49-xk5v9"
Jul 02 13:45:56.393781 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:56.393613    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/ce9206ce-a65b-4da4-9391-e64748d0518d-telemeter-client-tls\") pod \"telemeter-client-685f676f49-xk5v9\" (UID: \"ce9206ce-a65b-4da4-9391-e64748d0518d\") " pod="openshift-monitoring/telemeter-client-685f676f49-xk5v9"
Jul 02 13:45:56.393781 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:56.393630    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ce9206ce-a65b-4da4-9391-e64748d0518d-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-685f676f49-xk5v9\" (UID: \"ce9206ce-a65b-4da4-9391-e64748d0518d\") " pod="openshift-monitoring/telemeter-client-685f676f49-xk5v9"
Jul 02 13:45:56.393781 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:56.393658    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-74jfw\" (UniqueName: \"kubernetes.io/projected/ce9206ce-a65b-4da4-9391-e64748d0518d-kube-api-access-74jfw\") pod \"telemeter-client-685f676f49-xk5v9\" (UID: \"ce9206ce-a65b-4da4-9391-e64748d0518d\") " pod="openshift-monitoring/telemeter-client-685f676f49-xk5v9"
Jul 02 13:45:56.393781 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:56.393675    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ce9206ce-a65b-4da4-9391-e64748d0518d-metrics-client-ca\") pod \"telemeter-client-685f676f49-xk5v9\" (UID: \"ce9206ce-a65b-4da4-9391-e64748d0518d\") " pod="openshift-monitoring/telemeter-client-685f676f49-xk5v9"
Jul 02 13:45:56.393781 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:56.393722    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/ce9206ce-a65b-4da4-9391-e64748d0518d-federate-client-tls\") pod \"telemeter-client-685f676f49-xk5v9\" (UID: \"ce9206ce-a65b-4da4-9391-e64748d0518d\") " pod="openshift-monitoring/telemeter-client-685f676f49-xk5v9"
Jul 02 13:45:56.393781 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:56.393750    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ce9206ce-a65b-4da4-9391-e64748d0518d-telemeter-trusted-ca-bundle\") pod \"telemeter-client-685f676f49-xk5v9\" (UID: \"ce9206ce-a65b-4da4-9391-e64748d0518d\") " pod="openshift-monitoring/telemeter-client-685f676f49-xk5v9"
Jul 02 13:45:56.394504 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:56.394439    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ce9206ce-a65b-4da4-9391-e64748d0518d-serving-certs-ca-bundle\") pod \"telemeter-client-685f676f49-xk5v9\" (UID: \"ce9206ce-a65b-4da4-9391-e64748d0518d\") " pod="openshift-monitoring/telemeter-client-685f676f49-xk5v9"
Jul 02 13:45:56.394781 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:56.394569    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ce9206ce-a65b-4da4-9391-e64748d0518d-telemeter-trusted-ca-bundle\") pod \"telemeter-client-685f676f49-xk5v9\" (UID: \"ce9206ce-a65b-4da4-9391-e64748d0518d\") " pod="openshift-monitoring/telemeter-client-685f676f49-xk5v9"
Jul 02 13:45:56.394862 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:56.394787    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ce9206ce-a65b-4da4-9391-e64748d0518d-metrics-client-ca\") pod \"telemeter-client-685f676f49-xk5v9\" (UID: \"ce9206ce-a65b-4da4-9391-e64748d0518d\") " pod="openshift-monitoring/telemeter-client-685f676f49-xk5v9"
Jul 02 13:45:56.396280 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:56.396255    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/ce9206ce-a65b-4da4-9391-e64748d0518d-secret-telemeter-client\") pod \"telemeter-client-685f676f49-xk5v9\" (UID: \"ce9206ce-a65b-4da4-9391-e64748d0518d\") " pod="openshift-monitoring/telemeter-client-685f676f49-xk5v9"
Jul 02 13:45:56.396515 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:56.396492    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/ce9206ce-a65b-4da4-9391-e64748d0518d-federate-client-tls\") pod \"telemeter-client-685f676f49-xk5v9\" (UID: \"ce9206ce-a65b-4da4-9391-e64748d0518d\") " pod="openshift-monitoring/telemeter-client-685f676f49-xk5v9"
Jul 02 13:45:56.397061 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:56.397045    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ce9206ce-a65b-4da4-9391-e64748d0518d-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-685f676f49-xk5v9\" (UID: \"ce9206ce-a65b-4da4-9391-e64748d0518d\") " pod="openshift-monitoring/telemeter-client-685f676f49-xk5v9"
Jul 02 13:45:56.397224 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:56.397207    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/ce9206ce-a65b-4da4-9391-e64748d0518d-telemeter-client-tls\") pod \"telemeter-client-685f676f49-xk5v9\" (UID: \"ce9206ce-a65b-4da4-9391-e64748d0518d\") " pod="openshift-monitoring/telemeter-client-685f676f49-xk5v9"
Jul 02 13:45:56.403652 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:56.403627    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-74jfw\" (UniqueName: \"kubernetes.io/projected/ce9206ce-a65b-4da4-9391-e64748d0518d-kube-api-access-74jfw\") pod \"telemeter-client-685f676f49-xk5v9\" (UID: \"ce9206ce-a65b-4da4-9391-e64748d0518d\") " pod="openshift-monitoring/telemeter-client-685f676f49-xk5v9"
Jul 02 13:45:56.494705 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:56.494613    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/cd5df5ba-88e2-4926-b63c-f9fe0cdb8faa-monitoring-plugin-cert\") pod \"monitoring-plugin-cd8656b8-5v7b8\" (UID: \"cd5df5ba-88e2-4926-b63c-f9fe0cdb8faa\") " pod="openshift-monitoring/monitoring-plugin-cd8656b8-5v7b8"
Jul 02 13:45:56.497588 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:56.497560    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/cd5df5ba-88e2-4926-b63c-f9fe0cdb8faa-monitoring-plugin-cert\") pod \"monitoring-plugin-cd8656b8-5v7b8\" (UID: \"cd5df5ba-88e2-4926-b63c-f9fe0cdb8faa\") " pod="openshift-monitoring/monitoring-plugin-cd8656b8-5v7b8"
Jul 02 13:45:56.532047 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:56.532014    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-685f676f49-xk5v9"
Jul 02 13:45:56.640474 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:56.640395    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-cd8656b8-5v7b8"
Jul 02 13:45:56.684764 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:56.684737    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-685f676f49-xk5v9"]
Jul 02 13:45:56.736745 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:45:56.736709    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce9206ce_a65b_4da4_9391_e64748d0518d.slice/crio-fe74bd849d73f06944dcde7a7b3efbffe74415440aa44327fd947a1497ec1af8 WatchSource:0}: Error finding container fe74bd849d73f06944dcde7a7b3efbffe74415440aa44327fd947a1497ec1af8: Status 404 returned error can't find the container with id fe74bd849d73f06944dcde7a7b3efbffe74415440aa44327fd947a1497ec1af8
Jul 02 13:45:56.817480 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:56.817448    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-685f676f49-xk5v9" event={"ID":"ce9206ce-a65b-4da4-9391-e64748d0518d","Type":"ContainerStarted","Data":"fe74bd849d73f06944dcde7a7b3efbffe74415440aa44327fd947a1497ec1af8"}
Jul 02 13:45:56.883611 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:56.883581    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-cd8656b8-5v7b8"]
Jul 02 13:45:56.886877 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:45:56.886844    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd5df5ba_88e2_4926_b63c_f9fe0cdb8faa.slice/crio-54a000afdda99327024372db833a390bdc42e3847e02a638a41a3750fe2052ed WatchSource:0}: Error finding container 54a000afdda99327024372db833a390bdc42e3847e02a638a41a3750fe2052ed: Status 404 returned error can't find the container with id 54a000afdda99327024372db833a390bdc42e3847e02a638a41a3750fe2052ed
Jul 02 13:45:57.822256 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:57.822218    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-cd8656b8-5v7b8" event={"ID":"cd5df5ba-88e2-4926-b63c-f9fe0cdb8faa","Type":"ContainerStarted","Data":"54a000afdda99327024372db833a390bdc42e3847e02a638a41a3750fe2052ed"}
Jul 02 13:45:57.825708 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:57.825678    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-57879888b8-q8md5" event={"ID":"277131f0-f6c3-4190-b01f-dabc843e7403","Type":"ContainerStarted","Data":"e0255844cc48972c08143e24d9ae84407af0d62dfb2b4eb4f0dedc9845462ba2"}
Jul 02 13:45:57.825876 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:57.825751    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-57879888b8-q8md5" event={"ID":"277131f0-f6c3-4190-b01f-dabc843e7403","Type":"ContainerStarted","Data":"ecaacbf8ac76e5e91286c792b909ec579026748fe4436aebc16e76aeb70fa8af"}
Jul 02 13:45:57.825876 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:57.825796    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-57879888b8-q8md5" event={"ID":"277131f0-f6c3-4190-b01f-dabc843e7403","Type":"ContainerStarted","Data":"a295ce3b86cd6ca053ab46687a4d5bc09874387bb7be3d0df50c2f77eb7b28b4"}
Jul 02 13:45:57.825985 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:57.825884    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-57879888b8-q8md5"
Jul 02 13:45:57.851902 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:57.851849    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-57879888b8-q8md5" podStartSLOduration=2.745022344 podStartE2EDuration="5.851806275s" totalImagesPullingTime="3.106783931s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 13:45:52 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:45:57.84969379 +0000 UTC m=+123.028156013" watchObservedRunningTime="2026-07-02 13:45:57.851806275 +0000 UTC m=+123.030268460"
Jul 02 13:45:59.835272 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:59.835239    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-685f676f49-xk5v9" event={"ID":"ce9206ce-a65b-4da4-9391-e64748d0518d","Type":"ContainerStarted","Data":"b8df015425207e1662d09268af2bb8c42f9e4659b69f0ac3334e70d4e8e5717e"}
Jul 02 13:45:59.836900 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:59.836865    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-cd8656b8-5v7b8" event={"ID":"cd5df5ba-88e2-4926-b63c-f9fe0cdb8faa","Type":"ContainerStarted","Data":"6394842f46e0409fd83d7dff55b000959f121d51c7b2cdac7690d48c8f1e45b2"}
Jul 02 13:45:59.837211 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:59.837159    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-cd8656b8-5v7b8"
Jul 02 13:45:59.842757 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:59.842734    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-cd8656b8-5v7b8"
Jul 02 13:45:59.859854 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:45:59.857349    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-cd8656b8-5v7b8" podStartSLOduration=2.755041557 podStartE2EDuration="4.857330102s" totalImagesPullingTime="2.102288545s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 13:45:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:45:59.854398494 +0000 UTC m=+125.032860683" watchObservedRunningTime="2026-07-02 13:45:59.857330102 +0000 UTC m=+125.035792286"
Jul 02 13:46:00.842242 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:46:00.842206    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-685f676f49-xk5v9" event={"ID":"ce9206ce-a65b-4da4-9391-e64748d0518d","Type":"ContainerStarted","Data":"59159f0a89798462d3b5cc35ccb954c20e39f54225f396f97a5f63b602f94294"}
Jul 02 13:46:00.842716 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:46:00.842305    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-685f676f49-xk5v9" event={"ID":"ce9206ce-a65b-4da4-9391-e64748d0518d","Type":"ContainerStarted","Data":"b9a954003e1a7371c863bf10d121386edcd5bec644a4e4c91e2dc152a6678d23"}
Jul 02 13:46:00.869341 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:46:00.869282    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-685f676f49-xk5v9" podStartSLOduration=1.6462258859999999 podStartE2EDuration="4.869264894s" totalImagesPullingTime="3.223039008s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 13:45:56 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:46:00.867230579 +0000 UTC m=+126.045692775" watchObservedRunningTime="2026-07-02 13:46:00.869264894 +0000 UTC m=+126.047727078"
Jul 02 13:46:01.602087 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:46:01.602048    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-59755cb769-vtqqc"]
Jul 02 13:46:01.604359 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:46:01.604341    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-59755cb769-vtqqc"
Jul 02 13:46:01.608410 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:46:01.608386    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 02 13:46:01.608410 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:46:01.608388    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 02 13:46:01.609607 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:46:01.609589    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 02 13:46:01.609764 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:46:01.609749    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 02 13:46:01.610048 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:46:01.610030    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 02 13:46:01.611184 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:46:01.611137    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 02 13:46:01.611512 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:46:01.611486    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 02 13:46:01.611512 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:46:01.611504    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-lr5m4\""
Jul 02 13:46:01.614156 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:46:01.614140    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 02 13:46:01.621104 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:46:01.621081    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-59755cb769-vtqqc"]
Jul 02 13:46:01.643772 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:46:01.643735    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6csh2\" (UniqueName: \"kubernetes.io/projected/3258e405-39f8-4a2b-83a7-17c4a6ebd33c-kube-api-access-6csh2\") pod \"console-59755cb769-vtqqc\" (UID: \"3258e405-39f8-4a2b-83a7-17c4a6ebd33c\") " pod="openshift-console/console-59755cb769-vtqqc"
Jul 02 13:46:01.643772 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:46:01.643773    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3258e405-39f8-4a2b-83a7-17c4a6ebd33c-trusted-ca-bundle\") pod \"console-59755cb769-vtqqc\" (UID: \"3258e405-39f8-4a2b-83a7-17c4a6ebd33c\") " pod="openshift-console/console-59755cb769-vtqqc"
Jul 02 13:46:01.644014 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:46:01.643802    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3258e405-39f8-4a2b-83a7-17c4a6ebd33c-console-config\") pod \"console-59755cb769-vtqqc\" (UID: \"3258e405-39f8-4a2b-83a7-17c4a6ebd33c\") " pod="openshift-console/console-59755cb769-vtqqc"
Jul 02 13:46:01.644014 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:46:01.643876    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3258e405-39f8-4a2b-83a7-17c4a6ebd33c-oauth-serving-cert\") pod \"console-59755cb769-vtqqc\" (UID: \"3258e405-39f8-4a2b-83a7-17c4a6ebd33c\") " pod="openshift-console/console-59755cb769-vtqqc"
Jul 02 13:46:01.644014 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:46:01.643974    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3258e405-39f8-4a2b-83a7-17c4a6ebd33c-console-oauth-config\") pod \"console-59755cb769-vtqqc\" (UID: \"3258e405-39f8-4a2b-83a7-17c4a6ebd33c\") " pod="openshift-console/console-59755cb769-vtqqc"
Jul 02 13:46:01.644014 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:46:01.644010    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3258e405-39f8-4a2b-83a7-17c4a6ebd33c-service-ca\") pod \"console-59755cb769-vtqqc\" (UID: \"3258e405-39f8-4a2b-83a7-17c4a6ebd33c\") " pod="openshift-console/console-59755cb769-vtqqc"
Jul 02 13:46:01.644236 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:46:01.644084    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3258e405-39f8-4a2b-83a7-17c4a6ebd33c-console-serving-cert\") pod \"console-59755cb769-vtqqc\" (UID: \"3258e405-39f8-4a2b-83a7-17c4a6ebd33c\") " pod="openshift-console/console-59755cb769-vtqqc"
Jul 02 13:46:01.745419 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:46:01.745386    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3258e405-39f8-4a2b-83a7-17c4a6ebd33c-console-serving-cert\") pod \"console-59755cb769-vtqqc\" (UID: \"3258e405-39f8-4a2b-83a7-17c4a6ebd33c\") " pod="openshift-console/console-59755cb769-vtqqc"
Jul 02 13:46:01.745419 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:46:01.745423    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6csh2\" (UniqueName: \"kubernetes.io/projected/3258e405-39f8-4a2b-83a7-17c4a6ebd33c-kube-api-access-6csh2\") pod \"console-59755cb769-vtqqc\" (UID: \"3258e405-39f8-4a2b-83a7-17c4a6ebd33c\") " pod="openshift-console/console-59755cb769-vtqqc"
Jul 02 13:46:01.745662 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:46:01.745444    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3258e405-39f8-4a2b-83a7-17c4a6ebd33c-trusted-ca-bundle\") pod \"console-59755cb769-vtqqc\" (UID: \"3258e405-39f8-4a2b-83a7-17c4a6ebd33c\") " pod="openshift-console/console-59755cb769-vtqqc"
Jul 02 13:46:01.745662 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:46:01.745565    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3258e405-39f8-4a2b-83a7-17c4a6ebd33c-console-config\") pod \"console-59755cb769-vtqqc\" (UID: \"3258e405-39f8-4a2b-83a7-17c4a6ebd33c\") " pod="openshift-console/console-59755cb769-vtqqc"
Jul 02 13:46:01.745662 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:46:01.745600    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3258e405-39f8-4a2b-83a7-17c4a6ebd33c-oauth-serving-cert\") pod \"console-59755cb769-vtqqc\" (UID: \"3258e405-39f8-4a2b-83a7-17c4a6ebd33c\") " pod="openshift-console/console-59755cb769-vtqqc"
Jul 02 13:46:01.745662 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:46:01.745652    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3258e405-39f8-4a2b-83a7-17c4a6ebd33c-console-oauth-config\") pod \"console-59755cb769-vtqqc\" (UID: \"3258e405-39f8-4a2b-83a7-17c4a6ebd33c\") " pod="openshift-console/console-59755cb769-vtqqc"
Jul 02 13:46:01.745869 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:46:01.745676    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3258e405-39f8-4a2b-83a7-17c4a6ebd33c-service-ca\") pod \"console-59755cb769-vtqqc\" (UID: \"3258e405-39f8-4a2b-83a7-17c4a6ebd33c\") " pod="openshift-console/console-59755cb769-vtqqc"
Jul 02 13:46:01.746372 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:46:01.746319    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3258e405-39f8-4a2b-83a7-17c4a6ebd33c-console-config\") pod \"console-59755cb769-vtqqc\" (UID: \"3258e405-39f8-4a2b-83a7-17c4a6ebd33c\") " pod="openshift-console/console-59755cb769-vtqqc"
Jul 02 13:46:01.746372 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:46:01.746364    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3258e405-39f8-4a2b-83a7-17c4a6ebd33c-service-ca\") pod \"console-59755cb769-vtqqc\" (UID: \"3258e405-39f8-4a2b-83a7-17c4a6ebd33c\") " pod="openshift-console/console-59755cb769-vtqqc"
Jul 02 13:46:01.746536 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:46:01.746384    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3258e405-39f8-4a2b-83a7-17c4a6ebd33c-trusted-ca-bundle\") pod \"console-59755cb769-vtqqc\" (UID: \"3258e405-39f8-4a2b-83a7-17c4a6ebd33c\") " pod="openshift-console/console-59755cb769-vtqqc"
Jul 02 13:46:01.746536 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:46:01.746410    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3258e405-39f8-4a2b-83a7-17c4a6ebd33c-oauth-serving-cert\") pod \"console-59755cb769-vtqqc\" (UID: \"3258e405-39f8-4a2b-83a7-17c4a6ebd33c\") " pod="openshift-console/console-59755cb769-vtqqc"
Jul 02 13:46:01.748111 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:46:01.748095    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3258e405-39f8-4a2b-83a7-17c4a6ebd33c-console-oauth-config\") pod \"console-59755cb769-vtqqc\" (UID: \"3258e405-39f8-4a2b-83a7-17c4a6ebd33c\") " pod="openshift-console/console-59755cb769-vtqqc"
Jul 02 13:46:01.748216 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:46:01.748199    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3258e405-39f8-4a2b-83a7-17c4a6ebd33c-console-serving-cert\") pod \"console-59755cb769-vtqqc\" (UID: \"3258e405-39f8-4a2b-83a7-17c4a6ebd33c\") " pod="openshift-console/console-59755cb769-vtqqc"
Jul 02 13:46:01.755240 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:46:01.755209    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6csh2\" (UniqueName: \"kubernetes.io/projected/3258e405-39f8-4a2b-83a7-17c4a6ebd33c-kube-api-access-6csh2\") pod \"console-59755cb769-vtqqc\" (UID: \"3258e405-39f8-4a2b-83a7-17c4a6ebd33c\") " pod="openshift-console/console-59755cb769-vtqqc"
Jul 02 13:46:01.914298 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:46:01.914209    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-59755cb769-vtqqc"
Jul 02 13:46:02.050429 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:46:02.050399    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-59755cb769-vtqqc"]
Jul 02 13:46:02.054610 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:46:02.054583    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3258e405_39f8_4a2b_83a7_17c4a6ebd33c.slice/crio-b4e812435af0a9b476b0bb855034b6ce73b80eae89b5bd131847d6e73c0fa383 WatchSource:0}: Error finding container b4e812435af0a9b476b0bb855034b6ce73b80eae89b5bd131847d6e73c0fa383: Status 404 returned error can't find the container with id b4e812435af0a9b476b0bb855034b6ce73b80eae89b5bd131847d6e73c0fa383
Jul 02 13:46:02.856266 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:46:02.856230    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-59755cb769-vtqqc" event={"ID":"3258e405-39f8-4a2b-83a7-17c4a6ebd33c","Type":"ContainerStarted","Data":"b4e812435af0a9b476b0bb855034b6ce73b80eae89b5bd131847d6e73c0fa383"}
Jul 02 13:46:03.838043 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:46:03.838016    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-57879888b8-q8md5"
Jul 02 13:46:05.076069 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:46:05.076022    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ad2afabf-42cb-4619-b249-a2b011e32565-metrics-certs\") pod \"network-metrics-daemon-29552\" (UID: \"ad2afabf-42cb-4619-b249-a2b011e32565\") " pod="openshift-multus/network-metrics-daemon-29552"
Jul 02 13:46:05.078445 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:46:05.078425    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ad2afabf-42cb-4619-b249-a2b011e32565-metrics-certs\") pod \"network-metrics-daemon-29552\" (UID: \"ad2afabf-42cb-4619-b249-a2b011e32565\") " pod="openshift-multus/network-metrics-daemon-29552"
Jul 02 13:46:05.227047 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:46:05.226956    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-f4vwh\""
Jul 02 13:46:05.234463 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:46:05.234440    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29552"
Jul 02 13:46:05.376794 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:46:05.376724    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-29552"]
Jul 02 13:46:05.379478 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:46:05.379450    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad2afabf_42cb_4619_b249_a2b011e32565.slice/crio-942ce1fba02a2d54588b97553a75931f83d35d4944faf55b76f1d5f813c994d6 WatchSource:0}: Error finding container 942ce1fba02a2d54588b97553a75931f83d35d4944faf55b76f1d5f813c994d6: Status 404 returned error can't find the container with id 942ce1fba02a2d54588b97553a75931f83d35d4944faf55b76f1d5f813c994d6
Jul 02 13:46:05.868263 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:46:05.868228    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-59755cb769-vtqqc" event={"ID":"3258e405-39f8-4a2b-83a7-17c4a6ebd33c","Type":"ContainerStarted","Data":"66220731a16bf519d5ae07f1579c02d57bb4b38ab2e9caffc6c80776e05a4174"}
Jul 02 13:46:05.869438 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:46:05.869409    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-29552" event={"ID":"ad2afabf-42cb-4619-b249-a2b011e32565","Type":"ContainerStarted","Data":"942ce1fba02a2d54588b97553a75931f83d35d4944faf55b76f1d5f813c994d6"}
Jul 02 13:46:05.891217 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:46:05.891167    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-59755cb769-vtqqc" podStartSLOduration=2.101423493 podStartE2EDuration="4.891143738s" totalImagesPullingTime="2.789720245s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 13:46:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:46:05.88969865 +0000 UTC m=+131.068160847" watchObservedRunningTime="2026-07-02 13:46:05.891143738 +0000 UTC m=+131.069605922"
Jul 02 13:46:06.874267 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:46:06.874226    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-29552" event={"ID":"ad2afabf-42cb-4619-b249-a2b011e32565","Type":"ContainerStarted","Data":"cec8a196f2a24d902bedf348300415b62b6ddd724660c8cc07a76ce6ba75c111"}
Jul 02 13:46:06.874654 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:46:06.874321    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-29552" event={"ID":"ad2afabf-42cb-4619-b249-a2b011e32565","Type":"ContainerStarted","Data":"1af8f94361ad571697f0c5a1eeae4610990e0433ab1ea2f372fab2870e8523a3"}
Jul 02 13:46:06.896669 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:46:06.896610    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-29552" podStartSLOduration=130.926257664 podStartE2EDuration="2m11.896593779s" totalImagesPullingTime="970.336115ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 13:43:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:46:06.895680254 +0000 UTC m=+132.074142438" watchObservedRunningTime="2026-07-02 13:46:06.896593779 +0000 UTC m=+132.075055963"
Jul 02 13:46:11.914533 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:46:11.914495    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-59755cb769-vtqqc"
Jul 02 13:46:11.914946 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:46:11.914564    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-59755cb769-vtqqc"
Jul 02 13:46:11.919917 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:46:11.919887    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-59755cb769-vtqqc"
Jul 02 13:46:12.898465 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:46:12.898438    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-59755cb769-vtqqc"
Jul 02 13:47:20.268639 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:47:20.268560    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-59755cb769-vtqqc"]
Jul 02 13:47:45.294643 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:47:45.294575    2558 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-59755cb769-vtqqc" podUID="3258e405-39f8-4a2b-83a7-17c4a6ebd33c" containerName="console" containerID="cri-o://66220731a16bf519d5ae07f1579c02d57bb4b38ab2e9caffc6c80776e05a4174" gracePeriod=15
Jul 02 13:47:45.536535 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:47:45.536511    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-59755cb769-vtqqc"
Jul 02 13:47:45.681839 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:47:45.681792    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3258e405-39f8-4a2b-83a7-17c4a6ebd33c-console-oauth-config\") pod \"3258e405-39f8-4a2b-83a7-17c4a6ebd33c\" (UID: \"3258e405-39f8-4a2b-83a7-17c4a6ebd33c\") "
Jul 02 13:47:45.682011 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:47:45.681881    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3258e405-39f8-4a2b-83a7-17c4a6ebd33c-service-ca\") pod \"3258e405-39f8-4a2b-83a7-17c4a6ebd33c\" (UID: \"3258e405-39f8-4a2b-83a7-17c4a6ebd33c\") "
Jul 02 13:47:45.682011 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:47:45.681910    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3258e405-39f8-4a2b-83a7-17c4a6ebd33c-oauth-serving-cert\") pod \"3258e405-39f8-4a2b-83a7-17c4a6ebd33c\" (UID: \"3258e405-39f8-4a2b-83a7-17c4a6ebd33c\") "
Jul 02 13:47:45.682011 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:47:45.681942    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3258e405-39f8-4a2b-83a7-17c4a6ebd33c-console-config\") pod \"3258e405-39f8-4a2b-83a7-17c4a6ebd33c\" (UID: \"3258e405-39f8-4a2b-83a7-17c4a6ebd33c\") "
Jul 02 13:47:45.682011 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:47:45.681977    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3258e405-39f8-4a2b-83a7-17c4a6ebd33c-trusted-ca-bundle\") pod \"3258e405-39f8-4a2b-83a7-17c4a6ebd33c\" (UID: \"3258e405-39f8-4a2b-83a7-17c4a6ebd33c\") "
Jul 02 13:47:45.682011 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:47:45.682003    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3258e405-39f8-4a2b-83a7-17c4a6ebd33c-console-serving-cert\") pod \"3258e405-39f8-4a2b-83a7-17c4a6ebd33c\" (UID: \"3258e405-39f8-4a2b-83a7-17c4a6ebd33c\") "
Jul 02 13:47:45.682257 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:47:45.682053    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6csh2\" (UniqueName: \"kubernetes.io/projected/3258e405-39f8-4a2b-83a7-17c4a6ebd33c-kube-api-access-6csh2\") pod \"3258e405-39f8-4a2b-83a7-17c4a6ebd33c\" (UID: \"3258e405-39f8-4a2b-83a7-17c4a6ebd33c\") "
Jul 02 13:47:45.682407 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:47:45.682380    2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3258e405-39f8-4a2b-83a7-17c4a6ebd33c-service-ca" (OuterVolumeSpecName: "service-ca") pod "3258e405-39f8-4a2b-83a7-17c4a6ebd33c" (UID: "3258e405-39f8-4a2b-83a7-17c4a6ebd33c"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 02 13:47:45.682407 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:47:45.682389    2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3258e405-39f8-4a2b-83a7-17c4a6ebd33c-console-config" (OuterVolumeSpecName: "console-config") pod "3258e405-39f8-4a2b-83a7-17c4a6ebd33c" (UID: "3258e405-39f8-4a2b-83a7-17c4a6ebd33c"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 02 13:47:45.682610 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:47:45.682374    2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3258e405-39f8-4a2b-83a7-17c4a6ebd33c-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "3258e405-39f8-4a2b-83a7-17c4a6ebd33c" (UID: "3258e405-39f8-4a2b-83a7-17c4a6ebd33c"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 02 13:47:45.682610 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:47:45.682427    2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3258e405-39f8-4a2b-83a7-17c4a6ebd33c-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "3258e405-39f8-4a2b-83a7-17c4a6ebd33c" (UID: "3258e405-39f8-4a2b-83a7-17c4a6ebd33c"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 02 13:47:45.684343 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:47:45.684319    2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3258e405-39f8-4a2b-83a7-17c4a6ebd33c-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "3258e405-39f8-4a2b-83a7-17c4a6ebd33c" (UID: "3258e405-39f8-4a2b-83a7-17c4a6ebd33c"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 02 13:47:45.684430 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:47:45.684342    2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3258e405-39f8-4a2b-83a7-17c4a6ebd33c-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "3258e405-39f8-4a2b-83a7-17c4a6ebd33c" (UID: "3258e405-39f8-4a2b-83a7-17c4a6ebd33c"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 02 13:47:45.684430 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:47:45.684344    2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3258e405-39f8-4a2b-83a7-17c4a6ebd33c-kube-api-access-6csh2" (OuterVolumeSpecName: "kube-api-access-6csh2") pod "3258e405-39f8-4a2b-83a7-17c4a6ebd33c" (UID: "3258e405-39f8-4a2b-83a7-17c4a6ebd33c"). InnerVolumeSpecName "kube-api-access-6csh2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 13:47:45.783555 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:47:45.783508    2558 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3258e405-39f8-4a2b-83a7-17c4a6ebd33c-console-config\") on node \"ip-10-0-133-209.ec2.internal\" DevicePath \"\""
Jul 02 13:47:45.783555 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:47:45.783549    2558 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3258e405-39f8-4a2b-83a7-17c4a6ebd33c-trusted-ca-bundle\") on node \"ip-10-0-133-209.ec2.internal\" DevicePath \"\""
Jul 02 13:47:45.783555 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:47:45.783561    2558 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3258e405-39f8-4a2b-83a7-17c4a6ebd33c-console-serving-cert\") on node \"ip-10-0-133-209.ec2.internal\" DevicePath \"\""
Jul 02 13:47:45.783555 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:47:45.783570    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6csh2\" (UniqueName: \"kubernetes.io/projected/3258e405-39f8-4a2b-83a7-17c4a6ebd33c-kube-api-access-6csh2\") on node \"ip-10-0-133-209.ec2.internal\" DevicePath \"\""
Jul 02 13:47:45.783852 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:47:45.783578    2558 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3258e405-39f8-4a2b-83a7-17c4a6ebd33c-console-oauth-config\") on node \"ip-10-0-133-209.ec2.internal\" DevicePath \"\""
Jul 02 13:47:45.783852 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:47:45.783588    2558 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3258e405-39f8-4a2b-83a7-17c4a6ebd33c-service-ca\") on node \"ip-10-0-133-209.ec2.internal\" DevicePath \"\""
Jul 02 13:47:45.783852 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:47:45.783596    2558 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3258e405-39f8-4a2b-83a7-17c4a6ebd33c-oauth-serving-cert\") on node \"ip-10-0-133-209.ec2.internal\" DevicePath \"\""
Jul 02 13:47:46.227703 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:47:46.227671    2558 generic.go:356] "Generic (PLEG): container finished" podID="3258e405-39f8-4a2b-83a7-17c4a6ebd33c" containerID="66220731a16bf519d5ae07f1579c02d57bb4b38ab2e9caffc6c80776e05a4174" exitCode=0
Jul 02 13:47:46.227890 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:47:46.227715    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-59755cb769-vtqqc"
Jul 02 13:47:46.227890 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:47:46.227736    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-59755cb769-vtqqc" event={"ID":"3258e405-39f8-4a2b-83a7-17c4a6ebd33c","Type":"ContainerDied","Data":"66220731a16bf519d5ae07f1579c02d57bb4b38ab2e9caffc6c80776e05a4174"}
Jul 02 13:47:46.227890 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:47:46.227772    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-59755cb769-vtqqc" event={"ID":"3258e405-39f8-4a2b-83a7-17c4a6ebd33c","Type":"ContainerDied","Data":"b4e812435af0a9b476b0bb855034b6ce73b80eae89b5bd131847d6e73c0fa383"}
Jul 02 13:47:46.227890 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:47:46.227792    2558 scope.go:117] "RemoveContainer" containerID="66220731a16bf519d5ae07f1579c02d57bb4b38ab2e9caffc6c80776e05a4174"
Jul 02 13:47:46.236994 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:47:46.236972    2558 scope.go:117] "RemoveContainer" containerID="66220731a16bf519d5ae07f1579c02d57bb4b38ab2e9caffc6c80776e05a4174"
Jul 02 13:47:46.237256 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:47:46.237237    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"66220731a16bf519d5ae07f1579c02d57bb4b38ab2e9caffc6c80776e05a4174\": container with ID starting with 66220731a16bf519d5ae07f1579c02d57bb4b38ab2e9caffc6c80776e05a4174 not found: ID does not exist" containerID="66220731a16bf519d5ae07f1579c02d57bb4b38ab2e9caffc6c80776e05a4174"
Jul 02 13:47:46.237304 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:47:46.237264    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"66220731a16bf519d5ae07f1579c02d57bb4b38ab2e9caffc6c80776e05a4174"} err="failed to get container status \"66220731a16bf519d5ae07f1579c02d57bb4b38ab2e9caffc6c80776e05a4174\": rpc error: code = NotFound desc = could not find container \"66220731a16bf519d5ae07f1579c02d57bb4b38ab2e9caffc6c80776e05a4174\": container with ID starting with 66220731a16bf519d5ae07f1579c02d57bb4b38ab2e9caffc6c80776e05a4174 not found: ID does not exist"
Jul 02 13:47:46.250426 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:47:46.250399    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-59755cb769-vtqqc"]
Jul 02 13:47:46.254979 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:47:46.254958    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-59755cb769-vtqqc"]
Jul 02 13:47:47.216565 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:47:47.216527    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3258e405-39f8-4a2b-83a7-17c4a6ebd33c" path="/var/lib/kubelet/pods/3258e405-39f8-4a2b-83a7-17c4a6ebd33c/volumes"
Jul 02 13:48:55.156878 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:48:55.156849    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8f2ll_3310d1f7-1643-4017-9edf-71fa702d2615/ovn-acl-logging/0.log"
Jul 02 13:48:55.162249 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:48:55.162227    2558 kubelet.go:1648] "Image garbage collection succeeded"
Jul 02 13:48:55.162461 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:48:55.162445    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8f2ll_3310d1f7-1643-4017-9edf-71fa702d2615/ovn-acl-logging/0.log"
Jul 02 13:50:42.839876 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:50:42.839848    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/llmisvc-controller-manager-67d95fdc48-2frl4"]
Jul 02 13:50:42.840305 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:50:42.840119    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3258e405-39f8-4a2b-83a7-17c4a6ebd33c" containerName="console"
Jul 02 13:50:42.840305 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:50:42.840130    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="3258e405-39f8-4a2b-83a7-17c4a6ebd33c" containerName="console"
Jul 02 13:50:42.840305 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:50:42.840178    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="3258e405-39f8-4a2b-83a7-17c4a6ebd33c" containerName="console"
Jul 02 13:50:42.842026 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:50:42.842010    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/llmisvc-controller-manager-67d95fdc48-2frl4"
Jul 02 13:50:42.844673 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:50:42.844653    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"openshift-service-ca.crt\""
Jul 02 13:50:42.844673 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:50:42.844660    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"llmisvc-webhook-server-cert\""
Jul 02 13:50:42.845678 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:50:42.845663    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"kube-root-ca.crt\""
Jul 02 13:50:42.845678 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:50:42.845676    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"llmisvc-controller-manager-dockercfg-qpkrd\""
Jul 02 13:50:42.857905 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:50:42.857883    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/llmisvc-controller-manager-67d95fdc48-2frl4"]
Jul 02 13:50:42.948380 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:50:42.948342    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/07f30b08-ece7-479a-9c59-839506520dc5-cert\") pod \"llmisvc-controller-manager-67d95fdc48-2frl4\" (UID: \"07f30b08-ece7-479a-9c59-839506520dc5\") " pod="kserve/llmisvc-controller-manager-67d95fdc48-2frl4"
Jul 02 13:50:42.948553 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:50:42.948409    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nbwdq\" (UniqueName: \"kubernetes.io/projected/07f30b08-ece7-479a-9c59-839506520dc5-kube-api-access-nbwdq\") pod \"llmisvc-controller-manager-67d95fdc48-2frl4\" (UID: \"07f30b08-ece7-479a-9c59-839506520dc5\") " pod="kserve/llmisvc-controller-manager-67d95fdc48-2frl4"
Jul 02 13:50:43.049236 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:50:43.049200    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/07f30b08-ece7-479a-9c59-839506520dc5-cert\") pod \"llmisvc-controller-manager-67d95fdc48-2frl4\" (UID: \"07f30b08-ece7-479a-9c59-839506520dc5\") " pod="kserve/llmisvc-controller-manager-67d95fdc48-2frl4"
Jul 02 13:50:43.049405 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:50:43.049251    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nbwdq\" (UniqueName: \"kubernetes.io/projected/07f30b08-ece7-479a-9c59-839506520dc5-kube-api-access-nbwdq\") pod \"llmisvc-controller-manager-67d95fdc48-2frl4\" (UID: \"07f30b08-ece7-479a-9c59-839506520dc5\") " pod="kserve/llmisvc-controller-manager-67d95fdc48-2frl4"
Jul 02 13:50:43.049405 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:50:43.049348    2558 secret.go:189] Couldn't get secret kserve/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found
Jul 02 13:50:43.049481 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:50:43.049425    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/07f30b08-ece7-479a-9c59-839506520dc5-cert podName:07f30b08-ece7-479a-9c59-839506520dc5 nodeName:}" failed. No retries permitted until 2026-07-02 13:50:43.549408302 +0000 UTC m=+408.727870464 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/07f30b08-ece7-479a-9c59-839506520dc5-cert") pod "llmisvc-controller-manager-67d95fdc48-2frl4" (UID: "07f30b08-ece7-479a-9c59-839506520dc5") : secret "llmisvc-webhook-server-cert" not found
Jul 02 13:50:43.061041 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:50:43.061006    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nbwdq\" (UniqueName: \"kubernetes.io/projected/07f30b08-ece7-479a-9c59-839506520dc5-kube-api-access-nbwdq\") pod \"llmisvc-controller-manager-67d95fdc48-2frl4\" (UID: \"07f30b08-ece7-479a-9c59-839506520dc5\") " pod="kserve/llmisvc-controller-manager-67d95fdc48-2frl4"
Jul 02 13:50:43.554803 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:50:43.554764    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/07f30b08-ece7-479a-9c59-839506520dc5-cert\") pod \"llmisvc-controller-manager-67d95fdc48-2frl4\" (UID: \"07f30b08-ece7-479a-9c59-839506520dc5\") " pod="kserve/llmisvc-controller-manager-67d95fdc48-2frl4"
Jul 02 13:50:43.557256 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:50:43.557234    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/07f30b08-ece7-479a-9c59-839506520dc5-cert\") pod \"llmisvc-controller-manager-67d95fdc48-2frl4\" (UID: \"07f30b08-ece7-479a-9c59-839506520dc5\") " pod="kserve/llmisvc-controller-manager-67d95fdc48-2frl4"
Jul 02 13:50:43.752075 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:50:43.752035    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/llmisvc-controller-manager-67d95fdc48-2frl4"
Jul 02 13:50:43.879094 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:50:43.879062    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/llmisvc-controller-manager-67d95fdc48-2frl4"]
Jul 02 13:50:43.882066 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:50:43.882034    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod07f30b08_ece7_479a_9c59_839506520dc5.slice/crio-7b965a6e3381f59822ba9cb284c344bff1da66c3b39557aa25162dfd7ff295a2 WatchSource:0}: Error finding container 7b965a6e3381f59822ba9cb284c344bff1da66c3b39557aa25162dfd7ff295a2: Status 404 returned error can't find the container with id 7b965a6e3381f59822ba9cb284c344bff1da66c3b39557aa25162dfd7ff295a2
Jul 02 13:50:43.883511 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:50:43.883489    2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 02 13:50:44.850218 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:50:44.850188    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-67d95fdc48-2frl4" event={"ID":"07f30b08-ece7-479a-9c59-839506520dc5","Type":"ContainerStarted","Data":"7b965a6e3381f59822ba9cb284c344bff1da66c3b39557aa25162dfd7ff295a2"}
Jul 02 13:50:46.860183 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:50:46.860144    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-67d95fdc48-2frl4" event={"ID":"07f30b08-ece7-479a-9c59-839506520dc5","Type":"ContainerStarted","Data":"d37cd10fe7c486b64bddcf38432f431cbab655b59c54dad54a71893c229e92ff"}
Jul 02 13:50:46.860589 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:50:46.860276    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/llmisvc-controller-manager-67d95fdc48-2frl4"
Jul 02 13:50:46.879267 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:50:46.879215    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/llmisvc-controller-manager-67d95fdc48-2frl4" podStartSLOduration=2.761118782 podStartE2EDuration="4.879198559s" totalImagesPullingTime="2.118079777s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 13:50:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:50:46.877486811 +0000 UTC m=+412.055948994" watchObservedRunningTime="2026-07-02 13:50:46.879198559 +0000 UTC m=+412.057660742"
Jul 02 13:51:17.865937 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:51:17.865857    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/llmisvc-controller-manager-67d95fdc48-2frl4"
Jul 02 13:53:41.790325 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:53:41.790293    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-40523-predictor-57f4887bdf-rkfb5"]
Jul 02 13:53:41.793400 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:53:41.793385    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-40523-predictor-57f4887bdf-rkfb5"
Jul 02 13:53:41.796389 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:53:41.796368    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 02 13:53:41.796501 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:53:41.796370    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-kxrcm\""
Jul 02 13:53:41.797501 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:53:41.797482    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-40523-predictor-serving-cert\""
Jul 02 13:53:41.797615 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:53:41.797523    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-40523-kube-rbac-proxy-sar-config\""
Jul 02 13:53:41.797615 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:53:41.797532    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 02 13:53:41.804944 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:53:41.804924    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-40523-predictor-57f4887bdf-rkfb5"]
Jul 02 13:53:41.943862 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:53:41.943803    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/87f3a61b-c663-45a0-ae75-64c42cbe1695-proxy-tls\") pod \"error-404-isvc-40523-predictor-57f4887bdf-rkfb5\" (UID: \"87f3a61b-c663-45a0-ae75-64c42cbe1695\") " pod="kserve-ci-e2e-test/error-404-isvc-40523-predictor-57f4887bdf-rkfb5"
Jul 02 13:53:41.944047 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:53:41.943922    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/87f3a61b-c663-45a0-ae75-64c42cbe1695-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-40523-predictor-57f4887bdf-rkfb5\" (UID: \"87f3a61b-c663-45a0-ae75-64c42cbe1695\") " pod="kserve-ci-e2e-test/error-404-isvc-40523-predictor-57f4887bdf-rkfb5"
Jul 02 13:53:41.944047 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:53:41.943960    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-59qvl\" (UniqueName: \"kubernetes.io/projected/87f3a61b-c663-45a0-ae75-64c42cbe1695-kube-api-access-59qvl\") pod \"error-404-isvc-40523-predictor-57f4887bdf-rkfb5\" (UID: \"87f3a61b-c663-45a0-ae75-64c42cbe1695\") " pod="kserve-ci-e2e-test/error-404-isvc-40523-predictor-57f4887bdf-rkfb5"
Jul 02 13:53:42.044455 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:53:42.044370    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/87f3a61b-c663-45a0-ae75-64c42cbe1695-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-40523-predictor-57f4887bdf-rkfb5\" (UID: \"87f3a61b-c663-45a0-ae75-64c42cbe1695\") " pod="kserve-ci-e2e-test/error-404-isvc-40523-predictor-57f4887bdf-rkfb5"
Jul 02 13:53:42.044455 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:53:42.044415    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-59qvl\" (UniqueName: \"kubernetes.io/projected/87f3a61b-c663-45a0-ae75-64c42cbe1695-kube-api-access-59qvl\") pod \"error-404-isvc-40523-predictor-57f4887bdf-rkfb5\" (UID: \"87f3a61b-c663-45a0-ae75-64c42cbe1695\") " pod="kserve-ci-e2e-test/error-404-isvc-40523-predictor-57f4887bdf-rkfb5"
Jul 02 13:53:42.044687 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:53:42.044464    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/87f3a61b-c663-45a0-ae75-64c42cbe1695-proxy-tls\") pod \"error-404-isvc-40523-predictor-57f4887bdf-rkfb5\" (UID: \"87f3a61b-c663-45a0-ae75-64c42cbe1695\") " pod="kserve-ci-e2e-test/error-404-isvc-40523-predictor-57f4887bdf-rkfb5"
Jul 02 13:53:42.045158 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:53:42.045139    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/87f3a61b-c663-45a0-ae75-64c42cbe1695-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-40523-predictor-57f4887bdf-rkfb5\" (UID: \"87f3a61b-c663-45a0-ae75-64c42cbe1695\") " pod="kserve-ci-e2e-test/error-404-isvc-40523-predictor-57f4887bdf-rkfb5"
Jul 02 13:53:42.047091 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:53:42.047073    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/87f3a61b-c663-45a0-ae75-64c42cbe1695-proxy-tls\") pod \"error-404-isvc-40523-predictor-57f4887bdf-rkfb5\" (UID: \"87f3a61b-c663-45a0-ae75-64c42cbe1695\") " pod="kserve-ci-e2e-test/error-404-isvc-40523-predictor-57f4887bdf-rkfb5"
Jul 02 13:53:42.054916 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:53:42.054891    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-59qvl\" (UniqueName: \"kubernetes.io/projected/87f3a61b-c663-45a0-ae75-64c42cbe1695-kube-api-access-59qvl\") pod \"error-404-isvc-40523-predictor-57f4887bdf-rkfb5\" (UID: \"87f3a61b-c663-45a0-ae75-64c42cbe1695\") " pod="kserve-ci-e2e-test/error-404-isvc-40523-predictor-57f4887bdf-rkfb5"
Jul 02 13:53:42.103772 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:53:42.103726    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-40523-predictor-57f4887bdf-rkfb5"
Jul 02 13:53:42.242607 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:53:42.242576    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-40523-predictor-57f4887bdf-rkfb5"]
Jul 02 13:53:42.245857 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:53:42.245811    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87f3a61b_c663_45a0_ae75_64c42cbe1695.slice/crio-8e851281a29fccca34b4ca181e8260054f0037ed07e4bad81cb8924b1e66f054 WatchSource:0}: Error finding container 8e851281a29fccca34b4ca181e8260054f0037ed07e4bad81cb8924b1e66f054: Status 404 returned error can't find the container with id 8e851281a29fccca34b4ca181e8260054f0037ed07e4bad81cb8924b1e66f054
Jul 02 13:53:42.491352 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:53:42.491319    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-40523-predictor-57f4887bdf-rkfb5" event={"ID":"87f3a61b-c663-45a0-ae75-64c42cbe1695","Type":"ContainerStarted","Data":"8e851281a29fccca34b4ca181e8260054f0037ed07e4bad81cb8924b1e66f054"}
Jul 02 13:53:55.187294 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:53:55.187263    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8f2ll_3310d1f7-1643-4017-9edf-71fa702d2615/ovn-acl-logging/0.log"
Jul 02 13:53:55.194213 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:53:55.194186    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8f2ll_3310d1f7-1643-4017-9edf-71fa702d2615/ovn-acl-logging/0.log"
Jul 02 13:53:55.552160 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:53:55.552061    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-40523-predictor-57f4887bdf-rkfb5" event={"ID":"87f3a61b-c663-45a0-ae75-64c42cbe1695","Type":"ContainerStarted","Data":"af47b799d5c91b44e0ee1ee95d416c3a8cc11faeb569387dee8a6d9efc313436"}
Jul 02 13:53:57.560215 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:53:57.560176    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-40523-predictor-57f4887bdf-rkfb5" event={"ID":"87f3a61b-c663-45a0-ae75-64c42cbe1695","Type":"ContainerStarted","Data":"fe62d3110c7210b809af3c8108fe27ab50b10b73dac81c1a828ec70a9df0be2a"}
Jul 02 13:53:57.560639 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:53:57.560421    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-40523-predictor-57f4887bdf-rkfb5"
Jul 02 13:53:57.560639 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:53:57.560556    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-40523-predictor-57f4887bdf-rkfb5"
Jul 02 13:53:57.561956 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:53:57.561923    2558 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-40523-predictor-57f4887bdf-rkfb5" podUID="87f3a61b-c663-45a0-ae75-64c42cbe1695" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.18:8080: connect: connection refused"
Jul 02 13:53:57.580031 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:53:57.579989    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-40523-predictor-57f4887bdf-rkfb5" podStartSLOduration=1.8962906290000001 podStartE2EDuration="16.579975935s" totalImagesPullingTime="14.683685306s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 13:53:41 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:53:57.579488596 +0000 UTC m=+602.757950783" watchObservedRunningTime="2026-07-02 13:53:57.579975935 +0000 UTC m=+602.758438119"
Jul 02 13:53:58.563792 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:53:58.563752    2558 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-40523-predictor-57f4887bdf-rkfb5" podUID="87f3a61b-c663-45a0-ae75-64c42cbe1695" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.18:8080: connect: connection refused"
Jul 02 13:54:03.567917 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:54:03.567889    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-40523-predictor-57f4887bdf-rkfb5"
Jul 02 13:54:03.568373 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:54:03.568341    2558 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-40523-predictor-57f4887bdf-rkfb5" podUID="87f3a61b-c663-45a0-ae75-64c42cbe1695" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.18:8080: connect: connection refused"
Jul 02 13:54:13.568742 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:54:13.568703    2558 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-40523-predictor-57f4887bdf-rkfb5" podUID="87f3a61b-c663-45a0-ae75-64c42cbe1695" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.18:8080: connect: connection refused"
Jul 02 13:54:23.568544 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:54:23.568455    2558 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-40523-predictor-57f4887bdf-rkfb5" podUID="87f3a61b-c663-45a0-ae75-64c42cbe1695" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.18:8080: connect: connection refused"
Jul 02 13:54:33.569234 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:54:33.569191    2558 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-40523-predictor-57f4887bdf-rkfb5" podUID="87f3a61b-c663-45a0-ae75-64c42cbe1695" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.18:8080: connect: connection refused"
Jul 02 13:54:43.569781 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:54:43.569748    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-40523-predictor-57f4887bdf-rkfb5"
Jul 02 13:55:15.913783 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:55:15.913748    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-68692-predictor-58887df78c-cmj9w"]
Jul 02 13:55:15.917112 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:55:15.917095    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-68692-predictor-58887df78c-cmj9w"
Jul 02 13:55:15.919862 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:55:15.919841    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-68692-predictor-serving-cert\""
Jul 02 13:55:15.919862 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:55:15.919855    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-68692-kube-rbac-proxy-sar-config\""
Jul 02 13:55:15.927382 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:55:15.927361    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-68692-predictor-58887df78c-cmj9w"]
Jul 02 13:55:15.973496 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:55:15.973466    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-40523-predictor-57f4887bdf-rkfb5"]
Jul 02 13:55:15.973721 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:55:15.973689    2558 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-40523-predictor-57f4887bdf-rkfb5" podUID="87f3a61b-c663-45a0-ae75-64c42cbe1695" containerName="kserve-container" containerID="cri-o://af47b799d5c91b44e0ee1ee95d416c3a8cc11faeb569387dee8a6d9efc313436" gracePeriod=30
Jul 02 13:55:15.973887 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:55:15.973744    2558 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-40523-predictor-57f4887bdf-rkfb5" podUID="87f3a61b-c663-45a0-ae75-64c42cbe1695" containerName="kube-rbac-proxy" containerID="cri-o://fe62d3110c7210b809af3c8108fe27ab50b10b73dac81c1a828ec70a9df0be2a" gracePeriod=30
Jul 02 13:55:16.079621 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:55:16.079586    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/733c3c23-c4a1-4afa-ae84-f6417564974d-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-68692-predictor-58887df78c-cmj9w\" (UID: \"733c3c23-c4a1-4afa-ae84-f6417564974d\") " pod="kserve-ci-e2e-test/error-404-isvc-68692-predictor-58887df78c-cmj9w"
Jul 02 13:55:16.079621 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:55:16.079629    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jg4l8\" (UniqueName: \"kubernetes.io/projected/733c3c23-c4a1-4afa-ae84-f6417564974d-kube-api-access-jg4l8\") pod \"error-404-isvc-68692-predictor-58887df78c-cmj9w\" (UID: \"733c3c23-c4a1-4afa-ae84-f6417564974d\") " pod="kserve-ci-e2e-test/error-404-isvc-68692-predictor-58887df78c-cmj9w"
Jul 02 13:55:16.079807 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:55:16.079653    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/733c3c23-c4a1-4afa-ae84-f6417564974d-proxy-tls\") pod \"error-404-isvc-68692-predictor-58887df78c-cmj9w\" (UID: \"733c3c23-c4a1-4afa-ae84-f6417564974d\") " pod="kserve-ci-e2e-test/error-404-isvc-68692-predictor-58887df78c-cmj9w"
Jul 02 13:55:16.180143 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:55:16.180047    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/733c3c23-c4a1-4afa-ae84-f6417564974d-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-68692-predictor-58887df78c-cmj9w\" (UID: \"733c3c23-c4a1-4afa-ae84-f6417564974d\") " pod="kserve-ci-e2e-test/error-404-isvc-68692-predictor-58887df78c-cmj9w"
Jul 02 13:55:16.180143 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:55:16.180105    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jg4l8\" (UniqueName: \"kubernetes.io/projected/733c3c23-c4a1-4afa-ae84-f6417564974d-kube-api-access-jg4l8\") pod \"error-404-isvc-68692-predictor-58887df78c-cmj9w\" (UID: \"733c3c23-c4a1-4afa-ae84-f6417564974d\") " pod="kserve-ci-e2e-test/error-404-isvc-68692-predictor-58887df78c-cmj9w"
Jul 02 13:55:16.180357 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:55:16.180153    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/733c3c23-c4a1-4afa-ae84-f6417564974d-proxy-tls\") pod \"error-404-isvc-68692-predictor-58887df78c-cmj9w\" (UID: \"733c3c23-c4a1-4afa-ae84-f6417564974d\") " pod="kserve-ci-e2e-test/error-404-isvc-68692-predictor-58887df78c-cmj9w"
Jul 02 13:55:16.180357 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:55:16.180268    2558 secret.go:189] Couldn't get secret kserve-ci-e2e-test/error-404-isvc-68692-predictor-serving-cert: secret "error-404-isvc-68692-predictor-serving-cert" not found
Jul 02 13:55:16.180357 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:55:16.180349    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/733c3c23-c4a1-4afa-ae84-f6417564974d-proxy-tls podName:733c3c23-c4a1-4afa-ae84-f6417564974d nodeName:}" failed. No retries permitted until 2026-07-02 13:55:16.680317748 +0000 UTC m=+681.858779926 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/733c3c23-c4a1-4afa-ae84-f6417564974d-proxy-tls") pod "error-404-isvc-68692-predictor-58887df78c-cmj9w" (UID: "733c3c23-c4a1-4afa-ae84-f6417564974d") : secret "error-404-isvc-68692-predictor-serving-cert" not found
Jul 02 13:55:16.180696 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:55:16.180675    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/733c3c23-c4a1-4afa-ae84-f6417564974d-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-68692-predictor-58887df78c-cmj9w\" (UID: \"733c3c23-c4a1-4afa-ae84-f6417564974d\") " pod="kserve-ci-e2e-test/error-404-isvc-68692-predictor-58887df78c-cmj9w"
Jul 02 13:55:16.195739 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:55:16.195715    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jg4l8\" (UniqueName: \"kubernetes.io/projected/733c3c23-c4a1-4afa-ae84-f6417564974d-kube-api-access-jg4l8\") pod \"error-404-isvc-68692-predictor-58887df78c-cmj9w\" (UID: \"733c3c23-c4a1-4afa-ae84-f6417564974d\") " pod="kserve-ci-e2e-test/error-404-isvc-68692-predictor-58887df78c-cmj9w"
Jul 02 13:55:16.684728 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:55:16.684691    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/733c3c23-c4a1-4afa-ae84-f6417564974d-proxy-tls\") pod \"error-404-isvc-68692-predictor-58887df78c-cmj9w\" (UID: \"733c3c23-c4a1-4afa-ae84-f6417564974d\") " pod="kserve-ci-e2e-test/error-404-isvc-68692-predictor-58887df78c-cmj9w"
Jul 02 13:55:16.687359 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:55:16.687319    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/733c3c23-c4a1-4afa-ae84-f6417564974d-proxy-tls\") pod \"error-404-isvc-68692-predictor-58887df78c-cmj9w\" (UID: \"733c3c23-c4a1-4afa-ae84-f6417564974d\") " pod="kserve-ci-e2e-test/error-404-isvc-68692-predictor-58887df78c-cmj9w"
Jul 02 13:55:16.827601 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:55:16.827562    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-68692-predictor-58887df78c-cmj9w"
Jul 02 13:55:16.846029 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:55:16.845998    2558 generic.go:356] "Generic (PLEG): container finished" podID="87f3a61b-c663-45a0-ae75-64c42cbe1695" containerID="fe62d3110c7210b809af3c8108fe27ab50b10b73dac81c1a828ec70a9df0be2a" exitCode=2
Jul 02 13:55:16.846170 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:55:16.846050    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-40523-predictor-57f4887bdf-rkfb5" event={"ID":"87f3a61b-c663-45a0-ae75-64c42cbe1695","Type":"ContainerDied","Data":"fe62d3110c7210b809af3c8108fe27ab50b10b73dac81c1a828ec70a9df0be2a"}
Jul 02 13:55:16.973506 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:55:16.973429    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-68692-predictor-58887df78c-cmj9w"]
Jul 02 13:55:16.976496 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:55:16.976467    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod733c3c23_c4a1_4afa_ae84_f6417564974d.slice/crio-18799d8ae0f697b694ab7a9ab394b399ed7dc69e29ed2786800e1236c40fb9fe WatchSource:0}: Error finding container 18799d8ae0f697b694ab7a9ab394b399ed7dc69e29ed2786800e1236c40fb9fe: Status 404 returned error can't find the container with id 18799d8ae0f697b694ab7a9ab394b399ed7dc69e29ed2786800e1236c40fb9fe
Jul 02 13:55:17.852257 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:55:17.852226    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-68692-predictor-58887df78c-cmj9w" event={"ID":"733c3c23-c4a1-4afa-ae84-f6417564974d","Type":"ContainerStarted","Data":"46a1327b47326cd2e28aff73045de246ac17433de7648bbc97b60b6b6e985be7"}
Jul 02 13:55:17.852426 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:55:17.852280    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-68692-predictor-58887df78c-cmj9w" event={"ID":"733c3c23-c4a1-4afa-ae84-f6417564974d","Type":"ContainerStarted","Data":"5247352aa77c9ad4ef510426a3325ca00b277ce4e8879df6eac32671f3f7829c"}
Jul 02 13:55:17.852426 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:55:17.852302    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-68692-predictor-58887df78c-cmj9w" event={"ID":"733c3c23-c4a1-4afa-ae84-f6417564974d","Type":"ContainerStarted","Data":"18799d8ae0f697b694ab7a9ab394b399ed7dc69e29ed2786800e1236c40fb9fe"}
Jul 02 13:55:17.852510 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:55:17.852472    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-68692-predictor-58887df78c-cmj9w"
Jul 02 13:55:17.852636 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:55:17.852613    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-68692-predictor-58887df78c-cmj9w"
Jul 02 13:55:17.853856 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:55:17.853807    2558 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-68692-predictor-58887df78c-cmj9w" podUID="733c3c23-c4a1-4afa-ae84-f6417564974d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.19:8080: connect: connection refused"
Jul 02 13:55:17.872782 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:55:17.872738    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-68692-predictor-58887df78c-cmj9w" podStartSLOduration=2.87272366 podStartE2EDuration="2.87272366s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 13:55:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:55:17.871311892 +0000 UTC m=+683.049774076" watchObservedRunningTime="2026-07-02 13:55:17.87272366 +0000 UTC m=+683.051185844"
Jul 02 13:55:18.564157 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:55:18.564113    2558 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-40523-predictor-57f4887bdf-rkfb5" podUID="87f3a61b-c663-45a0-ae75-64c42cbe1695" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.18:8643/healthz\": dial tcp 10.133.0.18:8643: connect: connection refused"
Jul 02 13:55:18.856151 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:55:18.856117    2558 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-68692-predictor-58887df78c-cmj9w" podUID="733c3c23-c4a1-4afa-ae84-f6417564974d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.19:8080: connect: connection refused"
Jul 02 13:55:19.221024 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:55:19.221001    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-40523-predictor-57f4887bdf-rkfb5"
Jul 02 13:55:19.307296 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:55:19.307266    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/87f3a61b-c663-45a0-ae75-64c42cbe1695-kube-rbac-proxy-sar-config\") pod \"87f3a61b-c663-45a0-ae75-64c42cbe1695\" (UID: \"87f3a61b-c663-45a0-ae75-64c42cbe1695\") "
Jul 02 13:55:19.307296 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:55:19.307307    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-59qvl\" (UniqueName: \"kubernetes.io/projected/87f3a61b-c663-45a0-ae75-64c42cbe1695-kube-api-access-59qvl\") pod \"87f3a61b-c663-45a0-ae75-64c42cbe1695\" (UID: \"87f3a61b-c663-45a0-ae75-64c42cbe1695\") "
Jul 02 13:55:19.307549 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:55:19.307358    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/87f3a61b-c663-45a0-ae75-64c42cbe1695-proxy-tls\") pod \"87f3a61b-c663-45a0-ae75-64c42cbe1695\" (UID: \"87f3a61b-c663-45a0-ae75-64c42cbe1695\") "
Jul 02 13:55:19.307663 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:55:19.307639    2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/87f3a61b-c663-45a0-ae75-64c42cbe1695-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "87f3a61b-c663-45a0-ae75-64c42cbe1695" (UID: "87f3a61b-c663-45a0-ae75-64c42cbe1695"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 02 13:55:19.309650 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:55:19.309622    2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/87f3a61b-c663-45a0-ae75-64c42cbe1695-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "87f3a61b-c663-45a0-ae75-64c42cbe1695" (UID: "87f3a61b-c663-45a0-ae75-64c42cbe1695"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 02 13:55:19.309719 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:55:19.309656    2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/87f3a61b-c663-45a0-ae75-64c42cbe1695-kube-api-access-59qvl" (OuterVolumeSpecName: "kube-api-access-59qvl") pod "87f3a61b-c663-45a0-ae75-64c42cbe1695" (UID: "87f3a61b-c663-45a0-ae75-64c42cbe1695"). InnerVolumeSpecName "kube-api-access-59qvl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 13:55:19.408299 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:55:19.408194    2558 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/87f3a61b-c663-45a0-ae75-64c42cbe1695-kube-rbac-proxy-sar-config\") on node \"ip-10-0-133-209.ec2.internal\" DevicePath \"\""
Jul 02 13:55:19.408299 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:55:19.408237    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-59qvl\" (UniqueName: \"kubernetes.io/projected/87f3a61b-c663-45a0-ae75-64c42cbe1695-kube-api-access-59qvl\") on node \"ip-10-0-133-209.ec2.internal\" DevicePath \"\""
Jul 02 13:55:19.408299 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:55:19.408247    2558 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/87f3a61b-c663-45a0-ae75-64c42cbe1695-proxy-tls\") on node \"ip-10-0-133-209.ec2.internal\" DevicePath \"\""
Jul 02 13:55:19.861062 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:55:19.861031    2558 generic.go:356] "Generic (PLEG): container finished" podID="87f3a61b-c663-45a0-ae75-64c42cbe1695" containerID="af47b799d5c91b44e0ee1ee95d416c3a8cc11faeb569387dee8a6d9efc313436" exitCode=0
Jul 02 13:55:19.861484 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:55:19.861076    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-40523-predictor-57f4887bdf-rkfb5"
Jul 02 13:55:19.861484 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:55:19.861080    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-40523-predictor-57f4887bdf-rkfb5" event={"ID":"87f3a61b-c663-45a0-ae75-64c42cbe1695","Type":"ContainerDied","Data":"af47b799d5c91b44e0ee1ee95d416c3a8cc11faeb569387dee8a6d9efc313436"}
Jul 02 13:55:19.861484 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:55:19.861127    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-40523-predictor-57f4887bdf-rkfb5" event={"ID":"87f3a61b-c663-45a0-ae75-64c42cbe1695","Type":"ContainerDied","Data":"8e851281a29fccca34b4ca181e8260054f0037ed07e4bad81cb8924b1e66f054"}
Jul 02 13:55:19.861484 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:55:19.861155    2558 scope.go:117] "RemoveContainer" containerID="fe62d3110c7210b809af3c8108fe27ab50b10b73dac81c1a828ec70a9df0be2a"
Jul 02 13:55:19.869512 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:55:19.869496    2558 scope.go:117] "RemoveContainer" containerID="af47b799d5c91b44e0ee1ee95d416c3a8cc11faeb569387dee8a6d9efc313436"
Jul 02 13:55:19.876798 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:55:19.876783    2558 scope.go:117] "RemoveContainer" containerID="fe62d3110c7210b809af3c8108fe27ab50b10b73dac81c1a828ec70a9df0be2a"
Jul 02 13:55:19.877066 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:55:19.877036    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fe62d3110c7210b809af3c8108fe27ab50b10b73dac81c1a828ec70a9df0be2a\": container with ID starting with fe62d3110c7210b809af3c8108fe27ab50b10b73dac81c1a828ec70a9df0be2a not found: ID does not exist" containerID="fe62d3110c7210b809af3c8108fe27ab50b10b73dac81c1a828ec70a9df0be2a"
Jul 02 13:55:19.877155 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:55:19.877075    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fe62d3110c7210b809af3c8108fe27ab50b10b73dac81c1a828ec70a9df0be2a"} err="failed to get container status \"fe62d3110c7210b809af3c8108fe27ab50b10b73dac81c1a828ec70a9df0be2a\": rpc error: code = NotFound desc = could not find container \"fe62d3110c7210b809af3c8108fe27ab50b10b73dac81c1a828ec70a9df0be2a\": container with ID starting with fe62d3110c7210b809af3c8108fe27ab50b10b73dac81c1a828ec70a9df0be2a not found: ID does not exist"
Jul 02 13:55:19.877155 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:55:19.877100    2558 scope.go:117] "RemoveContainer" containerID="af47b799d5c91b44e0ee1ee95d416c3a8cc11faeb569387dee8a6d9efc313436"
Jul 02 13:55:19.877352 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:55:19.877331    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"af47b799d5c91b44e0ee1ee95d416c3a8cc11faeb569387dee8a6d9efc313436\": container with ID starting with af47b799d5c91b44e0ee1ee95d416c3a8cc11faeb569387dee8a6d9efc313436 not found: ID does not exist" containerID="af47b799d5c91b44e0ee1ee95d416c3a8cc11faeb569387dee8a6d9efc313436"
Jul 02 13:55:19.877397 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:55:19.877359    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"af47b799d5c91b44e0ee1ee95d416c3a8cc11faeb569387dee8a6d9efc313436"} err="failed to get container status \"af47b799d5c91b44e0ee1ee95d416c3a8cc11faeb569387dee8a6d9efc313436\": rpc error: code = NotFound desc = could not find container \"af47b799d5c91b44e0ee1ee95d416c3a8cc11faeb569387dee8a6d9efc313436\": container with ID starting with af47b799d5c91b44e0ee1ee95d416c3a8cc11faeb569387dee8a6d9efc313436 not found: ID does not exist"
Jul 02 13:55:19.883801 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:55:19.883779    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-40523-predictor-57f4887bdf-rkfb5"]
Jul 02 13:55:19.889121 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:55:19.889100    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-40523-predictor-57f4887bdf-rkfb5"]
Jul 02 13:55:21.217260 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:55:21.217228    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="87f3a61b-c663-45a0-ae75-64c42cbe1695" path="/var/lib/kubelet/pods/87f3a61b-c663-45a0-ae75-64c42cbe1695/volumes"
Jul 02 13:55:23.860785 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:55:23.860759    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-68692-predictor-58887df78c-cmj9w"
Jul 02 13:55:23.861230 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:55:23.861207    2558 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-68692-predictor-58887df78c-cmj9w" podUID="733c3c23-c4a1-4afa-ae84-f6417564974d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.19:8080: connect: connection refused"
Jul 02 13:55:33.862085 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:55:33.862043    2558 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-68692-predictor-58887df78c-cmj9w" podUID="733c3c23-c4a1-4afa-ae84-f6417564974d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.19:8080: connect: connection refused"
Jul 02 13:55:43.861925 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:55:43.861881    2558 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-68692-predictor-58887df78c-cmj9w" podUID="733c3c23-c4a1-4afa-ae84-f6417564974d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.19:8080: connect: connection refused"
Jul 02 13:55:53.861947 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:55:53.861910    2558 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-68692-predictor-58887df78c-cmj9w" podUID="733c3c23-c4a1-4afa-ae84-f6417564974d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.19:8080: connect: connection refused"
Jul 02 13:56:03.861970 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:56:03.861942    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-68692-predictor-58887df78c-cmj9w"
Jul 02 13:56:11.980766 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:56:11.980730    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-88827-predictor-66b8576947-28xrd"]
Jul 02 13:56:11.981221 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:56:11.981036    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="87f3a61b-c663-45a0-ae75-64c42cbe1695" containerName="kserve-container"
Jul 02 13:56:11.981221 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:56:11.981050    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="87f3a61b-c663-45a0-ae75-64c42cbe1695" containerName="kserve-container"
Jul 02 13:56:11.981221 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:56:11.981069    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="87f3a61b-c663-45a0-ae75-64c42cbe1695" containerName="kube-rbac-proxy"
Jul 02 13:56:11.981221 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:56:11.981074    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="87f3a61b-c663-45a0-ae75-64c42cbe1695" containerName="kube-rbac-proxy"
Jul 02 13:56:11.981221 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:56:11.981119    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="87f3a61b-c663-45a0-ae75-64c42cbe1695" containerName="kserve-container"
Jul 02 13:56:11.981221 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:56:11.981127    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="87f3a61b-c663-45a0-ae75-64c42cbe1695" containerName="kube-rbac-proxy"
Jul 02 13:56:11.985350 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:56:11.985333    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-88827-predictor-66b8576947-28xrd"
Jul 02 13:56:11.988058 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:56:11.988036    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-88827-predictor-serving-cert\""
Jul 02 13:56:11.988219 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:56:11.988199    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-88827-kube-rbac-proxy-sar-config\""
Jul 02 13:56:11.996626 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:56:11.996600    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-88827-predictor-66b8576947-28xrd"]
Jul 02 13:56:12.115780 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:56:12.115744    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f12742bd-4c26-4ef1-9c8d-e41010794a1c-proxy-tls\") pod \"error-404-isvc-88827-predictor-66b8576947-28xrd\" (UID: \"f12742bd-4c26-4ef1-9c8d-e41010794a1c\") " pod="kserve-ci-e2e-test/error-404-isvc-88827-predictor-66b8576947-28xrd"
Jul 02 13:56:12.115780 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:56:12.115781    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/f12742bd-4c26-4ef1-9c8d-e41010794a1c-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-88827-predictor-66b8576947-28xrd\" (UID: \"f12742bd-4c26-4ef1-9c8d-e41010794a1c\") " pod="kserve-ci-e2e-test/error-404-isvc-88827-predictor-66b8576947-28xrd"
Jul 02 13:56:12.116019 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:56:12.115930    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lllkn\" (UniqueName: \"kubernetes.io/projected/f12742bd-4c26-4ef1-9c8d-e41010794a1c-kube-api-access-lllkn\") pod \"error-404-isvc-88827-predictor-66b8576947-28xrd\" (UID: \"f12742bd-4c26-4ef1-9c8d-e41010794a1c\") " pod="kserve-ci-e2e-test/error-404-isvc-88827-predictor-66b8576947-28xrd"
Jul 02 13:56:12.216678 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:56:12.216634    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lllkn\" (UniqueName: \"kubernetes.io/projected/f12742bd-4c26-4ef1-9c8d-e41010794a1c-kube-api-access-lllkn\") pod \"error-404-isvc-88827-predictor-66b8576947-28xrd\" (UID: \"f12742bd-4c26-4ef1-9c8d-e41010794a1c\") " pod="kserve-ci-e2e-test/error-404-isvc-88827-predictor-66b8576947-28xrd"
Jul 02 13:56:12.216919 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:56:12.216702    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f12742bd-4c26-4ef1-9c8d-e41010794a1c-proxy-tls\") pod \"error-404-isvc-88827-predictor-66b8576947-28xrd\" (UID: \"f12742bd-4c26-4ef1-9c8d-e41010794a1c\") " pod="kserve-ci-e2e-test/error-404-isvc-88827-predictor-66b8576947-28xrd"
Jul 02 13:56:12.216919 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:56:12.216728    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/f12742bd-4c26-4ef1-9c8d-e41010794a1c-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-88827-predictor-66b8576947-28xrd\" (UID: \"f12742bd-4c26-4ef1-9c8d-e41010794a1c\") " pod="kserve-ci-e2e-test/error-404-isvc-88827-predictor-66b8576947-28xrd"
Jul 02 13:56:12.216919 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:56:12.216887    2558 secret.go:189] Couldn't get secret kserve-ci-e2e-test/error-404-isvc-88827-predictor-serving-cert: secret "error-404-isvc-88827-predictor-serving-cert" not found
Jul 02 13:56:12.217094 ip-10-0-133-209 kubenswrapper[2558]: E0702 13:56:12.216969    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f12742bd-4c26-4ef1-9c8d-e41010794a1c-proxy-tls podName:f12742bd-4c26-4ef1-9c8d-e41010794a1c nodeName:}" failed. No retries permitted until 2026-07-02 13:56:12.716949946 +0000 UTC m=+737.895412108 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/f12742bd-4c26-4ef1-9c8d-e41010794a1c-proxy-tls") pod "error-404-isvc-88827-predictor-66b8576947-28xrd" (UID: "f12742bd-4c26-4ef1-9c8d-e41010794a1c") : secret "error-404-isvc-88827-predictor-serving-cert" not found
Jul 02 13:56:12.217446 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:56:12.217424    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/f12742bd-4c26-4ef1-9c8d-e41010794a1c-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-88827-predictor-66b8576947-28xrd\" (UID: \"f12742bd-4c26-4ef1-9c8d-e41010794a1c\") " pod="kserve-ci-e2e-test/error-404-isvc-88827-predictor-66b8576947-28xrd"
Jul 02 13:56:12.226973 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:56:12.226947    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lllkn\" (UniqueName: \"kubernetes.io/projected/f12742bd-4c26-4ef1-9c8d-e41010794a1c-kube-api-access-lllkn\") pod \"error-404-isvc-88827-predictor-66b8576947-28xrd\" (UID: \"f12742bd-4c26-4ef1-9c8d-e41010794a1c\") " pod="kserve-ci-e2e-test/error-404-isvc-88827-predictor-66b8576947-28xrd"
Jul 02 13:56:12.721711 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:56:12.721669    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f12742bd-4c26-4ef1-9c8d-e41010794a1c-proxy-tls\") pod \"error-404-isvc-88827-predictor-66b8576947-28xrd\" (UID: \"f12742bd-4c26-4ef1-9c8d-e41010794a1c\") " pod="kserve-ci-e2e-test/error-404-isvc-88827-predictor-66b8576947-28xrd"
Jul 02 13:56:12.724155 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:56:12.724125    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f12742bd-4c26-4ef1-9c8d-e41010794a1c-proxy-tls\") pod \"error-404-isvc-88827-predictor-66b8576947-28xrd\" (UID: \"f12742bd-4c26-4ef1-9c8d-e41010794a1c\") " pod="kserve-ci-e2e-test/error-404-isvc-88827-predictor-66b8576947-28xrd"
Jul 02 13:56:12.896016 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:56:12.895965    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-88827-predictor-66b8576947-28xrd"
Jul 02 13:56:13.026929 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:56:13.026897    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-88827-predictor-66b8576947-28xrd"]
Jul 02 13:56:13.030622 ip-10-0-133-209 kubenswrapper[2558]: W0702 13:56:13.030596    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf12742bd_4c26_4ef1_9c8d_e41010794a1c.slice/crio-ec5e441a40a2ae800003bd5ac898259df59c55aaa567c58e61ac4b137659b4f6 WatchSource:0}: Error finding container ec5e441a40a2ae800003bd5ac898259df59c55aaa567c58e61ac4b137659b4f6: Status 404 returned error can't find the container with id ec5e441a40a2ae800003bd5ac898259df59c55aaa567c58e61ac4b137659b4f6
Jul 02 13:56:13.062033 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:56:13.062006    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-88827-predictor-66b8576947-28xrd" event={"ID":"f12742bd-4c26-4ef1-9c8d-e41010794a1c","Type":"ContainerStarted","Data":"ec5e441a40a2ae800003bd5ac898259df59c55aaa567c58e61ac4b137659b4f6"}
Jul 02 13:56:14.067506 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:56:14.067469    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-88827-predictor-66b8576947-28xrd" event={"ID":"f12742bd-4c26-4ef1-9c8d-e41010794a1c","Type":"ContainerStarted","Data":"9b7dd54c807b4a05f7ef133d48d661dab037fede9dfcd05e42b47ae8bbb80e2d"}
Jul 02 13:56:14.067958 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:56:14.067526    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-88827-predictor-66b8576947-28xrd" event={"ID":"f12742bd-4c26-4ef1-9c8d-e41010794a1c","Type":"ContainerStarted","Data":"6d75f1b15d984de571ded8fc5e9ea9cbbac049091bb1bce124e0a6d2209cc598"}
Jul 02 13:56:14.067958 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:56:14.067709    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-88827-predictor-66b8576947-28xrd"
Jul 02 13:56:14.067958 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:56:14.067847    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-88827-predictor-66b8576947-28xrd"
Jul 02 13:56:14.069096 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:56:14.069071    2558 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-88827-predictor-66b8576947-28xrd" podUID="f12742bd-4c26-4ef1-9c8d-e41010794a1c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.20:8080: connect: connection refused"
Jul 02 13:56:14.088794 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:56:14.088748    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-88827-predictor-66b8576947-28xrd" podStartSLOduration=3.088731664 podStartE2EDuration="3.088731664s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 13:56:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:56:14.087238904 +0000 UTC m=+739.265701088" watchObservedRunningTime="2026-07-02 13:56:14.088731664 +0000 UTC m=+739.267193847"
Jul 02 13:56:15.072462 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:56:15.072422    2558 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-88827-predictor-66b8576947-28xrd" podUID="f12742bd-4c26-4ef1-9c8d-e41010794a1c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.20:8080: connect: connection refused"
Jul 02 13:56:20.078030 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:56:20.077997    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-88827-predictor-66b8576947-28xrd"
Jul 02 13:56:20.078625 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:56:20.078503    2558 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-88827-predictor-66b8576947-28xrd" podUID="f12742bd-4c26-4ef1-9c8d-e41010794a1c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.20:8080: connect: connection refused"
Jul 02 13:56:30.078581 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:56:30.078538    2558 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-88827-predictor-66b8576947-28xrd" podUID="f12742bd-4c26-4ef1-9c8d-e41010794a1c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.20:8080: connect: connection refused"
Jul 02 13:56:40.078604 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:56:40.078558    2558 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-88827-predictor-66b8576947-28xrd" podUID="f12742bd-4c26-4ef1-9c8d-e41010794a1c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.20:8080: connect: connection refused"
Jul 02 13:56:50.078988 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:56:50.078942    2558 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-88827-predictor-66b8576947-28xrd" podUID="f12742bd-4c26-4ef1-9c8d-e41010794a1c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.20:8080: connect: connection refused"
Jul 02 13:57:00.079396 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:57:00.079360    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-88827-predictor-66b8576947-28xrd"
Jul 02 13:58:55.214984 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:58:55.214952    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8f2ll_3310d1f7-1643-4017-9edf-71fa702d2615/ovn-acl-logging/0.log"
Jul 02 13:58:55.218043 ip-10-0-133-209 kubenswrapper[2558]: I0702 13:58:55.218020    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8f2ll_3310d1f7-1643-4017-9edf-71fa702d2615/ovn-acl-logging/0.log"
Jul 02 14:03:55.239669 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:03:55.239634    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8f2ll_3310d1f7-1643-4017-9edf-71fa702d2615/ovn-acl-logging/0.log"
Jul 02 14:03:55.242221 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:03:55.240858    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8f2ll_3310d1f7-1643-4017-9edf-71fa702d2615/ovn-acl-logging/0.log"
Jul 02 14:04:30.663768 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:04:30.663738    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-c675b-predictor-5547c66cc5-ztplz"]
Jul 02 14:04:30.667139 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:04:30.667123    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-c675b-predictor-5547c66cc5-ztplz"
Jul 02 14:04:30.669606 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:04:30.669579    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-c675b-predictor-serving-cert\""
Jul 02 14:04:30.669714 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:04:30.669589    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-c675b-kube-rbac-proxy-sar-config\""
Jul 02 14:04:30.679451 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:04:30.679431    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-c675b-predictor-5547c66cc5-ztplz"]
Jul 02 14:04:30.705405 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:04:30.705371    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/6bd57168-96c6-4681-8758-f1026962e078-proxy-tls\") pod \"error-404-isvc-c675b-predictor-5547c66cc5-ztplz\" (UID: \"6bd57168-96c6-4681-8758-f1026962e078\") " pod="kserve-ci-e2e-test/error-404-isvc-c675b-predictor-5547c66cc5-ztplz"
Jul 02 14:04:30.705556 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:04:30.705452    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9lz8z\" (UniqueName: \"kubernetes.io/projected/6bd57168-96c6-4681-8758-f1026962e078-kube-api-access-9lz8z\") pod \"error-404-isvc-c675b-predictor-5547c66cc5-ztplz\" (UID: \"6bd57168-96c6-4681-8758-f1026962e078\") " pod="kserve-ci-e2e-test/error-404-isvc-c675b-predictor-5547c66cc5-ztplz"
Jul 02 14:04:30.705556 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:04:30.705489    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/6bd57168-96c6-4681-8758-f1026962e078-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-c675b-predictor-5547c66cc5-ztplz\" (UID: \"6bd57168-96c6-4681-8758-f1026962e078\") " pod="kserve-ci-e2e-test/error-404-isvc-c675b-predictor-5547c66cc5-ztplz"
Jul 02 14:04:30.709785 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:04:30.709761    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-68692-predictor-58887df78c-cmj9w"]
Jul 02 14:04:30.709999 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:04:30.709972    2558 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-68692-predictor-58887df78c-cmj9w" podUID="733c3c23-c4a1-4afa-ae84-f6417564974d" containerName="kserve-container" containerID="cri-o://5247352aa77c9ad4ef510426a3325ca00b277ce4e8879df6eac32671f3f7829c" gracePeriod=30
Jul 02 14:04:30.710124 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:04:30.710010    2558 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-68692-predictor-58887df78c-cmj9w" podUID="733c3c23-c4a1-4afa-ae84-f6417564974d" containerName="kube-rbac-proxy" containerID="cri-o://46a1327b47326cd2e28aff73045de246ac17433de7648bbc97b60b6b6e985be7" gracePeriod=30
Jul 02 14:04:30.806208 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:04:30.806171    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/6bd57168-96c6-4681-8758-f1026962e078-proxy-tls\") pod \"error-404-isvc-c675b-predictor-5547c66cc5-ztplz\" (UID: \"6bd57168-96c6-4681-8758-f1026962e078\") " pod="kserve-ci-e2e-test/error-404-isvc-c675b-predictor-5547c66cc5-ztplz"
Jul 02 14:04:30.806208 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:04:30.806215    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9lz8z\" (UniqueName: \"kubernetes.io/projected/6bd57168-96c6-4681-8758-f1026962e078-kube-api-access-9lz8z\") pod \"error-404-isvc-c675b-predictor-5547c66cc5-ztplz\" (UID: \"6bd57168-96c6-4681-8758-f1026962e078\") " pod="kserve-ci-e2e-test/error-404-isvc-c675b-predictor-5547c66cc5-ztplz"
Jul 02 14:04:30.806426 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:04:30.806239    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/6bd57168-96c6-4681-8758-f1026962e078-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-c675b-predictor-5547c66cc5-ztplz\" (UID: \"6bd57168-96c6-4681-8758-f1026962e078\") " pod="kserve-ci-e2e-test/error-404-isvc-c675b-predictor-5547c66cc5-ztplz"
Jul 02 14:04:30.806426 ip-10-0-133-209 kubenswrapper[2558]: E0702 14:04:30.806317    2558 secret.go:189] Couldn't get secret kserve-ci-e2e-test/error-404-isvc-c675b-predictor-serving-cert: secret "error-404-isvc-c675b-predictor-serving-cert" not found
Jul 02 14:04:30.806426 ip-10-0-133-209 kubenswrapper[2558]: E0702 14:04:30.806387    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6bd57168-96c6-4681-8758-f1026962e078-proxy-tls podName:6bd57168-96c6-4681-8758-f1026962e078 nodeName:}" failed. No retries permitted until 2026-07-02 14:04:31.306367207 +0000 UTC m=+1236.484829369 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/6bd57168-96c6-4681-8758-f1026962e078-proxy-tls") pod "error-404-isvc-c675b-predictor-5547c66cc5-ztplz" (UID: "6bd57168-96c6-4681-8758-f1026962e078") : secret "error-404-isvc-c675b-predictor-serving-cert" not found
Jul 02 14:04:30.806931 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:04:30.806914    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/6bd57168-96c6-4681-8758-f1026962e078-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-c675b-predictor-5547c66cc5-ztplz\" (UID: \"6bd57168-96c6-4681-8758-f1026962e078\") " pod="kserve-ci-e2e-test/error-404-isvc-c675b-predictor-5547c66cc5-ztplz"
Jul 02 14:04:30.815703 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:04:30.815678    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9lz8z\" (UniqueName: \"kubernetes.io/projected/6bd57168-96c6-4681-8758-f1026962e078-kube-api-access-9lz8z\") pod \"error-404-isvc-c675b-predictor-5547c66cc5-ztplz\" (UID: \"6bd57168-96c6-4681-8758-f1026962e078\") " pod="kserve-ci-e2e-test/error-404-isvc-c675b-predictor-5547c66cc5-ztplz"
Jul 02 14:04:30.878427 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:04:30.878394    2558 generic.go:356] "Generic (PLEG): container finished" podID="733c3c23-c4a1-4afa-ae84-f6417564974d" containerID="46a1327b47326cd2e28aff73045de246ac17433de7648bbc97b60b6b6e985be7" exitCode=2
Jul 02 14:04:30.878583 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:04:30.878450    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-68692-predictor-58887df78c-cmj9w" event={"ID":"733c3c23-c4a1-4afa-ae84-f6417564974d","Type":"ContainerDied","Data":"46a1327b47326cd2e28aff73045de246ac17433de7648bbc97b60b6b6e985be7"}
Jul 02 14:04:31.310632 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:04:31.310587    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/6bd57168-96c6-4681-8758-f1026962e078-proxy-tls\") pod \"error-404-isvc-c675b-predictor-5547c66cc5-ztplz\" (UID: \"6bd57168-96c6-4681-8758-f1026962e078\") " pod="kserve-ci-e2e-test/error-404-isvc-c675b-predictor-5547c66cc5-ztplz"
Jul 02 14:04:31.313182 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:04:31.313157    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/6bd57168-96c6-4681-8758-f1026962e078-proxy-tls\") pod \"error-404-isvc-c675b-predictor-5547c66cc5-ztplz\" (UID: \"6bd57168-96c6-4681-8758-f1026962e078\") " pod="kserve-ci-e2e-test/error-404-isvc-c675b-predictor-5547c66cc5-ztplz"
Jul 02 14:04:31.577104 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:04:31.577011    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-c675b-predictor-5547c66cc5-ztplz"
Jul 02 14:04:31.709761 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:04:31.709688    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-c675b-predictor-5547c66cc5-ztplz"]
Jul 02 14:04:31.712211 ip-10-0-133-209 kubenswrapper[2558]: W0702 14:04:31.712182    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6bd57168_96c6_4681_8758_f1026962e078.slice/crio-b0c2d284ad6114736def2aa5028d6c4e4325ae63489ffe4d9567a76452d505b1 WatchSource:0}: Error finding container b0c2d284ad6114736def2aa5028d6c4e4325ae63489ffe4d9567a76452d505b1: Status 404 returned error can't find the container with id b0c2d284ad6114736def2aa5028d6c4e4325ae63489ffe4d9567a76452d505b1
Jul 02 14:04:31.883438 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:04:31.883403    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-c675b-predictor-5547c66cc5-ztplz" event={"ID":"6bd57168-96c6-4681-8758-f1026962e078","Type":"ContainerStarted","Data":"4fd1f2d96214a20a93cae652306635ca5bab22554941179524b8d55de4ff335c"}
Jul 02 14:04:31.883627 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:04:31.883459    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-c675b-predictor-5547c66cc5-ztplz" event={"ID":"6bd57168-96c6-4681-8758-f1026962e078","Type":"ContainerStarted","Data":"99dbefc89edce8ffae6c67c5cb27ec7da8a93d2a3c38db4eba053b265f36915d"}
Jul 02 14:04:31.883627 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:04:31.883485    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-c675b-predictor-5547c66cc5-ztplz"
Jul 02 14:04:31.883627 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:04:31.883514    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-c675b-predictor-5547c66cc5-ztplz" event={"ID":"6bd57168-96c6-4681-8758-f1026962e078","Type":"ContainerStarted","Data":"b0c2d284ad6114736def2aa5028d6c4e4325ae63489ffe4d9567a76452d505b1"}
Jul 02 14:04:31.883627 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:04:31.883535    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-c675b-predictor-5547c66cc5-ztplz"
Jul 02 14:04:31.885017 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:04:31.884990    2558 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-c675b-predictor-5547c66cc5-ztplz" podUID="6bd57168-96c6-4681-8758-f1026962e078" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.21:8080: connect: connection refused"
Jul 02 14:04:31.904952 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:04:31.904901    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-c675b-predictor-5547c66cc5-ztplz" podStartSLOduration=1.904882321 podStartE2EDuration="1.904882321s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 14:04:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:04:31.903254346 +0000 UTC m=+1237.081716531" watchObservedRunningTime="2026-07-02 14:04:31.904882321 +0000 UTC m=+1237.083344504"
Jul 02 14:04:32.886844 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:04:32.886793    2558 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-c675b-predictor-5547c66cc5-ztplz" podUID="6bd57168-96c6-4681-8758-f1026962e078" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.21:8080: connect: connection refused"
Jul 02 14:04:33.857226 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:04:33.857077    2558 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-68692-predictor-58887df78c-cmj9w" podUID="733c3c23-c4a1-4afa-ae84-f6417564974d" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.19:8643/healthz\": dial tcp 10.133.0.19:8643: connect: connection refused"
Jul 02 14:04:33.861668 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:04:33.861632    2558 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-68692-predictor-58887df78c-cmj9w" podUID="733c3c23-c4a1-4afa-ae84-f6417564974d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.19:8080: connect: connection refused"
Jul 02 14:04:33.892527 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:04:33.892499    2558 generic.go:356] "Generic (PLEG): container finished" podID="733c3c23-c4a1-4afa-ae84-f6417564974d" containerID="5247352aa77c9ad4ef510426a3325ca00b277ce4e8879df6eac32671f3f7829c" exitCode=0
Jul 02 14:04:33.892970 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:04:33.892568    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-68692-predictor-58887df78c-cmj9w" event={"ID":"733c3c23-c4a1-4afa-ae84-f6417564974d","Type":"ContainerDied","Data":"5247352aa77c9ad4ef510426a3325ca00b277ce4e8879df6eac32671f3f7829c"}
Jul 02 14:04:33.962480 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:04:33.962458    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-68692-predictor-58887df78c-cmj9w"
Jul 02 14:04:34.032147 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:04:34.032054    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/733c3c23-c4a1-4afa-ae84-f6417564974d-kube-rbac-proxy-sar-config\") pod \"733c3c23-c4a1-4afa-ae84-f6417564974d\" (UID: \"733c3c23-c4a1-4afa-ae84-f6417564974d\") "
Jul 02 14:04:34.032147 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:04:34.032096    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jg4l8\" (UniqueName: \"kubernetes.io/projected/733c3c23-c4a1-4afa-ae84-f6417564974d-kube-api-access-jg4l8\") pod \"733c3c23-c4a1-4afa-ae84-f6417564974d\" (UID: \"733c3c23-c4a1-4afa-ae84-f6417564974d\") "
Jul 02 14:04:34.032332 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:04:34.032178    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/733c3c23-c4a1-4afa-ae84-f6417564974d-proxy-tls\") pod \"733c3c23-c4a1-4afa-ae84-f6417564974d\" (UID: \"733c3c23-c4a1-4afa-ae84-f6417564974d\") "
Jul 02 14:04:34.032472 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:04:34.032450    2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/733c3c23-c4a1-4afa-ae84-f6417564974d-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "733c3c23-c4a1-4afa-ae84-f6417564974d" (UID: "733c3c23-c4a1-4afa-ae84-f6417564974d"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 02 14:04:34.034443 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:04:34.034412    2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/733c3c23-c4a1-4afa-ae84-f6417564974d-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "733c3c23-c4a1-4afa-ae84-f6417564974d" (UID: "733c3c23-c4a1-4afa-ae84-f6417564974d"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 02 14:04:34.034443 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:04:34.034426    2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/733c3c23-c4a1-4afa-ae84-f6417564974d-kube-api-access-jg4l8" (OuterVolumeSpecName: "kube-api-access-jg4l8") pod "733c3c23-c4a1-4afa-ae84-f6417564974d" (UID: "733c3c23-c4a1-4afa-ae84-f6417564974d"). InnerVolumeSpecName "kube-api-access-jg4l8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 14:04:34.133144 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:04:34.133107    2558 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/733c3c23-c4a1-4afa-ae84-f6417564974d-kube-rbac-proxy-sar-config\") on node \"ip-10-0-133-209.ec2.internal\" DevicePath \"\""
Jul 02 14:04:34.133144 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:04:34.133137    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jg4l8\" (UniqueName: \"kubernetes.io/projected/733c3c23-c4a1-4afa-ae84-f6417564974d-kube-api-access-jg4l8\") on node \"ip-10-0-133-209.ec2.internal\" DevicePath \"\""
Jul 02 14:04:34.133144 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:04:34.133147    2558 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/733c3c23-c4a1-4afa-ae84-f6417564974d-proxy-tls\") on node \"ip-10-0-133-209.ec2.internal\" DevicePath \"\""
Jul 02 14:04:34.897514 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:04:34.897477    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-68692-predictor-58887df78c-cmj9w" event={"ID":"733c3c23-c4a1-4afa-ae84-f6417564974d","Type":"ContainerDied","Data":"18799d8ae0f697b694ab7a9ab394b399ed7dc69e29ed2786800e1236c40fb9fe"}
Jul 02 14:04:34.897514 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:04:34.897515    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-68692-predictor-58887df78c-cmj9w"
Jul 02 14:04:34.897993 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:04:34.897546    2558 scope.go:117] "RemoveContainer" containerID="46a1327b47326cd2e28aff73045de246ac17433de7648bbc97b60b6b6e985be7"
Jul 02 14:04:34.906195 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:04:34.906179    2558 scope.go:117] "RemoveContainer" containerID="5247352aa77c9ad4ef510426a3325ca00b277ce4e8879df6eac32671f3f7829c"
Jul 02 14:04:34.918282 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:04:34.918259    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-68692-predictor-58887df78c-cmj9w"]
Jul 02 14:04:34.922372 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:04:34.922352    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-68692-predictor-58887df78c-cmj9w"]
Jul 02 14:04:35.216814 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:04:35.216732    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="733c3c23-c4a1-4afa-ae84-f6417564974d" path="/var/lib/kubelet/pods/733c3c23-c4a1-4afa-ae84-f6417564974d/volumes"
Jul 02 14:04:37.895314 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:04:37.895283    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-c675b-predictor-5547c66cc5-ztplz"
Jul 02 14:04:37.895934 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:04:37.895905    2558 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-c675b-predictor-5547c66cc5-ztplz" podUID="6bd57168-96c6-4681-8758-f1026962e078" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.21:8080: connect: connection refused"
Jul 02 14:04:47.896589 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:04:47.896502    2558 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-c675b-predictor-5547c66cc5-ztplz" podUID="6bd57168-96c6-4681-8758-f1026962e078" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.21:8080: connect: connection refused"
Jul 02 14:04:57.896767 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:04:57.896718    2558 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-c675b-predictor-5547c66cc5-ztplz" podUID="6bd57168-96c6-4681-8758-f1026962e078" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.21:8080: connect: connection refused"
Jul 02 14:05:07.895932 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:05:07.895890    2558 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-c675b-predictor-5547c66cc5-ztplz" podUID="6bd57168-96c6-4681-8758-f1026962e078" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.21:8080: connect: connection refused"
Jul 02 14:05:17.897003 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:05:17.896970    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-c675b-predictor-5547c66cc5-ztplz"
Jul 02 14:05:26.702745 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:05:26.702712    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-88827-predictor-66b8576947-28xrd"]
Jul 02 14:05:26.703262 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:05:26.702957    2558 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-88827-predictor-66b8576947-28xrd" podUID="f12742bd-4c26-4ef1-9c8d-e41010794a1c" containerName="kserve-container" containerID="cri-o://6d75f1b15d984de571ded8fc5e9ea9cbbac049091bb1bce124e0a6d2209cc598" gracePeriod=30
Jul 02 14:05:26.703262 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:05:26.703019    2558 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-88827-predictor-66b8576947-28xrd" podUID="f12742bd-4c26-4ef1-9c8d-e41010794a1c" containerName="kube-rbac-proxy" containerID="cri-o://9b7dd54c807b4a05f7ef133d48d661dab037fede9dfcd05e42b47ae8bbb80e2d" gracePeriod=30
Jul 02 14:05:26.868603 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:05:26.868556    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-c559a-predictor-7c98468fff-9t7fx"]
Jul 02 14:05:26.868891 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:05:26.868879    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="733c3c23-c4a1-4afa-ae84-f6417564974d" containerName="kserve-container"
Jul 02 14:05:26.868939 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:05:26.868893    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="733c3c23-c4a1-4afa-ae84-f6417564974d" containerName="kserve-container"
Jul 02 14:05:26.868939 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:05:26.868902    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="733c3c23-c4a1-4afa-ae84-f6417564974d" containerName="kube-rbac-proxy"
Jul 02 14:05:26.868939 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:05:26.868908    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="733c3c23-c4a1-4afa-ae84-f6417564974d" containerName="kube-rbac-proxy"
Jul 02 14:05:26.869030 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:05:26.868963    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="733c3c23-c4a1-4afa-ae84-f6417564974d" containerName="kube-rbac-proxy"
Jul 02 14:05:26.869030 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:05:26.868972    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="733c3c23-c4a1-4afa-ae84-f6417564974d" containerName="kserve-container"
Jul 02 14:05:26.871792 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:05:26.871776    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-c559a-predictor-7c98468fff-9t7fx"
Jul 02 14:05:26.874296 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:05:26.874279    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-c559a-predictor-serving-cert\""
Jul 02 14:05:26.874355 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:05:26.874335    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-c559a-kube-rbac-proxy-sar-config\""
Jul 02 14:05:26.882721 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:05:26.882696    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-c559a-predictor-7c98468fff-9t7fx"]
Jul 02 14:05:26.939375 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:05:26.939335    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cv5q2\" (UniqueName: \"kubernetes.io/projected/e1ed2cb5-74ad-404f-a6e4-68b4432279b3-kube-api-access-cv5q2\") pod \"error-404-isvc-c559a-predictor-7c98468fff-9t7fx\" (UID: \"e1ed2cb5-74ad-404f-a6e4-68b4432279b3\") " pod="kserve-ci-e2e-test/error-404-isvc-c559a-predictor-7c98468fff-9t7fx"
Jul 02 14:05:26.939551 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:05:26.939410    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/e1ed2cb5-74ad-404f-a6e4-68b4432279b3-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-c559a-predictor-7c98468fff-9t7fx\" (UID: \"e1ed2cb5-74ad-404f-a6e4-68b4432279b3\") " pod="kserve-ci-e2e-test/error-404-isvc-c559a-predictor-7c98468fff-9t7fx"
Jul 02 14:05:26.939551 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:05:26.939445    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e1ed2cb5-74ad-404f-a6e4-68b4432279b3-proxy-tls\") pod \"error-404-isvc-c559a-predictor-7c98468fff-9t7fx\" (UID: \"e1ed2cb5-74ad-404f-a6e4-68b4432279b3\") " pod="kserve-ci-e2e-test/error-404-isvc-c559a-predictor-7c98468fff-9t7fx"
Jul 02 14:05:27.040722 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:05:27.040625    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/e1ed2cb5-74ad-404f-a6e4-68b4432279b3-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-c559a-predictor-7c98468fff-9t7fx\" (UID: \"e1ed2cb5-74ad-404f-a6e4-68b4432279b3\") " pod="kserve-ci-e2e-test/error-404-isvc-c559a-predictor-7c98468fff-9t7fx"
Jul 02 14:05:27.040722 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:05:27.040668    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e1ed2cb5-74ad-404f-a6e4-68b4432279b3-proxy-tls\") pod \"error-404-isvc-c559a-predictor-7c98468fff-9t7fx\" (UID: \"e1ed2cb5-74ad-404f-a6e4-68b4432279b3\") " pod="kserve-ci-e2e-test/error-404-isvc-c559a-predictor-7c98468fff-9t7fx"
Jul 02 14:05:27.040722 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:05:27.040715    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cv5q2\" (UniqueName: \"kubernetes.io/projected/e1ed2cb5-74ad-404f-a6e4-68b4432279b3-kube-api-access-cv5q2\") pod \"error-404-isvc-c559a-predictor-7c98468fff-9t7fx\" (UID: \"e1ed2cb5-74ad-404f-a6e4-68b4432279b3\") " pod="kserve-ci-e2e-test/error-404-isvc-c559a-predictor-7c98468fff-9t7fx"
Jul 02 14:05:27.041448 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:05:27.041382    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/e1ed2cb5-74ad-404f-a6e4-68b4432279b3-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-c559a-predictor-7c98468fff-9t7fx\" (UID: \"e1ed2cb5-74ad-404f-a6e4-68b4432279b3\") " pod="kserve-ci-e2e-test/error-404-isvc-c559a-predictor-7c98468fff-9t7fx"
Jul 02 14:05:27.043262 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:05:27.043240    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e1ed2cb5-74ad-404f-a6e4-68b4432279b3-proxy-tls\") pod \"error-404-isvc-c559a-predictor-7c98468fff-9t7fx\" (UID: \"e1ed2cb5-74ad-404f-a6e4-68b4432279b3\") " pod="kserve-ci-e2e-test/error-404-isvc-c559a-predictor-7c98468fff-9t7fx"
Jul 02 14:05:27.049487 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:05:27.049462    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cv5q2\" (UniqueName: \"kubernetes.io/projected/e1ed2cb5-74ad-404f-a6e4-68b4432279b3-kube-api-access-cv5q2\") pod \"error-404-isvc-c559a-predictor-7c98468fff-9t7fx\" (UID: \"e1ed2cb5-74ad-404f-a6e4-68b4432279b3\") " pod="kserve-ci-e2e-test/error-404-isvc-c559a-predictor-7c98468fff-9t7fx"
Jul 02 14:05:27.088419 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:05:27.088385    2558 generic.go:356] "Generic (PLEG): container finished" podID="f12742bd-4c26-4ef1-9c8d-e41010794a1c" containerID="9b7dd54c807b4a05f7ef133d48d661dab037fede9dfcd05e42b47ae8bbb80e2d" exitCode=2
Jul 02 14:05:27.088578 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:05:27.088457    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-88827-predictor-66b8576947-28xrd" event={"ID":"f12742bd-4c26-4ef1-9c8d-e41010794a1c","Type":"ContainerDied","Data":"9b7dd54c807b4a05f7ef133d48d661dab037fede9dfcd05e42b47ae8bbb80e2d"}
Jul 02 14:05:27.182130 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:05:27.182090    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-c559a-predictor-7c98468fff-9t7fx"
Jul 02 14:05:27.317010 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:05:27.316797    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-c559a-predictor-7c98468fff-9t7fx"]
Jul 02 14:05:27.319697 ip-10-0-133-209 kubenswrapper[2558]: W0702 14:05:27.319669    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1ed2cb5_74ad_404f_a6e4_68b4432279b3.slice/crio-930d925b9ce7564b248e4144051f3d7d97003afd961e4674fd0e724b1a94a6e3 WatchSource:0}: Error finding container 930d925b9ce7564b248e4144051f3d7d97003afd961e4674fd0e724b1a94a6e3: Status 404 returned error can't find the container with id 930d925b9ce7564b248e4144051f3d7d97003afd961e4674fd0e724b1a94a6e3
Jul 02 14:05:28.094454 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:05:28.094417    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-c559a-predictor-7c98468fff-9t7fx" event={"ID":"e1ed2cb5-74ad-404f-a6e4-68b4432279b3","Type":"ContainerStarted","Data":"bb649180cc20e631c63e0eef026d92ff1e79a68f617e24332766d871b3eae825"}
Jul 02 14:05:28.094872 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:05:28.094476    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-c559a-predictor-7c98468fff-9t7fx" event={"ID":"e1ed2cb5-74ad-404f-a6e4-68b4432279b3","Type":"ContainerStarted","Data":"448ea2940b912eec4f7d1af292c1dea7322ad415f946bc798623745d17c78c1e"}
Jul 02 14:05:28.094872 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:05:28.094500    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-c559a-predictor-7c98468fff-9t7fx"
Jul 02 14:05:28.094872 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:05:28.094519    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-c559a-predictor-7c98468fff-9t7fx" event={"ID":"e1ed2cb5-74ad-404f-a6e4-68b4432279b3","Type":"ContainerStarted","Data":"930d925b9ce7564b248e4144051f3d7d97003afd961e4674fd0e724b1a94a6e3"}
Jul 02 14:05:28.094872 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:05:28.094658    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-c559a-predictor-7c98468fff-9t7fx"
Jul 02 14:05:28.096130 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:05:28.096104    2558 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-c559a-predictor-7c98468fff-9t7fx" podUID="e1ed2cb5-74ad-404f-a6e4-68b4432279b3" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.22:8080: connect: connection refused"
Jul 02 14:05:28.112944 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:05:28.112902    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-c559a-predictor-7c98468fff-9t7fx" podStartSLOduration=2.112887261 podStartE2EDuration="2.112887261s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 14:05:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:05:28.111424401 +0000 UTC m=+1293.289886608" watchObservedRunningTime="2026-07-02 14:05:28.112887261 +0000 UTC m=+1293.291349445"
Jul 02 14:05:29.098810 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:05:29.098771    2558 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-c559a-predictor-7c98468fff-9t7fx" podUID="e1ed2cb5-74ad-404f-a6e4-68b4432279b3" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.22:8080: connect: connection refused"
Jul 02 14:05:29.958622 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:05:29.958600    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-88827-predictor-66b8576947-28xrd"
Jul 02 14:05:30.067293 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:05:30.067207    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/f12742bd-4c26-4ef1-9c8d-e41010794a1c-kube-rbac-proxy-sar-config\") pod \"f12742bd-4c26-4ef1-9c8d-e41010794a1c\" (UID: \"f12742bd-4c26-4ef1-9c8d-e41010794a1c\") "
Jul 02 14:05:30.067293 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:05:30.067244    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lllkn\" (UniqueName: \"kubernetes.io/projected/f12742bd-4c26-4ef1-9c8d-e41010794a1c-kube-api-access-lllkn\") pod \"f12742bd-4c26-4ef1-9c8d-e41010794a1c\" (UID: \"f12742bd-4c26-4ef1-9c8d-e41010794a1c\") "
Jul 02 14:05:30.067293 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:05:30.067268    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f12742bd-4c26-4ef1-9c8d-e41010794a1c-proxy-tls\") pod \"f12742bd-4c26-4ef1-9c8d-e41010794a1c\" (UID: \"f12742bd-4c26-4ef1-9c8d-e41010794a1c\") "
Jul 02 14:05:30.067639 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:05:30.067609    2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f12742bd-4c26-4ef1-9c8d-e41010794a1c-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "f12742bd-4c26-4ef1-9c8d-e41010794a1c" (UID: "f12742bd-4c26-4ef1-9c8d-e41010794a1c"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 02 14:05:30.069465 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:05:30.069435    2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f12742bd-4c26-4ef1-9c8d-e41010794a1c-kube-api-access-lllkn" (OuterVolumeSpecName: "kube-api-access-lllkn") pod "f12742bd-4c26-4ef1-9c8d-e41010794a1c" (UID: "f12742bd-4c26-4ef1-9c8d-e41010794a1c"). InnerVolumeSpecName "kube-api-access-lllkn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 14:05:30.069565 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:05:30.069460    2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f12742bd-4c26-4ef1-9c8d-e41010794a1c-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "f12742bd-4c26-4ef1-9c8d-e41010794a1c" (UID: "f12742bd-4c26-4ef1-9c8d-e41010794a1c"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 02 14:05:30.103607 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:05:30.103578    2558 generic.go:356] "Generic (PLEG): container finished" podID="f12742bd-4c26-4ef1-9c8d-e41010794a1c" containerID="6d75f1b15d984de571ded8fc5e9ea9cbbac049091bb1bce124e0a6d2209cc598" exitCode=0
Jul 02 14:05:30.104000 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:05:30.103632    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-88827-predictor-66b8576947-28xrd"
Jul 02 14:05:30.104000 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:05:30.103672    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-88827-predictor-66b8576947-28xrd" event={"ID":"f12742bd-4c26-4ef1-9c8d-e41010794a1c","Type":"ContainerDied","Data":"6d75f1b15d984de571ded8fc5e9ea9cbbac049091bb1bce124e0a6d2209cc598"}
Jul 02 14:05:30.104000 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:05:30.103728    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-88827-predictor-66b8576947-28xrd" event={"ID":"f12742bd-4c26-4ef1-9c8d-e41010794a1c","Type":"ContainerDied","Data":"ec5e441a40a2ae800003bd5ac898259df59c55aaa567c58e61ac4b137659b4f6"}
Jul 02 14:05:30.104000 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:05:30.103757    2558 scope.go:117] "RemoveContainer" containerID="9b7dd54c807b4a05f7ef133d48d661dab037fede9dfcd05e42b47ae8bbb80e2d"
Jul 02 14:05:30.112577 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:05:30.112558    2558 scope.go:117] "RemoveContainer" containerID="6d75f1b15d984de571ded8fc5e9ea9cbbac049091bb1bce124e0a6d2209cc598"
Jul 02 14:05:30.119895 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:05:30.119869    2558 scope.go:117] "RemoveContainer" containerID="9b7dd54c807b4a05f7ef133d48d661dab037fede9dfcd05e42b47ae8bbb80e2d"
Jul 02 14:05:30.120157 ip-10-0-133-209 kubenswrapper[2558]: E0702 14:05:30.120140    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9b7dd54c807b4a05f7ef133d48d661dab037fede9dfcd05e42b47ae8bbb80e2d\": container with ID starting with 9b7dd54c807b4a05f7ef133d48d661dab037fede9dfcd05e42b47ae8bbb80e2d not found: ID does not exist" containerID="9b7dd54c807b4a05f7ef133d48d661dab037fede9dfcd05e42b47ae8bbb80e2d"
Jul 02 14:05:30.120207 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:05:30.120166    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9b7dd54c807b4a05f7ef133d48d661dab037fede9dfcd05e42b47ae8bbb80e2d"} err="failed to get container status \"9b7dd54c807b4a05f7ef133d48d661dab037fede9dfcd05e42b47ae8bbb80e2d\": rpc error: code = NotFound desc = could not find container \"9b7dd54c807b4a05f7ef133d48d661dab037fede9dfcd05e42b47ae8bbb80e2d\": container with ID starting with 9b7dd54c807b4a05f7ef133d48d661dab037fede9dfcd05e42b47ae8bbb80e2d not found: ID does not exist"
Jul 02 14:05:30.120207 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:05:30.120184    2558 scope.go:117] "RemoveContainer" containerID="6d75f1b15d984de571ded8fc5e9ea9cbbac049091bb1bce124e0a6d2209cc598"
Jul 02 14:05:30.120382 ip-10-0-133-209 kubenswrapper[2558]: E0702 14:05:30.120369    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6d75f1b15d984de571ded8fc5e9ea9cbbac049091bb1bce124e0a6d2209cc598\": container with ID starting with 6d75f1b15d984de571ded8fc5e9ea9cbbac049091bb1bce124e0a6d2209cc598 not found: ID does not exist" containerID="6d75f1b15d984de571ded8fc5e9ea9cbbac049091bb1bce124e0a6d2209cc598"
Jul 02 14:05:30.120414 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:05:30.120386    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6d75f1b15d984de571ded8fc5e9ea9cbbac049091bb1bce124e0a6d2209cc598"} err="failed to get container status \"6d75f1b15d984de571ded8fc5e9ea9cbbac049091bb1bce124e0a6d2209cc598\": rpc error: code = NotFound desc = could not find container \"6d75f1b15d984de571ded8fc5e9ea9cbbac049091bb1bce124e0a6d2209cc598\": container with ID starting with 6d75f1b15d984de571ded8fc5e9ea9cbbac049091bb1bce124e0a6d2209cc598 not found: ID does not exist"
Jul 02 14:05:30.125438 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:05:30.125418    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-88827-predictor-66b8576947-28xrd"]
Jul 02 14:05:30.129513 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:05:30.129494    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-88827-predictor-66b8576947-28xrd"]
Jul 02 14:05:30.168103 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:05:30.168070    2558 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/f12742bd-4c26-4ef1-9c8d-e41010794a1c-kube-rbac-proxy-sar-config\") on node \"ip-10-0-133-209.ec2.internal\" DevicePath \"\""
Jul 02 14:05:30.168103 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:05:30.168098    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lllkn\" (UniqueName: \"kubernetes.io/projected/f12742bd-4c26-4ef1-9c8d-e41010794a1c-kube-api-access-lllkn\") on node \"ip-10-0-133-209.ec2.internal\" DevicePath \"\""
Jul 02 14:05:30.168103 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:05:30.168111    2558 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f12742bd-4c26-4ef1-9c8d-e41010794a1c-proxy-tls\") on node \"ip-10-0-133-209.ec2.internal\" DevicePath \"\""
Jul 02 14:05:31.216769 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:05:31.216733    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f12742bd-4c26-4ef1-9c8d-e41010794a1c" path="/var/lib/kubelet/pods/f12742bd-4c26-4ef1-9c8d-e41010794a1c/volumes"
Jul 02 14:05:34.102943 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:05:34.102912    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-c559a-predictor-7c98468fff-9t7fx"
Jul 02 14:05:34.103386 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:05:34.103357    2558 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-c559a-predictor-7c98468fff-9t7fx" podUID="e1ed2cb5-74ad-404f-a6e4-68b4432279b3" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.22:8080: connect: connection refused"
Jul 02 14:05:40.782706 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:05:40.782624    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-c675b-predictor-5547c66cc5-ztplz"]
Jul 02 14:05:40.783140 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:05:40.782886    2558 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-c675b-predictor-5547c66cc5-ztplz" podUID="6bd57168-96c6-4681-8758-f1026962e078" containerName="kserve-container" containerID="cri-o://99dbefc89edce8ffae6c67c5cb27ec7da8a93d2a3c38db4eba053b265f36915d" gracePeriod=30
Jul 02 14:05:40.783140 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:05:40.782886    2558 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-c675b-predictor-5547c66cc5-ztplz" podUID="6bd57168-96c6-4681-8758-f1026962e078" containerName="kube-rbac-proxy" containerID="cri-o://4fd1f2d96214a20a93cae652306635ca5bab22554941179524b8d55de4ff335c" gracePeriod=30
Jul 02 14:05:40.947417 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:05:40.947369    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-7b7ab-predictor-7d65578975-ghbgv"]
Jul 02 14:05:40.947864 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:05:40.947810    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f12742bd-4c26-4ef1-9c8d-e41010794a1c" containerName="kube-rbac-proxy"
Jul 02 14:05:40.947864 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:05:40.947863    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="f12742bd-4c26-4ef1-9c8d-e41010794a1c" containerName="kube-rbac-proxy"
Jul 02 14:05:40.948052 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:05:40.947911    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f12742bd-4c26-4ef1-9c8d-e41010794a1c" containerName="kserve-container"
Jul 02 14:05:40.948052 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:05:40.947921    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="f12742bd-4c26-4ef1-9c8d-e41010794a1c" containerName="kserve-container"
Jul 02 14:05:40.948052 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:05:40.948014    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="f12742bd-4c26-4ef1-9c8d-e41010794a1c" containerName="kserve-container"
Jul 02 14:05:40.948052 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:05:40.948031    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="f12742bd-4c26-4ef1-9c8d-e41010794a1c" containerName="kube-rbac-proxy"
Jul 02 14:05:40.951602 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:05:40.951575    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-7b7ab-predictor-7d65578975-ghbgv"
Jul 02 14:05:40.954394 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:05:40.954375    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-7b7ab-predictor-serving-cert\""
Jul 02 14:05:40.954484 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:05:40.954387    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-7b7ab-kube-rbac-proxy-sar-config\""
Jul 02 14:05:40.962000 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:05:40.961978    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-7b7ab-predictor-7d65578975-ghbgv"]
Jul 02 14:05:41.056241 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:05:41.056148    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/bf5c8734-5d7c-47a7-a030-d10ef03f926a-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-7b7ab-predictor-7d65578975-ghbgv\" (UID: \"bf5c8734-5d7c-47a7-a030-d10ef03f926a\") " pod="kserve-ci-e2e-test/error-404-isvc-7b7ab-predictor-7d65578975-ghbgv"
Jul 02 14:05:41.056241 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:05:41.056202    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/bf5c8734-5d7c-47a7-a030-d10ef03f926a-proxy-tls\") pod \"error-404-isvc-7b7ab-predictor-7d65578975-ghbgv\" (UID: \"bf5c8734-5d7c-47a7-a030-d10ef03f926a\") " pod="kserve-ci-e2e-test/error-404-isvc-7b7ab-predictor-7d65578975-ghbgv"
Jul 02 14:05:41.056431 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:05:41.056270    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z77jr\" (UniqueName: \"kubernetes.io/projected/bf5c8734-5d7c-47a7-a030-d10ef03f926a-kube-api-access-z77jr\") pod \"error-404-isvc-7b7ab-predictor-7d65578975-ghbgv\" (UID: \"bf5c8734-5d7c-47a7-a030-d10ef03f926a\") " pod="kserve-ci-e2e-test/error-404-isvc-7b7ab-predictor-7d65578975-ghbgv"
Jul 02 14:05:41.147082 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:05:41.147038    2558 generic.go:356] "Generic (PLEG): container finished" podID="6bd57168-96c6-4681-8758-f1026962e078" containerID="4fd1f2d96214a20a93cae652306635ca5bab22554941179524b8d55de4ff335c" exitCode=2
Jul 02 14:05:41.147249 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:05:41.147105    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-c675b-predictor-5547c66cc5-ztplz" event={"ID":"6bd57168-96c6-4681-8758-f1026962e078","Type":"ContainerDied","Data":"4fd1f2d96214a20a93cae652306635ca5bab22554941179524b8d55de4ff335c"}
Jul 02 14:05:41.157416 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:05:41.157383    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/bf5c8734-5d7c-47a7-a030-d10ef03f926a-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-7b7ab-predictor-7d65578975-ghbgv\" (UID: \"bf5c8734-5d7c-47a7-a030-d10ef03f926a\") " pod="kserve-ci-e2e-test/error-404-isvc-7b7ab-predictor-7d65578975-ghbgv"
Jul 02 14:05:41.157566 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:05:41.157423    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/bf5c8734-5d7c-47a7-a030-d10ef03f926a-proxy-tls\") pod \"error-404-isvc-7b7ab-predictor-7d65578975-ghbgv\" (UID: \"bf5c8734-5d7c-47a7-a030-d10ef03f926a\") " pod="kserve-ci-e2e-test/error-404-isvc-7b7ab-predictor-7d65578975-ghbgv"
Jul 02 14:05:41.157566 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:05:41.157457    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z77jr\" (UniqueName: \"kubernetes.io/projected/bf5c8734-5d7c-47a7-a030-d10ef03f926a-kube-api-access-z77jr\") pod \"error-404-isvc-7b7ab-predictor-7d65578975-ghbgv\" (UID: \"bf5c8734-5d7c-47a7-a030-d10ef03f926a\") " pod="kserve-ci-e2e-test/error-404-isvc-7b7ab-predictor-7d65578975-ghbgv"
Jul 02 14:05:41.157566 ip-10-0-133-209 kubenswrapper[2558]: E0702 14:05:41.157549    2558 secret.go:189] Couldn't get secret kserve-ci-e2e-test/error-404-isvc-7b7ab-predictor-serving-cert: secret "error-404-isvc-7b7ab-predictor-serving-cert" not found
Jul 02 14:05:41.157713 ip-10-0-133-209 kubenswrapper[2558]: E0702 14:05:41.157611    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bf5c8734-5d7c-47a7-a030-d10ef03f926a-proxy-tls podName:bf5c8734-5d7c-47a7-a030-d10ef03f926a nodeName:}" failed. No retries permitted until 2026-07-02 14:05:41.657594705 +0000 UTC m=+1306.836056884 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/bf5c8734-5d7c-47a7-a030-d10ef03f926a-proxy-tls") pod "error-404-isvc-7b7ab-predictor-7d65578975-ghbgv" (UID: "bf5c8734-5d7c-47a7-a030-d10ef03f926a") : secret "error-404-isvc-7b7ab-predictor-serving-cert" not found
Jul 02 14:05:41.158072 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:05:41.158053    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/bf5c8734-5d7c-47a7-a030-d10ef03f926a-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-7b7ab-predictor-7d65578975-ghbgv\" (UID: \"bf5c8734-5d7c-47a7-a030-d10ef03f926a\") " pod="kserve-ci-e2e-test/error-404-isvc-7b7ab-predictor-7d65578975-ghbgv"
Jul 02 14:05:41.166663 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:05:41.166639    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z77jr\" (UniqueName: \"kubernetes.io/projected/bf5c8734-5d7c-47a7-a030-d10ef03f926a-kube-api-access-z77jr\") pod \"error-404-isvc-7b7ab-predictor-7d65578975-ghbgv\" (UID: \"bf5c8734-5d7c-47a7-a030-d10ef03f926a\") " pod="kserve-ci-e2e-test/error-404-isvc-7b7ab-predictor-7d65578975-ghbgv"
Jul 02 14:05:41.660411 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:05:41.660371    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/bf5c8734-5d7c-47a7-a030-d10ef03f926a-proxy-tls\") pod \"error-404-isvc-7b7ab-predictor-7d65578975-ghbgv\" (UID: \"bf5c8734-5d7c-47a7-a030-d10ef03f926a\") " pod="kserve-ci-e2e-test/error-404-isvc-7b7ab-predictor-7d65578975-ghbgv"
Jul 02 14:05:41.663075 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:05:41.663039    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/bf5c8734-5d7c-47a7-a030-d10ef03f926a-proxy-tls\") pod \"error-404-isvc-7b7ab-predictor-7d65578975-ghbgv\" (UID: \"bf5c8734-5d7c-47a7-a030-d10ef03f926a\") " pod="kserve-ci-e2e-test/error-404-isvc-7b7ab-predictor-7d65578975-ghbgv"
Jul 02 14:05:41.861939 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:05:41.861893    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-7b7ab-predictor-7d65578975-ghbgv"
Jul 02 14:05:41.999500 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:05:41.999474    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-7b7ab-predictor-7d65578975-ghbgv"]
Jul 02 14:05:42.001884 ip-10-0-133-209 kubenswrapper[2558]: W0702 14:05:42.001855    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf5c8734_5d7c_47a7_a030_d10ef03f926a.slice/crio-5a03346db84d747af0f5a13b08c7cd134155ab19632b0a5cb26916e82690bd14 WatchSource:0}: Error finding container 5a03346db84d747af0f5a13b08c7cd134155ab19632b0a5cb26916e82690bd14: Status 404 returned error can't find the container with id 5a03346db84d747af0f5a13b08c7cd134155ab19632b0a5cb26916e82690bd14
Jul 02 14:05:42.154165 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:05:42.154125    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-7b7ab-predictor-7d65578975-ghbgv" event={"ID":"bf5c8734-5d7c-47a7-a030-d10ef03f926a","Type":"ContainerStarted","Data":"f56a800075a501efa7730a254374de5cd3b09ea34c43dd8dc650e411367bc04c"}
Jul 02 14:05:42.154336 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:05:42.154197    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-7b7ab-predictor-7d65578975-ghbgv" event={"ID":"bf5c8734-5d7c-47a7-a030-d10ef03f926a","Type":"ContainerStarted","Data":"267f3cf87698f810157861c6c10d5cf7afc3bcd427aa216496ea5b412e289b2c"}
Jul 02 14:05:42.154336 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:05:42.154227    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-7b7ab-predictor-7d65578975-ghbgv"
Jul 02 14:05:42.154336 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:05:42.154261    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-7b7ab-predictor-7d65578975-ghbgv" event={"ID":"bf5c8734-5d7c-47a7-a030-d10ef03f926a","Type":"ContainerStarted","Data":"5a03346db84d747af0f5a13b08c7cd134155ab19632b0a5cb26916e82690bd14"}
Jul 02 14:05:42.174144 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:05:42.174031    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-7b7ab-predictor-7d65578975-ghbgv" podStartSLOduration=2.174014851 podStartE2EDuration="2.174014851s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 14:05:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:05:42.172440881 +0000 UTC m=+1307.350903066" watchObservedRunningTime="2026-07-02 14:05:42.174014851 +0000 UTC m=+1307.352477034"
Jul 02 14:05:42.887839 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:05:42.887777    2558 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-c675b-predictor-5547c66cc5-ztplz" podUID="6bd57168-96c6-4681-8758-f1026962e078" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.21:8643/healthz\": dial tcp 10.133.0.21:8643: connect: connection refused"
Jul 02 14:05:43.158936 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:05:43.158857    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-7b7ab-predictor-7d65578975-ghbgv"
Jul 02 14:05:43.160144 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:05:43.160117    2558 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-7b7ab-predictor-7d65578975-ghbgv" podUID="bf5c8734-5d7c-47a7-a030-d10ef03f926a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.23:8080: connect: connection refused"
Jul 02 14:05:44.106682 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:05:44.106197    2558 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-c559a-predictor-7c98468fff-9t7fx" podUID="e1ed2cb5-74ad-404f-a6e4-68b4432279b3" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.22:8080: connect: connection refused"
Jul 02 14:05:44.139183 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:05:44.139153    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-c675b-predictor-5547c66cc5-ztplz"
Jul 02 14:05:44.164116 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:05:44.164080    2558 generic.go:356] "Generic (PLEG): container finished" podID="6bd57168-96c6-4681-8758-f1026962e078" containerID="99dbefc89edce8ffae6c67c5cb27ec7da8a93d2a3c38db4eba053b265f36915d" exitCode=0
Jul 02 14:05:44.164285 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:05:44.164139    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-c675b-predictor-5547c66cc5-ztplz"
Jul 02 14:05:44.164285 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:05:44.164159    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-c675b-predictor-5547c66cc5-ztplz" event={"ID":"6bd57168-96c6-4681-8758-f1026962e078","Type":"ContainerDied","Data":"99dbefc89edce8ffae6c67c5cb27ec7da8a93d2a3c38db4eba053b265f36915d"}
Jul 02 14:05:44.164285 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:05:44.164228    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-c675b-predictor-5547c66cc5-ztplz" event={"ID":"6bd57168-96c6-4681-8758-f1026962e078","Type":"ContainerDied","Data":"b0c2d284ad6114736def2aa5028d6c4e4325ae63489ffe4d9567a76452d505b1"}
Jul 02 14:05:44.164285 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:05:44.164255    2558 scope.go:117] "RemoveContainer" containerID="4fd1f2d96214a20a93cae652306635ca5bab22554941179524b8d55de4ff335c"
Jul 02 14:05:44.164747 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:05:44.164575    2558 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-7b7ab-predictor-7d65578975-ghbgv" podUID="bf5c8734-5d7c-47a7-a030-d10ef03f926a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.23:8080: connect: connection refused"
Jul 02 14:05:44.172655 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:05:44.172549    2558 scope.go:117] "RemoveContainer" containerID="99dbefc89edce8ffae6c67c5cb27ec7da8a93d2a3c38db4eba053b265f36915d"
Jul 02 14:05:44.180717 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:05:44.180696    2558 scope.go:117] "RemoveContainer" containerID="4fd1f2d96214a20a93cae652306635ca5bab22554941179524b8d55de4ff335c"
Jul 02 14:05:44.180998 ip-10-0-133-209 kubenswrapper[2558]: E0702 14:05:44.180981    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4fd1f2d96214a20a93cae652306635ca5bab22554941179524b8d55de4ff335c\": container with ID starting with 4fd1f2d96214a20a93cae652306635ca5bab22554941179524b8d55de4ff335c not found: ID does not exist" containerID="4fd1f2d96214a20a93cae652306635ca5bab22554941179524b8d55de4ff335c"
Jul 02 14:05:44.181065 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:05:44.181008    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4fd1f2d96214a20a93cae652306635ca5bab22554941179524b8d55de4ff335c"} err="failed to get container status \"4fd1f2d96214a20a93cae652306635ca5bab22554941179524b8d55de4ff335c\": rpc error: code = NotFound desc = could not find container \"4fd1f2d96214a20a93cae652306635ca5bab22554941179524b8d55de4ff335c\": container with ID starting with 4fd1f2d96214a20a93cae652306635ca5bab22554941179524b8d55de4ff335c not found: ID does not exist"
Jul 02 14:05:44.181065 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:05:44.181025    2558 scope.go:117] "RemoveContainer" containerID="99dbefc89edce8ffae6c67c5cb27ec7da8a93d2a3c38db4eba053b265f36915d"
Jul 02 14:05:44.181252 ip-10-0-133-209 kubenswrapper[2558]: E0702 14:05:44.181234    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"99dbefc89edce8ffae6c67c5cb27ec7da8a93d2a3c38db4eba053b265f36915d\": container with ID starting with 99dbefc89edce8ffae6c67c5cb27ec7da8a93d2a3c38db4eba053b265f36915d not found: ID does not exist" containerID="99dbefc89edce8ffae6c67c5cb27ec7da8a93d2a3c38db4eba053b265f36915d"
Jul 02 14:05:44.181309 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:05:44.181255    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"99dbefc89edce8ffae6c67c5cb27ec7da8a93d2a3c38db4eba053b265f36915d"} err="failed to get container status \"99dbefc89edce8ffae6c67c5cb27ec7da8a93d2a3c38db4eba053b265f36915d\": rpc error: code = NotFound desc = could not find container \"99dbefc89edce8ffae6c67c5cb27ec7da8a93d2a3c38db4eba053b265f36915d\": container with ID starting with 99dbefc89edce8ffae6c67c5cb27ec7da8a93d2a3c38db4eba053b265f36915d not found: ID does not exist"
Jul 02 14:05:44.183519 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:05:44.183503    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/6bd57168-96c6-4681-8758-f1026962e078-proxy-tls\") pod \"6bd57168-96c6-4681-8758-f1026962e078\" (UID: \"6bd57168-96c6-4681-8758-f1026962e078\") "
Jul 02 14:05:44.183583 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:05:44.183538    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9lz8z\" (UniqueName: \"kubernetes.io/projected/6bd57168-96c6-4681-8758-f1026962e078-kube-api-access-9lz8z\") pod \"6bd57168-96c6-4681-8758-f1026962e078\" (UID: \"6bd57168-96c6-4681-8758-f1026962e078\") "
Jul 02 14:05:44.183583 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:05:44.183563    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/6bd57168-96c6-4681-8758-f1026962e078-kube-rbac-proxy-sar-config\") pod \"6bd57168-96c6-4681-8758-f1026962e078\" (UID: \"6bd57168-96c6-4681-8758-f1026962e078\") "
Jul 02 14:05:44.183957 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:05:44.183938    2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6bd57168-96c6-4681-8758-f1026962e078-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "6bd57168-96c6-4681-8758-f1026962e078" (UID: "6bd57168-96c6-4681-8758-f1026962e078"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 02 14:05:44.185768 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:05:44.185743    2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6bd57168-96c6-4681-8758-f1026962e078-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "6bd57168-96c6-4681-8758-f1026962e078" (UID: "6bd57168-96c6-4681-8758-f1026962e078"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 02 14:05:44.185935 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:05:44.185790    2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6bd57168-96c6-4681-8758-f1026962e078-kube-api-access-9lz8z" (OuterVolumeSpecName: "kube-api-access-9lz8z") pod "6bd57168-96c6-4681-8758-f1026962e078" (UID: "6bd57168-96c6-4681-8758-f1026962e078"). InnerVolumeSpecName "kube-api-access-9lz8z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 14:05:44.284649 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:05:44.284533    2558 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/6bd57168-96c6-4681-8758-f1026962e078-kube-rbac-proxy-sar-config\") on node \"ip-10-0-133-209.ec2.internal\" DevicePath \"\""
Jul 02 14:05:44.284649 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:05:44.284572    2558 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/6bd57168-96c6-4681-8758-f1026962e078-proxy-tls\") on node \"ip-10-0-133-209.ec2.internal\" DevicePath \"\""
Jul 02 14:05:44.284649 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:05:44.284589    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9lz8z\" (UniqueName: \"kubernetes.io/projected/6bd57168-96c6-4681-8758-f1026962e078-kube-api-access-9lz8z\") on node \"ip-10-0-133-209.ec2.internal\" DevicePath \"\""
Jul 02 14:05:44.485234 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:05:44.485199    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-c675b-predictor-5547c66cc5-ztplz"]
Jul 02 14:05:44.488972 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:05:44.488948    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-c675b-predictor-5547c66cc5-ztplz"]
Jul 02 14:05:45.216902 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:05:45.216870    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6bd57168-96c6-4681-8758-f1026962e078" path="/var/lib/kubelet/pods/6bd57168-96c6-4681-8758-f1026962e078/volumes"
Jul 02 14:05:49.169324 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:05:49.169296    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-7b7ab-predictor-7d65578975-ghbgv"
Jul 02 14:05:49.169766 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:05:49.169740    2558 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-7b7ab-predictor-7d65578975-ghbgv" podUID="bf5c8734-5d7c-47a7-a030-d10ef03f926a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.23:8080: connect: connection refused"
Jul 02 14:05:54.104063 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:05:54.104019    2558 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-c559a-predictor-7c98468fff-9t7fx" podUID="e1ed2cb5-74ad-404f-a6e4-68b4432279b3" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.22:8080: connect: connection refused"
Jul 02 14:05:59.169947 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:05:59.169903    2558 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-7b7ab-predictor-7d65578975-ghbgv" podUID="bf5c8734-5d7c-47a7-a030-d10ef03f926a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.23:8080: connect: connection refused"
Jul 02 14:06:04.103852 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:06:04.103787    2558 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-c559a-predictor-7c98468fff-9t7fx" podUID="e1ed2cb5-74ad-404f-a6e4-68b4432279b3" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.22:8080: connect: connection refused"
Jul 02 14:06:09.170169 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:06:09.170121    2558 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-7b7ab-predictor-7d65578975-ghbgv" podUID="bf5c8734-5d7c-47a7-a030-d10ef03f926a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.23:8080: connect: connection refused"
Jul 02 14:06:14.104541 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:06:14.104508    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-c559a-predictor-7c98468fff-9t7fx"
Jul 02 14:06:19.169786 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:06:19.169693    2558 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-7b7ab-predictor-7d65578975-ghbgv" podUID="bf5c8734-5d7c-47a7-a030-d10ef03f926a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.23:8080: connect: connection refused"
Jul 02 14:06:29.170447 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:06:29.170410    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-7b7ab-predictor-7d65578975-ghbgv"
Jul 02 14:06:36.956731 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:06:36.956689    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-c559a-predictor-7c98468fff-9t7fx"]
Jul 02 14:06:36.957221 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:06:36.957036    2558 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-c559a-predictor-7c98468fff-9t7fx" podUID="e1ed2cb5-74ad-404f-a6e4-68b4432279b3" containerName="kserve-container" containerID="cri-o://448ea2940b912eec4f7d1af292c1dea7322ad415f946bc798623745d17c78c1e" gracePeriod=30
Jul 02 14:06:36.957221 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:06:36.957125    2558 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-c559a-predictor-7c98468fff-9t7fx" podUID="e1ed2cb5-74ad-404f-a6e4-68b4432279b3" containerName="kube-rbac-proxy" containerID="cri-o://bb649180cc20e631c63e0eef026d92ff1e79a68f617e24332766d871b3eae825" gracePeriod=30
Jul 02 14:06:37.113713 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:06:37.113684    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-6d218-predictor-fc586b6c5-77mmt"]
Jul 02 14:06:37.114016 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:06:37.114003    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6bd57168-96c6-4681-8758-f1026962e078" containerName="kube-rbac-proxy"
Jul 02 14:06:37.114063 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:06:37.114018    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="6bd57168-96c6-4681-8758-f1026962e078" containerName="kube-rbac-proxy"
Jul 02 14:06:37.114063 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:06:37.114031    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6bd57168-96c6-4681-8758-f1026962e078" containerName="kserve-container"
Jul 02 14:06:37.114063 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:06:37.114036    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="6bd57168-96c6-4681-8758-f1026962e078" containerName="kserve-container"
Jul 02 14:06:37.114150 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:06:37.114087    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="6bd57168-96c6-4681-8758-f1026962e078" containerName="kube-rbac-proxy"
Jul 02 14:06:37.114150 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:06:37.114096    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="6bd57168-96c6-4681-8758-f1026962e078" containerName="kserve-container"
Jul 02 14:06:37.116984 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:06:37.116968    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-6d218-predictor-fc586b6c5-77mmt"
Jul 02 14:06:37.119559 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:06:37.119532    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-6d218-kube-rbac-proxy-sar-config\""
Jul 02 14:06:37.119559 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:06:37.119539    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-6d218-predictor-serving-cert\""
Jul 02 14:06:37.126961 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:06:37.126930    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-6d218-predictor-fc586b6c5-77mmt"]
Jul 02 14:06:37.207311 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:06:37.207220    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/aff672a5-50a2-46cb-85fd-41c7230f155f-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-6d218-predictor-fc586b6c5-77mmt\" (UID: \"aff672a5-50a2-46cb-85fd-41c7230f155f\") " pod="kserve-ci-e2e-test/error-404-isvc-6d218-predictor-fc586b6c5-77mmt"
Jul 02 14:06:37.207311 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:06:37.207273    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d4tbf\" (UniqueName: \"kubernetes.io/projected/aff672a5-50a2-46cb-85fd-41c7230f155f-kube-api-access-d4tbf\") pod \"error-404-isvc-6d218-predictor-fc586b6c5-77mmt\" (UID: \"aff672a5-50a2-46cb-85fd-41c7230f155f\") " pod="kserve-ci-e2e-test/error-404-isvc-6d218-predictor-fc586b6c5-77mmt"
Jul 02 14:06:37.207311 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:06:37.207291    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/aff672a5-50a2-46cb-85fd-41c7230f155f-proxy-tls\") pod \"error-404-isvc-6d218-predictor-fc586b6c5-77mmt\" (UID: \"aff672a5-50a2-46cb-85fd-41c7230f155f\") " pod="kserve-ci-e2e-test/error-404-isvc-6d218-predictor-fc586b6c5-77mmt"
Jul 02 14:06:37.308617 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:06:37.308574    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/aff672a5-50a2-46cb-85fd-41c7230f155f-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-6d218-predictor-fc586b6c5-77mmt\" (UID: \"aff672a5-50a2-46cb-85fd-41c7230f155f\") " pod="kserve-ci-e2e-test/error-404-isvc-6d218-predictor-fc586b6c5-77mmt"
Jul 02 14:06:37.308805 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:06:37.308641    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d4tbf\" (UniqueName: \"kubernetes.io/projected/aff672a5-50a2-46cb-85fd-41c7230f155f-kube-api-access-d4tbf\") pod \"error-404-isvc-6d218-predictor-fc586b6c5-77mmt\" (UID: \"aff672a5-50a2-46cb-85fd-41c7230f155f\") " pod="kserve-ci-e2e-test/error-404-isvc-6d218-predictor-fc586b6c5-77mmt"
Jul 02 14:06:37.308805 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:06:37.308665    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/aff672a5-50a2-46cb-85fd-41c7230f155f-proxy-tls\") pod \"error-404-isvc-6d218-predictor-fc586b6c5-77mmt\" (UID: \"aff672a5-50a2-46cb-85fd-41c7230f155f\") " pod="kserve-ci-e2e-test/error-404-isvc-6d218-predictor-fc586b6c5-77mmt"
Jul 02 14:06:37.308942 ip-10-0-133-209 kubenswrapper[2558]: E0702 14:06:37.308804    2558 secret.go:189] Couldn't get secret kserve-ci-e2e-test/error-404-isvc-6d218-predictor-serving-cert: secret "error-404-isvc-6d218-predictor-serving-cert" not found
Jul 02 14:06:37.308942 ip-10-0-133-209 kubenswrapper[2558]: E0702 14:06:37.308910    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aff672a5-50a2-46cb-85fd-41c7230f155f-proxy-tls podName:aff672a5-50a2-46cb-85fd-41c7230f155f nodeName:}" failed. No retries permitted until 2026-07-02 14:06:37.808888649 +0000 UTC m=+1362.987350811 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/aff672a5-50a2-46cb-85fd-41c7230f155f-proxy-tls") pod "error-404-isvc-6d218-predictor-fc586b6c5-77mmt" (UID: "aff672a5-50a2-46cb-85fd-41c7230f155f") : secret "error-404-isvc-6d218-predictor-serving-cert" not found
Jul 02 14:06:37.309289 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:06:37.309267    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/aff672a5-50a2-46cb-85fd-41c7230f155f-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-6d218-predictor-fc586b6c5-77mmt\" (UID: \"aff672a5-50a2-46cb-85fd-41c7230f155f\") " pod="kserve-ci-e2e-test/error-404-isvc-6d218-predictor-fc586b6c5-77mmt"
Jul 02 14:06:37.318014 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:06:37.317993    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d4tbf\" (UniqueName: \"kubernetes.io/projected/aff672a5-50a2-46cb-85fd-41c7230f155f-kube-api-access-d4tbf\") pod \"error-404-isvc-6d218-predictor-fc586b6c5-77mmt\" (UID: \"aff672a5-50a2-46cb-85fd-41c7230f155f\") " pod="kserve-ci-e2e-test/error-404-isvc-6d218-predictor-fc586b6c5-77mmt"
Jul 02 14:06:37.368615 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:06:37.368580    2558 generic.go:356] "Generic (PLEG): container finished" podID="e1ed2cb5-74ad-404f-a6e4-68b4432279b3" containerID="bb649180cc20e631c63e0eef026d92ff1e79a68f617e24332766d871b3eae825" exitCode=2
Jul 02 14:06:37.368771 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:06:37.368648    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-c559a-predictor-7c98468fff-9t7fx" event={"ID":"e1ed2cb5-74ad-404f-a6e4-68b4432279b3","Type":"ContainerDied","Data":"bb649180cc20e631c63e0eef026d92ff1e79a68f617e24332766d871b3eae825"}
Jul 02 14:06:37.813760 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:06:37.813717    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/aff672a5-50a2-46cb-85fd-41c7230f155f-proxy-tls\") pod \"error-404-isvc-6d218-predictor-fc586b6c5-77mmt\" (UID: \"aff672a5-50a2-46cb-85fd-41c7230f155f\") " pod="kserve-ci-e2e-test/error-404-isvc-6d218-predictor-fc586b6c5-77mmt"
Jul 02 14:06:37.816407 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:06:37.816374    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/aff672a5-50a2-46cb-85fd-41c7230f155f-proxy-tls\") pod \"error-404-isvc-6d218-predictor-fc586b6c5-77mmt\" (UID: \"aff672a5-50a2-46cb-85fd-41c7230f155f\") " pod="kserve-ci-e2e-test/error-404-isvc-6d218-predictor-fc586b6c5-77mmt"
Jul 02 14:06:38.027627 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:06:38.027592    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-6d218-predictor-fc586b6c5-77mmt"
Jul 02 14:06:38.180785 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:06:38.180762    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-6d218-predictor-fc586b6c5-77mmt"]
Jul 02 14:06:38.183345 ip-10-0-133-209 kubenswrapper[2558]: W0702 14:06:38.183317    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaff672a5_50a2_46cb_85fd_41c7230f155f.slice/crio-6b2d4a5d3146008197e7fb9ecfc9e33241ee84440ac8afc0f37178bb968c2c9d WatchSource:0}: Error finding container 6b2d4a5d3146008197e7fb9ecfc9e33241ee84440ac8afc0f37178bb968c2c9d: Status 404 returned error can't find the container with id 6b2d4a5d3146008197e7fb9ecfc9e33241ee84440ac8afc0f37178bb968c2c9d
Jul 02 14:06:38.374462 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:06:38.374429    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-6d218-predictor-fc586b6c5-77mmt" event={"ID":"aff672a5-50a2-46cb-85fd-41c7230f155f","Type":"ContainerStarted","Data":"11e71549f021cee55cc2645d572751aa4ec7d34ae1f138e4a673aee594f81cfc"}
Jul 02 14:06:38.374629 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:06:38.374488    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-6d218-predictor-fc586b6c5-77mmt" event={"ID":"aff672a5-50a2-46cb-85fd-41c7230f155f","Type":"ContainerStarted","Data":"08dd49f330c35db51b14609e9a5ffbfcaf2f7f7f2e3adf8ebcd340d47b101d29"}
Jul 02 14:06:38.374629 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:06:38.374517    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-6d218-predictor-fc586b6c5-77mmt" event={"ID":"aff672a5-50a2-46cb-85fd-41c7230f155f","Type":"ContainerStarted","Data":"6b2d4a5d3146008197e7fb9ecfc9e33241ee84440ac8afc0f37178bb968c2c9d"}
Jul 02 14:06:38.374629 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:06:38.374580    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-6d218-predictor-fc586b6c5-77mmt"
Jul 02 14:06:38.394665 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:06:38.394613    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-6d218-predictor-fc586b6c5-77mmt" podStartSLOduration=1.3945982080000001 podStartE2EDuration="1.394598208s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 14:06:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:06:38.3923704 +0000 UTC m=+1363.570832587" watchObservedRunningTime="2026-07-02 14:06:38.394598208 +0000 UTC m=+1363.573060391"
Jul 02 14:06:39.099584 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:06:39.099534    2558 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-c559a-predictor-7c98468fff-9t7fx" podUID="e1ed2cb5-74ad-404f-a6e4-68b4432279b3" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.22:8643/healthz\": dial tcp 10.133.0.22:8643: connect: connection refused"
Jul 02 14:06:39.378136 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:06:39.378048    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-6d218-predictor-fc586b6c5-77mmt"
Jul 02 14:06:39.379359 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:06:39.379328    2558 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-6d218-predictor-fc586b6c5-77mmt" podUID="aff672a5-50a2-46cb-85fd-41c7230f155f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.24:8080: connect: connection refused"
Jul 02 14:06:40.205076 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:06:40.205052    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-c559a-predictor-7c98468fff-9t7fx"
Jul 02 14:06:40.233687 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:06:40.233663    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cv5q2\" (UniqueName: \"kubernetes.io/projected/e1ed2cb5-74ad-404f-a6e4-68b4432279b3-kube-api-access-cv5q2\") pod \"e1ed2cb5-74ad-404f-a6e4-68b4432279b3\" (UID: \"e1ed2cb5-74ad-404f-a6e4-68b4432279b3\") "
Jul 02 14:06:40.233859 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:06:40.233699    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e1ed2cb5-74ad-404f-a6e4-68b4432279b3-proxy-tls\") pod \"e1ed2cb5-74ad-404f-a6e4-68b4432279b3\" (UID: \"e1ed2cb5-74ad-404f-a6e4-68b4432279b3\") "
Jul 02 14:06:40.233859 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:06:40.233741    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/e1ed2cb5-74ad-404f-a6e4-68b4432279b3-kube-rbac-proxy-sar-config\") pod \"e1ed2cb5-74ad-404f-a6e4-68b4432279b3\" (UID: \"e1ed2cb5-74ad-404f-a6e4-68b4432279b3\") "
Jul 02 14:06:40.234213 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:06:40.234186    2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e1ed2cb5-74ad-404f-a6e4-68b4432279b3-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "e1ed2cb5-74ad-404f-a6e4-68b4432279b3" (UID: "e1ed2cb5-74ad-404f-a6e4-68b4432279b3"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 02 14:06:40.236053 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:06:40.236026    2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e1ed2cb5-74ad-404f-a6e4-68b4432279b3-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "e1ed2cb5-74ad-404f-a6e4-68b4432279b3" (UID: "e1ed2cb5-74ad-404f-a6e4-68b4432279b3"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 02 14:06:40.236162 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:06:40.236050    2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e1ed2cb5-74ad-404f-a6e4-68b4432279b3-kube-api-access-cv5q2" (OuterVolumeSpecName: "kube-api-access-cv5q2") pod "e1ed2cb5-74ad-404f-a6e4-68b4432279b3" (UID: "e1ed2cb5-74ad-404f-a6e4-68b4432279b3"). InnerVolumeSpecName "kube-api-access-cv5q2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 14:06:40.335406 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:06:40.335317    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cv5q2\" (UniqueName: \"kubernetes.io/projected/e1ed2cb5-74ad-404f-a6e4-68b4432279b3-kube-api-access-cv5q2\") on node \"ip-10-0-133-209.ec2.internal\" DevicePath \"\""
Jul 02 14:06:40.335406 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:06:40.335347    2558 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e1ed2cb5-74ad-404f-a6e4-68b4432279b3-proxy-tls\") on node \"ip-10-0-133-209.ec2.internal\" DevicePath \"\""
Jul 02 14:06:40.335406 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:06:40.335356    2558 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/e1ed2cb5-74ad-404f-a6e4-68b4432279b3-kube-rbac-proxy-sar-config\") on node \"ip-10-0-133-209.ec2.internal\" DevicePath \"\""
Jul 02 14:06:40.383406 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:06:40.383376    2558 generic.go:356] "Generic (PLEG): container finished" podID="e1ed2cb5-74ad-404f-a6e4-68b4432279b3" containerID="448ea2940b912eec4f7d1af292c1dea7322ad415f946bc798623745d17c78c1e" exitCode=0
Jul 02 14:06:40.383526 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:06:40.383414    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-c559a-predictor-7c98468fff-9t7fx" event={"ID":"e1ed2cb5-74ad-404f-a6e4-68b4432279b3","Type":"ContainerDied","Data":"448ea2940b912eec4f7d1af292c1dea7322ad415f946bc798623745d17c78c1e"}
Jul 02 14:06:40.383526 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:06:40.383432    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-c559a-predictor-7c98468fff-9t7fx"
Jul 02 14:06:40.383526 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:06:40.383482    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-c559a-predictor-7c98468fff-9t7fx" event={"ID":"e1ed2cb5-74ad-404f-a6e4-68b4432279b3","Type":"ContainerDied","Data":"930d925b9ce7564b248e4144051f3d7d97003afd961e4674fd0e724b1a94a6e3"}
Jul 02 14:06:40.383526 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:06:40.383512    2558 scope.go:117] "RemoveContainer" containerID="bb649180cc20e631c63e0eef026d92ff1e79a68f617e24332766d871b3eae825"
Jul 02 14:06:40.384077 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:06:40.384048    2558 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-6d218-predictor-fc586b6c5-77mmt" podUID="aff672a5-50a2-46cb-85fd-41c7230f155f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.24:8080: connect: connection refused"
Jul 02 14:06:40.392319 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:06:40.392302    2558 scope.go:117] "RemoveContainer" containerID="448ea2940b912eec4f7d1af292c1dea7322ad415f946bc798623745d17c78c1e"
Jul 02 14:06:40.399634 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:06:40.399617    2558 scope.go:117] "RemoveContainer" containerID="bb649180cc20e631c63e0eef026d92ff1e79a68f617e24332766d871b3eae825"
Jul 02 14:06:40.399919 ip-10-0-133-209 kubenswrapper[2558]: E0702 14:06:40.399899    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bb649180cc20e631c63e0eef026d92ff1e79a68f617e24332766d871b3eae825\": container with ID starting with bb649180cc20e631c63e0eef026d92ff1e79a68f617e24332766d871b3eae825 not found: ID does not exist" containerID="bb649180cc20e631c63e0eef026d92ff1e79a68f617e24332766d871b3eae825"
Jul 02 14:06:40.399979 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:06:40.399931    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bb649180cc20e631c63e0eef026d92ff1e79a68f617e24332766d871b3eae825"} err="failed to get container status \"bb649180cc20e631c63e0eef026d92ff1e79a68f617e24332766d871b3eae825\": rpc error: code = NotFound desc = could not find container \"bb649180cc20e631c63e0eef026d92ff1e79a68f617e24332766d871b3eae825\": container with ID starting with bb649180cc20e631c63e0eef026d92ff1e79a68f617e24332766d871b3eae825 not found: ID does not exist"
Jul 02 14:06:40.399979 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:06:40.399951    2558 scope.go:117] "RemoveContainer" containerID="448ea2940b912eec4f7d1af292c1dea7322ad415f946bc798623745d17c78c1e"
Jul 02 14:06:40.400181 ip-10-0-133-209 kubenswrapper[2558]: E0702 14:06:40.400162    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"448ea2940b912eec4f7d1af292c1dea7322ad415f946bc798623745d17c78c1e\": container with ID starting with 448ea2940b912eec4f7d1af292c1dea7322ad415f946bc798623745d17c78c1e not found: ID does not exist" containerID="448ea2940b912eec4f7d1af292c1dea7322ad415f946bc798623745d17c78c1e"
Jul 02 14:06:40.400222 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:06:40.400188    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"448ea2940b912eec4f7d1af292c1dea7322ad415f946bc798623745d17c78c1e"} err="failed to get container status \"448ea2940b912eec4f7d1af292c1dea7322ad415f946bc798623745d17c78c1e\": rpc error: code = NotFound desc = could not find container \"448ea2940b912eec4f7d1af292c1dea7322ad415f946bc798623745d17c78c1e\": container with ID starting with 448ea2940b912eec4f7d1af292c1dea7322ad415f946bc798623745d17c78c1e not found: ID does not exist"
Jul 02 14:06:40.405281 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:06:40.405259    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-c559a-predictor-7c98468fff-9t7fx"]
Jul 02 14:06:40.409046 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:06:40.409027    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-c559a-predictor-7c98468fff-9t7fx"]
Jul 02 14:06:41.217101 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:06:41.217071    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e1ed2cb5-74ad-404f-a6e4-68b4432279b3" path="/var/lib/kubelet/pods/e1ed2cb5-74ad-404f-a6e4-68b4432279b3/volumes"
Jul 02 14:06:45.389047 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:06:45.389013    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-6d218-predictor-fc586b6c5-77mmt"
Jul 02 14:06:45.389568 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:06:45.389540    2558 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-6d218-predictor-fc586b6c5-77mmt" podUID="aff672a5-50a2-46cb-85fd-41c7230f155f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.24:8080: connect: connection refused"
Jul 02 14:06:55.390081 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:06:55.390043    2558 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-6d218-predictor-fc586b6c5-77mmt" podUID="aff672a5-50a2-46cb-85fd-41c7230f155f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.24:8080: connect: connection refused"
Jul 02 14:07:05.390160 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:07:05.390123    2558 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-6d218-predictor-fc586b6c5-77mmt" podUID="aff672a5-50a2-46cb-85fd-41c7230f155f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.24:8080: connect: connection refused"
Jul 02 14:07:15.390341 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:07:15.390304    2558 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-6d218-predictor-fc586b6c5-77mmt" podUID="aff672a5-50a2-46cb-85fd-41c7230f155f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.24:8080: connect: connection refused"
Jul 02 14:07:25.390562 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:07:25.390532    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-6d218-predictor-fc586b6c5-77mmt"
Jul 02 14:08:55.263908 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:08:55.263739    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8f2ll_3310d1f7-1643-4017-9edf-71fa702d2615/ovn-acl-logging/0.log"
Jul 02 14:08:55.267041 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:08:55.263978    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8f2ll_3310d1f7-1643-4017-9edf-71fa702d2615/ovn-acl-logging/0.log"
Jul 02 14:13:55.286881 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:13:55.286749    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8f2ll_3310d1f7-1643-4017-9edf-71fa702d2615/ovn-acl-logging/0.log"
Jul 02 14:13:55.291767 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:13:55.287307    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8f2ll_3310d1f7-1643-4017-9edf-71fa702d2615/ovn-acl-logging/0.log"
Jul 02 14:14:55.958799 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:14:55.958760    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-09121-predictor-78d4984875-8wwtg"]
Jul 02 14:14:55.959422 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:14:55.959222    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e1ed2cb5-74ad-404f-a6e4-68b4432279b3" containerName="kserve-container"
Jul 02 14:14:55.959422 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:14:55.959242    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="e1ed2cb5-74ad-404f-a6e4-68b4432279b3" containerName="kserve-container"
Jul 02 14:14:55.959422 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:14:55.959265    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e1ed2cb5-74ad-404f-a6e4-68b4432279b3" containerName="kube-rbac-proxy"
Jul 02 14:14:55.959422 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:14:55.959273    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="e1ed2cb5-74ad-404f-a6e4-68b4432279b3" containerName="kube-rbac-proxy"
Jul 02 14:14:55.959422 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:14:55.959342    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="e1ed2cb5-74ad-404f-a6e4-68b4432279b3" containerName="kube-rbac-proxy"
Jul 02 14:14:55.959422 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:14:55.959355    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="e1ed2cb5-74ad-404f-a6e4-68b4432279b3" containerName="kserve-container"
Jul 02 14:14:55.962452 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:14:55.962430    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-09121-predictor-78d4984875-8wwtg"
Jul 02 14:14:55.965209 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:14:55.965190    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-09121-predictor-serving-cert\""
Jul 02 14:14:55.965321 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:14:55.965191    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-09121-kube-rbac-proxy-sar-config\""
Jul 02 14:14:55.973131 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:14:55.973107    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-09121-predictor-78d4984875-8wwtg"]
Jul 02 14:14:56.009229 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:14:56.009190    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-7b7ab-predictor-7d65578975-ghbgv"]
Jul 02 14:14:56.009466 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:14:56.009441    2558 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-7b7ab-predictor-7d65578975-ghbgv" podUID="bf5c8734-5d7c-47a7-a030-d10ef03f926a" containerName="kserve-container" containerID="cri-o://267f3cf87698f810157861c6c10d5cf7afc3bcd427aa216496ea5b412e289b2c" gracePeriod=30
Jul 02 14:14:56.009581 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:14:56.009485    2558 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-7b7ab-predictor-7d65578975-ghbgv" podUID="bf5c8734-5d7c-47a7-a030-d10ef03f926a" containerName="kube-rbac-proxy" containerID="cri-o://f56a800075a501efa7730a254374de5cd3b09ea34c43dd8dc650e411367bc04c" gracePeriod=30
Jul 02 14:14:56.054233 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:14:56.054205    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/3506c5ec-7374-43af-a888-c0e6eb72dd6d-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-09121-predictor-78d4984875-8wwtg\" (UID: \"3506c5ec-7374-43af-a888-c0e6eb72dd6d\") " pod="kserve-ci-e2e-test/error-404-isvc-09121-predictor-78d4984875-8wwtg"
Jul 02 14:14:56.054387 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:14:56.054252    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jd9vz\" (UniqueName: \"kubernetes.io/projected/3506c5ec-7374-43af-a888-c0e6eb72dd6d-kube-api-access-jd9vz\") pod \"error-404-isvc-09121-predictor-78d4984875-8wwtg\" (UID: \"3506c5ec-7374-43af-a888-c0e6eb72dd6d\") " pod="kserve-ci-e2e-test/error-404-isvc-09121-predictor-78d4984875-8wwtg"
Jul 02 14:14:56.054387 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:14:56.054272    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3506c5ec-7374-43af-a888-c0e6eb72dd6d-proxy-tls\") pod \"error-404-isvc-09121-predictor-78d4984875-8wwtg\" (UID: \"3506c5ec-7374-43af-a888-c0e6eb72dd6d\") " pod="kserve-ci-e2e-test/error-404-isvc-09121-predictor-78d4984875-8wwtg"
Jul 02 14:14:56.155230 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:14:56.155194    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jd9vz\" (UniqueName: \"kubernetes.io/projected/3506c5ec-7374-43af-a888-c0e6eb72dd6d-kube-api-access-jd9vz\") pod \"error-404-isvc-09121-predictor-78d4984875-8wwtg\" (UID: \"3506c5ec-7374-43af-a888-c0e6eb72dd6d\") " pod="kserve-ci-e2e-test/error-404-isvc-09121-predictor-78d4984875-8wwtg"
Jul 02 14:14:56.155230 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:14:56.155235    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3506c5ec-7374-43af-a888-c0e6eb72dd6d-proxy-tls\") pod \"error-404-isvc-09121-predictor-78d4984875-8wwtg\" (UID: \"3506c5ec-7374-43af-a888-c0e6eb72dd6d\") " pod="kserve-ci-e2e-test/error-404-isvc-09121-predictor-78d4984875-8wwtg"
Jul 02 14:14:56.155463 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:14:56.155291    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/3506c5ec-7374-43af-a888-c0e6eb72dd6d-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-09121-predictor-78d4984875-8wwtg\" (UID: \"3506c5ec-7374-43af-a888-c0e6eb72dd6d\") " pod="kserve-ci-e2e-test/error-404-isvc-09121-predictor-78d4984875-8wwtg"
Jul 02 14:14:56.155463 ip-10-0-133-209 kubenswrapper[2558]: E0702 14:14:56.155438    2558 secret.go:189] Couldn't get secret kserve-ci-e2e-test/error-404-isvc-09121-predictor-serving-cert: secret "error-404-isvc-09121-predictor-serving-cert" not found
Jul 02 14:14:56.155570 ip-10-0-133-209 kubenswrapper[2558]: E0702 14:14:56.155511    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3506c5ec-7374-43af-a888-c0e6eb72dd6d-proxy-tls podName:3506c5ec-7374-43af-a888-c0e6eb72dd6d nodeName:}" failed. No retries permitted until 2026-07-02 14:14:56.655487833 +0000 UTC m=+1861.833950022 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/3506c5ec-7374-43af-a888-c0e6eb72dd6d-proxy-tls") pod "error-404-isvc-09121-predictor-78d4984875-8wwtg" (UID: "3506c5ec-7374-43af-a888-c0e6eb72dd6d") : secret "error-404-isvc-09121-predictor-serving-cert" not found
Jul 02 14:14:56.155944 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:14:56.155927    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/3506c5ec-7374-43af-a888-c0e6eb72dd6d-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-09121-predictor-78d4984875-8wwtg\" (UID: \"3506c5ec-7374-43af-a888-c0e6eb72dd6d\") " pod="kserve-ci-e2e-test/error-404-isvc-09121-predictor-78d4984875-8wwtg"
Jul 02 14:14:56.165456 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:14:56.165416    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jd9vz\" (UniqueName: \"kubernetes.io/projected/3506c5ec-7374-43af-a888-c0e6eb72dd6d-kube-api-access-jd9vz\") pod \"error-404-isvc-09121-predictor-78d4984875-8wwtg\" (UID: \"3506c5ec-7374-43af-a888-c0e6eb72dd6d\") " pod="kserve-ci-e2e-test/error-404-isvc-09121-predictor-78d4984875-8wwtg"
Jul 02 14:14:56.234381 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:14:56.234289    2558 generic.go:356] "Generic (PLEG): container finished" podID="bf5c8734-5d7c-47a7-a030-d10ef03f926a" containerID="f56a800075a501efa7730a254374de5cd3b09ea34c43dd8dc650e411367bc04c" exitCode=2
Jul 02 14:14:56.234381 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:14:56.234345    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-7b7ab-predictor-7d65578975-ghbgv" event={"ID":"bf5c8734-5d7c-47a7-a030-d10ef03f926a","Type":"ContainerDied","Data":"f56a800075a501efa7730a254374de5cd3b09ea34c43dd8dc650e411367bc04c"}
Jul 02 14:14:56.659200 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:14:56.659165    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3506c5ec-7374-43af-a888-c0e6eb72dd6d-proxy-tls\") pod \"error-404-isvc-09121-predictor-78d4984875-8wwtg\" (UID: \"3506c5ec-7374-43af-a888-c0e6eb72dd6d\") " pod="kserve-ci-e2e-test/error-404-isvc-09121-predictor-78d4984875-8wwtg"
Jul 02 14:14:56.661750 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:14:56.661719    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3506c5ec-7374-43af-a888-c0e6eb72dd6d-proxy-tls\") pod \"error-404-isvc-09121-predictor-78d4984875-8wwtg\" (UID: \"3506c5ec-7374-43af-a888-c0e6eb72dd6d\") " pod="kserve-ci-e2e-test/error-404-isvc-09121-predictor-78d4984875-8wwtg"
Jul 02 14:14:56.873249 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:14:56.873203    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-09121-predictor-78d4984875-8wwtg"
Jul 02 14:14:57.009869 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:14:57.009844    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-09121-predictor-78d4984875-8wwtg"]
Jul 02 14:14:57.011731 ip-10-0-133-209 kubenswrapper[2558]: W0702 14:14:57.011700    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3506c5ec_7374_43af_a888_c0e6eb72dd6d.slice/crio-a736878da6c36ab8f6f0f12f4a9386189815380edde79c5cb597c51dff88b1e7 WatchSource:0}: Error finding container a736878da6c36ab8f6f0f12f4a9386189815380edde79c5cb597c51dff88b1e7: Status 404 returned error can't find the container with id a736878da6c36ab8f6f0f12f4a9386189815380edde79c5cb597c51dff88b1e7
Jul 02 14:14:57.240416 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:14:57.240307    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-09121-predictor-78d4984875-8wwtg" event={"ID":"3506c5ec-7374-43af-a888-c0e6eb72dd6d","Type":"ContainerStarted","Data":"0a19afe6c42f95c0fba6d51d18d149442c34775f593e9d337a1de983c101db10"}
Jul 02 14:14:57.240416 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:14:57.240381    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-09121-predictor-78d4984875-8wwtg" event={"ID":"3506c5ec-7374-43af-a888-c0e6eb72dd6d","Type":"ContainerStarted","Data":"cdb55adac258f7a997104d245b23836960189ceddbb32481424e04c96f7a60b2"}
Jul 02 14:14:57.240416 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:14:57.240413    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-09121-predictor-78d4984875-8wwtg" event={"ID":"3506c5ec-7374-43af-a888-c0e6eb72dd6d","Type":"ContainerStarted","Data":"a736878da6c36ab8f6f0f12f4a9386189815380edde79c5cb597c51dff88b1e7"}
Jul 02 14:14:57.240619 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:14:57.240438    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-09121-predictor-78d4984875-8wwtg"
Jul 02 14:14:57.240619 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:14:57.240458    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-09121-predictor-78d4984875-8wwtg"
Jul 02 14:14:57.241625 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:14:57.241606    2558 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-09121-predictor-78d4984875-8wwtg" podUID="3506c5ec-7374-43af-a888-c0e6eb72dd6d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.25:8080: connect: connection refused"
Jul 02 14:14:57.261266 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:14:57.261208    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-09121-predictor-78d4984875-8wwtg" podStartSLOduration=2.2611905820000002 podStartE2EDuration="2.261190582s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 14:14:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:14:57.260183893 +0000 UTC m=+1862.438646077" watchObservedRunningTime="2026-07-02 14:14:57.261190582 +0000 UTC m=+1862.439652762"
Jul 02 14:14:58.245341 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:14:58.245301    2558 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-09121-predictor-78d4984875-8wwtg" podUID="3506c5ec-7374-43af-a888-c0e6eb72dd6d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.25:8080: connect: connection refused"
Jul 02 14:14:59.164738 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:14:59.164706    2558 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-7b7ab-predictor-7d65578975-ghbgv" podUID="bf5c8734-5d7c-47a7-a030-d10ef03f926a" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.23:8643/healthz\": dial tcp 10.133.0.23:8643: connect: connection refused"
Jul 02 14:14:59.170034 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:14:59.170011    2558 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-7b7ab-predictor-7d65578975-ghbgv" podUID="bf5c8734-5d7c-47a7-a030-d10ef03f926a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.23:8080: connect: connection refused"
Jul 02 14:14:59.250994 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:14:59.250958    2558 generic.go:356] "Generic (PLEG): container finished" podID="bf5c8734-5d7c-47a7-a030-d10ef03f926a" containerID="267f3cf87698f810157861c6c10d5cf7afc3bcd427aa216496ea5b412e289b2c" exitCode=0
Jul 02 14:14:59.251346 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:14:59.251020    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-7b7ab-predictor-7d65578975-ghbgv" event={"ID":"bf5c8734-5d7c-47a7-a030-d10ef03f926a","Type":"ContainerDied","Data":"267f3cf87698f810157861c6c10d5cf7afc3bcd427aa216496ea5b412e289b2c"}
Jul 02 14:14:59.357269 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:14:59.357243    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-7b7ab-predictor-7d65578975-ghbgv"
Jul 02 14:14:59.481498 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:14:59.481411    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z77jr\" (UniqueName: \"kubernetes.io/projected/bf5c8734-5d7c-47a7-a030-d10ef03f926a-kube-api-access-z77jr\") pod \"bf5c8734-5d7c-47a7-a030-d10ef03f926a\" (UID: \"bf5c8734-5d7c-47a7-a030-d10ef03f926a\") "
Jul 02 14:14:59.481498 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:14:59.481465    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/bf5c8734-5d7c-47a7-a030-d10ef03f926a-kube-rbac-proxy-sar-config\") pod \"bf5c8734-5d7c-47a7-a030-d10ef03f926a\" (UID: \"bf5c8734-5d7c-47a7-a030-d10ef03f926a\") "
Jul 02 14:14:59.481498 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:14:59.481501    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/bf5c8734-5d7c-47a7-a030-d10ef03f926a-proxy-tls\") pod \"bf5c8734-5d7c-47a7-a030-d10ef03f926a\" (UID: \"bf5c8734-5d7c-47a7-a030-d10ef03f926a\") "
Jul 02 14:14:59.481894 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:14:59.481862    2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bf5c8734-5d7c-47a7-a030-d10ef03f926a-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "bf5c8734-5d7c-47a7-a030-d10ef03f926a" (UID: "bf5c8734-5d7c-47a7-a030-d10ef03f926a"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 02 14:14:59.483751 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:14:59.483729    2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bf5c8734-5d7c-47a7-a030-d10ef03f926a-kube-api-access-z77jr" (OuterVolumeSpecName: "kube-api-access-z77jr") pod "bf5c8734-5d7c-47a7-a030-d10ef03f926a" (UID: "bf5c8734-5d7c-47a7-a030-d10ef03f926a"). InnerVolumeSpecName "kube-api-access-z77jr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 14:14:59.483884 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:14:59.483752    2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bf5c8734-5d7c-47a7-a030-d10ef03f926a-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "bf5c8734-5d7c-47a7-a030-d10ef03f926a" (UID: "bf5c8734-5d7c-47a7-a030-d10ef03f926a"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 02 14:14:59.582595 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:14:59.582553    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z77jr\" (UniqueName: \"kubernetes.io/projected/bf5c8734-5d7c-47a7-a030-d10ef03f926a-kube-api-access-z77jr\") on node \"ip-10-0-133-209.ec2.internal\" DevicePath \"\""
Jul 02 14:14:59.582595 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:14:59.582586    2558 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/bf5c8734-5d7c-47a7-a030-d10ef03f926a-kube-rbac-proxy-sar-config\") on node \"ip-10-0-133-209.ec2.internal\" DevicePath \"\""
Jul 02 14:14:59.582595 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:14:59.582596    2558 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/bf5c8734-5d7c-47a7-a030-d10ef03f926a-proxy-tls\") on node \"ip-10-0-133-209.ec2.internal\" DevicePath \"\""
Jul 02 14:15:00.256388 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:15:00.256359    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-7b7ab-predictor-7d65578975-ghbgv"
Jul 02 14:15:00.256388 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:15:00.256376    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-7b7ab-predictor-7d65578975-ghbgv" event={"ID":"bf5c8734-5d7c-47a7-a030-d10ef03f926a","Type":"ContainerDied","Data":"5a03346db84d747af0f5a13b08c7cd134155ab19632b0a5cb26916e82690bd14"}
Jul 02 14:15:00.256866 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:15:00.256442    2558 scope.go:117] "RemoveContainer" containerID="f56a800075a501efa7730a254374de5cd3b09ea34c43dd8dc650e411367bc04c"
Jul 02 14:15:00.264858 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:15:00.264816    2558 scope.go:117] "RemoveContainer" containerID="267f3cf87698f810157861c6c10d5cf7afc3bcd427aa216496ea5b412e289b2c"
Jul 02 14:15:00.278000 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:15:00.277979    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-7b7ab-predictor-7d65578975-ghbgv"]
Jul 02 14:15:00.282731 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:15:00.282709    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-7b7ab-predictor-7d65578975-ghbgv"]
Jul 02 14:15:01.216616 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:15:01.216575    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bf5c8734-5d7c-47a7-a030-d10ef03f926a" path="/var/lib/kubelet/pods/bf5c8734-5d7c-47a7-a030-d10ef03f926a/volumes"
Jul 02 14:15:03.250162 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:15:03.250133    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-09121-predictor-78d4984875-8wwtg"
Jul 02 14:15:03.250577 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:15:03.250547    2558 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-09121-predictor-78d4984875-8wwtg" podUID="3506c5ec-7374-43af-a888-c0e6eb72dd6d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.25:8080: connect: connection refused"
Jul 02 14:15:13.251211 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:15:13.251174    2558 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-09121-predictor-78d4984875-8wwtg" podUID="3506c5ec-7374-43af-a888-c0e6eb72dd6d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.25:8080: connect: connection refused"
Jul 02 14:15:23.250661 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:15:23.250575    2558 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-09121-predictor-78d4984875-8wwtg" podUID="3506c5ec-7374-43af-a888-c0e6eb72dd6d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.25:8080: connect: connection refused"
Jul 02 14:15:33.251406 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:15:33.251359    2558 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-09121-predictor-78d4984875-8wwtg" podUID="3506c5ec-7374-43af-a888-c0e6eb72dd6d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.25:8080: connect: connection refused"
Jul 02 14:15:43.251920 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:15:43.251881    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-09121-predictor-78d4984875-8wwtg"
Jul 02 14:15:51.959456 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:15:51.959419    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-8d2ee-predictor-6f9b7fbf65-8r9xm"]
Jul 02 14:15:51.959951 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:15:51.959735    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bf5c8734-5d7c-47a7-a030-d10ef03f926a" containerName="kserve-container"
Jul 02 14:15:51.959951 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:15:51.959747    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf5c8734-5d7c-47a7-a030-d10ef03f926a" containerName="kserve-container"
Jul 02 14:15:51.959951 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:15:51.959756    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bf5c8734-5d7c-47a7-a030-d10ef03f926a" containerName="kube-rbac-proxy"
Jul 02 14:15:51.959951 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:15:51.959762    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf5c8734-5d7c-47a7-a030-d10ef03f926a" containerName="kube-rbac-proxy"
Jul 02 14:15:51.959951 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:15:51.959813    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="bf5c8734-5d7c-47a7-a030-d10ef03f926a" containerName="kserve-container"
Jul 02 14:15:51.959951 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:15:51.959836    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="bf5c8734-5d7c-47a7-a030-d10ef03f926a" containerName="kube-rbac-proxy"
Jul 02 14:15:51.963016 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:15:51.962999    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-8d2ee-predictor-6f9b7fbf65-8r9xm"
Jul 02 14:15:51.965466 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:15:51.965443    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-8d2ee-predictor-serving-cert\""
Jul 02 14:15:51.965695 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:15:51.965678    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-8d2ee-kube-rbac-proxy-sar-config\""
Jul 02 14:15:51.973933 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:15:51.973901    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-8d2ee-predictor-6f9b7fbf65-8r9xm"]
Jul 02 14:15:52.003860 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:15:52.003804    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9rq9s\" (UniqueName: \"kubernetes.io/projected/373c3199-3158-4e4c-8eab-6105fc0d4734-kube-api-access-9rq9s\") pod \"error-404-isvc-8d2ee-predictor-6f9b7fbf65-8r9xm\" (UID: \"373c3199-3158-4e4c-8eab-6105fc0d4734\") " pod="kserve-ci-e2e-test/error-404-isvc-8d2ee-predictor-6f9b7fbf65-8r9xm"
Jul 02 14:15:52.004059 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:15:52.003889    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/373c3199-3158-4e4c-8eab-6105fc0d4734-proxy-tls\") pod \"error-404-isvc-8d2ee-predictor-6f9b7fbf65-8r9xm\" (UID: \"373c3199-3158-4e4c-8eab-6105fc0d4734\") " pod="kserve-ci-e2e-test/error-404-isvc-8d2ee-predictor-6f9b7fbf65-8r9xm"
Jul 02 14:15:52.004059 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:15:52.003909    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/373c3199-3158-4e4c-8eab-6105fc0d4734-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-8d2ee-predictor-6f9b7fbf65-8r9xm\" (UID: \"373c3199-3158-4e4c-8eab-6105fc0d4734\") " pod="kserve-ci-e2e-test/error-404-isvc-8d2ee-predictor-6f9b7fbf65-8r9xm"
Jul 02 14:15:52.054923 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:15:52.054887    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-6d218-predictor-fc586b6c5-77mmt"]
Jul 02 14:15:52.055168 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:15:52.055136    2558 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-6d218-predictor-fc586b6c5-77mmt" podUID="aff672a5-50a2-46cb-85fd-41c7230f155f" containerName="kserve-container" containerID="cri-o://08dd49f330c35db51b14609e9a5ffbfcaf2f7f7f2e3adf8ebcd340d47b101d29" gracePeriod=30
Jul 02 14:15:52.055300 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:15:52.055173    2558 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-6d218-predictor-fc586b6c5-77mmt" podUID="aff672a5-50a2-46cb-85fd-41c7230f155f" containerName="kube-rbac-proxy" containerID="cri-o://11e71549f021cee55cc2645d572751aa4ec7d34ae1f138e4a673aee594f81cfc" gracePeriod=30
Jul 02 14:15:52.105184 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:15:52.105150    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9rq9s\" (UniqueName: \"kubernetes.io/projected/373c3199-3158-4e4c-8eab-6105fc0d4734-kube-api-access-9rq9s\") pod \"error-404-isvc-8d2ee-predictor-6f9b7fbf65-8r9xm\" (UID: \"373c3199-3158-4e4c-8eab-6105fc0d4734\") " pod="kserve-ci-e2e-test/error-404-isvc-8d2ee-predictor-6f9b7fbf65-8r9xm"
Jul 02 14:15:52.105404 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:15:52.105204    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/373c3199-3158-4e4c-8eab-6105fc0d4734-proxy-tls\") pod \"error-404-isvc-8d2ee-predictor-6f9b7fbf65-8r9xm\" (UID: \"373c3199-3158-4e4c-8eab-6105fc0d4734\") " pod="kserve-ci-e2e-test/error-404-isvc-8d2ee-predictor-6f9b7fbf65-8r9xm"
Jul 02 14:15:52.105404 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:15:52.105224    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/373c3199-3158-4e4c-8eab-6105fc0d4734-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-8d2ee-predictor-6f9b7fbf65-8r9xm\" (UID: \"373c3199-3158-4e4c-8eab-6105fc0d4734\") " pod="kserve-ci-e2e-test/error-404-isvc-8d2ee-predictor-6f9b7fbf65-8r9xm"
Jul 02 14:15:52.105404 ip-10-0-133-209 kubenswrapper[2558]: E0702 14:15:52.105329    2558 secret.go:189] Couldn't get secret kserve-ci-e2e-test/error-404-isvc-8d2ee-predictor-serving-cert: secret "error-404-isvc-8d2ee-predictor-serving-cert" not found
Jul 02 14:15:52.105404 ip-10-0-133-209 kubenswrapper[2558]: E0702 14:15:52.105399    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/373c3199-3158-4e4c-8eab-6105fc0d4734-proxy-tls podName:373c3199-3158-4e4c-8eab-6105fc0d4734 nodeName:}" failed. No retries permitted until 2026-07-02 14:15:52.605377379 +0000 UTC m=+1917.783839546 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/373c3199-3158-4e4c-8eab-6105fc0d4734-proxy-tls") pod "error-404-isvc-8d2ee-predictor-6f9b7fbf65-8r9xm" (UID: "373c3199-3158-4e4c-8eab-6105fc0d4734") : secret "error-404-isvc-8d2ee-predictor-serving-cert" not found
Jul 02 14:15:52.105908 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:15:52.105891    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/373c3199-3158-4e4c-8eab-6105fc0d4734-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-8d2ee-predictor-6f9b7fbf65-8r9xm\" (UID: \"373c3199-3158-4e4c-8eab-6105fc0d4734\") " pod="kserve-ci-e2e-test/error-404-isvc-8d2ee-predictor-6f9b7fbf65-8r9xm"
Jul 02 14:15:52.114625 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:15:52.114599    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9rq9s\" (UniqueName: \"kubernetes.io/projected/373c3199-3158-4e4c-8eab-6105fc0d4734-kube-api-access-9rq9s\") pod \"error-404-isvc-8d2ee-predictor-6f9b7fbf65-8r9xm\" (UID: \"373c3199-3158-4e4c-8eab-6105fc0d4734\") " pod="kserve-ci-e2e-test/error-404-isvc-8d2ee-predictor-6f9b7fbf65-8r9xm"
Jul 02 14:15:52.455128 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:15:52.455082    2558 generic.go:356] "Generic (PLEG): container finished" podID="aff672a5-50a2-46cb-85fd-41c7230f155f" containerID="11e71549f021cee55cc2645d572751aa4ec7d34ae1f138e4a673aee594f81cfc" exitCode=2
Jul 02 14:15:52.455128 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:15:52.455130    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-6d218-predictor-fc586b6c5-77mmt" event={"ID":"aff672a5-50a2-46cb-85fd-41c7230f155f","Type":"ContainerDied","Data":"11e71549f021cee55cc2645d572751aa4ec7d34ae1f138e4a673aee594f81cfc"}
Jul 02 14:15:52.609281 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:15:52.609243    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/373c3199-3158-4e4c-8eab-6105fc0d4734-proxy-tls\") pod \"error-404-isvc-8d2ee-predictor-6f9b7fbf65-8r9xm\" (UID: \"373c3199-3158-4e4c-8eab-6105fc0d4734\") " pod="kserve-ci-e2e-test/error-404-isvc-8d2ee-predictor-6f9b7fbf65-8r9xm"
Jul 02 14:15:52.611744 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:15:52.611713    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/373c3199-3158-4e4c-8eab-6105fc0d4734-proxy-tls\") pod \"error-404-isvc-8d2ee-predictor-6f9b7fbf65-8r9xm\" (UID: \"373c3199-3158-4e4c-8eab-6105fc0d4734\") " pod="kserve-ci-e2e-test/error-404-isvc-8d2ee-predictor-6f9b7fbf65-8r9xm"
Jul 02 14:15:52.873261 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:15:52.873222    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-8d2ee-predictor-6f9b7fbf65-8r9xm"
Jul 02 14:15:53.005818 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:15:53.005793    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-8d2ee-predictor-6f9b7fbf65-8r9xm"]
Jul 02 14:15:53.008269 ip-10-0-133-209 kubenswrapper[2558]: W0702 14:15:53.008237    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod373c3199_3158_4e4c_8eab_6105fc0d4734.slice/crio-9013e508e55ff4ec4c5d4e8aace82996886d26df806d8398734b44cd05632834 WatchSource:0}: Error finding container 9013e508e55ff4ec4c5d4e8aace82996886d26df806d8398734b44cd05632834: Status 404 returned error can't find the container with id 9013e508e55ff4ec4c5d4e8aace82996886d26df806d8398734b44cd05632834
Jul 02 14:15:53.465877 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:15:53.465835    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-8d2ee-predictor-6f9b7fbf65-8r9xm" event={"ID":"373c3199-3158-4e4c-8eab-6105fc0d4734","Type":"ContainerStarted","Data":"8ede763652faf80e1b94a87ced1c1201e1ad017720952cbbbbd14e244a1548c3"}
Jul 02 14:15:53.466051 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:15:53.465934    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-8d2ee-predictor-6f9b7fbf65-8r9xm" event={"ID":"373c3199-3158-4e4c-8eab-6105fc0d4734","Type":"ContainerStarted","Data":"7db11b1b2f00c99285e19b224e5fd1fa41d2bf0a13247a3d1b4ca5eaca4c276b"}
Jul 02 14:15:53.466051 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:15:53.465964    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-8d2ee-predictor-6f9b7fbf65-8r9xm"
Jul 02 14:15:53.466051 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:15:53.465985    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-8d2ee-predictor-6f9b7fbf65-8r9xm" event={"ID":"373c3199-3158-4e4c-8eab-6105fc0d4734","Type":"ContainerStarted","Data":"9013e508e55ff4ec4c5d4e8aace82996886d26df806d8398734b44cd05632834"}
Jul 02 14:15:53.466051 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:15:53.466003    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-8d2ee-predictor-6f9b7fbf65-8r9xm"
Jul 02 14:15:53.467280 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:15:53.467256    2558 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-8d2ee-predictor-6f9b7fbf65-8r9xm" podUID="373c3199-3158-4e4c-8eab-6105fc0d4734" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.26:8080: connect: connection refused"
Jul 02 14:15:53.486601 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:15:53.486549    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-8d2ee-predictor-6f9b7fbf65-8r9xm" podStartSLOduration=2.486535184 podStartE2EDuration="2.486535184s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 14:15:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:15:53.48584837 +0000 UTC m=+1918.664310547" watchObservedRunningTime="2026-07-02 14:15:53.486535184 +0000 UTC m=+1918.664997368"
Jul 02 14:15:54.469869 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:15:54.469800    2558 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-8d2ee-predictor-6f9b7fbf65-8r9xm" podUID="373c3199-3158-4e4c-8eab-6105fc0d4734" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.26:8080: connect: connection refused"
Jul 02 14:15:55.206950 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:15:55.206926    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-6d218-predictor-fc586b6c5-77mmt"
Jul 02 14:15:55.330802 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:15:55.330700    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/aff672a5-50a2-46cb-85fd-41c7230f155f-proxy-tls\") pod \"aff672a5-50a2-46cb-85fd-41c7230f155f\" (UID: \"aff672a5-50a2-46cb-85fd-41c7230f155f\") "
Jul 02 14:15:55.330802 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:15:55.330754    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/aff672a5-50a2-46cb-85fd-41c7230f155f-kube-rbac-proxy-sar-config\") pod \"aff672a5-50a2-46cb-85fd-41c7230f155f\" (UID: \"aff672a5-50a2-46cb-85fd-41c7230f155f\") "
Jul 02 14:15:55.330802 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:15:55.330773    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d4tbf\" (UniqueName: \"kubernetes.io/projected/aff672a5-50a2-46cb-85fd-41c7230f155f-kube-api-access-d4tbf\") pod \"aff672a5-50a2-46cb-85fd-41c7230f155f\" (UID: \"aff672a5-50a2-46cb-85fd-41c7230f155f\") "
Jul 02 14:15:55.331169 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:15:55.331143    2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aff672a5-50a2-46cb-85fd-41c7230f155f-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "aff672a5-50a2-46cb-85fd-41c7230f155f" (UID: "aff672a5-50a2-46cb-85fd-41c7230f155f"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 02 14:15:55.333054 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:15:55.333025    2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aff672a5-50a2-46cb-85fd-41c7230f155f-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "aff672a5-50a2-46cb-85fd-41c7230f155f" (UID: "aff672a5-50a2-46cb-85fd-41c7230f155f"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 02 14:15:55.333054 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:15:55.333041    2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aff672a5-50a2-46cb-85fd-41c7230f155f-kube-api-access-d4tbf" (OuterVolumeSpecName: "kube-api-access-d4tbf") pod "aff672a5-50a2-46cb-85fd-41c7230f155f" (UID: "aff672a5-50a2-46cb-85fd-41c7230f155f"). InnerVolumeSpecName "kube-api-access-d4tbf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 14:15:55.334957 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:15:55.334935    2558 scope.go:117] "RemoveContainer" containerID="11e71549f021cee55cc2645d572751aa4ec7d34ae1f138e4a673aee594f81cfc"
Jul 02 14:15:55.347958 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:15:55.347937    2558 scope.go:117] "RemoveContainer" containerID="08dd49f330c35db51b14609e9a5ffbfcaf2f7f7f2e3adf8ebcd340d47b101d29"
Jul 02 14:15:55.431513 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:15:55.431474    2558 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/aff672a5-50a2-46cb-85fd-41c7230f155f-proxy-tls\") on node \"ip-10-0-133-209.ec2.internal\" DevicePath \"\""
Jul 02 14:15:55.431513 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:15:55.431510    2558 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/aff672a5-50a2-46cb-85fd-41c7230f155f-kube-rbac-proxy-sar-config\") on node \"ip-10-0-133-209.ec2.internal\" DevicePath \"\""
Jul 02 14:15:55.431723 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:15:55.431525    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d4tbf\" (UniqueName: \"kubernetes.io/projected/aff672a5-50a2-46cb-85fd-41c7230f155f-kube-api-access-d4tbf\") on node \"ip-10-0-133-209.ec2.internal\" DevicePath \"\""
Jul 02 14:15:55.474156 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:15:55.474122    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-6d218-predictor-fc586b6c5-77mmt"
Jul 02 14:15:55.474544 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:15:55.474159    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-6d218-predictor-fc586b6c5-77mmt" event={"ID":"aff672a5-50a2-46cb-85fd-41c7230f155f","Type":"ContainerDied","Data":"08dd49f330c35db51b14609e9a5ffbfcaf2f7f7f2e3adf8ebcd340d47b101d29"}
Jul 02 14:15:55.474544 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:15:55.474225    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-6d218-predictor-fc586b6c5-77mmt" event={"ID":"aff672a5-50a2-46cb-85fd-41c7230f155f","Type":"ContainerDied","Data":"6b2d4a5d3146008197e7fb9ecfc9e33241ee84440ac8afc0f37178bb968c2c9d"}
Jul 02 14:15:55.509065 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:15:55.509030    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-6d218-predictor-fc586b6c5-77mmt"]
Jul 02 14:15:55.514020 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:15:55.513993    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-6d218-predictor-fc586b6c5-77mmt"]
Jul 02 14:15:57.216798 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:15:57.216763    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aff672a5-50a2-46cb-85fd-41c7230f155f" path="/var/lib/kubelet/pods/aff672a5-50a2-46cb-85fd-41c7230f155f/volumes"
Jul 02 14:15:59.474359 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:15:59.474323    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-8d2ee-predictor-6f9b7fbf65-8r9xm"
Jul 02 14:15:59.474974 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:15:59.474942    2558 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-8d2ee-predictor-6f9b7fbf65-8r9xm" podUID="373c3199-3158-4e4c-8eab-6105fc0d4734" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.26:8080: connect: connection refused"
Jul 02 14:16:06.263815 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:16:06.263778    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-78590-predictor-64948cf447-bmtpr"]
Jul 02 14:16:06.264229 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:16:06.264090    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aff672a5-50a2-46cb-85fd-41c7230f155f" containerName="kube-rbac-proxy"
Jul 02 14:16:06.264229 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:16:06.264104    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="aff672a5-50a2-46cb-85fd-41c7230f155f" containerName="kube-rbac-proxy"
Jul 02 14:16:06.264229 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:16:06.264116    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aff672a5-50a2-46cb-85fd-41c7230f155f" containerName="kserve-container"
Jul 02 14:16:06.264229 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:16:06.264121    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="aff672a5-50a2-46cb-85fd-41c7230f155f" containerName="kserve-container"
Jul 02 14:16:06.264229 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:16:06.264176    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="aff672a5-50a2-46cb-85fd-41c7230f155f" containerName="kserve-container"
Jul 02 14:16:06.264229 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:16:06.264185    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="aff672a5-50a2-46cb-85fd-41c7230f155f" containerName="kube-rbac-proxy"
Jul 02 14:16:06.268295 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:16:06.268277    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-78590-predictor-64948cf447-bmtpr"
Jul 02 14:16:06.271236 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:16:06.271209    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-78590-kube-rbac-proxy-sar-config\""
Jul 02 14:16:06.271358 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:16:06.271240    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-78590-predictor-serving-cert\""
Jul 02 14:16:06.280696 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:16:06.280667    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-78590-predictor-64948cf447-bmtpr"]
Jul 02 14:16:06.311291 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:16:06.311263    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-09121-predictor-78d4984875-8wwtg"]
Jul 02 14:16:06.311547 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:16:06.311522    2558 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-09121-predictor-78d4984875-8wwtg" podUID="3506c5ec-7374-43af-a888-c0e6eb72dd6d" containerName="kube-rbac-proxy" containerID="cri-o://0a19afe6c42f95c0fba6d51d18d149442c34775f593e9d337a1de983c101db10" gracePeriod=30
Jul 02 14:16:06.311547 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:16:06.311516    2558 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-09121-predictor-78d4984875-8wwtg" podUID="3506c5ec-7374-43af-a888-c0e6eb72dd6d" containerName="kserve-container" containerID="cri-o://cdb55adac258f7a997104d245b23836960189ceddbb32481424e04c96f7a60b2" gracePeriod=30
Jul 02 14:16:06.322236 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:16:06.322211    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lmtkp\" (UniqueName: \"kubernetes.io/projected/99bd2872-b70a-4d62-bddf-1947e802d6ed-kube-api-access-lmtkp\") pod \"error-404-isvc-78590-predictor-64948cf447-bmtpr\" (UID: \"99bd2872-b70a-4d62-bddf-1947e802d6ed\") " pod="kserve-ci-e2e-test/error-404-isvc-78590-predictor-64948cf447-bmtpr"
Jul 02 14:16:06.322322 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:16:06.322251    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/99bd2872-b70a-4d62-bddf-1947e802d6ed-proxy-tls\") pod \"error-404-isvc-78590-predictor-64948cf447-bmtpr\" (UID: \"99bd2872-b70a-4d62-bddf-1947e802d6ed\") " pod="kserve-ci-e2e-test/error-404-isvc-78590-predictor-64948cf447-bmtpr"
Jul 02 14:16:06.322322 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:16:06.322285    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/99bd2872-b70a-4d62-bddf-1947e802d6ed-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-78590-predictor-64948cf447-bmtpr\" (UID: \"99bd2872-b70a-4d62-bddf-1947e802d6ed\") " pod="kserve-ci-e2e-test/error-404-isvc-78590-predictor-64948cf447-bmtpr"
Jul 02 14:16:06.423021 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:16:06.422987    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lmtkp\" (UniqueName: \"kubernetes.io/projected/99bd2872-b70a-4d62-bddf-1947e802d6ed-kube-api-access-lmtkp\") pod \"error-404-isvc-78590-predictor-64948cf447-bmtpr\" (UID: \"99bd2872-b70a-4d62-bddf-1947e802d6ed\") " pod="kserve-ci-e2e-test/error-404-isvc-78590-predictor-64948cf447-bmtpr"
Jul 02 14:16:06.423021 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:16:06.423023    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/99bd2872-b70a-4d62-bddf-1947e802d6ed-proxy-tls\") pod \"error-404-isvc-78590-predictor-64948cf447-bmtpr\" (UID: \"99bd2872-b70a-4d62-bddf-1947e802d6ed\") " pod="kserve-ci-e2e-test/error-404-isvc-78590-predictor-64948cf447-bmtpr"
Jul 02 14:16:06.423255 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:16:06.423052    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/99bd2872-b70a-4d62-bddf-1947e802d6ed-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-78590-predictor-64948cf447-bmtpr\" (UID: \"99bd2872-b70a-4d62-bddf-1947e802d6ed\") " pod="kserve-ci-e2e-test/error-404-isvc-78590-predictor-64948cf447-bmtpr"
Jul 02 14:16:06.423652 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:16:06.423633    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/99bd2872-b70a-4d62-bddf-1947e802d6ed-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-78590-predictor-64948cf447-bmtpr\" (UID: \"99bd2872-b70a-4d62-bddf-1947e802d6ed\") " pod="kserve-ci-e2e-test/error-404-isvc-78590-predictor-64948cf447-bmtpr"
Jul 02 14:16:06.425486 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:16:06.425464    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/99bd2872-b70a-4d62-bddf-1947e802d6ed-proxy-tls\") pod \"error-404-isvc-78590-predictor-64948cf447-bmtpr\" (UID: \"99bd2872-b70a-4d62-bddf-1947e802d6ed\") " pod="kserve-ci-e2e-test/error-404-isvc-78590-predictor-64948cf447-bmtpr"
Jul 02 14:16:06.432134 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:16:06.432112    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lmtkp\" (UniqueName: \"kubernetes.io/projected/99bd2872-b70a-4d62-bddf-1947e802d6ed-kube-api-access-lmtkp\") pod \"error-404-isvc-78590-predictor-64948cf447-bmtpr\" (UID: \"99bd2872-b70a-4d62-bddf-1947e802d6ed\") " pod="kserve-ci-e2e-test/error-404-isvc-78590-predictor-64948cf447-bmtpr"
Jul 02 14:16:06.518237 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:16:06.518147    2558 generic.go:356] "Generic (PLEG): container finished" podID="3506c5ec-7374-43af-a888-c0e6eb72dd6d" containerID="0a19afe6c42f95c0fba6d51d18d149442c34775f593e9d337a1de983c101db10" exitCode=2
Jul 02 14:16:06.518237 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:16:06.518215    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-09121-predictor-78d4984875-8wwtg" event={"ID":"3506c5ec-7374-43af-a888-c0e6eb72dd6d","Type":"ContainerDied","Data":"0a19afe6c42f95c0fba6d51d18d149442c34775f593e9d337a1de983c101db10"}
Jul 02 14:16:06.578503 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:16:06.578454    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-78590-predictor-64948cf447-bmtpr"
Jul 02 14:16:06.716384 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:16:06.716360    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-78590-predictor-64948cf447-bmtpr"]
Jul 02 14:16:06.718366 ip-10-0-133-209 kubenswrapper[2558]: W0702 14:16:06.718338    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99bd2872_b70a_4d62_bddf_1947e802d6ed.slice/crio-695248d5fa827762d417ba7cba6625463924b0a669cea1efc8cd19295a175d74 WatchSource:0}: Error finding container 695248d5fa827762d417ba7cba6625463924b0a669cea1efc8cd19295a175d74: Status 404 returned error can't find the container with id 695248d5fa827762d417ba7cba6625463924b0a669cea1efc8cd19295a175d74
Jul 02 14:16:07.523575 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:16:07.523537    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-78590-predictor-64948cf447-bmtpr" event={"ID":"99bd2872-b70a-4d62-bddf-1947e802d6ed","Type":"ContainerStarted","Data":"a7a56fc4cb3a8297bd405e70525c03c180aa751927e43fadd4c1a257afcc6350"}
Jul 02 14:16:07.524047 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:16:07.523599    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-78590-predictor-64948cf447-bmtpr" event={"ID":"99bd2872-b70a-4d62-bddf-1947e802d6ed","Type":"ContainerStarted","Data":"75419986d4d07e002f13d80e30a94dc144a2669a8f82f63107a240fa57af4f8f"}
Jul 02 14:16:07.524047 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:16:07.523620    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-78590-predictor-64948cf447-bmtpr" event={"ID":"99bd2872-b70a-4d62-bddf-1947e802d6ed","Type":"ContainerStarted","Data":"695248d5fa827762d417ba7cba6625463924b0a669cea1efc8cd19295a175d74"}
Jul 02 14:16:07.524047 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:16:07.523642    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-78590-predictor-64948cf447-bmtpr"
Jul 02 14:16:07.524047 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:16:07.523758    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-78590-predictor-64948cf447-bmtpr"
Jul 02 14:16:07.525169 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:16:07.525144    2558 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-78590-predictor-64948cf447-bmtpr" podUID="99bd2872-b70a-4d62-bddf-1947e802d6ed" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.27:8080: connect: connection refused"
Jul 02 14:16:07.545697 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:16:07.545631    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-78590-predictor-64948cf447-bmtpr" podStartSLOduration=1.545612794 podStartE2EDuration="1.545612794s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 14:16:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:16:07.544186331 +0000 UTC m=+1932.722648515" watchObservedRunningTime="2026-07-02 14:16:07.545612794 +0000 UTC m=+1932.724074979"
Jul 02 14:16:08.246189 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:16:08.246148    2558 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-09121-predictor-78d4984875-8wwtg" podUID="3506c5ec-7374-43af-a888-c0e6eb72dd6d" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.25:8643/healthz\": dial tcp 10.133.0.25:8643: connect: connection refused"
Jul 02 14:16:08.528141 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:16:08.528052    2558 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-78590-predictor-64948cf447-bmtpr" podUID="99bd2872-b70a-4d62-bddf-1947e802d6ed" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.27:8080: connect: connection refused"
Jul 02 14:16:09.474897 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:16:09.474860    2558 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-8d2ee-predictor-6f9b7fbf65-8r9xm" podUID="373c3199-3158-4e4c-8eab-6105fc0d4734" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.26:8080: connect: connection refused"
Jul 02 14:16:09.533190 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:16:09.533160    2558 generic.go:356] "Generic (PLEG): container finished" podID="3506c5ec-7374-43af-a888-c0e6eb72dd6d" containerID="cdb55adac258f7a997104d245b23836960189ceddbb32481424e04c96f7a60b2" exitCode=0
Jul 02 14:16:09.533528 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:16:09.533228    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-09121-predictor-78d4984875-8wwtg" event={"ID":"3506c5ec-7374-43af-a888-c0e6eb72dd6d","Type":"ContainerDied","Data":"cdb55adac258f7a997104d245b23836960189ceddbb32481424e04c96f7a60b2"}
Jul 02 14:16:09.661278 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:16:09.661255    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-09121-predictor-78d4984875-8wwtg"
Jul 02 14:16:09.751198 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:16:09.751103    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/3506c5ec-7374-43af-a888-c0e6eb72dd6d-kube-rbac-proxy-sar-config\") pod \"3506c5ec-7374-43af-a888-c0e6eb72dd6d\" (UID: \"3506c5ec-7374-43af-a888-c0e6eb72dd6d\") "
Jul 02 14:16:09.751198 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:16:09.751148    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3506c5ec-7374-43af-a888-c0e6eb72dd6d-proxy-tls\") pod \"3506c5ec-7374-43af-a888-c0e6eb72dd6d\" (UID: \"3506c5ec-7374-43af-a888-c0e6eb72dd6d\") "
Jul 02 14:16:09.751423 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:16:09.751237    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jd9vz\" (UniqueName: \"kubernetes.io/projected/3506c5ec-7374-43af-a888-c0e6eb72dd6d-kube-api-access-jd9vz\") pod \"3506c5ec-7374-43af-a888-c0e6eb72dd6d\" (UID: \"3506c5ec-7374-43af-a888-c0e6eb72dd6d\") "
Jul 02 14:16:09.751503 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:16:09.751482    2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3506c5ec-7374-43af-a888-c0e6eb72dd6d-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "3506c5ec-7374-43af-a888-c0e6eb72dd6d" (UID: "3506c5ec-7374-43af-a888-c0e6eb72dd6d"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 02 14:16:09.753314 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:16:09.753289    2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3506c5ec-7374-43af-a888-c0e6eb72dd6d-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "3506c5ec-7374-43af-a888-c0e6eb72dd6d" (UID: "3506c5ec-7374-43af-a888-c0e6eb72dd6d"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 02 14:16:09.753405 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:16:09.753362    2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3506c5ec-7374-43af-a888-c0e6eb72dd6d-kube-api-access-jd9vz" (OuterVolumeSpecName: "kube-api-access-jd9vz") pod "3506c5ec-7374-43af-a888-c0e6eb72dd6d" (UID: "3506c5ec-7374-43af-a888-c0e6eb72dd6d"). InnerVolumeSpecName "kube-api-access-jd9vz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 14:16:09.852752 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:16:09.852717    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jd9vz\" (UniqueName: \"kubernetes.io/projected/3506c5ec-7374-43af-a888-c0e6eb72dd6d-kube-api-access-jd9vz\") on node \"ip-10-0-133-209.ec2.internal\" DevicePath \"\""
Jul 02 14:16:09.852752 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:16:09.852742    2558 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/3506c5ec-7374-43af-a888-c0e6eb72dd6d-kube-rbac-proxy-sar-config\") on node \"ip-10-0-133-209.ec2.internal\" DevicePath \"\""
Jul 02 14:16:09.852752 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:16:09.852752    2558 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3506c5ec-7374-43af-a888-c0e6eb72dd6d-proxy-tls\") on node \"ip-10-0-133-209.ec2.internal\" DevicePath \"\""
Jul 02 14:16:10.538552 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:16:10.538523    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-09121-predictor-78d4984875-8wwtg"
Jul 02 14:16:10.538988 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:16:10.538527    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-09121-predictor-78d4984875-8wwtg" event={"ID":"3506c5ec-7374-43af-a888-c0e6eb72dd6d","Type":"ContainerDied","Data":"a736878da6c36ab8f6f0f12f4a9386189815380edde79c5cb597c51dff88b1e7"}
Jul 02 14:16:10.538988 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:16:10.538678    2558 scope.go:117] "RemoveContainer" containerID="0a19afe6c42f95c0fba6d51d18d149442c34775f593e9d337a1de983c101db10"
Jul 02 14:16:10.554727 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:16:10.554702    2558 scope.go:117] "RemoveContainer" containerID="cdb55adac258f7a997104d245b23836960189ceddbb32481424e04c96f7a60b2"
Jul 02 14:16:10.569026 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:16:10.568999    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-09121-predictor-78d4984875-8wwtg"]
Jul 02 14:16:10.573772 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:16:10.573745    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-09121-predictor-78d4984875-8wwtg"]
Jul 02 14:16:11.216476 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:16:11.216440    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3506c5ec-7374-43af-a888-c0e6eb72dd6d" path="/var/lib/kubelet/pods/3506c5ec-7374-43af-a888-c0e6eb72dd6d/volumes"
Jul 02 14:16:13.533411 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:16:13.533381    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-78590-predictor-64948cf447-bmtpr"
Jul 02 14:16:13.533852 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:16:13.533808    2558 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-78590-predictor-64948cf447-bmtpr" podUID="99bd2872-b70a-4d62-bddf-1947e802d6ed" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.27:8080: connect: connection refused"
Jul 02 14:16:19.475151 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:16:19.475108    2558 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-8d2ee-predictor-6f9b7fbf65-8r9xm" podUID="373c3199-3158-4e4c-8eab-6105fc0d4734" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.26:8080: connect: connection refused"
Jul 02 14:16:23.534639 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:16:23.534602    2558 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-78590-predictor-64948cf447-bmtpr" podUID="99bd2872-b70a-4d62-bddf-1947e802d6ed" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.27:8080: connect: connection refused"
Jul 02 14:16:29.475508 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:16:29.475468    2558 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-8d2ee-predictor-6f9b7fbf65-8r9xm" podUID="373c3199-3158-4e4c-8eab-6105fc0d4734" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.26:8080: connect: connection refused"
Jul 02 14:16:33.534485 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:16:33.534447    2558 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-78590-predictor-64948cf447-bmtpr" podUID="99bd2872-b70a-4d62-bddf-1947e802d6ed" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.27:8080: connect: connection refused"
Jul 02 14:16:39.474967 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:16:39.474939    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-8d2ee-predictor-6f9b7fbf65-8r9xm"
Jul 02 14:16:43.534080 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:16:43.534044    2558 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-78590-predictor-64948cf447-bmtpr" podUID="99bd2872-b70a-4d62-bddf-1947e802d6ed" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.27:8080: connect: connection refused"
Jul 02 14:16:53.534728 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:16:53.534653    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-78590-predictor-64948cf447-bmtpr"
Jul 02 14:18:55.310074 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:18:55.309956    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8f2ll_3310d1f7-1643-4017-9edf-71fa702d2615/ovn-acl-logging/0.log"
Jul 02 14:18:55.314210 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:18:55.310739    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8f2ll_3310d1f7-1643-4017-9edf-71fa702d2615/ovn-acl-logging/0.log"
Jul 02 14:23:55.333219 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:23:55.333107    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8f2ll_3310d1f7-1643-4017-9edf-71fa702d2615/ovn-acl-logging/0.log"
Jul 02 14:23:55.337187 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:23:55.334590    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8f2ll_3310d1f7-1643-4017-9edf-71fa702d2615/ovn-acl-logging/0.log"
Jul 02 14:25:21.001800 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:25:21.001766    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-78590-predictor-64948cf447-bmtpr"]
Jul 02 14:25:21.002361 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:25:21.002026    2558 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-78590-predictor-64948cf447-bmtpr" podUID="99bd2872-b70a-4d62-bddf-1947e802d6ed" containerName="kserve-container" containerID="cri-o://75419986d4d07e002f13d80e30a94dc144a2669a8f82f63107a240fa57af4f8f" gracePeriod=30
Jul 02 14:25:21.002361 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:25:21.002070    2558 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-78590-predictor-64948cf447-bmtpr" podUID="99bd2872-b70a-4d62-bddf-1947e802d6ed" containerName="kube-rbac-proxy" containerID="cri-o://a7a56fc4cb3a8297bd405e70525c03c180aa751927e43fadd4c1a257afcc6350" gracePeriod=30
Jul 02 14:25:21.032763 ip-10-0-133-209 kubenswrapper[2558]: E0702 14:25:21.032718    2558 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99bd2872_b70a_4d62_bddf_1947e802d6ed.slice/crio-a7a56fc4cb3a8297bd405e70525c03c180aa751927e43fadd4c1a257afcc6350.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99bd2872_b70a_4d62_bddf_1947e802d6ed.slice/crio-conmon-a7a56fc4cb3a8297bd405e70525c03c180aa751927e43fadd4c1a257afcc6350.scope\": RecentStats: unable to find data in memory cache]"
Jul 02 14:25:21.565631 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:25:21.565584    2558 generic.go:356] "Generic (PLEG): container finished" podID="99bd2872-b70a-4d62-bddf-1947e802d6ed" containerID="a7a56fc4cb3a8297bd405e70525c03c180aa751927e43fadd4c1a257afcc6350" exitCode=2
Jul 02 14:25:21.565805 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:25:21.565653    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-78590-predictor-64948cf447-bmtpr" event={"ID":"99bd2872-b70a-4d62-bddf-1947e802d6ed","Type":"ContainerDied","Data":"a7a56fc4cb3a8297bd405e70525c03c180aa751927e43fadd4c1a257afcc6350"}
Jul 02 14:25:23.529235 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:25:23.529192    2558 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-78590-predictor-64948cf447-bmtpr" podUID="99bd2872-b70a-4d62-bddf-1947e802d6ed" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.27:8643/healthz\": dial tcp 10.133.0.27:8643: connect: connection refused"
Jul 02 14:25:23.534727 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:25:23.534702    2558 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-78590-predictor-64948cf447-bmtpr" podUID="99bd2872-b70a-4d62-bddf-1947e802d6ed" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.27:8080: connect: connection refused"
Jul 02 14:25:24.255349 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:25:24.255325    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-78590-predictor-64948cf447-bmtpr"
Jul 02 14:25:24.330966 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:25:24.330872    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/99bd2872-b70a-4d62-bddf-1947e802d6ed-proxy-tls\") pod \"99bd2872-b70a-4d62-bddf-1947e802d6ed\" (UID: \"99bd2872-b70a-4d62-bddf-1947e802d6ed\") "
Jul 02 14:25:24.330966 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:25:24.330932    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lmtkp\" (UniqueName: \"kubernetes.io/projected/99bd2872-b70a-4d62-bddf-1947e802d6ed-kube-api-access-lmtkp\") pod \"99bd2872-b70a-4d62-bddf-1947e802d6ed\" (UID: \"99bd2872-b70a-4d62-bddf-1947e802d6ed\") "
Jul 02 14:25:24.330966 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:25:24.330960    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/99bd2872-b70a-4d62-bddf-1947e802d6ed-kube-rbac-proxy-sar-config\") pod \"99bd2872-b70a-4d62-bddf-1947e802d6ed\" (UID: \"99bd2872-b70a-4d62-bddf-1947e802d6ed\") "
Jul 02 14:25:24.331435 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:25:24.331405    2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/99bd2872-b70a-4d62-bddf-1947e802d6ed-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "99bd2872-b70a-4d62-bddf-1947e802d6ed" (UID: "99bd2872-b70a-4d62-bddf-1947e802d6ed"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 02 14:25:24.333136 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:25:24.333109    2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/99bd2872-b70a-4d62-bddf-1947e802d6ed-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "99bd2872-b70a-4d62-bddf-1947e802d6ed" (UID: "99bd2872-b70a-4d62-bddf-1947e802d6ed"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 02 14:25:24.333239 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:25:24.333168    2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/99bd2872-b70a-4d62-bddf-1947e802d6ed-kube-api-access-lmtkp" (OuterVolumeSpecName: "kube-api-access-lmtkp") pod "99bd2872-b70a-4d62-bddf-1947e802d6ed" (UID: "99bd2872-b70a-4d62-bddf-1947e802d6ed"). InnerVolumeSpecName "kube-api-access-lmtkp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 14:25:24.432338 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:25:24.432288    2558 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/99bd2872-b70a-4d62-bddf-1947e802d6ed-proxy-tls\") on node \"ip-10-0-133-209.ec2.internal\" DevicePath \"\""
Jul 02 14:25:24.432338 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:25:24.432332    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lmtkp\" (UniqueName: \"kubernetes.io/projected/99bd2872-b70a-4d62-bddf-1947e802d6ed-kube-api-access-lmtkp\") on node \"ip-10-0-133-209.ec2.internal\" DevicePath \"\""
Jul 02 14:25:24.432338 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:25:24.432343    2558 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/99bd2872-b70a-4d62-bddf-1947e802d6ed-kube-rbac-proxy-sar-config\") on node \"ip-10-0-133-209.ec2.internal\" DevicePath \"\""
Jul 02 14:25:24.577904 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:25:24.577866    2558 generic.go:356] "Generic (PLEG): container finished" podID="99bd2872-b70a-4d62-bddf-1947e802d6ed" containerID="75419986d4d07e002f13d80e30a94dc144a2669a8f82f63107a240fa57af4f8f" exitCode=0
Jul 02 14:25:24.577904 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:25:24.577910    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-78590-predictor-64948cf447-bmtpr" event={"ID":"99bd2872-b70a-4d62-bddf-1947e802d6ed","Type":"ContainerDied","Data":"75419986d4d07e002f13d80e30a94dc144a2669a8f82f63107a240fa57af4f8f"}
Jul 02 14:25:24.578425 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:25:24.577932    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-78590-predictor-64948cf447-bmtpr"
Jul 02 14:25:24.578425 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:25:24.577965    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-78590-predictor-64948cf447-bmtpr" event={"ID":"99bd2872-b70a-4d62-bddf-1947e802d6ed","Type":"ContainerDied","Data":"695248d5fa827762d417ba7cba6625463924b0a669cea1efc8cd19295a175d74"}
Jul 02 14:25:24.578425 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:25:24.577993    2558 scope.go:117] "RemoveContainer" containerID="a7a56fc4cb3a8297bd405e70525c03c180aa751927e43fadd4c1a257afcc6350"
Jul 02 14:25:24.586101 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:25:24.586082    2558 scope.go:117] "RemoveContainer" containerID="75419986d4d07e002f13d80e30a94dc144a2669a8f82f63107a240fa57af4f8f"
Jul 02 14:25:24.593122 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:25:24.593104    2558 scope.go:117] "RemoveContainer" containerID="a7a56fc4cb3a8297bd405e70525c03c180aa751927e43fadd4c1a257afcc6350"
Jul 02 14:25:24.593368 ip-10-0-133-209 kubenswrapper[2558]: E0702 14:25:24.593344    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a7a56fc4cb3a8297bd405e70525c03c180aa751927e43fadd4c1a257afcc6350\": container with ID starting with a7a56fc4cb3a8297bd405e70525c03c180aa751927e43fadd4c1a257afcc6350 not found: ID does not exist" containerID="a7a56fc4cb3a8297bd405e70525c03c180aa751927e43fadd4c1a257afcc6350"
Jul 02 14:25:24.593432 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:25:24.593380    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a7a56fc4cb3a8297bd405e70525c03c180aa751927e43fadd4c1a257afcc6350"} err="failed to get container status \"a7a56fc4cb3a8297bd405e70525c03c180aa751927e43fadd4c1a257afcc6350\": rpc error: code = NotFound desc = could not find container \"a7a56fc4cb3a8297bd405e70525c03c180aa751927e43fadd4c1a257afcc6350\": container with ID starting with a7a56fc4cb3a8297bd405e70525c03c180aa751927e43fadd4c1a257afcc6350 not found: ID does not exist"
Jul 02 14:25:24.593432 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:25:24.593405    2558 scope.go:117] "RemoveContainer" containerID="75419986d4d07e002f13d80e30a94dc144a2669a8f82f63107a240fa57af4f8f"
Jul 02 14:25:24.593659 ip-10-0-133-209 kubenswrapper[2558]: E0702 14:25:24.593641    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"75419986d4d07e002f13d80e30a94dc144a2669a8f82f63107a240fa57af4f8f\": container with ID starting with 75419986d4d07e002f13d80e30a94dc144a2669a8f82f63107a240fa57af4f8f not found: ID does not exist" containerID="75419986d4d07e002f13d80e30a94dc144a2669a8f82f63107a240fa57af4f8f"
Jul 02 14:25:24.593703 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:25:24.593665    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"75419986d4d07e002f13d80e30a94dc144a2669a8f82f63107a240fa57af4f8f"} err="failed to get container status \"75419986d4d07e002f13d80e30a94dc144a2669a8f82f63107a240fa57af4f8f\": rpc error: code = NotFound desc = could not find container \"75419986d4d07e002f13d80e30a94dc144a2669a8f82f63107a240fa57af4f8f\": container with ID starting with 75419986d4d07e002f13d80e30a94dc144a2669a8f82f63107a240fa57af4f8f not found: ID does not exist"
Jul 02 14:25:24.598985 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:25:24.598963    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-78590-predictor-64948cf447-bmtpr"]
Jul 02 14:25:24.603435 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:25:24.603412    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-78590-predictor-64948cf447-bmtpr"]
Jul 02 14:25:25.216969 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:25:25.216934    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="99bd2872-b70a-4d62-bddf-1947e802d6ed" path="/var/lib/kubelet/pods/99bd2872-b70a-4d62-bddf-1947e802d6ed/volumes"
Jul 02 14:28:55.358028 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:28:55.358003    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8f2ll_3310d1f7-1643-4017-9edf-71fa702d2615/ovn-acl-logging/0.log"
Jul 02 14:28:55.360390 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:28:55.359295    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8f2ll_3310d1f7-1643-4017-9edf-71fa702d2615/ovn-acl-logging/0.log"
Jul 02 14:33:11.762002 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:33:11.761964    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-8d2ee-predictor-6f9b7fbf65-8r9xm"]
Jul 02 14:33:11.762578 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:33:11.762184    2558 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-8d2ee-predictor-6f9b7fbf65-8r9xm" podUID="373c3199-3158-4e4c-8eab-6105fc0d4734" containerName="kserve-container" containerID="cri-o://7db11b1b2f00c99285e19b224e5fd1fa41d2bf0a13247a3d1b4ca5eaca4c276b" gracePeriod=30
Jul 02 14:33:11.762578 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:33:11.762215    2558 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-8d2ee-predictor-6f9b7fbf65-8r9xm" podUID="373c3199-3158-4e4c-8eab-6105fc0d4734" containerName="kube-rbac-proxy" containerID="cri-o://8ede763652faf80e1b94a87ced1c1201e1ad017720952cbbbbd14e244a1548c3" gracePeriod=30
Jul 02 14:33:12.260584 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:33:12.260551    2558 generic.go:356] "Generic (PLEG): container finished" podID="373c3199-3158-4e4c-8eab-6105fc0d4734" containerID="8ede763652faf80e1b94a87ced1c1201e1ad017720952cbbbbd14e244a1548c3" exitCode=2
Jul 02 14:33:12.260774 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:33:12.260609    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-8d2ee-predictor-6f9b7fbf65-8r9xm" event={"ID":"373c3199-3158-4e4c-8eab-6105fc0d4734","Type":"ContainerDied","Data":"8ede763652faf80e1b94a87ced1c1201e1ad017720952cbbbbd14e244a1548c3"}
Jul 02 14:33:14.501102 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:33:14.501080    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-8d2ee-predictor-6f9b7fbf65-8r9xm"
Jul 02 14:33:14.652744 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:33:14.652669    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9rq9s\" (UniqueName: \"kubernetes.io/projected/373c3199-3158-4e4c-8eab-6105fc0d4734-kube-api-access-9rq9s\") pod \"373c3199-3158-4e4c-8eab-6105fc0d4734\" (UID: \"373c3199-3158-4e4c-8eab-6105fc0d4734\") "
Jul 02 14:33:14.652744 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:33:14.652701    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/373c3199-3158-4e4c-8eab-6105fc0d4734-kube-rbac-proxy-sar-config\") pod \"373c3199-3158-4e4c-8eab-6105fc0d4734\" (UID: \"373c3199-3158-4e4c-8eab-6105fc0d4734\") "
Jul 02 14:33:14.652959 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:33:14.652760    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/373c3199-3158-4e4c-8eab-6105fc0d4734-proxy-tls\") pod \"373c3199-3158-4e4c-8eab-6105fc0d4734\" (UID: \"373c3199-3158-4e4c-8eab-6105fc0d4734\") "
Jul 02 14:33:14.653179 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:33:14.653146    2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/373c3199-3158-4e4c-8eab-6105fc0d4734-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "373c3199-3158-4e4c-8eab-6105fc0d4734" (UID: "373c3199-3158-4e4c-8eab-6105fc0d4734"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 02 14:33:14.654979 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:33:14.654953    2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/373c3199-3158-4e4c-8eab-6105fc0d4734-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "373c3199-3158-4e4c-8eab-6105fc0d4734" (UID: "373c3199-3158-4e4c-8eab-6105fc0d4734"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 02 14:33:14.655069 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:33:14.654969    2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/373c3199-3158-4e4c-8eab-6105fc0d4734-kube-api-access-9rq9s" (OuterVolumeSpecName: "kube-api-access-9rq9s") pod "373c3199-3158-4e4c-8eab-6105fc0d4734" (UID: "373c3199-3158-4e4c-8eab-6105fc0d4734"). InnerVolumeSpecName "kube-api-access-9rq9s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 14:33:14.753427 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:33:14.753395    2558 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/373c3199-3158-4e4c-8eab-6105fc0d4734-proxy-tls\") on node \"ip-10-0-133-209.ec2.internal\" DevicePath \"\""
Jul 02 14:33:14.753427 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:33:14.753423    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9rq9s\" (UniqueName: \"kubernetes.io/projected/373c3199-3158-4e4c-8eab-6105fc0d4734-kube-api-access-9rq9s\") on node \"ip-10-0-133-209.ec2.internal\" DevicePath \"\""
Jul 02 14:33:14.753606 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:33:14.753436    2558 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/373c3199-3158-4e4c-8eab-6105fc0d4734-kube-rbac-proxy-sar-config\") on node \"ip-10-0-133-209.ec2.internal\" DevicePath \"\""
Jul 02 14:33:15.272435 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:33:15.272398    2558 generic.go:356] "Generic (PLEG): container finished" podID="373c3199-3158-4e4c-8eab-6105fc0d4734" containerID="7db11b1b2f00c99285e19b224e5fd1fa41d2bf0a13247a3d1b4ca5eaca4c276b" exitCode=0
Jul 02 14:33:15.272554 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:33:15.272459    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-8d2ee-predictor-6f9b7fbf65-8r9xm"
Jul 02 14:33:15.272554 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:33:15.272462    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-8d2ee-predictor-6f9b7fbf65-8r9xm" event={"ID":"373c3199-3158-4e4c-8eab-6105fc0d4734","Type":"ContainerDied","Data":"7db11b1b2f00c99285e19b224e5fd1fa41d2bf0a13247a3d1b4ca5eaca4c276b"}
Jul 02 14:33:15.272642 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:33:15.272570    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-8d2ee-predictor-6f9b7fbf65-8r9xm" event={"ID":"373c3199-3158-4e4c-8eab-6105fc0d4734","Type":"ContainerDied","Data":"9013e508e55ff4ec4c5d4e8aace82996886d26df806d8398734b44cd05632834"}
Jul 02 14:33:15.272642 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:33:15.272603    2558 scope.go:117] "RemoveContainer" containerID="8ede763652faf80e1b94a87ced1c1201e1ad017720952cbbbbd14e244a1548c3"
Jul 02 14:33:15.280669 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:33:15.280648    2558 scope.go:117] "RemoveContainer" containerID="7db11b1b2f00c99285e19b224e5fd1fa41d2bf0a13247a3d1b4ca5eaca4c276b"
Jul 02 14:33:15.287633 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:33:15.287615    2558 scope.go:117] "RemoveContainer" containerID="8ede763652faf80e1b94a87ced1c1201e1ad017720952cbbbbd14e244a1548c3"
Jul 02 14:33:15.287909 ip-10-0-133-209 kubenswrapper[2558]: E0702 14:33:15.287886    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8ede763652faf80e1b94a87ced1c1201e1ad017720952cbbbbd14e244a1548c3\": container with ID starting with 8ede763652faf80e1b94a87ced1c1201e1ad017720952cbbbbd14e244a1548c3 not found: ID does not exist" containerID="8ede763652faf80e1b94a87ced1c1201e1ad017720952cbbbbd14e244a1548c3"
Jul 02 14:33:15.287983 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:33:15.287922    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8ede763652faf80e1b94a87ced1c1201e1ad017720952cbbbbd14e244a1548c3"} err="failed to get container status \"8ede763652faf80e1b94a87ced1c1201e1ad017720952cbbbbd14e244a1548c3\": rpc error: code = NotFound desc = could not find container \"8ede763652faf80e1b94a87ced1c1201e1ad017720952cbbbbd14e244a1548c3\": container with ID starting with 8ede763652faf80e1b94a87ced1c1201e1ad017720952cbbbbd14e244a1548c3 not found: ID does not exist"
Jul 02 14:33:15.287983 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:33:15.287939    2558 scope.go:117] "RemoveContainer" containerID="7db11b1b2f00c99285e19b224e5fd1fa41d2bf0a13247a3d1b4ca5eaca4c276b"
Jul 02 14:33:15.288173 ip-10-0-133-209 kubenswrapper[2558]: E0702 14:33:15.288157    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7db11b1b2f00c99285e19b224e5fd1fa41d2bf0a13247a3d1b4ca5eaca4c276b\": container with ID starting with 7db11b1b2f00c99285e19b224e5fd1fa41d2bf0a13247a3d1b4ca5eaca4c276b not found: ID does not exist" containerID="7db11b1b2f00c99285e19b224e5fd1fa41d2bf0a13247a3d1b4ca5eaca4c276b"
Jul 02 14:33:15.288212 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:33:15.288179    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7db11b1b2f00c99285e19b224e5fd1fa41d2bf0a13247a3d1b4ca5eaca4c276b"} err="failed to get container status \"7db11b1b2f00c99285e19b224e5fd1fa41d2bf0a13247a3d1b4ca5eaca4c276b\": rpc error: code = NotFound desc = could not find container \"7db11b1b2f00c99285e19b224e5fd1fa41d2bf0a13247a3d1b4ca5eaca4c276b\": container with ID starting with 7db11b1b2f00c99285e19b224e5fd1fa41d2bf0a13247a3d1b4ca5eaca4c276b not found: ID does not exist"
Jul 02 14:33:15.303137 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:33:15.303114    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-8d2ee-predictor-6f9b7fbf65-8r9xm"]
Jul 02 14:33:15.307569 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:33:15.307547    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-8d2ee-predictor-6f9b7fbf65-8r9xm"]
Jul 02 14:33:15.471211 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:33:15.471164    2558 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-8d2ee-predictor-6f9b7fbf65-8r9xm" podUID="373c3199-3158-4e4c-8eab-6105fc0d4734" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.26:8643/healthz\": context deadline exceeded"
Jul 02 14:33:17.217087 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:33:17.217012    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="373c3199-3158-4e4c-8eab-6105fc0d4734" path="/var/lib/kubelet/pods/373c3199-3158-4e4c-8eab-6105fc0d4734/volumes"
Jul 02 14:33:44.165390 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:33:44.165353    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-p8hbs/must-gather-gvhth"]
Jul 02 14:33:44.165899 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:33:44.165609    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="99bd2872-b70a-4d62-bddf-1947e802d6ed" containerName="kserve-container"
Jul 02 14:33:44.165899 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:33:44.165619    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="99bd2872-b70a-4d62-bddf-1947e802d6ed" containerName="kserve-container"
Jul 02 14:33:44.165899 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:33:44.165630    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="373c3199-3158-4e4c-8eab-6105fc0d4734" containerName="kube-rbac-proxy"
Jul 02 14:33:44.165899 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:33:44.165636    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="373c3199-3158-4e4c-8eab-6105fc0d4734" containerName="kube-rbac-proxy"
Jul 02 14:33:44.165899 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:33:44.165642    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3506c5ec-7374-43af-a888-c0e6eb72dd6d" containerName="kube-rbac-proxy"
Jul 02 14:33:44.165899 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:33:44.165648    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="3506c5ec-7374-43af-a888-c0e6eb72dd6d" containerName="kube-rbac-proxy"
Jul 02 14:33:44.165899 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:33:44.165653    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="99bd2872-b70a-4d62-bddf-1947e802d6ed" containerName="kube-rbac-proxy"
Jul 02 14:33:44.165899 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:33:44.165658    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="99bd2872-b70a-4d62-bddf-1947e802d6ed" containerName="kube-rbac-proxy"
Jul 02 14:33:44.165899 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:33:44.165671    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="373c3199-3158-4e4c-8eab-6105fc0d4734" containerName="kserve-container"
Jul 02 14:33:44.165899 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:33:44.165676    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="373c3199-3158-4e4c-8eab-6105fc0d4734" containerName="kserve-container"
Jul 02 14:33:44.165899 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:33:44.165681    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3506c5ec-7374-43af-a888-c0e6eb72dd6d" containerName="kserve-container"
Jul 02 14:33:44.165899 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:33:44.165685    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="3506c5ec-7374-43af-a888-c0e6eb72dd6d" containerName="kserve-container"
Jul 02 14:33:44.165899 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:33:44.165728    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="99bd2872-b70a-4d62-bddf-1947e802d6ed" containerName="kserve-container"
Jul 02 14:33:44.165899 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:33:44.165736    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="373c3199-3158-4e4c-8eab-6105fc0d4734" containerName="kserve-container"
Jul 02 14:33:44.165899 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:33:44.165742    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="99bd2872-b70a-4d62-bddf-1947e802d6ed" containerName="kube-rbac-proxy"
Jul 02 14:33:44.165899 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:33:44.165749    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="3506c5ec-7374-43af-a888-c0e6eb72dd6d" containerName="kserve-container"
Jul 02 14:33:44.165899 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:33:44.165754    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="373c3199-3158-4e4c-8eab-6105fc0d4734" containerName="kube-rbac-proxy"
Jul 02 14:33:44.165899 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:33:44.165759    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="3506c5ec-7374-43af-a888-c0e6eb72dd6d" containerName="kube-rbac-proxy"
Jul 02 14:33:44.168462 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:33:44.168444    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-p8hbs/must-gather-gvhth"
Jul 02 14:33:44.171029 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:33:44.171009    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-p8hbs\"/\"default-dockercfg-4f72d\""
Jul 02 14:33:44.171269 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:33:44.171257    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-p8hbs\"/\"openshift-service-ca.crt\""
Jul 02 14:33:44.172308 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:33:44.172290    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-p8hbs\"/\"kube-root-ca.crt\""
Jul 02 14:33:44.182561 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:33:44.182531    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-p8hbs/must-gather-gvhth"]
Jul 02 14:33:44.369435 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:33:44.369396    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/7e9e292a-9710-4fe7-b8f6-d274e3f9586a-must-gather-output\") pod \"must-gather-gvhth\" (UID: \"7e9e292a-9710-4fe7-b8f6-d274e3f9586a\") " pod="openshift-must-gather-p8hbs/must-gather-gvhth"
Jul 02 14:33:44.369435 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:33:44.369435    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l6vnd\" (UniqueName: \"kubernetes.io/projected/7e9e292a-9710-4fe7-b8f6-d274e3f9586a-kube-api-access-l6vnd\") pod \"must-gather-gvhth\" (UID: \"7e9e292a-9710-4fe7-b8f6-d274e3f9586a\") " pod="openshift-must-gather-p8hbs/must-gather-gvhth"
Jul 02 14:33:44.470566 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:33:44.470471    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/7e9e292a-9710-4fe7-b8f6-d274e3f9586a-must-gather-output\") pod \"must-gather-gvhth\" (UID: \"7e9e292a-9710-4fe7-b8f6-d274e3f9586a\") " pod="openshift-must-gather-p8hbs/must-gather-gvhth"
Jul 02 14:33:44.470566 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:33:44.470512    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l6vnd\" (UniqueName: \"kubernetes.io/projected/7e9e292a-9710-4fe7-b8f6-d274e3f9586a-kube-api-access-l6vnd\") pod \"must-gather-gvhth\" (UID: \"7e9e292a-9710-4fe7-b8f6-d274e3f9586a\") " pod="openshift-must-gather-p8hbs/must-gather-gvhth"
Jul 02 14:33:44.470863 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:33:44.470811    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/7e9e292a-9710-4fe7-b8f6-d274e3f9586a-must-gather-output\") pod \"must-gather-gvhth\" (UID: \"7e9e292a-9710-4fe7-b8f6-d274e3f9586a\") " pod="openshift-must-gather-p8hbs/must-gather-gvhth"
Jul 02 14:33:44.480561 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:33:44.480541    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l6vnd\" (UniqueName: \"kubernetes.io/projected/7e9e292a-9710-4fe7-b8f6-d274e3f9586a-kube-api-access-l6vnd\") pod \"must-gather-gvhth\" (UID: \"7e9e292a-9710-4fe7-b8f6-d274e3f9586a\") " pod="openshift-must-gather-p8hbs/must-gather-gvhth"
Jul 02 14:33:44.777754 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:33:44.777643    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-p8hbs/must-gather-gvhth"
Jul 02 14:33:44.908708 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:33:44.908683    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-p8hbs/must-gather-gvhth"]
Jul 02 14:33:44.911231 ip-10-0-133-209 kubenswrapper[2558]: W0702 14:33:44.911205    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7e9e292a_9710_4fe7_b8f6_d274e3f9586a.slice/crio-f3f843b8fbb892df23040d4e05a622d11f9145202aef0eda859578a3fe2c1504 WatchSource:0}: Error finding container f3f843b8fbb892df23040d4e05a622d11f9145202aef0eda859578a3fe2c1504: Status 404 returned error can't find the container with id f3f843b8fbb892df23040d4e05a622d11f9145202aef0eda859578a3fe2c1504
Jul 02 14:33:44.913170 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:33:44.913153    2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 02 14:33:45.381061 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:33:45.381029    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-p8hbs/must-gather-gvhth" event={"ID":"7e9e292a-9710-4fe7-b8f6-d274e3f9586a","Type":"ContainerStarted","Data":"f3f843b8fbb892df23040d4e05a622d11f9145202aef0eda859578a3fe2c1504"}
Jul 02 14:33:46.392609 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:33:46.392562    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-p8hbs/must-gather-gvhth" event={"ID":"7e9e292a-9710-4fe7-b8f6-d274e3f9586a","Type":"ContainerStarted","Data":"3287c22a4096d872c1450cc67f0a0f77e013f6ed1f38c748e322c2de88e432fb"}
Jul 02 14:33:46.393119 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:33:46.392625    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-p8hbs/must-gather-gvhth" event={"ID":"7e9e292a-9710-4fe7-b8f6-d274e3f9586a","Type":"ContainerStarted","Data":"041a2903314199609e38575e7483f7ae59fbc8f1a235c3c04c3ca89b29517668"}
Jul 02 14:33:46.414022 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:33:46.413971    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-p8hbs/must-gather-gvhth" podStartSLOduration=1.580681322 podStartE2EDuration="2.413952502s" totalImagesPullingTime="833.27118ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 14:33:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:33:46.41173785 +0000 UTC m=+2991.590200035" watchObservedRunningTime="2026-07-02 14:33:46.413952502 +0000 UTC m=+2991.592414683"
Jul 02 14:33:47.251663 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:33:47.251633    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-2tjcl_6ebfc25f-6722-4f73-98b8-18b416460437/global-pull-secret-syncer/0.log"
Jul 02 14:33:47.438672 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:33:47.438640    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-m4ndl_23638fc3-c572-4760-8e60-e82de1c6f290/konnectivity-agent/0.log"
Jul 02 14:33:47.492734 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:33:47.492697    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-133-209.ec2.internal_4d8a265dbb955726a248c0c5b7ee4114/haproxy/0.log"
Jul 02 14:33:51.640053 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:33:51.640021    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-cd8656b8-5v7b8_cd5df5ba-88e2-4926-b63c-f9fe0cdb8faa/monitoring-plugin/0.log"
Jul 02 14:33:51.699010 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:33:51.698976    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-bmljc_4f2e0e73-0517-440f-95b7-28364750732f/node-exporter/0.log"
Jul 02 14:33:51.730135 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:33:51.730106    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-bmljc_4f2e0e73-0517-440f-95b7-28364750732f/kube-rbac-proxy/0.log"
Jul 02 14:33:51.766582 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:33:51.766552    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-bmljc_4f2e0e73-0517-440f-95b7-28364750732f/init-textfile/0.log"
Jul 02 14:33:52.448376 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:33:52.448339    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-685f676f49-xk5v9_ce9206ce-a65b-4da4-9391-e64748d0518d/telemeter-client/0.log"
Jul 02 14:33:52.476565 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:33:52.476526    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-685f676f49-xk5v9_ce9206ce-a65b-4da4-9391-e64748d0518d/reload/0.log"
Jul 02 14:33:52.507257 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:33:52.507230    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-685f676f49-xk5v9_ce9206ce-a65b-4da4-9391-e64748d0518d/kube-rbac-proxy/0.log"
Jul 02 14:33:52.543306 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:33:52.543275    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-57879888b8-q8md5_277131f0-f6c3-4190-b01f-dabc843e7403/thanos-query/0.log"
Jul 02 14:33:52.567430 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:33:52.567396    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-57879888b8-q8md5_277131f0-f6c3-4190-b01f-dabc843e7403/kube-rbac-proxy-web/0.log"
Jul 02 14:33:52.602587 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:33:52.602551    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-57879888b8-q8md5_277131f0-f6c3-4190-b01f-dabc843e7403/kube-rbac-proxy/0.log"
Jul 02 14:33:52.637634 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:33:52.637598    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-57879888b8-q8md5_277131f0-f6c3-4190-b01f-dabc843e7403/prom-label-proxy/0.log"
Jul 02 14:33:52.660902 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:33:52.660879    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-57879888b8-q8md5_277131f0-f6c3-4190-b01f-dabc843e7403/kube-rbac-proxy-rules/0.log"
Jul 02 14:33:52.692176 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:33:52.692148    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-57879888b8-q8md5_277131f0-f6c3-4190-b01f-dabc843e7403/kube-rbac-proxy-metrics/0.log"
Jul 02 14:33:55.389383 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:33:55.389258    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8f2ll_3310d1f7-1643-4017-9edf-71fa702d2615/ovn-acl-logging/0.log"
Jul 02 14:33:55.394481 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:33:55.392123    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8f2ll_3310d1f7-1643-4017-9edf-71fa702d2615/ovn-acl-logging/0.log"
Jul 02 14:33:55.559204 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:33:55.559168    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-69d9f679db-g87ql_bf9d29ad-19aa-4844-8742-d5f10adcaad8/volume-data-source-validator/0.log"
Jul 02 14:33:55.886870 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:33:55.886812    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-p8hbs/perf-node-gather-daemonset-w4p82"]
Jul 02 14:33:55.892079 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:33:55.892050    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-p8hbs/perf-node-gather-daemonset-w4p82"
Jul 02 14:33:55.905233 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:33:55.905207    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-p8hbs/perf-node-gather-daemonset-w4p82"]
Jul 02 14:33:55.981698 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:33:55.981667    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6175820b-4de3-49c0-b02d-21f2aa56ee8e-podres\") pod \"perf-node-gather-daemonset-w4p82\" (UID: \"6175820b-4de3-49c0-b02d-21f2aa56ee8e\") " pod="openshift-must-gather-p8hbs/perf-node-gather-daemonset-w4p82"
Jul 02 14:33:55.981912 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:33:55.981715    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6175820b-4de3-49c0-b02d-21f2aa56ee8e-proc\") pod \"perf-node-gather-daemonset-w4p82\" (UID: \"6175820b-4de3-49c0-b02d-21f2aa56ee8e\") " pod="openshift-must-gather-p8hbs/perf-node-gather-daemonset-w4p82"
Jul 02 14:33:55.981912 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:33:55.981779    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6175820b-4de3-49c0-b02d-21f2aa56ee8e-sys\") pod \"perf-node-gather-daemonset-w4p82\" (UID: \"6175820b-4de3-49c0-b02d-21f2aa56ee8e\") " pod="openshift-must-gather-p8hbs/perf-node-gather-daemonset-w4p82"
Jul 02 14:33:55.981912 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:33:55.981881    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6175820b-4de3-49c0-b02d-21f2aa56ee8e-lib-modules\") pod \"perf-node-gather-daemonset-w4p82\" (UID: \"6175820b-4de3-49c0-b02d-21f2aa56ee8e\") " pod="openshift-must-gather-p8hbs/perf-node-gather-daemonset-w4p82"
Jul 02 14:33:55.981912 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:33:55.981908    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tcw2d\" (UniqueName: \"kubernetes.io/projected/6175820b-4de3-49c0-b02d-21f2aa56ee8e-kube-api-access-tcw2d\") pod \"perf-node-gather-daemonset-w4p82\" (UID: \"6175820b-4de3-49c0-b02d-21f2aa56ee8e\") " pod="openshift-must-gather-p8hbs/perf-node-gather-daemonset-w4p82"
Jul 02 14:33:56.083229 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:33:56.083171    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6175820b-4de3-49c0-b02d-21f2aa56ee8e-sys\") pod \"perf-node-gather-daemonset-w4p82\" (UID: \"6175820b-4de3-49c0-b02d-21f2aa56ee8e\") " pod="openshift-must-gather-p8hbs/perf-node-gather-daemonset-w4p82"
Jul 02 14:33:56.083421 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:33:56.083243    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6175820b-4de3-49c0-b02d-21f2aa56ee8e-lib-modules\") pod \"perf-node-gather-daemonset-w4p82\" (UID: \"6175820b-4de3-49c0-b02d-21f2aa56ee8e\") " pod="openshift-must-gather-p8hbs/perf-node-gather-daemonset-w4p82"
Jul 02 14:33:56.083421 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:33:56.083273    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tcw2d\" (UniqueName: \"kubernetes.io/projected/6175820b-4de3-49c0-b02d-21f2aa56ee8e-kube-api-access-tcw2d\") pod \"perf-node-gather-daemonset-w4p82\" (UID: \"6175820b-4de3-49c0-b02d-21f2aa56ee8e\") " pod="openshift-must-gather-p8hbs/perf-node-gather-daemonset-w4p82"
Jul 02 14:33:56.083421 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:33:56.083327    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6175820b-4de3-49c0-b02d-21f2aa56ee8e-podres\") pod \"perf-node-gather-daemonset-w4p82\" (UID: \"6175820b-4de3-49c0-b02d-21f2aa56ee8e\") " pod="openshift-must-gather-p8hbs/perf-node-gather-daemonset-w4p82"
Jul 02 14:33:56.083421 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:33:56.083381    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6175820b-4de3-49c0-b02d-21f2aa56ee8e-proc\") pod \"perf-node-gather-daemonset-w4p82\" (UID: \"6175820b-4de3-49c0-b02d-21f2aa56ee8e\") " pod="openshift-must-gather-p8hbs/perf-node-gather-daemonset-w4p82"
Jul 02 14:33:56.083637 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:33:56.083474    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6175820b-4de3-49c0-b02d-21f2aa56ee8e-proc\") pod \"perf-node-gather-daemonset-w4p82\" (UID: \"6175820b-4de3-49c0-b02d-21f2aa56ee8e\") " pod="openshift-must-gather-p8hbs/perf-node-gather-daemonset-w4p82"
Jul 02 14:33:56.083637 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:33:56.083535    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6175820b-4de3-49c0-b02d-21f2aa56ee8e-sys\") pod \"perf-node-gather-daemonset-w4p82\" (UID: \"6175820b-4de3-49c0-b02d-21f2aa56ee8e\") " pod="openshift-must-gather-p8hbs/perf-node-gather-daemonset-w4p82"
Jul 02 14:33:56.083744 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:33:56.083651    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6175820b-4de3-49c0-b02d-21f2aa56ee8e-lib-modules\") pod \"perf-node-gather-daemonset-w4p82\" (UID: \"6175820b-4de3-49c0-b02d-21f2aa56ee8e\") " pod="openshift-must-gather-p8hbs/perf-node-gather-daemonset-w4p82"
Jul 02 14:33:56.083993 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:33:56.083963    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6175820b-4de3-49c0-b02d-21f2aa56ee8e-podres\") pod \"perf-node-gather-daemonset-w4p82\" (UID: \"6175820b-4de3-49c0-b02d-21f2aa56ee8e\") " pod="openshift-must-gather-p8hbs/perf-node-gather-daemonset-w4p82"
Jul 02 14:33:56.092736 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:33:56.092688    2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tcw2d\" (UniqueName: \"kubernetes.io/projected/6175820b-4de3-49c0-b02d-21f2aa56ee8e-kube-api-access-tcw2d\") pod \"perf-node-gather-daemonset-w4p82\" (UID: \"6175820b-4de3-49c0-b02d-21f2aa56ee8e\") " pod="openshift-must-gather-p8hbs/perf-node-gather-daemonset-w4p82"
Jul 02 14:33:56.206200 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:33:56.206103    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-p8hbs/perf-node-gather-daemonset-w4p82"
Jul 02 14:33:56.364086 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:33:56.363707    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-p8hbs/perf-node-gather-daemonset-w4p82"]
Jul 02 14:33:56.443502 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:33:56.443465    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-p8hbs/perf-node-gather-daemonset-w4p82" event={"ID":"6175820b-4de3-49c0-b02d-21f2aa56ee8e","Type":"ContainerStarted","Data":"1e93700109411ad41196aa9666c9fb2c8fb2f3950e6ff91e6e30e79400d782c6"}
Jul 02 14:33:56.729535 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:33:56.729505    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-zcqdj_2bf21a2c-20d4-47ce-922b-b48612f3c438/dns/0.log"
Jul 02 14:33:56.768635 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:33:56.768609    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-zcqdj_2bf21a2c-20d4-47ce-922b-b48612f3c438/kube-rbac-proxy/0.log"
Jul 02 14:33:56.845188 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:33:56.845156    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-g9pll_859dd06a-dc53-48ca-a2ea-9fea2bad0253/dns-node-resolver/0.log"
Jul 02 14:33:57.449901 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:33:57.449854    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-p8hbs/perf-node-gather-daemonset-w4p82" event={"ID":"6175820b-4de3-49c0-b02d-21f2aa56ee8e","Type":"ContainerStarted","Data":"8ee3a4bbfdb8b7f51c592f9b084046d841f4e79392f3b7bd14cdd0061c99bda5"}
Jul 02 14:33:57.450380 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:33:57.450027    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-p8hbs/perf-node-gather-daemonset-w4p82"
Jul 02 14:33:57.483126 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:33:57.483078    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-p8hbs/perf-node-gather-daemonset-w4p82" podStartSLOduration=2.483061894 podStartE2EDuration="2.483061894s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 14:33:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:33:57.478494406 +0000 UTC m=+3002.656956592" watchObservedRunningTime="2026-07-02 14:33:57.483061894 +0000 UTC m=+3002.661524077"
Jul 02 14:33:57.616423 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:33:57.616383    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-t99mm_de0396fd-b15d-4e52-b0af-7e40954f53db/node-ca/0.log"
Jul 02 14:33:58.501052 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:33:58.501019    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-75dffdcbd7-bsn5q_a4cc908b-b2a6-409e-8377-9ed59a79ff27/router/0.log"
Jul 02 14:33:58.911247 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:33:58.911212    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-gcr27_7b4fd04b-30e6-4599-b877-2719d6d08231/serve-healthcheck-canary/0.log"
Jul 02 14:33:59.472364 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:33:59.472337    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-nv5c5_a5dbc04b-cfab-4330-b693-5cf3fa6c7263/kube-rbac-proxy/0.log"
Jul 02 14:33:59.498647 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:33:59.498618    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-nv5c5_a5dbc04b-cfab-4330-b693-5cf3fa6c7263/exporter/0.log"
Jul 02 14:33:59.526710 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:33:59.526679    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-nv5c5_a5dbc04b-cfab-4330-b693-5cf3fa6c7263/extractor/0.log"
Jul 02 14:34:01.639738 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:34:01.639693    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_llmisvc-controller-manager-67d95fdc48-2frl4_07f30b08-ece7-479a-9c59-839506520dc5/manager/0.log"
Jul 02 14:34:03.466814 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:34:03.466782    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-p8hbs/perf-node-gather-daemonset-w4p82"
Jul 02 14:34:07.382993 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:34:07.382955    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-976zg_364e6172-44ee-4b85-a48c-a0f29679abce/kube-multus/0.log"
Jul 02 14:34:07.601197 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:34:07.601168    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-s85vc_e1521a8d-402a-4bee-bc35-bfa382300084/kube-multus-additional-cni-plugins/0.log"
Jul 02 14:34:07.625404 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:34:07.625371    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-s85vc_e1521a8d-402a-4bee-bc35-bfa382300084/egress-router-binary-copy/0.log"
Jul 02 14:34:07.647757 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:34:07.647728    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-s85vc_e1521a8d-402a-4bee-bc35-bfa382300084/cni-plugins/0.log"
Jul 02 14:34:07.668143 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:34:07.668117    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-s85vc_e1521a8d-402a-4bee-bc35-bfa382300084/bond-cni-plugin/0.log"
Jul 02 14:34:07.688883 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:34:07.688854    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-s85vc_e1521a8d-402a-4bee-bc35-bfa382300084/routeoverride-cni/0.log"
Jul 02 14:34:07.710964 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:34:07.710934    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-s85vc_e1521a8d-402a-4bee-bc35-bfa382300084/whereabouts-cni-bincopy/0.log"
Jul 02 14:34:07.734223 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:34:07.734193    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-s85vc_e1521a8d-402a-4bee-bc35-bfa382300084/whereabouts-cni/0.log"
Jul 02 14:34:07.942522 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:34:07.942484    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-29552_ad2afabf-42cb-4619-b249-a2b011e32565/network-metrics-daemon/0.log"
Jul 02 14:34:07.962910 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:34:07.962880    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-29552_ad2afabf-42cb-4619-b249-a2b011e32565/kube-rbac-proxy/0.log"
Jul 02 14:34:08.795786 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:34:08.795756    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8f2ll_3310d1f7-1643-4017-9edf-71fa702d2615/ovn-controller/0.log"
Jul 02 14:34:08.813526 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:34:08.813491    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8f2ll_3310d1f7-1643-4017-9edf-71fa702d2615/ovn-acl-logging/1.log"
Jul 02 14:34:08.813526 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:34:08.813531    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8f2ll_3310d1f7-1643-4017-9edf-71fa702d2615/ovn-acl-logging/0.log"
Jul 02 14:34:08.836545 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:34:08.836519    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8f2ll_3310d1f7-1643-4017-9edf-71fa702d2615/kube-rbac-proxy-node/0.log"
Jul 02 14:34:08.859977 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:34:08.859950    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8f2ll_3310d1f7-1643-4017-9edf-71fa702d2615/kube-rbac-proxy-ovn-metrics/0.log"
Jul 02 14:34:08.879466 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:34:08.879438    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8f2ll_3310d1f7-1643-4017-9edf-71fa702d2615/northd/0.log"
Jul 02 14:34:08.901747 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:34:08.901724    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8f2ll_3310d1f7-1643-4017-9edf-71fa702d2615/nbdb/0.log"
Jul 02 14:34:08.924135 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:34:08.924109    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8f2ll_3310d1f7-1643-4017-9edf-71fa702d2615/sbdb/0.log"
Jul 02 14:34:08.992322 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:34:08.992287    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8f2ll_3310d1f7-1643-4017-9edf-71fa702d2615/ovnkube-controller/0.log"
Jul 02 14:34:10.470782 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:34:10.470754    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-q9w6j_06e4af50-04c6-4c58-af10-0e0a28c1a261/network-check-target-container/0.log"
Jul 02 14:34:11.360545 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:34:11.360516    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-lxhdr_7e1c7379-372d-48f3-a865-d0510d6c27a9/iptables-alerter/0.log"
Jul 02 14:34:12.093617 ip-10-0-133-209 kubenswrapper[2558]: I0702 14:34:12.093577    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-node-tuning-operator_tuned-scnm4_823f515b-2ffa-4001-a325-1acc4a714ca3/tuned/0.log"